bird of maps

competency N

evaluate programs and services on specified criteria


Introduction

Progress often extends from examining and evaluating one's options. In professional situations, the ability to evaluate current and prospective programs, services, and tools is crucial. It is a fundamental part of informed decision-making, and has a direct influence on the quality of the services that are provided for our customers and patrons.

There are two types of evaluation that are relevant to information professionals. Primarily, it is necessary to evaluate how well specific services match the goals of the organization, and by extension, also the needs of the patrons. The organization's overarching goals can often serve as a broad criteria against which the evaluation of specific services can be measured.

The second, equally important level at which to evaluate services concerns the public-facing services that an organization's users interact with directly. An example may be the evaluation of a chat reference service. This type of evaluation also applies when examining prospective and future services or tools, and it may be advantageous to examine how another organization's parallel attempt can be applied to one's own. Services provided by information professionals that should be frequently evaluated include events and programs, reference and research tools and assistance, operational procedures, and any specific resources that are utilized to provide these services.

The criteria for evaluating a service, resource, or tools for the delivery of services will be developed through the lens of a specific need. The criteria must also be rooted in organizational policies, our experiences, and our observations. To measure a service against these criteria, research is required, since an evaluation involves examining how successful something is at meeting the needs of users in question. This may not always be the patrons, since information professionals themselves must also utilize and acquire resources in order to provide services. Evaluation tools used to acquire information to measure against the evaluation criteria may include usage statistics, first-hand trials, professional literature and articles, and reviews of resources, events, and vendors.

Evidence

To demonstrate my competency in evaluating a web-based service using specified criteria, I am submitting an essay that I composed for the final exam of my Information Retrieval course. In this essay, I evaluate the user interface of the academic database Library Literature and Information Science Full Text, produced by WilsonWeb. The criteria that I used for my evaluation comes from Donald A. Norman's book The Design of Everyday Things, which discusses the concepts of good design. Using Norman's ideas, I conducted a thorough evaluation of the main interface of the database from the perspective of an intended user of this resource.

Another valuable and important service provided by information organizations are digital exhibits of items in their collections. In my History of Books and Libraries course, I critiqued three online exhibits about the history of the book. Digital exhibits provide more users with access to items in a collection and are a valuable contribution to information seekers - just as digitized materials are. I developed the criteria for the evaluations myself, based on experience and personal judgement, and focused on how well the digital exhibition is able to convey both the intellectual content and the physical information of the items in the exhibits. It was necessary to examine both types of information in this exercise because the while the intellectual content has value, the physical items themselves are also highly relevant to studying the book as an object.

Finally, to illustrate my competency in evaluating project-based tools, I am submitting excerpts from logs kept during my internship with the California Digital Library (CDL). The internship centered around preparations for an international conference called iPRES that was hosted by CDL. One of my duties was to locate and evaluate open source conference management software. The first step in this process was to identify the software features that were needed for our purposes at that time, which included paper submission and review and registration capabilities. As the planning process advanced, our needs for the software changed, and we ultimately did not select a product from the list presented in this evidence.

Conclusion

The ability to construct and adhere to specific criteria for the evaluation of programs, services, and the tools for their delivery is a tremendously important professional skill. With the evidentiary items I submitted above, I have demonstrated that I have fulfilled this competency. Because evaluation is such a key component of progress, I look forward to participating in this activity throughout my career.

References

Norman, D. A. (1990). The design of everyday things. New York: Currency and Doubleday.