Evaluations and Metrics

OK, so your library is one of the 7 Wonders of the Modern World. Now how do you prove it to your funders and your community? Evaluations and metrics! Or maybe you’re a little more humble and you want a better sense of your library’s strengths and weaknesses, or you want to know whether you’ve done what you set out to do with a particular service or project. How do you find out? Evaluations and metrics!

Why Measure the Impact of Technology in Your Library?

  • Again, it’s all about the money. You want to demonstrate that technology is expensive, but also that it’s worth the expense. How is your community benefiting? How are the politicians and town council members benefiting? How are their constituents’ lives improving? You can use general, abstract arguments, but numbers and stories have more of an impact.
  • If you don’t measure benefits and outcomes, you’re driving blind. You and your staff may have an intuitive sense that a service is succeeding, but intuitions are sometimes wrong. And even if you’re right, there’s always room for improvement.

Key Actions

  • If you are interested in measuring statistics related to your online databases, electronic journals and e-books, start by taking a look at the E-metrics site, especially the tutorials and the catalog. It also offers measurements related to patron training, public computer usage and other technology services.
  • If you want to know who visits your Web site and how they use it, check out this article on Web analytics from Idealware.
  • Plan to conduct a lot of online surveys of patrons and staff. There’s an article from TechSoup that will tell you how to choose the right tool. There are also dozens of articles, such as this one, that provide advice on writing good survey questions.

Technology Metrics

There are hundreds, if not thousands, of ways to obtain concrete and specific information about your technology services. Some common metrics include:

  • Who’s visiting your web site?
  • How often are they visiting?
  • What do they do when they’re on your site?
  • How many laptop users are there in your library on an average day?

Evaluation Methods

If you’re manufacturing widgets, it’s fairly easy to determine the types of things you should measure. You want to know:

  • The number of widgets produced per hour and per day
  • The average cost of each widget and the average profit margin
  • Something about the reliability, quality and safety of your widgets

Libraries, on the other hand, deal with outcomes and results that are much less tangible. We’re trying to teach people and helping them to teach themselves. We’re encouraging folks to adjust their behaviors and attitudes. We’re helping them build new skills. But how do you bring these vague, high-flown aspirations down into more concrete, specific language that politicians and bureaucrats will understand? Try following these two evaluation steps.

STEP LINK AND LEARN
1. Decide what it is you want to measure. This is sometimes known as the outcome, the result or the return on investment. Defining precisely and in detail what exactly you’re trying to do is tough. The first part of evaluation is figuring out exactly what we’re trying to change. For a fuller of definition of an outcome, see this FAQ from the Institute of Museums and Libraries, especially the second and third questions (“What is an ‘outcome’ and how do you evaluate (measure) them?” and “What is the difference between outputs and outcomes?”).
2. Decide how you’ll measure that outcome or result. In other words, what evidence and data will you collect? There might be a single measurement, or you might measure several things. These measurements are often referred to as success indicators. The good ones are usually concrete, observable and countable, but some projects also rely heavily on collecting stories. For more information on success indicators, refer again to the Outcomes FAQ from IMLS, especially the section titled “What is an indicator?”

Further Resources

For more suggestions on technology evaluation and measurement, check out the Further Resources section.