The Measure of Program Success? It’s complicated, kind of

So what started out as one post became a series where I discussed my thoughts about measuring program success. It began with my assertion that trying to determine book circulation was not a good measure. Then yesterday I discussed how program attendance wasn’t even necessarily the best measure. But library administrators want to know that the time and money you are spending on teen programming has value, that it is a good way for them to invest their staff and money. So we need to have a good way to communicate with administrators and the communities we serve that we are in some way being successful with our teen programming.

I’m not going to lie, the number one way that admin will measure this is going to be statistics, which means attendance statistics. So even though just yesterday I discussed that this was not necessarily the best measuring stick, it will in fact be your primary measuring stick. There really is no getting around that and it can be a useful tool, especially if we have also investigated whether or not we are accounting for variables such as day, time, competition, and our marketing strategies. But this should not be our only tool, we can provide additional information to create a fuller picture.

There are other ways we can measure program success as well.

Quality

The quality of a program is actually quite independent of book circulation or attendance. The problem is that quality is not a measurable thing, it’s subjective not objective. You can invite a great author of a popular book series to come to your library. You can decorate the library to the nines and serve cupcakes and punch. The author can give a kick ass talk. This is all quality. But all of that can happen with a low attendance. We once had John Scalzi visit my library: he’s popular, funny, and it was one of the best programs ever. Sadly, there were maybe 10 people in attendance. Those 10 people had the best time ever and thought it was the best program ever, but that number 10 looks bad on a piece of paper. What we have to do to properly measure program success is look beyond a simple number to get a more complete idea of what is happening in our library programs.

This is purely a personal reflection, but it helps me to know that no matter what the outcome of a program as measured by attendance may be that I still put together a quality program. This doesn’t necessarily fly with administrators, but it helps me to keep in mind that I’m still a good librarian, there are just other details that I need to continue working out like scheduling and/or marketing. Just as some of the best books can be the least circulating titles, some of our best programs may in fact be our least attended.

Meeting Your Goals

When you set about doing your library programming, you should have goals. Not just numerical goals, but goals that help guide you in your program planning. You want to be able to answer not only why you do library programming, but why you are doing THIS library program. One of the goals I use is the 40 Developmental Assets system. I try to plan programs that help me equip my teen patrons with as many assets as possible. Other goals may include things like introducing teens to STEM concepts, providing opportunities for social interaction, etc. If I can show that the program did in fact help me meet my stated goals, then I can better evaluate if the program was successful.

In addition to having individual program goals, you should have some overall goals for your teen programming as a whole. These can include both yearly and long term goals. Sometimes they are measurable, say reaching a numerical goal of providing 52 programs in the fiscal year. You can further break this down if you would like by saying things like making sure that 30% of them were STEM related. Again, discussing how our programs how meet these stated goals helps us to have a better rubric for evaluating their success.

You can also demonstrate that you are meeting library goals by showing how teen programming helps support and fulfill the library’s overall mission statement. When you discuss the success of a library program just don’t report on numbers, discuss how the program(s) meet the needs of teens, how they help fulfill the library’s mission to the community, and how they provide alternative educational and recreational experiences for teens. Programming isn’t just about numbers, it’s about reaching a variety of goals. If we are reaching those goals, then we can better demonstrate that our programming is successful.

Ask the Teens!

One of the best ways, however, to find out if the program is successful in the eyes of the teens that you serve is to ask them. Get their feedback. I’m actually not a big fan of having teens fill out evaluations after every program, this always makes it seem too much like completing an assignment to me. But I do make a note of any feedback I get and keep it on hand to put in annual reports and send it out in emails. Share positive feedback from teen patrons frequently and keep your co-workers and administration informed about not only what is happening in teen services, but how well the teens are responding to it. Keeping others in the feedback loop is important.

I’m also a big fan of a big, yearly survey. I have a pretty standard survey that I have put together that I use regularly and just adjust to fit my needs. I ask about everything from the collection to programming, giving lots of opportunities for feedback. There is an example of the form I use in The Whole Library Handbook: Teen Services below.

Teen Programs

Did you attend any of the teen programs in the last year?     Yes     No

If yes, what did you think of the programs?         (Excellent)  A     B     C     D     F (Poor)

What type of library programs are you interested in?

Crafts   Trivia   Games   Speakers   Book Discussion    None     Other ________________

Suggestions for future programs:

What is the best day of the week for you to attend programs?    

Monday     Tuesday     Wednesday     Thursday     Friday

What is the best time of day for you to attend programs?

3-4 PM     4-5 PM     5-6 PM     6-7 PM     7-8 PM

How do you hear about our programs?

Website     Newspaper     School     Friends     In Library     Other  _________________

Source: https://www.teenlibrariantoolbox.com/2012/06/cut-through-the-static-get-feedback/

You could also get feedback through things like a focus group or Teen Advisory Group. I actually recommend using multiple tools to get more and different types of feedback. Save this feedback and incorporate it into your annual report to library administration; It adds context and meaning to the numbers on a page. The feedback of our teen patrons and invested adults help tell a more complete story. It’s not just numbers, it’s numbers plus quality plus goals plus positive feedback. If you ask me, that is a better formula for measuring program success.

Program Success = Attendance (Accounting for Variables) + Quality + Goals Met + Positive Feedback + Cost Per Person Ratios (see below)

A note about cost per patron ratios: Each program has a cost. Keep a track of any supplies purchased plus the amount of time spent planning, setting up, executing and cleaning up from a program. Take your total program cost (Material/Performer Cost + Staff Time Cost) and total it. This is your Program Total Cost. Divide your program total cost by the number of teens you had in attendance and you get your Cost Per Person Ratio. Generally speaking, the lower per person cost the better.

At the end of the day, we can use this information to write a story about our programming success, giving our administrators more information than simple head counts. Programming isn’t just about numbers, it’s about the experience.

Part 1: The Measure of Program Success? Probably not book circulation

Part 2: The Measure of Program Success? Probably not attendance either

Filed under: Professional Development