Evaluation, assessment, research & impact

Around the same time I noticed that a number of academic libraries were posting for new (or newish) “assessment librarians,” I went to a cool lecture by Dr. Eliza Dresang about a project teaming LIS researchers with children’s librarians to investigate impact of early literacy programming.

After the lecture, a local children’s librarian extraordinaire and I began a conversation – still ongoing – about assessment & impact research in public libraries. I’m a firm believer that in order to a) provide the best possible service to the community, and b) justify funding, libraries ought to be doing assessment beyond mere program evaluation.

Unfortunately, library school “research methods” courses seem generally weak, and there is limited professional development on research methods for professional librarians. Even excellent library programs often result in needlessly-biased evaluation reports that could have provided more valid evidence if only the methods have been stronger. Even senior librarians in public libraries confuse evaluation with assessment with research (yes there is ample grey area in there, but the terms are not synonymous), and fall into the trap of trying to demonstrate impact & value by counting things/measuring productivity.*

Adding to the challenge, few public libraries are intimately connected with professional academic researchers, and few librarians have the time to learn how to conduct unbiased program evaluations, let alone develop high-quality impact assessment skills.

In my mind, large public library systems should consider taking a page out of academic libraries’ new book and hiring internal research staff to demonstrate value and investigate impact. What’s more, government bodies that oversee libraries (e.g., the BC Public Libraries Services Branch) should be hiring staff to a) support library-based assessment & research, and b) coordinate, liaise with & conduct research on the value and impact of public library services.

I know that asking more more staff seems expensive, and there have been a couple of years of belt-tightening in a row at this point, but some of the best things the library sector could do to improve our ability to advocate for funding are to

  1. provide evidence of impact and
  2. ensure that services are relevant to the community

-Greyson

*To those nodding along with my concerns but unsure of how to move beyond these common problems, I recommend Markless & Streatfield’s Evaluating the Impact of Your Library, published by CILIP’s Facet Press. It does a great job of walking one through that process of mid-level assessment between basic program eval/library stats and full-fledged long-term impact research.

Advertisements

2 Comments

Filed under academic libraries, funding, government, inclusion/exclusion, LIS education, public libraries, The Profession

2 responses to “Evaluation, assessment, research & impact

  1. willimen

    Hi Greystone,

    Thanks for this posting – it has been something that I have been grappling with for a while. As an ‘old’ sociologist, I have been distressed by this issue – which I would say is found throughout our profession (academic, public, and special libraries).

    You are right, libraries and librarians are not utilizing the right research methods to adequately draw causal connections between concepts (which guide policy and thus has real life consequences). Many times output measures (counting heads) are used to show impact.

    However, I must caution – that logic modeling and other outcome based measures are usually created a-priori (before the fact), by people doing the research – not the communities who are actually impacted. Instead we should be actively involving target groups etc. in evaluation process –

    https://sjlibrarian.wordpress.com/2010/03/08/community-led-evaluation-impact-on-community/

    https://sjlibrarian.wordpress.com/2010/09/27/making-the-case-for-public-libraries/

    We, as librarians, should be questioning our role in ensuring that great research is conducted, which are both reliable and valid (I am not sure if bias can ever be totally removed). What is the role of the academic librarian in linking academics on campus with other librarians working in the community, on reduced budgets, compared to our similarly educated librarians in the ivory tower? Should academic librarians be trying to bridge the gap that exists between academia and the tax payers that fund them?

    I don’t think that we need to create research librarians to fill the role – other professions already have the expertise. We just need the hubris to know what our roles as librarians is in this process.

    ~ Ken

    • greyson

      Hi Ken,

      Thanks for the comment. I almost linked to the first of the 2 posts you cite in your comment, as I think some of the best working examples we have of actual impact assessment in public libraries today are coming from the community development library folk. I decided to try not to make my original post too long, but I’m glad you brought it up because asking the right questions and having the right methods are essential parts of the research process.

      I’m torn about training librarians in research vs partnering with researchers, but really I don’t think it’s an either/or situation. Both would be very helpful in this regard. That said, all librarians do not necessarily need to be cross-trained as researchers, and all researchers will not “get” libraries or be the kind of researcher qualified to ethically conduct community-based research on needs or values.

      I do think we need public-library-sector-led (or at least public library-as-truly-equal-partner) research to investigate the impact of our work, as researchers in academia necessarily have their own agendas and motivations. And I think to get that we really need to employ/fund some researchers to focus on us, whether this be librarians-who-are-researchers or researchers-who-really-get-libraries.

      Greyson

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s