Conference Season 2011: Librarianship Researchers, we need to toughen up

May/June are the epicentre of what I call “conference season.” For me that means looking longingly at my garden as I leave on weekends that should be dedicated to barbecues and street hockey with the kid. However, it also means lots of intensive time for rapid-fire thinking. As a research-embedded health librarian, I often go to non-LIS-type conferences that are aimed at health researchers. While I really like my librarian colleagues, this spring I’ve been thinking about the juxtoposition of how we present research in LIS fora versus health fora.

To my fellow Health Librarianship Researchers: We need to toughen up.

(Frankly, this should probably be addressed to all librarianship researchers, not just those in health, but health is my current niche and where my illustrative examples come from today)

1) Stand up for your methods!

When I am listening to a health librarian give a research presentation, it is all too common for me to end up cringing at what is either weak research methodology or weak defense of methods (or both).We need to deal with this.

I’ll give a specific example from a recent conference, because the speaker in this example is a very well-respected tenured LIS faculty member in an ALA-accredited institution, who has a long and established track record of important research and advocacy for libraries, and thus I think fair game for public critique of research presentations. Dr. Marshall gave a keynote talk at CHLA/ABSC on a study that will be very important to health libraries and librarians – especially in clinical settings such as hospitals. Yes, this is the much-needed and highly anticipated update to the Rochester Study! Very exciting stuff.

Now, obviously this talk was a conference presentation, and in that kind of setting there’s never enough time to fully describe methods. That’s part of why we have time for questions afterward – so audience members can ask about areas of particular interest that were not explained in the talk.

In the question period after the talk, when people asked about methods, however, I found that even this prominent LIS faculty member was a bit wishy-washy. For example, rather than defending the reliability of her research, or explaining why reliability was perhaps not the appropriate question to ask about the qualitative portion of her investigation, the speaker demurred, basically saying that well, nothing’s perfect and we all do our best.

WHAT? I mean, yes nothing’s perfect and I’m sure we all do try to do our best, but that is how you respond to someone questioning your methods? When I mentally place this faculty member at my workplace, presenting to the faculty in my home department, she gets torn apart. Sitting there in the audience, I had a vision of the dreaded librarian “niceness” working to discredit our field in the face of other disciplines.

To give this speaker the benefit of the doubt, she might present and defend completely differently in front of another audience – say, an audience of economists. Also, she could have been having an off-day, or any number of things. BUT, this isn’t the only example I’ve seen of this type of thing, and she’s not doing librarians any favours by being soft in front of members of our own discipline. As a leader in our field, she should be modelling rigourous research and the ability to explain and defend it for us.

2) Policy-based evidence: We need to recognize and avoid it

I saw a few examples of this during the current conference season, but I feel bad pointing specific fingers because I don’t want to be “mean” or discouraging to novice researchers. (Yes, I am aware that this is the dreaded librarian “niceness” manifesting in me, and I don’t have the distance to know if it’s good or bad.)  However, I don’t think my naming a particular presentation from a particular conference is that useful, as you can probably conjure up your own examples of library policy-based evidence without much effort.

Here’s what I see: Librarians do a lot of surveys. Especially Masters-level academic librarians, who are supposed to do some research and base policy decisions on some sort of evidence. User surveys are pretty common, and this is reinforced by our love of LIBQUAL+. We also have this idea that survey research is “simple” and thus a masters-level professional can do it just fine with no methodological problems. This I would dispute. Surveys *can* be simple, just as many other research methods can be simple. But surveys are also really easy to do badly. And we do a fair amount of bad, or at least biased, surveys.

At pretty much every library conference I attend, I see presentations of surveys with conclusions that do not follow from the actual results, and/or surveys that were clearly (albeit often not purposely) designed to justify a particular policy move. This is certainly not unique to libraries, or health libraries. Lots of fields generate policy-based evidence. The federal government does it at times. (*ahem* Long form census -> National Household Survey)

But when I see librarians doing things like:

  1. presenting surveys with extremely low response rates, and
  2. no demographic information to assume this small sample is representative of the whole population, then
  3. basing conclusions on the responses of the majority of a tiny minority of the whole, with no discussion of response bias,

I am frankly appalled. I know ML/IS research methods classes tend to be generic and weak, but that we in the profession continue to reward shoddy research methods with conference presentations and other support is horrible. And doing a huge disservice to our profession. Not only are we probably making poor decisions based on lousy research, but we are completely undermining our own efforts to position librarians as professionals with research expertise.

Yet, even I am reluctant to rake someone, especially a first-time presenter, over the coals in the open question period after a low-quality presentation. It’s “mean.” I feel peer pressure not to ask the same questions of my librarian peers that I would ask to my researcher peers back at home/work. I reassure myself that maybe I will talk to an individual privately afterward, if I can catch her. But honestly, this often doesn’t happen.

What do we do? How do we toughen up? How do we get others in our profession to toughen up?

I think one thing that would help would be for our visible and prominant leaders in the field to engage in more public methodological debate regarding LIS research, personally. If we’re going to do research and position ourselves as reseach experts (or even just research-competent), we need to sharpen our chops.

-Greyson

Follow-up post: here

Advertisements

6 Comments

Filed under academic libraries, research, The Profession

6 responses to “Conference Season 2011: Librarianship Researchers, we need to toughen up

  1. willimen

    Hi Greystone,

    I would even build on this argument and say that most LIS graduates – who enter the academic stream within universities – are vastly under-prepared to conduct reliable and valid research in this setting. They are entering institutions where ‘some’ other faculty have vast knowledge and training in research methods. Most other faculty posses both a masters degree in their respective field AND A PhD! This extra 4-6 or more years of education, with some departments specifically focusing their curriculum on qant. and qualit. research methodologies, provides these faculty with the research with the tools to conduct well designed research projects.

    One thing most academic librarians do have going for them is reduced educational costs, in their respective institutions. I would highly recommend most academic librarians, who want to be treated as fellow academics, to enroll in some of the research courses offered at their respective universities and colleges.

    P.S. convenience sampling of grad students is a cop-out.

    ~ Ken

  2. Pingback: Update on withdrawn CIHR trials policy | Social Justice Librarian

  3. Martha

    It’s been a long time since I’ve contributed, even with a comment but I have to chime in because I couldn’t agree more. Questionable research methods are certainly not limited to health librarianship or to conference presentations. Our journals are replete with them. Reports and studies on the information needs of this or that population make ridiculous and embarrassing claims based on incomplete and unreliable data.

    In the last few months I have seen a proliferation of postings for assessment librarians. I don’t know if this will help to improve the current laxity in research in our field. Perhaps not but I hope it will. I am the first one to recognize that I don’t know enough about research methods but that is no excuse. If we are going to take something on, like conducting anything beyond a case study (which still needs to be conducted properly), then we need to learn more about reliable data gathering AND about analysing this data. If we don’t know how to do it, then why not collaborate with those who do? And, no, I do not mean LIS students taking a research methods course. I know that most librarian positions, even those with faculty status, do not come with a lot of time allotted for doing research, but again, that is no excuse. Doing crappy research just discredits our profession. And yes, we need to stop being so nice and start being more critical. I think we can take it, if we can’t, then we need to toughen up.

  4. Pingback: PL 23/11: Soft research « Plinius

  5. Pingback: Toughening ourselves up as librarian-researchers: Follow up Post #1 | Social Justice Librarian

  6. Thanks, a great argument!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s