I originally wrote this in 2008, but never really knew what to do with it? I am not against the use of quantitative statistics. I actually quite enjoy doing multivariate stats, although I really haven’t touched it much since entering the library profession.
Using statistics makes sense for libraries. Statistics provide our funders, boards, and senior administrators with a snapshot of the inputs and outputs occurring in libraries. I hope to write a future posting which talks about the importance of using the narrative to ensure we are also capturing the impacts and outcomes of library services.
I hope this makes sense.. but here it goes.
Librarians, like most other professionals, have traditionally collected numbers, especially descriptive statistics, as a primary means of measurement, evaluation, and to justify or change current or proposed operations. While quantitative statistics have served a valuable function within the traditional library environment, there are many drawbacks to using numbers to represent the attitudes and behaviours of patrons.
Using Quantitative Measures
While the collection of numbers is usually viewed as a non-biased methodology for collecting information, numeric indicators have a number of drawbacks. Quantitative methodology is a deductive approach, where the researcher acts:
- as the expert who determines the questions used to collect information or data, and
- questions are usually generated by referring to other library studies (conducting a literature review), or by relying on their own expertise, to determine the concepts to measure.
When this occurs, we as library staff, define the concepts are important to measure. Staff also decide:
- what questions to ask,
- how to ask them, and
- how to measure them.
This process should, but does not always involve, clearly pre-determining the concepts to measure. This can provide a one time snapshot, or a more long term picture of a social phenomena.
Numbers permits us to test hypotheses, predictions, and causal connections between the measured concepts. Under specific circumstances the use of statistical procedures allows for sample (e.g. small number of old age library users) findings to be inferred to populations (e.g. all old age users in the library system).
Ultimately, the numeric basis of quantitative research is one of its major weaknesses. While concepts are asked and defined by library staff, so they can measure them, the use of common terminology is not always consistent.
(Example #1 – After a program library staff may ask the participants what they thought of it through the use of a five point scale: 1=Very bad / 2=Bad / 3=Neutral / 4=Good / 5=Very Good. One person may have had a horrible time, but only interpret the experience as “bad”, while another might have been mildly annoyed by the person sitting beside them and then indicate their experience was “very bad”. Therefore, the response depends on the definition the individual places on the concept created by the survey constructor – not how the survey constructor defined the concept).
(Example #2 – The number of library users in one library system can be defined as the number of people that check out books, while another system may measure the same concept based on the number of people who enter their buildings. Therefore, when numeric data is collected and compared, between branches and other systems, it is very important that library staff constantly ensure that apples are being compared with apples – not oranges.)
By pre-determining the concepts to measure and compare, the librarian is viewed as the expert who knows, prior to data collection, which concepts or variables are important. This process is very inflexible, and does not provide members of the community the opportunity to provide information about how they see the world, outside the prescribed measurement tool created by the librarian.
By far the most dangerous consequences of the improper use of quantitative statistics occurs when people collecting and interpreting data conclude that the findings are causal or predictive. With quantitative statistics, the type of number used (nominal, ordinal, interval-ratio) determines which research questions can be asked, what questions can be answered, and what types of analysis can be performed. For example, people may talk about the “correlation” between concepts, although correlation does not show causality – it is a measure of association – and is much more accurate when occurring between concepts measured at the interval-ratio level (not nominal level data (frequencies or whole numbers) which are primarily collected in libraries).
In addition, small samples should only be generalized to large scale populations, when library staff can tell the sample drawn is representative of the entire library system (e.g. remember the old age user example discussed above. Survey results of a sample of old aged users can only be generalized to a population of old aged users, if the sample is reflective of the population). If sample characteristics do not reflect the population, there is a danger of introducing bias into the results, and interpretations – which drive library policy (e.g. only older library users who are highly mobile filled out the survey, since the survey was conducted in winter and those with mobility issues could not come to the branch because of all the snow). This is a real threat to library systems if done incorrectly, since a small innocent survey – which was improperly interpreted, is relied upon to direct future library services and policies.