Research Article

A Content Analysis of Research Impact Guides: The Presence of Responsible Use of Metrics and Bias in Research Assessment Content

---

Abstract

Introduction: Responsible use of metrics is a trending topic and is becoming increasingly important in research impact services. However, the inclusion of responsible use of metrics content, particularly concerning bias in research assessment, in research impact-related guides remains an unexplored area in the LIS literature.

Methods: In this study, the author performed a content analysis of the explanatory text of different page types within Association of Research Libraries (ARL) research impact-related guides, including Introductory pages, Journal-level Metrics pages, Article or Citation-level Metrics pages, Author-level Metrics pages, and Responsible Use of Metrics pages. The author gathered data for this analysis with Voyant, utilizing its tools to search for keywords across four categories: Bias, Personal Characteristics, Diversity, Equity, Inclusion, and Justice (DEIJ), and Responsible Use of Metrics. The resulting data was analyzed by page type in Tableau. 

Results: Among the four keyword categories, keywords in the Responsible Use of Metrics category appeared more frequently across all page types compared to the other keywords included in this study. However, when guide authors used keywords in the Bias category, the text surrounding the keywords was found to be related to bias in research assessment, especially on Responsible Use of Metrics pages and Introductory pages.

Discussion: Guide authors addressing bias in research assessment in research impact guides structure the information similarly: they strategically use keywords in the Bias and Responsible Use of Metrics categories in the explanatory text on the Introductory and Responsible Use of Metrics pages, while rarely using keywords in other categories reviewed in this study. Including text on the responsible use of metrics and bias in research assessment should serve as best practice when developing guides related to Research Impact. The study highlights the need to develop training resources that introduce the concept of equitable research impact services, as all four categories are essential when creating content about bias in research assessment. 

Conclusion: Librarians developing and maintaining research impact-related guides should include content on the responsible use of metrics and bias in research assessment in their resource guides, workshops, and consultations to ensure equitable research impact services. These services should not only incorporate the principles of the responsible use of metrics but also address bias in research assessment at any stage it may occur, recognize who may be affected, and acknowledge suggested mitigation strategies.

Keywords: academic libraries, research impact, equity and inclusion, resource guides, responsible use of metrics, Voyant Tools, content analysis

How to Cite:

Craft-Morgan, S. (2026). A Content Analysis of Research Impact Guides: The Presence of Responsible Use of Metrics and Bias in Research Assessment Content. Journal of Librarianship and Scholarly Communication, 14(1), eP19042. https://doi.org/10.31274/jlsc.19042

Rights:

© 2026 The Author(s). License: CC BY 4.0