Abstract
PURPOSE/RESEARCH QUESTION
During the past two decades, academic libraries have significantly invested in scholarly communication through the allocation of staffing, resources, and establishing institutional repositories. Despite these expenditures, quantifying the actual outcomes or impact of these scholarly communication activities has remained elusive, particularly given emergent new technologies and paradigms for creating and sharing work, diverse research output, rapid dissemination and preservation of the scholarly record, innovative publishing models, and disagreement over scholarly impact measurement.
To better understand the full range of perspectives on the assessment of scholarly communication, librarians, campus stakeholders, and assessment experts participated in focus groups or interviews to identify the metrics that might be used to assess the success of scholarly communication programs. This paper will present the metrics suggested by these different groups, potential common themes, ad considerations in the assessment process. This project was made possible in part by the Institute of Museum and Library Services (LG-35-19-0066-19).
DESIGN, METHODOLOGY OR APPROACH
Focus groups were conducted and recorded with 20 librarians, including mostly scholarly communication practitioners and a few assessment experts. Participants identified the scholarly communication services offered at their institutions and rated them in terms of maturity of service and staffing models. A facilitator then led the focus groups to discuss scripted questions using examples compiled during the survey. As a second phase of the research, the facilitator/evaluator conducted and recorded one-on-one interviews with campus stakeholders using a guided script. In the third phase of the project, a forum was held; many of the focus group participants presented services from their institutions, and campus stakeholders served on a panel. During the forum, various guided break-out discussions centered on how to assess scholarly communication services.
After these stages concluded, the authors analyzed transcribed interviews to develop a normalized codebook across all phases of participation. This paper presents the themes drawn from the collected qualitative data, while specifically highlighting the metrics that have been determined to be helpful for coding the types of services institutions are offering.
FINDINGS & LIMITATIONS (AS APPLICABLE)
Three major themes emerged from the study: Education & Outreach, Support for Open Access, and Impact. Suggested metrics were in alignment with these critical areas.
The study focused on Carnegie-classified public M1 universities, which frequently have finite resources and must therefore be selective about the services they offer. Campus external stakeholders were drawn from offices of research, sponsored programs, or grants development and support. Thus, the perspectives from these groups center on funder mandates and compliance issues; external stakeholders representing other functions of the academy would have likely resulted in different metrics and areas of emphasis.
CONCLUSIONS AND HOW FINDINGS HAVE BEEN APPLIED
Participants pointed out the inadequacies of reporting for both short-term and long-term effects of scholarly communication programs. External stakeholders pointed out that there are significant gaps in reporting between funded and unfunded research, and indicated that libraries were well poised to assist in these areas.
Metrics from this study have been incorporated into the white paper and rubrics that will be disseminated in May 2022.