Skip to Main Content

Research Metrics

A guide for research impact tools, databases, and metrics

Why Metrics?

Research metrics, also called bibliometrics or citation metrics, are quantitative measurements designed to help you or others evaluate research outputs. They might include measurements such as times an article is cited, journal impact factor measurements, social media mentions, the h-index, news media mentions, and more. Some of these metrics are more traditional than others, but all of them can be used demonstrate the impact of your work.

 

 

Research metrics, when used responsibly, can help you tell a story about your research career; research metrics can help you enhance your impact in your field and beyond.

"Research Impact Graphic" by Carrie Price, Welch Medical Library, Johns Hopkins University is licensed under CC BY-NC 4.0

Responsible Use and Ethics

Leiden Manifesto for Research Metrics

The Leiden Manifesto for Research Metrics was published in 2015 by five experts urging responsible use in metrics, named after the conference where the idea came to fruition. They promote the following ten principles to guide research evaluation:

  1. Quantitative evaluation should support qualitative, expert assessment.
  2. Measure performance against the research missions of the institution, group, or researcher.
  3. Protect excellence in locally relevant research.
  4. Keep data collection and analytical processes open, transparent, and simple.
  5. Allow those evaluated to verify data and analysis.
  6. Account for variation by field in publication and citation practices.
  7. Base assessment of individual researchers on a qualitative judgment of their portfolio.
  8. Avoid misplaced concreteness and false precision.
  9. Recognize the systemic effects of assessment and indicators.
  10. Scrutinize indicators regularly and update them.

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature News 520(7548), 429–431. https://doi.org/10.1038/520429a

Metric Tide Report

The Metric Tide Report, published in 2015 commissioned by the Higher Education Funding Council for England (UK), is a report of the independent review of the role of metrics in research assessment and management. Traditional metrics have long been used as indicators for research and researcher impact. Their use can be problematic when taken out of context with uncritical acceptance. Responsible metrics should be considered and understood in the following dimensions:

  • Robustness - Is the metric using the best available and accurate data?
  • Humility - Quantitative evaluation can complement, but not replace, expert assessment. 
  • Transparency - Is the collection of data and its analysis open to scrutiny?
  • Diversity - Does the metric represent the landscape of research in any given field, and use appropriate indicators to support research and researchers?
  • Reflexivity - Is the use of bibliometric analysis dynamic and open to change?

Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. If we as a sector can’t take full advantage of the possibilities of big data, then who can?

Wilsdon, J., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. https://doi.org/10.13140/RG.2.1.4929.1363

SF DORA

The San Francisco Declaration on Research Assessment (SF DORA) recognizes the need to improve the ways in which the outputs of scholarly research are evaluated. The declaration was developed in 2012 during the Annual Meeting of the American Society for Cell Biology.

The declaration is a worldwide initiative covering all scholarly disciplines and stakeholders.

Signing the 2013 San Francisco Declaration on Research Assessment (DORA) is an important  way for individuals and organizations to publicly acknowledge their commitment to improve research by strengthening research assessment.

The DORA roadmap for the next two years will focus on three strategic goals to enable signatories to take action:

  1. Increase awareness of the need to develop credible alternatives to the inappropriate uses of metrics in research assessment.
  2. Research and promote tools and processes that facilitate best practice in research assessment.
  3. Extend the reach and impact of DORA’s work across scholarly disciplines and in new areas of the world.

From SF DORA at www.sfdora.org.

Read the declaration at SF DORA: https://sfdora.org/read/.

Sign the declaration at SF DORA: https://sfdora.org/sign/.

Further Reading on Responsible Use: A Selected Bibliography