Citation Research and Impact Metrics

Citation research, impact metrics, and research analytics are ways to attempt to assess the performance or impact of research, by analyzing how published items are cited in publications, or other forms of usage metrics (sometimes called alternative metrics) such as number of tweets. Citations can be analyzed at a variety of levels:

• Individual journals
• Individual articles
• The output of an individual author
• The output of all authors associated with an institution

This guide does not attempt to list every type of metric that exists, instead, we concentrate on ones to which the ASU community has easy access or those we think might be most useful.

When using metrics, keep in mind these Rules of Thumb:

1. Use at least two different metrics for assessment.
   Each metric has its strengths and weaknesses. Selecting metrics that balance each other reduces the possibility of inadvertent favoritism or penalization.
2. Compare "apples to apples" not "apples to oranges."
   Do not mix scores from different metrics as each metric uses different sources to obtain data. Compare Web of Science counts to other Web of Science counts; do not compare a Web of Science count for Article A to the Google Scholar count of Article B. Additionally, do not mix scores across different subject fields as citation behavior varies considerably; a low citation count in one field may actually be considered a high count in another.
3. Include qualitative assessment in addition to numerical metrics.
   As tempting as just using the raw data may be, the numbers must be put in context. Is the citation count due to positive or negative reasons? How does the count compare to others in the same subject field, the same journal and the same timeframe? Is the count increasing or decreasing with each successive year? Are there weaknesses in the metric that would favor or penalize the item or person under review?

Critical Resource