Library advice regarding the responsible use of quantitative research indicators

Library Services provides support and guidance to The OU research community regarding quantitative research indicators, the preferred term for metrics/bibliometrics. 

The use of these indicators has generated much discussion and we believe that it is vitally important they are used responsibly. Accordingly, the following advice outlines our approach to the responsible use of such indicators, representing current good practice and acting as a guide for future activities. It draws on good practice being adopted at institutions worldwide and documents such as the Leiden Manifesto and The Metric Tide, which we recommend people consult. The OU signed the Declaration on Research Assessment (DORA) in November 2020 and we recommend this statement should also be consulted.

Quantitative research indicators should support expert judgement but not replace it

We believe that quantitative research indicators should be used to support expert judgement in the evaluation of research, specifically to inform assessments and challenge preconceptions. The use of quantitative indicators should not replace expert judgement. This is because quantitative data, whilst often useful, cannot give a complete or accurate assessment of research alone. 

Note that it may not always be appropriate to use quantitative research indicators in the analysis of research.

Use reliable sources of quantitative research indicators

We believe that sources of quantitative research indicators should be selected with due consideration for the quality, breadth, depth and transparency of their data. All sources of such data have their limitations and these should be acknowledged. Please note that access to sources of data may be limited by cost or by a supplier's decision to discontinue their product.

Provide tailored analyses 

We believe that choice of data source and indicators should be informed by factors such as the question being asked about the research and the entities being analysed.

For example:

• Different disciplines have different citation, publication and research cultures, which need to be taken into account
• Different researchers have different career lengths, which need to be taken into account
• Particular care is needed when carrying out assessments on a small-scale (i.e. on an individual researcher or on a small set of publications)

Provide transparent analyses

We believe the criteria, data and processes used in analyses must be transparent and available on request to those being analysed. The sources of data should be acknowledged and individuals should be able to review and, if necessary, correct data about their research.

Use multiple indicators

No one indicator can give a complete picture of a piece of research or a researcher, so we believe a selection of appropriate and complementary indicators should be used.

Any biases (e.g. relating to discipline, publication type, career length) in indicators should be acknowledged.

Scrutinize indicators and stay up to date 

Research evaluation is a complex and changing area. We believe that indicators should be scrutinized regularly to ensure their ongoing suitability. Efforts should be made to stay abreast of changes in good practice, sources of data available and indicators available. 

If you would like any more information on the responsible use of quantitative research indicators, please contact the Library Research Support team. 

Final version of this statement 3 May 2017. Page last updated 11 March 2022.