Skip to main content
Log in

Aggregation properties of relative impact and other classical indicators: Convexity issues and the Yule-Simpson paradox

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Among classical bibliometric indicators, direct and relative impact measures for countries or other players in science are appealing and standard. Yet, as shown in this article, they may exhibit undesirable statistical properties, or at least ones that pose questions of interpretation in evaluation and benchmarking contexts. In this article, we address two such properties namely sensitivity to the Yule-Simpson effect, and a problem related to convexity. The Yule-Simpson effect can occur for direct impacts and, in a variant form, for relative impact, causing an apparent incoherence between field values and the aggregate (all-fields) value. For relative impacts, it may result in a severe form of ‘out-range’ of aggregate values, where a player’s relative impact shifts from ‘good’ to ‘bad’, or conversely. Out-range and lack of convexity in general are typical of relative impact indicators. Using empirical data, we suggest that, for relative impact measures, ‘out-range’ due to lack of convexity is not exceptional. The Yule-Simpson effect is less frequent, and especially occurs for small players with particular specialisation profiles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aksnes, D., Sandström, U. (2006), National citation indicators, their methodological foundation and political interpretation, Book of Abstracts, 9th International Conference on Science & Technology Indicators, September 2006, Leuven, Belgium, pp. 3–5.

  • Balassa, B. (1965), Trade liberalization and ‘revealed’ comparative advantage. Manchester School, 33: 99–123.

    Article  Google Scholar 

  • Braun, T., Glänzel, W., Schubert, A. (1985), Scientometric Indicators. A 32 Country Comparison of Publication Productivity and Citation Impact. Singapore: World Scientific Publishing Co.

    Google Scholar 

  • Brusoni, S., Geuna, A. (2004), Specialisation and integration: Combining patents and publications data to map the ’structure’ of specialised knowledge. In: W. Glänzel, H. Moed, U. Schmoch (Eds), Handbook of Quantitative Science and Technology Research, Dordrecht: Kluwer Academic Publishers, pp. 733–758.

    Google Scholar 

  • CEST (2004), Les institutions du Domaine des Ecoles polytechniques fédérales, Profils de recherche et comparaisons internationales, Indicateurs bibliométriques 1981–2002, Centre de compétence scientométrie du CEST, CEST 2004/5, www.cest.ch

  • David, H. A., Edwards, A. F. W. (2001), Yule’s Paradox (Simpson’s paradox), In: Annotated Readings in the History of Statistics (pp. 137–140): Springer.

  • Egghe, L., Rousseau, R. (1996), Average and global impact of a set of journals. Scientometrics, 36: 97–107.

    Article  Google Scholar 

  • Egghe, L., Rousseau, R. (2002), A general frame-work for relative impact indicators. Canadian Journal of Information and Library Science — Revue canadienne des Sciences de l’Information et de Bibliothéconomie, 27: 29–48.

    Google Scholar 

  • Garfield, E. (1967), Primordial concepts, citation indexing and historio-bibliography. Journal of Library History, 2: 235–249.

    Google Scholar 

  • Garfield, E. (1998), Random thoughts on citationology. Its theory and practice — Comments on theories of citation? Scientometrics, 43: 69–76.

    Article  Google Scholar 

  • Glänzel, W., De Lange, C. (1997), Modelling and measuring multilateral co-authorship in international scientific collaboration. Part I. Development of a new model using a series expansion approach. Scientometrics, 40: 594–604. Part II. A comparative study on the extent and change of international scientific collaboration links. Scientometrics, 40: 605–626.

    Google Scholar 

  • Grupp, H. (1998), Foundations of the Economics of Innovation: Theory, Measurement and Practice. Cheltenham: Elgar.

    Google Scholar 

  • Luukkonen, T., Tijssen, R. J. W., Persson, O., Sivertsen, G. (1993), The measurement of international scientific collaboration. Scientometrics, 28: 15–36.

    Article  Google Scholar 

  • May, R. M. (1997), The scientific wealth of nations. Science, 275: 793–796.

    Article  Google Scholar 

  • Murugesan, P., Moravcsik, M. J. (1978), Variation of the nature of citation measures with journal and scientific specialties. Journal of the American Society for Information Science, 29: 141–155.

    Article  Google Scholar 

  • OST (2004), in Note Méthodologique B-5, Indicateurs de sciences et de technologies, Rapport de l’Observatoire des Sciences et des Techniques (L. Esterle, G. Filliatreau (Eds)), 2004.

  • Ramanana-Rahary, S., Zitt; M., Rousseau, R. (2007), Aggregation properties of relative impact and other classical indicators: convexity issues and the Yule-Simpson paradox. In: Proceedings of ISSI 2007, D. Torres-Salinas, H. F. Moed (Eds), Madrid, CINDOC-CSIC, pp. 643–654.

    Google Scholar 

  • Schubert, A., Braun, T. (1986), Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics, 9: 281–291.

    Article  Google Scholar 

  • Tijssen, R. J. W., De Leeuw, J., Van Raan, A. F. J. (1987), Quasi-correspondence analysis on scientometric transaction matrices. Scientometrics, 11: 351–366.

    Article  Google Scholar 

  • Van Raan, A. F. J. (1996), Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics,36: 397–420.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2004), Measuring Science. In: W. Glänzel, H. Moed, U. Schmoch (Eds.), Handbook of Quantitative Science and Technology Research, Dordrecht: Kluwer Academic Publishers, pp. 19–50.

    Google Scholar 

  • Vinkler, P. (2003), Relations of relative scientometric indicators. Scientometrics, 58(3): 687–694.

    Article  Google Scholar 

  • Yule, G. U. (1903), Notes on the theory of association of attributes in statistics. Biometrika, 2: 121–134.

    Article  Google Scholar 

  • Zitt, M., Bassecoulard, E., Okubo, Y. (2000), Shadows of the past in international cooperation: Collaboration profiles of the top five producers of science. Scientometrics, 47: 627–657.

    Article  Google Scholar 

  • Zitt, M., Ramanana-Rahary, S., Bassecoulard, E. (2003), Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation. Scientometrics, 56: 259–282.

    Article  Google Scholar 

  • Zitt, M., Ramanana-Rahary, S., Bassecoulard, E. (2005), Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation. Scientometrics, 63: 373–401.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Suzy Ramanana-Rahary.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ramanana-Rahary, S., Zitt, M. & Rousseau, R. Aggregation properties of relative impact and other classical indicators: Convexity issues and the Yule-Simpson paradox. Scientometrics 79, 311–327 (2009). https://doi.org/10.1007/s11192-009-0420-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-009-0420-4

Keywords

Navigation