4.0 Article

Integrated Impact Indicators Compared With Impact Factors: An Alternative Research Design With Policy Implications

向作者/读者索取更多资源

In bibliometrics, the association of impact with central-tendency statistics is mistaken. Impacts add up, and citation curves therefore should be integrated instead of averaged. For example, the journals MIS Quarterly and Journal of the American Society for Information Science and Technology differ by a factor of 2 in terms of their respective impact factors (IF), but the journal with the lower IF has the higher impact. Using percentile ranks (e. g., top-1%, top-10%, etc.), an Integrated Impact Indicator (I3) can be based on integration of the citation curves, but after normalization of the citation curves to the same scale. The results across document sets can be compared as percentages of the total impact of a reference set. Total number of citations, however, should not be used instead because the shape of the citation curves is then not appreciated. I3 can be applied to any document set and any citation window. The results of the integration (summation) are fully decomposable in terms of journals or institutional units such as nations, universities, and so on because percentile ranks are determined at the paper level. In this study, we first compare I3 with IFs for the journals in two Institute for Scientific Information subject categories (Information Science & Library Science and Multidisciplinary Sciences). The library and information science set is additionally decomposed in terms of nations. Policy implications of this possible paradigm shift in citation impact analysis are specified.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.0
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

Article Computer Science, Interdisciplinary Applications

Empirical analysis of recent temporal dynamics of research fields: Annual publications in chemistry and related areas as an example

Lutz Bornmann, Robin Haunschild

Summary: This paper presents a methodological approach to analyze the dynamics of scientific progress at the level of research fields, focusing on the annual number of publications in chemistry. The study reveals that research on optical phenomena and electrochemical technologies has emerged as key topics in recent years.

JOURNAL OF INFORMETRICS (2022)

Article Communication

Are there biases in decisions to tweet on scientific papers? A plea for conducting an experimental Twitter study. Technical note

Lutz Bornmann, Robin Haunschild, Alexander Tekles

Summary: Tweets serve as alternative metrics to measure research impact or attention, but biases must not influence tweet decisions for Twitter data to be used in research evaluation. Investigating the existence of biases requires an experimental design with controlled manipulations. This comment proposes an experimental approach to study scientists' decision to tweet about a paper, describing a study design adaptable to various social media platforms.

PROFESIONAL DE LA INFORMACION (2022)

Article Information Science & Library Science

Research calls, competition for funding and inefficiency

Antonio Osorio, Lutz Bornmann

Summary: This article explores four ways to reduce inefficiency in research funding calls, and suggests that reducing the dependence on fund-seeking activities and narrowing the subject scope of the calls may be particularly effective.

RESEARCH EVALUATION (2022)

Article Computer Science, Interdisciplinary Applications

Relevance of document types in the scores' calculation of a specific field-normalized indicator: Are the scores strongly dependent on or nearly independent of the document type handling?

Robin Haunschild, Lutz Bornmann

Summary: This study investigates the differences in field-normalized scores under different document type handling methods. By comparing the results at the individual publication level, the country level, and the institutional level, it is found that different handling methods result in significantly different scores. Therefore, caution should be exercised when comparing normalized scores obtained from different handling procedures.

SCIENTOMETRICS (2022)

Article Multidisciplinary Sciences

How relevant is climate change research for climate change policy? An empirical analysis based on Overton data

Lutz Bornmann, Robin Haunschild, Kevin Boyack, Werner Marx, Jan C. Minx

Summary: This study examines the connection between climate change research and policy-making. It found that climate change papers cited in policy documents receive more citations on average. The study also observed an impact of international climate policy cycles on policy document publication and identified similar research fields between the scientific and policy communities.

PLOS ONE (2022)

Article Communication

Trajectories and regimes in research versus knowledge evaluations: Contributions to an evolutionary theory of citation

Loet Leydesdorff, Lin Zhang, Paul Wouters

Summary: Citation analysis provides models for studying the evolutionary dynamics of scholarly communication. This study proposes distinguishing between institutional research evaluation and knowledge evaluation in citation analysis, and discusses the role of redundancy as a measure of communication potential. It uses indicators at the journal level to analyze citation environments in both the cited and citing directions, and confirms that interdisciplinarity cannot be captured by one-dimensional citation analysis. The study raises questions about developing indicators of interdisciplinarity and discusses the normative implications of the results.

PROFESIONAL DE LA INFORMACION (2023)

Article Computer Science, Information Systems

Revolutions in science: The proposal of an approach for the identification of most important researchers, institutions and countries based on co-citation reference publication year spectroscopy exemplified at research on physical modelling of Earth's climate

Lutz Bornmann, Robin Haunschild, Werner Marx

Summary: Reference Publication Year Spectroscopy (RPYS) is a bibliometric method used to reveal the historical roots of research topics or fields. It identifies the most frequently referenced publications within a specific reference publication year, instead of the most highly cited papers. This study demonstrates how RPYS can be applied to identify important researchers, institutions, and countries in breakthrough research, using the example of climate modeling and global warming prediction.

JOURNAL OF INFORMATION SCIENCE (2023)

Article Multidisciplinary Sciences

Anchoring effects in the assessment of papers: An empirical survey of citing authors

Lutz Bornmann, Christian Ganser, Alexander Tekles

Summary: In this study, we empirically investigated the influence of numerical information as anchors, such as citation impact, on the assessment of cited papers. We conducted a survey among corresponding authors and assigned them to different treatment groups receiving various additional numerical information. Our results suggest that the assessment of paper quality is influenced by the citation impact information, but not by other numerical information like an access code or journal impact.

PLOS ONE (2023)

Article Computer Science, Interdisciplinary Applications

John Mearsheimer's academic roots: a reference publication year spectroscopy of a political scientist's oeuvre

Anton Gruber, Alexander Tekles, Lutz Bornmann

Summary: This paper aims to analyze the academic roots of political scientist John J. Mearsheimer using the method of Reference Publication Year Spectroscopy (RPYS). By compiling a list of his most cited works and examining temporal peaks, the study reveals both the foundation of his theory of International Relations and his focus on current events and conflicts.

SCIENTOMETRICS (2023)

Article Computer Science, Interdisciplinary Applications

Which papers cited which tweets? An exploratory analysis based on Scopus data

Robin Haunschild, Lutz Bornmann

Summary: Many altmetric studies have examined the frequency of paper mentions on Twitter. This study investigates the citations of tweets in papers to assess their potential relevance. However, the findings suggest a low number of citations and indicate that tweets are more often used as study objects rather than influential content. The subject areas with the most citations of tweets are Social Sciences, Arts and Humanities, and Medicine, with COVID-19/corona pandemic being the dominant topic.

JOURNAL OF INFORMETRICS (2023)

Article Computer Science, Interdisciplinary Applications

Efficiency of universities and research-focused institutions worldwide: The introduction of a new input indicator reflecting institutional staff numbers

Lutz Bornmann, Sabine Gralka, Felix de Moya Anegon, Klaus Wohlrabe

Summary: One of the core indicators in scientometrics is the number of papers published by a unit within a given period. To properly assess such indicators, it is necessary to consider the unit's available resources. This study introduces a new input indicator, the number of unique authors mentioned on the institutions' papers, and calculates efficiency scores for over 3100 institutions from the higher education sector using this new indicator and output indicators. The results show a strong correlation between the new input indicator and institutional staff numbers.

JOURNAL OF INFORMETRICS (2023)

Article Multidisciplinary Sciences

Are quality assessments in science affected by anchoring effects? The proposal of a follow-up study

Lutz Bornmann, Christian Ganser

Summary: This study aims to empirically examine the assessment of scientific papers using the anchoring-and-adjustment heuristic. By surveying corresponding authors and analyzing their adjustments to the anchor value, the research investigates whether the evaluation can be influenced by both quality-related information and unrelated numerical factors. The study seeks to shed light on whether bibliometrics or other numerical measures create the social order they intend to measure.

PLOS ONE (2023)

Article Computer Science, Interdisciplinary Applications

Identification of potential young talented individuals in the natural and life sciences: A bibliometric approach

Robin Haunschild, Lutz Bornmann

Summary: This study proposes a citation-free method for identifying potential young talented individuals based on their early publication performance. Three different indicators and their combinations were used to define potential talent. The best performing indicator combination was applied to identify young potentially talented individuals who published their first paper between 2007-2011, resulting in a set of 46,200 individuals that can be downloaded for free.

JOURNAL OF INFORMETRICS (2023)

Article Computer Science, Interdisciplinary Applications

What do we know about the disruption index in scientometrics? An overview of the literature

Christian Leibel, Lutz Bornmann

Summary: The paper provides a comprehensive review of the original disruption index (DI1) and its variants in scientometrics. It explains the technical and theoretical properties of these indices and examines their validity and limitations. The review highlights the higher convergent validity of modified index variants compared to DI1 and emphasizes the need for further research on the validity of disruption scores.

SCIENTOMETRICS (2023)

Article Management

Have Quadruple and Quintuple Helices Emerged? Metaphors, Project Titles, and Empirical Research

Helen Lawton Smith, Loet Leydesdorff

Summary: The Triple Helix model, proposing the interactions among universities, industry, and government, has been the basis for studying entrepreneurship. However, additional helix structures may add little to the discourse. This article responds to criticisms and discusses convenience, presentation, and the need for a more unified theory model.

TRIPLE HELIX (2022)

暂无数据