4.1 Article

A SIMULATION OF DISAGREEMENT FOR CONTROL OF RATIONAL CHEATING IN PEER REVIEW

期刊

ADVANCES IN COMPLEX SYSTEMS
卷 16, 期 7, 页码 -

出版社

WORLD SCIENTIFIC PUBL CO PTE LTD
DOI: 10.1142/S0219525913500045

关键词

Artificial social systems; peer review; agent-based simulation; trust reliability and reputation

资金

  1. European Union [315874]
  2. Spanish MICINN
  3. European Commission FEDER funds
  4. Universitat de Valencia funds [CSD2006-00046, TIN2009-14475-C04-04, UV-INV-AE11-40990.62]
  5. FuturICT coordination action

向作者/读者索取更多资源

Understanding the peer review process could help research and shed light on the mechanisms that underlie crowdsourcing. In this paper, we present an agent-based model of peer review built on three entities - the paper, the scientist and the conference. The system is implemented on a BDI platform (Jason) that allows to define a rich model of scoring, evaluating and selecting papers for conferences. Then, we propose a programme committee update mechanism based on disagreement control that is able to remove reviewers applying a strategy aimed to prevent papers better than their own to be accepted (rational cheating). We analyze a homogeneous scenario, where all conferences aim to the same level of quality, and a heterogeneous scenario, in which conferences request different qualities, showing how this affects the update mechanism proposed. We also present a first step toward an empirical validation of our model that compares the amount of disagreements found in real conferences with that obtained in our simulations.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.1
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

Article Computer Science, Artificial Intelligence

Modeling and implementing distributed data mining strategies in JaCa-DDM

Xavier Limon, Alejandro Guerra-Hernandez, Nicandro Cruz-Ramirez, Francisco Grimaldo

KNOWLEDGE AND INFORMATION SYSTEMS (2019)

Editorial Material Multidisciplinary Sciences

Unlock ways to share data on peer review

Flaminio Squazzoni, Petra Ahrweiler, Tiago Barros, Federico Bianchi, Aliaksandr Birukou, Harry J. J. Blom, Giangiacomo Bravo, Stephen Cowley, Virginia Dignum, Pierpaolo Dondio, Francisco Grimaldo, Lynsey Haire, Jason Hoyt, Phil Hurst, Rachael Lammey, Catriona MacCallum, Ana Marusic, Bahar Mehmani, Hollydawn Murray, Duncan Nicholas, Giorgio Pedrazzi, Iratxe Puebla, Peter Rodgers, Tony Ross-Hellauer, Marco Seeber, Kalpana Shankar, Joris Van Rossum, Michael Willis

NATURE (2020)

Review Biology

Large-scale language analysis of peer review reports

Ivan Buljan, Daniel Garcia-Costa, Francisco Grimaldo, Flaminio Squazzoni, Ana Marusic

Article Mathematics

Measuring Arithmetic Word Problem Complexity through Reading Comprehension and Learning Analytics

Maria T. Sanz, Emilia Lopez-Inesta, Daniel Garcia-Costa, Francisco Grimaldo

MATHEMATICS (2020)

Article Multidisciplinary Sciences

Peer review and gender bias: A study on 145 scholarly journals

Flaminio Squazzoni, Giangiacomo Bravo, Mike Farjam, Ana Marusic, Bahar Mehmani, Michael Willis, Aliaksandr Birukou, Pierpaolo Dondio, Francisco Grimaldo

Summary: This study analyzed gender bias in 145 journals and found that manuscripts written by women as solo authors or coauthored by women were treated more favorably by referees and editors. The results suggest that peer review and editorial processes do not penalize manuscripts by women, and increasing gender diversity in editorial teams and referee pools could help encourage more women to participate.

SCIENCE ADVANCES (2021)

Article Multidisciplinary Sciences

Gender gap in journal submissions and peer review during the first wave of the COVID-19 pandemic. A study on 2329 Elsevier journals

Flaminio Squazzoni, Giangiacomo Bravo, Francisco Grimaldo, Daniel Garcia-Costa, Mike Farjam, Bahar Mehmani

Summary: During the early months of the COVID-19 pandemic, women submitted proportionally fewer scholarly manuscripts than men, especially among younger cohorts of female academics. However, women took on a greater service responsibility for journals in terms of peer-review activities, except in the field of health & medicine where men had a more prominent role due to the impact of COVID-19 research.

PLOS ONE (2021)

Review Multidisciplinary Sciences

Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals

Daniel Garcia-Costa, Flaminio Squazzoni, Bahar Mehmani, Francisco Grimaldo

Summary: This study tested a quality assessment tool on a sample of 1.3 million reports submitted to 740 Elsevier journals. The results showed that the developmental standards of peer review are shared across research areas but with significant differences. Reports submitted to social science and economics journals have the highest developmental standards. Reports from junior reviewers, women, and reviewers from Western Europe are generally more developmental than those from senior, men, and reviewers working in academic institutions outside Western regions.
Article Computer Science, Interdisciplinary Applications

Measuring the effect of reviewers on manuscript change: A study on a sample of submissions to Royal Society journals (2006-2017)

Federico Bianchi, Daniel Garcia-Costa, Francisco Grimaldo, Flaminio Squazzoni

Summary: Peer review is crucial for the public trust in academic journals, ensuring rigorous research and enhancing the value of manuscripts through feedback. The number of reviewers has a significant impact on manuscript changes and the likelihood of citation post-publication. Further research with larger and more representative journal samples is needed to develop systematic measures that capture the complexity of peer review.

JOURNAL OF INFORMETRICS (2022)

Review Computer Science, Interdisciplinary Applications

Linguistic and semantic characteristics of articles and peer review reports in Social Sciences and Medical and Health Sciences: analysis of articles published in Open Research Central

Andrijana Perkovic Palos, Antonija Mijatovic, Ivan Buljan, Daniel Garcia-Costa, Elena Alvarez-Garcia, Francisco Grimaldo, Ana Marusic

Summary: We compared articles, peer review processes, and editorial processes between Medical and Health Sciences and Social Sciences. Our analysis was based on data from the Open Research Central (ORC) portal, which hosts various journal platforms for post-publication peer review. We found differences in the structure and content of articles, as well as the tone and length of peer review reports, between the two disciplines. However, the characteristics of the peer review process were similar for both types of articles, possibly due to the use of uniform policies and procedures by the publication platform.

SCIENTOMETRICS (2023)

Article Green & Sustainable Science & Technology

Hands-on Learning: Assessing the Impact of a Mobile Robot Platform in Engineering Learning Environments

Adrian Suarez, Daniel Garcia-Costa, Joaquin Perez, Emilia Lopez-Inesta, Francisco Grimaldo, Jose Torres

Summary: As the world changes, students need to acquire a diverse set of competencies and skills that focus on sustainability beyond environmental matters. Using educational tools that emulate real-life tasks related to students' future careers can significantly boost their motivation. Adopting teaching techniques that align more closely with the professional work of engineering empowers students and contributes to the advancement of science and technology. This study analyzes the impact of using a mobile robot platform as a teaching approach that merges problem-based learning with hands-on learning. The evaluation shows that students positively perceive the impact of this learning strategy.

SUSTAINABILITY (2023)

Article Multidisciplinary Sciences

Tool to assess recognition and understanding of elements in Summary of Findings Table for health evidence synthesis: a cross-sectional study

Jakov Matas, Ruzica Tokalic, Daniel Garcia-Costa, Emilia Lopez-Inesta, Elena alvarez-Garcia, Francisco Grimaldo, Ana Marusic

Summary: This study examines how medical students interact with Summary of Findings (SoF) tables and identifies their difficulties in critical thinking and understanding study outcomes. The findings suggest that students struggle with complex questions and tend to focus on answering simpler questions.

SCIENTIFIC REPORTS (2023)

Article History & Philosophy Of Science

CITIZEN SCIENCE PROJECTS An opportunity for education in scientific literacy and sustainability

Emilia Lopez-Inesta, Miguel Angel Queiruga-Dios, Daniel Garcia-Costa, Francisco Grimaldo

Summary: Since the term citizen science first emerged, the number of projects involving citizens in science has increased and spread to various areas. It is becoming a mechanism for involving society, stimulating people's interest in science, and contributing to their scientific literacy.

METODE SCIENCE STUDIES JOURNAL (2022)

Article Social Sciences, Interdisciplinary

On the Macroeconomic Effect of Extortion: An Agent-Based Approach

Alejandro Platas-Lopez, Alejandro Guerra-Hernandez, Francisco Grimaldo

Summary: This study proposes an agent-based approach to simulate the impact of extortion on macroeconomic aggregates. Results show that even low levels of propensity towards extortion can lead to significant negative effects, and there is no police efficiency level that can achieve healthy macroeconomic signals in the absence of extortion.

JASSS-THE JOURNAL OF ARTIFICIAL SOCIETIES AND SOCIAL SIMULATION (2021)

Article Mathematics, Interdisciplinary Applications

Windowing as a Sub-Sampling Method for Distributed Data Mining

David Martinez-Galicia, Alejandro Guerra-Hernandez, Nicandro Cruz-Ramirez, Xavier Limon, Francisco Grimaldo

MATHEMATICAL AND COMPUTATIONAL APPLICATIONS (2020)

Proceedings Paper Computer Science, Artificial Intelligence

Micro-Foundations of Macroeconomic Dynamics: The Agent-Based BAM Model

Alejandro Platas-Lopez, Alejandro Guerra-Hernandez, Federico Cecconi, Mario Paolucci, Francisco Grimaldo

ARTIFICIAL INTELLIGENCE RESEARCH AND DEVELOPMENT (2019)

暂无数据