Review
Engineering, Environmental
Jun-Jie Zhu, Meiqi Yang, Zhiyong Jason Ren
Summary: Machine learning is widely used in environmental research, but inadequate studies can lead to incorrect conclusions. This study provides a compilation of common pitfalls and best practice guidelines for environmental machine learning research, based on literature analysis and the authors' own experience. By analyzing highly cited research articles, the study identifies key areas of concern and provides evidence-based data analysis to address misconceptions and improve the rigor of data preprocessing and model development standards.
ENVIRONMENTAL SCIENCE & TECHNOLOGY
(2023)
Review
Biochemistry & Molecular Biology
Samantha Dodbele, Nebibe Mutlu, Jeremy E. Wilusz
Summary: Circular RNAs (circRNAs) are a class of RNA with covalently linked ends, with most mature circRNAs expressed at low levels, but some having known physiological functions and higher expression levels. Research on this type of RNA requires consideration of method limitations and potential risks, and practical advice for robust identification, validation, and functional characterization is provided.
Article
Computer Science, Artificial Intelligence
Hansika Hewamalage, Klaus Ackermann, Christoph Bergmeir
Summary: Recent trends in ML and DL have shown their competitiveness in time series forecasting due to the availability of massive time series data. However, non-stationarities challenge the capabilities of data-driven ML models. The lack of knowledge on forecast evaluation among ML researchers leads to flawed evaluation practices, suggesting methods that are not competitive to be seemingly competitive. This work provides a tutorial-like compilation of details and best practices in forecast evaluation to bridge the knowledge gap.
DATA MINING AND KNOWLEDGE DISCOVERY
(2023)
Article
Radiology, Nuclear Medicine & Medical Imaging
Tyler J. Bradshaw, Ronald Boellaard, Joyita Dutta, Abhinav K. Jha, Paul Jacobs, Quanzheng Li, Chi Liu, Arkadiusz Sitek, Babak Saboury, Peter J. H. Scott, Piotr J. Slomka, John J. Sunderland, Richard L. Wahl, Fereshteh Yousefirizi, Sven Zuehlsdorff, Arman Rahmim, Irene Buvat
Summary: This article provides recommendations on technical best practices for developing AI algorithms in nuclear medicine, aiming to help users and developers avoid pitfalls of AI.
JOURNAL OF NUCLEAR MEDICINE
(2022)
Article
Chemistry, Physical
Fernande Grandjean, Gary J. Long
Summary: This paper discusses the best recommended practices and protocols for measuring, analyzing, and presenting Mossbauer spectra, using various iron-containing compounds as examples. It focuses on techniques for spectrometer calibration, minimizing spectral line widths, and selecting optimal spectral velocity ranges.
CHEMISTRY OF MATERIALS
(2021)
Article
Chemistry, Physical
Aditya Bhan, W. Nicholas Delgass
Summary: Catalysis, as a transdisciplinary field, has made significant progress in the synthesis, characterization, kinetics, and theory of materials and molecules. It plays a crucial role in energy conversion and storage, environmental remediation, and medicine. By following best practices and standards, we can enhance the clarity, reproducibility, and rigor in catalysis research and practice.
JOURNAL OF CATALYSIS
(2022)
Editorial Material
Green & Sustainable Science & Technology
Lyu Zhou, Xiaobo Yin, Qiaoqiang Gan
Summary: Radiative cooling is an energy-efficient technology that can dissipate excess heat without the need for additional energy input. However, the lack of transparency and standardization in reporting radiative cooling performance poses a risk of misjudging the actual breakthroughs. This Comment addresses the common pitfalls in performance measurement and suggests best practices for future endeavors to promote practical applications.
NATURE SUSTAINABILITY
(2023)
Editorial Material
Chemistry, Multidisciplinary
Nongnuch Artrith, Keith T. Butler, Francois-Xavier Coudert, Seungwu Han, Olexandr Isayev, Anubhav Jain, Aron Walsh
Summary: In chemistry research, statistical tools based on machine learning are being integrated to train reliable, repeatable, and reproducible models. Guidelines for machine learning reports are recommended to ensure the quality of the models.
Review
Biochemical Research Methods
Janis Shin, Veronica Porubsky, James Carothers, Herbert M. Sauro
Summary: The reproducibility of scientific research is crucial and can be enhanced by using software engineering strategies and adhering to modeling principles. Existing standards and well-established model repositories can greatly improve the reproducibility of published models. Open-source publication of source code, data, and documentation is recommended, especially for models published in executable programming languages. Container-based solutions are recommended for complex models to easily replicate the software dependencies and run-time context.
CURRENT OPINION IN BIOTECHNOLOGY
(2023)
Article
Environmental Studies
Maggie Cascadden, Thomas Gunton, Murray Rutherford
Summary: This paper presents a comprehensive best practices framework for developing and managing IBAs from the perspective of impacted communities, with 10 general best practice criteria, 44 sub-criteria, and 89 indicators. These criteria can guide the negotiation, implementation, and management of IBAs, as well as conducting ex post IBA evaluations.
Review
Cardiac & Cardiovascular Systems
Robert J. Lederman, Adam B. Greenbaum, Jaffar M. Khan, Christopher G. Bruce, Vasilis C. Babaliaros, Toby Rogers
Summary: Transcaval aortic access is a versatile technique for large-bore arterial access via the abdominal aorta and inferior vena cava. It has been used for various procedures and offers advantages in certain situations. However, the dissemination of this technique has been hindered by the lack of commercially available devices. This review provides comprehensive guidance on transcaval access and closure techniques to help operators improve their proficiency.
JACC-CARDIOVASCULAR INTERVENTIONS
(2023)
Review
Chemistry, Physical
Zhanzhao Li, Jinyoung Yoon, Rui Zhang, Farshad Rajabipour, Wil V. Srubar, Ismaila Dabo, Aleksandra Radlinska
Summary: Concrete science has made progress, but concrete formulation remains challenging. Machine learning has transformative potential and has been widely used in concrete research. It is necessary to understand methodological limitations and formulate best practices to fully exploit the capabilities of machine learning models.
NPJ COMPUTATIONAL MATERIALS
(2022)
Article
Mathematical & Computational Biology
Ricardo Sanchez, Beth Ann Griffin, Joseph Pane, Daniel F. McCaffrey
Summary: This paper outlines key steps for implementing a code quality assurance process to ensure the quality of final data, code, analyses, and results, including adherence to best practices, clear documentation, and regular testing and review.
STATISTICS IN MEDICINE
(2021)
Editorial Material
Green & Sustainable Science & Technology
Yaoxin Zhang, Swee Ching Tan
Summary: Consensus is needed to accurately evaluate the performance and potential of emerging water production technologies, and this article provides recommendations on how to achieve a fair basis for comparison and align research input with actual demand.
NATURE SUSTAINABILITY
(2022)
Article
Computer Science, Information Systems
Luise Pufahl, Francesca Zerbato, Barbara Weber, Ingo Weber
Summary: The design and analysis of process models play a crucial role in improving organizations across industries. However, the healthcare sector faces unique challenges in process modeling due to its complexity. This paper identifies common challenges in healthcare process modeling and proposes BPMN best practices to address these challenges.
INFORMATION SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Zhengying Liu, Adrien Pavao, Zhen Xu, Sergio Escalera, Fabio Ferreira, Isabelle Guyon, Sirui Hong, Frank Hutter, Rongrong Ji, Julio C. S. Jacques, Ge Li, Marius Lindauer, Zhipeng Luo, Meysam Madadi, Thomas Nierhoff, Kangning Niu, Chunguang Pan, Danny Stoll, Sebastien Treguer, Jin Wang, Peng Wang, Chenglin Wu, Youcheng Xiong, Arber Zela, Yang Zhang
Summary: This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, where deep learning methods dominated in the setting that pushed for quick results through code submissions on hidden tasks. The study found that popular Neural Architecture Search (NAS) was impractical in this context.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Computer Science, Artificial Intelligence
Lucas Zimmer, Marius Lindauer, Frank Hutter
Summary: Auto-PyTorch combines neural architecture search and hyperparameter optimization to enable fully automated deep learning, achieving state-of-the-art performance on tabular benchmarks. Additionally, a new benchmark on DNN learning curves called LCBench is introduced, along with extensive ablation studies of Auto-PyTorch on typical AutoML benchmarks.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Computer Science, Artificial Intelligence
Jack Parker-Holder, Raghu Rajan, Xingyou Song, Andre Biedenkapp, Yingjie Miao, Theresa Eimer, Baohe Zhang, Vu Nguyen, Roberto Calandra, Aleksandra Faust, Frank Hutter, Marius Lindauer
Summary: The combination of Reinforcement Learning (RL) with deep learning has led to impressive achievements, but the success of RL agents is sensitive to design choices and manual tuning. AutoML has shown promise in automating design choices, and AutoRL is emerging as an important research area. This survey aims to unify the field of AutoRL, provide a common taxonomy, and discuss open problems for future researchers.
JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH
(2022)
Proceedings Paper
Automation & Control Systems
Iman Nematollahi, Erick Rosete-Beas, Seyed Mahdi B. Azad, Raghu Rajan, Frank Hutter, Wolfram Burgard
Summary: To achieve autonomous skill acquisition, a transformation-based 3D video prediction (T3VIP) approach is proposed, which learns the physical rules governing the 3D world dynamics and is able to predict and reason about future outcomes. The model captures observational cues from image and point cloud domains, and incorporates automatic hyperparameter optimization to leverage the 2D and 3D observational signals. The model produces interpretable 3D models for predicting future depth videos and outperforms 2D baselines in RGB video prediction and visuomotor control.
2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)
(2022)
Article
Computer Science, Artificial Intelligence
Steven Adriaensen, Andre Biedenkapp, Gresa Shala, Noor Awad, Theresa Eimer, Marius Lindauer, Frank Flutter
Summary: The performance of algorithms often relies on their parameter configuration. Automated algorithm configuration methods can alleviate the task of manually tuning parameters, but the learned configuration remains static. A promising approach is to automatically learn dynamic parameter adaptation policies from data. This article provides a comprehensive account of this new field of automated dynamic algorithm configuration, presenting recent advances and laying the foundation for future research.
JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH
(2022)
Proceedings Paper
Computer Science, Cybernetics
Andre Biedenkapp, Nguyen Dang, Martin S. Krejca, Frank Hutter, Carola Doerr
Summary: The performance of evolutionary algorithms and other randomized search heuristics can be improved by choosing non-static parameters. However, we still lack understanding of the best approaches for dynamic parameter setting. This study extends a benchmark with known control policies and demonstrates its usefulness in analyzing the behavior of a reinforcement learning approach for dynamic algorithm configuration.
PROCEEDINGS OF THE 2022 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE (GECCO'22)
(2022)
Article
Automation & Control Systems
Marius Lindauer, Katharina Eggensperger, Matthias Feurer, Andre Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, Rene Sass, Frank Hutter
Summary: Algorithm parameters, especially hyperparameters, play a crucial role in the performance of machine learning algorithms. SMAC3 offers a versatile Bayesian Optimization framework to help users determine optimal hyperparameter configurations. It can significantly improve performance with just a few evaluations and is suitable for various use cases.
JOURNAL OF MACHINE LEARNING RESEARCH
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Samuel G. Mueller, Frank Hutter
Summary: Automatic augmentation methods have become crucial for strong model performance in vision tasks. TrivialAugment, a simple baseline without parameters, outperforms previous methods almost for free. Through experiments and ablation studies, the key requirements for TrivialAugment's performance are revealed, providing a simple interface for widespread adoption and proposing best practices for future progress.
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Artur Souza, Luigi Nardi, Leonardo B. Oliveira, Kunle Olukotun, Marius Lindauer, Frank Hutter
Summary: BOPrO is a Bayesian Optimization method that allows users to inject their knowledge about the input space to improve optimization performance. It outperforms state-of-the-art methods in terms of speed and achieves new performance levels in real-world applications.
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2021: RESEARCH TRACK, PT III
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Andre Biedenkapp, Raghu Rajan, Frank Hutter, Marius Lindauer
Summary: Reinforcement learning is a powerful method of learning behavior through interaction with the environment, but traditional methods may struggle to make decisions proactively. The TempoRL approach introduces skip connections between states and skip-policy learning for repeated actions, showing significant acceleration in learning successful policies compared to vanilla Q-learning.
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Baohe Zhang, Raghu Rajan, Luis Pineda, Nathan Lambert, Andre Biedenkapp, Kurtland Chua, Frank Hutter, Roberto Calandra
Summary: Model-based Reinforcement Learning (MBRL) is a promising framework for learning control efficiently. Automatic hyperparameter optimization (HPO) can significantly improve performance, and dynamically tuning multiple MBRL hyperparameters during training further enhances performance. These insights contribute to understanding the effects of various hyperparameters on training stability and rewards.
24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Andre Biedenkapp, H. Furkan Bozkurt, Theresa Eimer, Frank Hutter, Marius Lindauer
ECAI 2020: 24TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE
(2020)
