Against the one method fits all data sets philosophy for comparison studies in methodological research

Many methodological comparison studies aim at identifying a single or a few best performing methods over a certain range of data sets. In this paper we take a different viewpoint by asking whether the research question of identifying the best performing method is what we should be striving for in the first place. We will argue that this research question implies assumptions which we do not consider warranted in methodological research, that a different research question would be more informative, and how this research question can be fruitfully investigated.

Automated summary

This paper challenges the idea of striving to identify the best performing method in methodological comparison studies. The author argues that this research question assumes certain unwarranted assumptions and suggests a more informative research question that can be fruitfully investigated.