4.5 Article

Recruitment, engagement and feedback in empirical software engineering studies in industrial contexts

Journal

INFORMATION AND SOFTWARE TECHNOLOGY
Volume 98, Issue -, Pages 161-172

Publisher

ELSEVIER
DOI: 10.1016/j.infsof.2017.12.001

Keywords

Empirical software engineering; Industry; Research; Grounded theory; Survey; Case study; Quasi-experiment; Challenges; Solutions; Recommendations

Funding

  1. Australian Research Council
  2. Ministry of Business, Innovation and Employment
  3. Foundation for Research, Science and Technology
  4. Ministry of Higher Education Malaysia
  5. BuildIT NZ
  6. Agile Alliance
  7. Software Process and Product Improvement for New Zealand Software Industry [UOAX0710]

Ask authors/readers for more resources

Context Research carried out in industrial contexts are recognized as important to the advancement of software engineering knowledge and practice. However, several challenges present themselves in the three key phases of research carried out in industrial contexts, recruitment, engagement and feedback. Objective: The aim of this paper is to report the challenges related to each of the three phases of research carried out in industrial contexts, and the associated solutions we have found useful from our combined body of industrial empirical software engineering research studies spanning four case studies, five grounded theory studies, seven survey studies and two quasi-experimental studies involving a total of over 400 industrial participants in the past decade. Method: We designed an instrument to gather details of our studies carried out in industrial contexts and performed thematic analysis to synthesise and draw out the most prominent challenges faced. Results: We present a set of recommendations around study design, conduct and reporting to try and mitigate some of these challenges as they apply specifically to industrial empirical research. Conclusion: These recommendations can guide researchers, novice and experienced, working in close collaboration with industry stakeholders to make the most of their industrial software engineering research.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.5
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Computer Science, Software Engineering

APIMatchmaker: Matching the Right APIs for Supporting the Development of Android Apps

Yanjie Zhao, Li Li, Haoyu Wang, Qiang He, John Grundy

Summary: Android developers often need to learn how to use different APIs suitable for their projects. Existing automated API recommendation approaches for Android app development are often redundant and have poor run-time performance. To improve this, we propose a new tool called APIMatchmaker, which recommends API usages by learning from similar real-world Android apps. Unlike existing approaches, we introduce a multi-dimensional, context-aware, collaborative filtering approach that considers both code similarity and app descriptions to ensure similar apps provide similar functions. Evaluation results show that APIMatchmaker outperforms state-of-the-art methods in recommending APIs for Android app development.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2023)

Article Computer Science, Theory & Methods

Engineering Blockchain-based Software Systems: Foundations, Survey, and Future Directions

Mahdi Fahmideh, John Grundy, Aakash Ahmad, Jun Shen, Jun Yan, Davoud Mougouei, Peng Wang, Aditya Ghose, Anuradha Gunawardana, Uwe Aickelin, Babak Abedin

Summary: This article presents a systematic literature review of the state-of-the-art in Blockchain-based Software (BBS) engineering research from the perspective of software engineering discipline. It provides a rich repertoire of development tasks, design principles, models, roles, challenges, and resolution techniques in BBS engineering.

ACM COMPUTING SURVEYS (2023)

Article Computer Science, Software Engineering

The Influence of Human Aspects on Requirements Engineering-related Activities: Software Practitioners' Perspective

Dulaji Hidellaarachchi, John Grundy, Rashina Hoda, Ingo Mueller

Summary: Requirements Engineering (RE)-related activities heavily rely on collaboration between various roles in software engineering (SE), making it highly human-dependent. This study aims to understand the influence of human aspects, particularly motivation and personality, on RE-related activities from the perspective of software practitioners. The findings highlight the importance of motivation, domain knowledge, attitude, communication skills, and personality in RE-related activities, providing insights into key motivational factors and desirable personality characteristics for effective involvement in RE. The study also identifies areas needing further investigation and provides recommendations for future research.

ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY (2023)

Article Computer Science, Software Engineering

Supporting Developers in Addressing Human-Centric Issues in Mobile Apps

Hourieh Khalajzadeh, Mojtaba Shahin, Humphrey O. Obie, Pragya Agrawal, John Grundy

Summary: Failure to consider the characteristics, limitations, and abilities of diverse end-users during mobile app development may lead to human-centric issues for end-users. This paper examines the human-centric issues reported by end-users through app reviews and discussed by developers on GitHub. It also investigates the feasibility and usefulness of an automated tool for detecting and classifying human-centric issues. The findings highlight the importance of addressing these issues and suggest possible future work to improve mobile app development.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2023)

Article Computer Science, Software Engineering

A Framework for Emotion-Oriented Requirements Change Handling in Agile Software Engineering

Kashumi Madampe, Rashina Hoda, John Grundy

Summary: This study investigates the impact of requirements changes on software practitioners and identifies key challenges, emotional impacts, influencing factors, and practices for better handling changes. It emphasizes the importance of synergy between agility, emotional intelligence, and cognitive intelligence in handling changes with positive emotions in socio-technical environments.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2023)

Article Computer Science, Software Engineering

The Emotional Roller Coaster of Responding to Requirements Changes in Software Engineering

Kashumi Madampe, Rashina Hoda, John Grundy

Summary: This study investigated the emotional responses of software practitioners to requirements changes (RCs). The findings showed that practitioners experienced different emotions at different stages of the RC handling life cycle, and these emotions had direct effects on cognition, productivity, and decision making. The study emphasized the importance of understanding the role of emotions in critical scenarios such as handling RCs.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2023)

Article Computer Science, Information Systems

Requirements engineering for artificial intelligence systems: A systematic mapping study

Khlood Ahmad, Mohamed Abdelrazek, Chetan Arora, Muneera Bano, John Grundy

Summary: This paper investigates current approaches for specifying requirements for AI systems, identifies available frameworks, methodologies, tools, and techniques used to model requirements, and finds existing challenges and limitations. The empirical studies on RE4AI focused on autonomous vehicles and managing data requirements, and areas such as ethics, trust, and explainability need further research.

INFORMATION AND SOFTWARE TECHNOLOGY (2023)

Article Computer Science, Software Engineering

Towards automated Android app internationalisation: An exploratory study

Pei Liu, Qingxin Xia, Kui Liu, Juncai Guo, Xin Wang, Jin Liu, John Grundy, Li Li

Summary: Android is the most popular mobile platform with over 2.5 billion active users worldwide. To accommodate users from different countries and languages, app developers need to add internationalization features to their apps. Automated approaches are in demand to reduce manual effort and improve efficiency.

JOURNAL OF SYSTEMS AND SOFTWARE (2023)

Article Computer Science, Software Engineering

Taming Android Fragmentation Through Lightweight Crowdsourced Testing

Xiaoyu Sun, Xiao Chen, Yonghui Liu, John Grundy, Li Li

Summary: Android fragmentation refers to the vast diversity of Android devices and OS versions, leading to compatibility bugs and poor user experiences. Existing tools can only detect certain types of compatibility issues, leaving many others undisclosed. To address this, we propose LazyCow, a lightweight crowdsourced testing approach, which effectively identifies and verifies API-induced compatibility issues. Experimental results demonstrate its usefulness and satisfaction among users.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2023)

Article Computer Science, Information Systems

OL-MEDC: An Online Approach for Cost-Effective Data Caching in Mobile Edge Computing Systems

Xiaoyu Xia, Feifei Chen, Qiang He, Guangming Cui, John Grundy, Mohamed Abdelrazek, Athman Bouguettaya, Hai Jin

Summary: Mobile Edge Computing (MEC) has emerged to overcome the limitations of cloud computing in providing low latency services. This study focuses on the Mobile Edge Data Caching (MEDC) problem from the app vendor's perspective, aiming to maximize caching revenue while considering costs. An approach called Online MEDC (OL-MEDC) is proposed, which formulates MEDC strategies for app vendors without requiring future information about data demands. OL-MEDC outperforms state-of-the-art approaches by at least 20.41% on average, as demonstrated by experimental results.

IEEE TRANSACTIONS ON MOBILE COMPUTING (2023)

Article Computer Science, Information Systems

Toward a Unified mHealth Platform: A Survey of Current User Challenges and Expectations

Ben Joseph Philip, Mohamed Abdelrazek, Scott Barnett, Alessio Bonti, John Grundy

Summary: mHealth apps have become widespread, but downloading and using multiple apps present challenges. This study analyzes user challenges and expectations to inform the design of better mHealth platforms. The findings suggest the need for more capable and unified platforms that can be customized to meet user needs.

IEEE ACCESS (2023)

Article Computer Science, Information Systems

Online User and Power Allocation in Dynamic NOMA-Based Mobile Edge Computing

Phu Lai, Qiang He, Feifei Chen, Mohamed Abdelrazek, John Hosking, John Grundy, Yun Yang

Summary: This study tackles the online user allocation problem in mobile edge computing systems. By using non-orthogonal multiple access, the aim is to minimize allocation delay and transmit power costs, thus increasing energy efficiency. The Lyapunov framework and a distributed game theory-based approach are adopted to achieve this objective and guarantee the data rate requirements of users over time.

IEEE TRANSACTIONS ON MOBILE COMPUTING (2023)

Review Computer Science, Information Systems

Systematic reviews in mobile app software engineering: A tertiary study

Samer Zein, Norsaremah Salleh, John Grundy

Summary: This paper provides an overview and analysis of secondary studies in the field of mobile app software engineering, finding that the trend is towards more specific areas such as architectural design models and context-aware testing. The study identifies research directions and useful practices for practitioners.

INFORMATION AND SOFTWARE TECHNOLOGY (2023)

Article Computer Science, Information Systems

Why and how bug blocking relations are breakable: An empirical study on breakable blocking bugs

Hao Ren, Yanhui Li, Lin Chen, Yuming Zhou, Changhai Nie

Summary: This study aims to explore the breakable blocking bugs (BBBs) through quantitative and qualitative analysis. The analysis reveals that BBBs have higher levels of involvement, longer fix time, and more complex source code compared to other bugs. The study also identifies four reasons for breaking blocking relationships between bugs and three measures adopted by developers to break these relationships.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Vulnerability detection based on federated learning

Chunyong Zhang, Tianxiang Yu, Bin Liu, Yang Xin

Summary: This paper proposes a vulnerability detection framework based on federated learning (VDBFL), which combines code property graph, graph neural networks, and convolutional neural networks to detect vulnerability code. The experimental results show that this method outperforms other vulnerability detection methods.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Collaborative software design and modeling in virtual reality

Martin Stancek, Ivan Polasek, Tibor Zalabai, Juraj Vincur, Rodi Jolak, Michel Chaudron

Summary: The aim of this research is to support distributed software design activities in Virtual Reality (VR). Using design science research methodology, a tool for collaborative design in VR is designed and evaluated. The efficiency of collaboration and recall of design information when using VR software design environment compared to non-VR environment are evaluated. Furthermore, the perceptions and preferences of users are collected to explore the opportunities and challenges of using VR software design environment.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Improving domain-specific neural code generation with few-shot meta-learning

Zhen Yang, Jacky Wai Keung, Zeyu Sun, Yunfei Zhao, Ge Li, Zhi Jin, Shuo Liu, Yishu Li

Summary: This paper presents MetaCoder, a meta-learning code generation approach that efficiently extracts general-purpose knowledge from large-scale source languages and rapidly adapts to domain-specific scenarios.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Automated code-based test case reuse for software product line testing

Pilsu Jung, Seonah Lee, Uicheon Lee

Summary: This study proposes an automated code-based approach (ActSPL) for reusing SPL test cases by utilizing source code and test cases. The results show that ActSPL achieves high precision and recall, and significantly reduces the time required for testing a new product.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Understanding the implementation issues when using deep learning frameworks

Chao Liu, Runfeng Cai, Yiqun Zhou, Xin Chen, Haibo Hu, Meng Yan

Summary: This paper conducts an empirical study on the implementation issues of deep learning frameworks, focusing on relevant questions on Stack Overflow. The study identifies various implementation issues and constructs a taxonomy, revealing that data processing, model setting, model training, and model prediction are the most common categories. The paper also provides suggestions for future research and aims to help developers and researchers understand these issues better.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Genetic model-based success probability prediction of quantum software development projects

Muhammad Azeem Akbar, Arif Ali Khan, Mohammad Shameem, Mohammad Nadeem

Summary: This study identifies key variables in quantum software development (QSD) and develops a model for predicting the success probability of QSD projects. The results show that as the QSD process matures, project success probability significantly increases and costs are notably reduced. The developed prediction model can help practitioners focus on key areas for successful implementation of QSD projects.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Developer and End-User Perspectives on Addressing Human Aspects in Mobile eHealth Apps

Md. Shamsujjoha, John Grundy, Hourieh Khalajzadeh, Qinghua Lu, Li Li

Summary: This paper investigates the challenges and benefits of incorporating human aspects into eHealth app development and usage from the perspectives of developers and end-users. The study used a mixed-method approach and gathered data from online surveys and interviews. The findings suggest that addressing human aspects throughout the app development life-cycle is beneficial for more effective eHealth apps.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Understanding how early-stage researchers leverage socio-technical affordances for distributed research

Yuchao Jiang, Boualem Benatallah, Marcos Baez

Summary: This paper reports on interviews and surveys with early-stage researchers (ESRs) and explores the potential of online research communities in supporting ESRs to learn from diverse perspectives and experiences. The results reveal the limited adoption of research communities for learning and identify unmet needs in their design. Design implications for future socio-technical systems are provided to support the development of research skills.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Deep learning-based software bug classification

Jyoti Prakash Meher, Sourav Biswas, Rajib Mall

Summary: Accurate bug classification is important for speeding up bug triage, code inspection, and repair tasks. To improve classification, this study proposes a novel bug classification approach based on deep learning. The approach includes building a bug taxonomy with eight bug classes using keywords, annotating a large set of bug resolution reports, and utilizing attention-based classification techniques. Experimental results show that the proposed technique outperforms existing methods in terms of F1-Score by an average of 16.88% on the considered dataset.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Software Engineering for Systems-of-Systems and Software Ecosystems

Rodrigo Santos, Eleni Constantinou, Pablo Antonino, Jan Bosch

Summary: In the last decade, software engineering has faced challenges beyond technical aspects. The field now considers technological, organizational, and social aspects together in research and practice to handle complexity and provide solutions to the industry's demands. Systems-of-systems (SoS) and software ecosystems (SECO) have emerged as topics of interest, bringing together researchers and practitioners to understand how to manage and engineer software-intensive systems in modern, complex, distributed, dynamic, and open environments.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Article Computer Science, Information Systems

Stratified random sampling for neural network test input selection

Zhuo Wu, Zan Wang, Junjie Chen, Hanmo You, Ming Yan, Lanjun Wang

Summary: In this paper, a statistical method called Stratified random Sampling with Optimum Allocation (SSOA) is proposed to provide an unbiased estimation of model accuracy with the smallest estimation variance. The unlabeled test set is first divided into strata based on predictive confidences. Then, two stratum accuracy variance estimation methods are designed to allocate the given budget to each stratum based on the optimum allocation strategy. Multiple experiments are conducted to evaluate the effectiveness and stability of SSOA by comparing it with baseline methods.

INFORMATION AND SOFTWARE TECHNOLOGY (2024)

Review Computer Science, Information Systems

The consolidation of game software engineering: A systematic literature review of software engineering for industry-scale computer games

Jorge Chueca, Javier Veron, Jaime Font, Francisca Perez, Carlos Cetina

INFORMATION AND SOFTWARE TECHNOLOGY (2024)