4.7 Article

Controlled experiment replication in evaluation of e-learning system's educational influence

Journal

COMPUTERS & EDUCATION
Volume 53, Issue 3, Pages 591-602

Publisher

PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.compedu.2009.03.014

Keywords

Authoring tools and methods; Distributed learning environments; Evaluation methodologies; Intelligent tutoring systems

Funding

  1. Ministry of Science, Education and Sports of the Republic of Croatia [177-0361994-1996]

Ask authors/readers for more resources

We believe that every effectiveness evaluation should be replicated at least in order to verify the original results and to indicate evaluated e-learning system's advantages or disadvantages. This paper presents the methodology for conducting controlled experiment replication, as well as, results of a controlled experiment and an internal replication that investigated the effectiveness of intelligent authoring shell eXtended Tutor-Expert System (xTEx-Sys). The initial and the replicated experiment were based on our approach that combines classical two-group experimental design and with factoral design. A trait that distinguishes this approach from others is the existence of arbitrary number of checkpoint-tests to determine the effectiveness in intermediate states. We call it a pre-and-post test control group experimental design with checkpoint-tests. The gained results revealed small or even negative effect sizes, which could be explained by the fact that the xTEx-Sys's domain knowledge presentation is rather novel for students and therefore difficult to grasp and apply in earlier phases of the experiment. In order to develop and improve the xTEx-Sys, further experiments must be conducted. (C) 2009 Elsevier Ltd. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Engineering, Marine

An Agent-Based Ship Firefighting Model

Dean Sumic, Lada Males, Marko Rosic

Summary: This paper presents a proof of principal for on-board fire monitoring and extinguishing software agents that may be used to upgrade present systems and contribute to an autonomous ship design, reducing the risks of human error and increasing maritime safety.

JOURNAL OF MARINE SCIENCE AND ENGINEERING (2021)

Article Computer Science, Artificial Intelligence

Evaluation of split-and-rephrase output of the knowledge extraction tool in the intelligent tutoring system

Ani Grubisic, Branko Zitko, Angelina Gaspar, Daniel Vasic, Arta Dodaj

Summary: Various approaches to text simplification have been proposed to increase readability, but balancing linguistic simplicity, naturalness, and informativeness is crucial. Simplified texts may improve reading comprehension with shorter sentences and limited vocabulary, but often lack coherence and style. Future research should focus on addressing these trade-offs.

EXPERT SYSTEMS WITH APPLICATIONS (2022)

Article Computer Science, Information Systems

Applications of Multi-Agent Systems in Unmanned Surface Vessels

Lada Males, Dean Sumic, Marko Rosic

Summary: This paper presents a review of the current state of implementation of multi-agent systems for performing the functions of unmanned surface vessels, based on the analysis of 24 relevant papers.

ELECTRONICS (2022)

Article Computer Science, Software Engineering

The evaluation of a semi-automatic authoring tool for knowledge extraction in the AC&NL Tutor

Ani Grubisic, Slavomir Stankov, Branko Zitko, Ines Saric-Grgic, Angelina Gaspar, Emil Brajkovic, Daniel Vasic

Summary: This paper describes and evaluates the performance of a semi-automatic authoring tool (SAAT) for knowledge extraction in the AC & NL Tutor. The study compares automatic annotation tasks to a gold standard and includes human-error analysis for improving error identification and correction. The main contributions of this research are an integrated approach to text preprocessing, knowledge extraction, and visualization, as well as a comprehensive evaluation of natural language processing tasks and knowledge extraction output.

JOURNAL OF UNIVERSAL COMPUTER SCIENCE (2023)

Article Education & Educational Research

Perceptions of Digital Learning and Teaching: The Case of a Croatian University Transition to an Emergency Digital Environment

Nikola Balic, Ani Grubisic, Andrina Granic

Summary: This research analyzes the self-perception of teachers and students regarding their digital readiness and attitudes towards digital and distance learning, as well as the potential influence of other factors. The findings indicate significant digital readiness gaps, considering gender and the field of study. The study emphasizes the disconnect between expectations and reality in terms of the digital transformation of the learning environment, highlighting the need to reduce existing gaps and improve the digital teaching and learning environment.

TECHNOLOGY KNOWLEDGE AND LEARNING (2023)

Article Education, Scientific Disciplines

Predicting Academic Performance of Students in a Computer Programming Course using Data Mining

Ivan Peraic, Ani Grubisic

Summary: The purpose of this study is to predict the academic performance of students in a Programming class course by analyzing data from the Moodle learning platform. Six machine learning classification techniques were used on two datasets, and multiple similar prediction models were found to perform best in binary and three-level grade categorization. The study highlights the impact of feature selection on prediction results.

INTERNATIONAL JOURNAL OF ENGINEERING EDUCATION (2023)

Proceedings Paper Computer Science, Artificial Intelligence

Development and Evaluation of a Learning Analytics Dashboard for Moodle Learning Management System

Ivan Peraic, Ani Grubisic

Summary: This paper presents the development and evaluation of the learning analytics dashboard for students (LAD-S), which provides feedback to students and teachers. The survey results show a high level of student satisfaction with various aspects of the LAD-S, as well as willingness to use and perceive its usefulness.

HCI INTERNATIONAL 2022 - LATE BREAKING PAPERS: INTERACTION IN NEW MEDIA, LEARNING AND GAMES (2022)

Proceedings Paper Computer Science, Artificial Intelligence

A simulation model of autonomous ship firefighting

Lada Males, Dean Sumic, Marko Rosic

Summary: This paper presents a simulation model for testing agent based on board firefighting system, aiming to address ship fire accidents. By developing appropriate simulation environments, we can conduct tests for automated firefighting systems and evaluate the results.

2022 8TH INTERNATIONAL CONFERENCE ON CONTROL, DECISION AND INFORMATION TECHNOLOGIES (CODIT'22) (2022)

Article Computer Science, Theory & Methods

The Impact of Note-Taking on the Learning Process in Intelligent Tutoring System Tutomat

Ines Saric-Grgic, Ani Grubisic, Branko Zitko

Summary: This research investigates the impact of note-taking practice on the learning process in Tutomat, an intelligent tutoring system. The analysis reveals that student interaction with Tutomat varies based on their note-taking practice, which can be detected using learning analytics variables and predicted with an accuracy of 85% through clustering.

APPLIED COMPUTER SYSTEMS (2021)

Article Engineering, Marine

Agent Based Onboard Firefighting System

Dean Sumic, Lada Males, Marko Rosic

Summary: This paper presents a model of agent-based architecture for fighting fires on ships, aiming to achieve safe autonomous vessels. The introduction of agent technology can exclude the human factor, reducing injuries and losses. Additionally, agent-based technology offers easy interoperability with other automated systems.

TRANSACTIONS ON MARITIME SCIENCE-TOMS (2021)

Article Engineering, Multidisciplinary

Automatic Question Generation Using Semantic Role Labeling for Morphologically Rich Languages

Daniel Vasic, Branko Zitko, Hrvoje Ljubic

Summary: This paper presents a novel approach to automatic question generation using semantic role labeling for morphologically rich languages, focusing on Croatian. It can be divided into two stages: the development of a new SRL model for Croatian and the implementation of this model in an AQG system. The results show high accuracy in argument classification and positive expert evaluation for educational purposes.

TEHNICKI VJESNIK-TECHNICAL GAZETTE (2021)

Article Education, Scientific Disciplines

Automated Software Testing Based on Semantic Distance

Tonci Dadic, Vlado Glavinic, Marko Rosic

INTERNATIONAL JOURNAL OF ENGINEERING EDUCATION (2020)

Article Computer Science, Interdisciplinary Applications

Supporting students' basic psychological needs and satisfaction in a blended learning environment through learning analytics

Elise Ameloot, Tijs Rotsaert, Thomas Ameloot, Bart Rienties, Tammy Schellens

Summary: This study investigates the impact of using learning analytics to support students' autonomy and competence needs in a blended learning environment. The findings show that teachers' adaptation based on learning analytics positively influences students' satisfaction with the adapted learning environment. However, students' basic psychological needs vary depending on the face-to-face workshop subject. The study emphasizes the importance of thoughtful blended learning course design and provides recommendations for effective learning analytics utilization in university settings.

COMPUTERS & EDUCATION (2024)

Article Computer Science, Interdisciplinary Applications

Enhancing elementary school students' computational thinking and programming learning with graphic organizers

Tzu-Chi Yang, Zhi-Shen Lin

Summary: Computational thinking is essential in the current era and learning programming is the most effective way to develop it. Introducing computational thinking and programming at an early age is recommended. Graphic organizers serve as a bridge between students' existing knowledge and new learning, enhancing their learning process. The study found that using graphic organizers improved elementary school students' computational thinking, programming skills, and learning experiences.

COMPUTERS & EDUCATION (2024)

Article Computer Science, Interdisciplinary Applications

Collaborative analytics-enhanced reflective assessment to foster conducive epistemic emotions in knowledge building

Yuqin Yang, Kaicheng Yuan, Gaoxia Zhu, Lizhen Jiao

Summary: The study finds that the design of collaborative analytics-enhanced reflective assessment can promote conducive epistemic emotions to knowledge building among undergraduates, and enriches our understanding of the relationships between metacognition, epistemic emotions, and knowledge building practices.

COMPUTERS & EDUCATION (2024)

Article Computer Science, Interdisciplinary Applications

A novel quantitative assessment of engagement in virtual reality: Task-unrelated thought is reduced compared to 2D videos.

Vishal Kiran Kuvar, Jeremy N. Bailenson, Caitlin Mills

Summary: Recent research suggests that students' minds often wander off-task during learning, regardless of the learning modality. This study explores the potential of virtual reality (VR) to reduce task-unrelated thoughts (TUT) and finds that learning with VR leads to lower TUT and better performance.

COMPUTERS & EDUCATION (2024)

Article Computer Science, Interdisciplinary Applications

Adopt or abandon: Facilitators and barriers of in-service teachers' integration of game learning analytics in K-12 classrooms?

Yiming Liu, Jeremy Tzi Dong Ng, Xiao Hu, Zhengyang Ma, Xiaoyan Lai

Summary: This study examines teachers' usage intention and behavior towards the Game Learning Analytics (GLA) system in K-12 classrooms. The study found that personal, environmental, and technological factors influenced teachers' intention and behavior, and that technostress moderated the intention-behavior relationship. The study also identified the heterogeneity of GLA usage among teachers with different individual characteristics.

COMPUTERS & EDUCATION (2024)

Article Computer Science, Interdisciplinary Applications

Supporting the monitoring of the digital capacity of schools through optimal shortening of the SELFIE tool

Romina Cachia, Artur Pokropek, Nikoleta Giannoutsou

Summary: This article introduces a shortened version of the European Commission's SELFIE tool for measuring the digital capacity of schools. Two shorter measurement instruments, called midi-SELFIE and mini-SELFIE, are proposed based on the original tool. The validity and uses of these shortened versions are explored through various cases and compared to the complete instrument. The results suggest that the shortened versions of SELFIE are reliable alternatives for specific purposes.

COMPUTERS & EDUCATION (2024)