Article
Computer Science, Artificial Intelligence
Xiaoshu Zhou, Fei Zhu, Peiyao Zhao
Summary: The method of prediction based on uncertainty exploration (SPE) improves the quality of exploration and reduces noise interference in deep reinforcement learning, leading to significant improvements in simulated environments.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Computer Science, Information Systems
Kaibiao Yang, Wenhan Dong, Ming Cai, Shengde Jia, Ri Liu
Summary: This paper proposes a method for unmanned combat air vehicle air combat maneuver decision based on the proximal policy optimization algorithm. The method is validated through a simulation experiment, demonstrating its effectiveness.
Article
Computer Science, Information Systems
Takato Okudo, Seiji Yamada
Summary: Human knowledge can reduce the number of iterations required in reinforcement learning, and the subgoal-based reward shaping method shows promise in certain domains. By learning the potential function through parameterization of a hyperparameter, we are able to accelerate value learning and obtain more effective results compared to baseline algorithms.
Article
Energy & Fuels
Zhao Liu, Jiateng Li, Pei Zhang, Zhenhuan Ding, Yanshun Zhao
Summary: This article proposes a novel framework based on the PPO reinforcement learning algorithm for AGC dynamic optimization, which aims to handle fluctuations and uncertainties in power systems and improve the frequency characteristic to meet control performance standards.
FRONTIERS IN ENERGY RESEARCH
(2022)
Article
Automation & Control Systems
Her-Terng Yau, Ping-Huan Kuo, Po-Chien Luan, Yung-Ruen Tseng
Summary: This article presents a DRL-based control method for nonlinear chaotic systems without prior knowledge of the system's equations. Experimental results demonstrate that the PPO algorithm is the most efficient and effective for controlling chaotic systems.
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL
(2023)
Article
Computer Science, Artificial Intelligence
Le Zhang, Yanshuo Zhang, Xin Zhao, Zexiao Zou
Summary: Image captioning involves generating captions for images using natural language. By applying the PPO algorithm to a state-of-the-art architecture like X-Transformer, improvements in system performance can be achieved. Experimental results suggest that combining PPO with dropout regularization may decrease performance, possibly due to the KL-divergence of RL policies. Using word-level baseline estimation instead of sentence-level baseline in the policy gradient estimator can lead to better results.
IMAGE AND VISION COMPUTING
(2021)
Article
Computer Science, Artificial Intelligence
Zhuang Shao, Fengqi Si, Huaijiang Wu, Xiaozhong Tong
Summary: The research defines a novel dynamic economic emission dispatcher problem and learning framework, transferring the optimization tasks offline and using multi-objective proximal policy optimization to significantly improve the speed and performance of the neural network dispatcher, showcasing its generalization capabilities.
APPLIED SOFT COMPUTING
(2021)
Article
Computer Science, Information Systems
Junwei Zhang, Zhenghao Zhang, Shuai Han, Shuai Lue
Summary: This paper discusses the exploration issue in the PPO algorithm and proposes an exploration enhancement mechanism based on uncertainty estimation. By applying the exploration enhancement theory to the PPO algorithm, the IEM-PPO algorithm is proposed, and it is evaluated in experiments using the MuJoCo physical simulator. The results show that the IEM-PPO algorithm outperforms PPO in terms of sample efficiency and cumulative reward.
INFORMATION SCIENCES
(2022)
Article
Computer Science, Artificial Intelligence
Meng Xu, Yechao She, Yang Jin, Jianping Wang
Summary: We propose a new method for policy fusion in deep reinforcement learning, which dynamically selects sub-tasks and reduces fusion bias. Experimental results show significant improvements in task duration, episode reward, and score difference.
ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY
(2023)
Article
Computer Science, Artificial Intelligence
Luckeciano C. Melo, Dicksiano C. Melo, Marcos R. O. A. Maximo
Summary: This study introduces a methodology based on deep reinforcement learning to improve running skills in a humanoid robot, achieving remarkable results with Proximal Policy Optimization. The approach outperforms existing technologies by approximately 50% in terms of sprint speed in the RoboCup 3D Soccer Simulation competition. Evaluation of training procedures and controllers in terms of speed, reliability, and human similarity were conducted, with a focus on encouraging symmetry in movements for top speed running policies. Key factors leading to surpassing previous results and suggestions for future research are discussed.
JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Xin Li, Haojie Lei, Li Zhang, Mingzhong Wang
Summary: This paper explores interpretable Deep Reinforcement Learning (DRL) by representing policy using Differentiable Inductive Logic Programming (DILP). The research focuses on the optimization perspective of DILP-based policy learning and proposes using Mirror Descent for policy optimization. The theoretical and empirical studies verify the effectiveness of the proposed approach.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Qingchun Zheng, Zhi Peng, Peihao Zhu, Yangyang Zhao, Ran Zhai, Wenpeng Ma
Summary: This paper proposes an object recognition grasping approach using Proximal Policy Optimization (PPO) with You Only Look Once v5 (YOLOv5) to overcome the problems of traditional grasping methods for mobile manipulators. The approach combines a vision recognition algorithm with a deep reinforcement learning algorithm to achieve object recognition grasping. Experimental results show that the proposed method outperforms the original YOLOv4 model in terms of object recognition speed and achieves higher detection precision and lower hardware requirements. The proposed method also outperforms the SAC and TRPO algorithms in object grasping, with the average reward of the PPO algorithm improved significantly compared to the other algorithms.
Article
Management
Bram J. De Moor, Joren Gijsbrechts, Robert N. Boute
Summary: This study demonstrates the feasibility of applying transfer learning to deep reinforcement learning for improving performance and training stability in inventory management. Additionally, potential-based reward shaping is implemented to manage inventory control efficiently.
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH
(2022)
Article
Computer Science, Information Systems
Rousslan Fernand Julien Dossa, Shengyi Huang, Santiago Ontanon, Takashi Matsubara
Summary: This paper investigates the impact of the optimization technique called "early stopping" on the performance of the PPO algorithm. The results show that PPO's performance is sensitive to the number of update iterations per epoch, and early stopping optimizations can dynamically adjust the update iterations, serving as a convenient alternative to tuning on K.
Article
Robotics
Zhuang Wang, Hui Li, Zhaoxin Wu, Haolin Wu
Summary: A pretrained PPO algorithm is proposed to solve the guidance problem of manned aircraft and unmanned aerial vehicles, with continuous action reward function and position reward function to increase training speed and trajectory performance.
INTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS
(2021)
Article
Engineering, Marine
Mogens Blanke, Dong T. Nguyen
Article
Engineering, Environmental
Dat H. Nguyen, Dong T. Nguyen, Ser T. Quek, Asgeir J. Sorensen
COLD REGIONS SCIENCE AND TECHNOLOGY
(2011)
Article
Automation & Control Systems
Dong T. Nguyen, Asgeir J. Sorensen
CONTROL ENGINEERING PRACTICE
(2009)
Article
Automation & Control Systems
Dat H. Nguyen, Dong T. Nguyen, Ser T. Quek, Asgeir J. Sorensen
CONTROL ENGINEERING PRACTICE
(2010)
Article
Engineering, Civil
Dong Trong Nguyen, Asgeir J. Sorensen
IEEE JOURNAL OF OCEANIC ENGINEERING
(2009)
Article
Automation & Control Systems
Trong Dong Nguyen, Asgeir J. Sorensen, Ser Tong Quek
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY
(2008)
Article
Engineering, Ocean
Dong T. Nguyen, Ser Tong Quek
JOURNAL OF OFFSHORE MECHANICS AND ARCTIC ENGINEERING-TRANSACTIONS OF THE ASME
(2010)
Article
Engineering, Marine
Dong Trong Nguyen, Marius Trodahl, Tom Arne Pedersen, Azzeddine Bakdi
Summary: This paper proposes a fuzzy logic-based method for evaluating the compliance of Collision-Avoidance Systems (CAS) in Autonomous Surface Vehicles (ASV). The evaluation systems were verified on simulated scenarios and found to provide variables that would be challenging or impossible to obtain by visual assessment.
Proceedings Paper
Automation & Control Systems
Chanjei Vasanthan, Dong T. Nguyen
Summary: This paper presents the development of an autonomous path planner based on supervised learning, addressing concerns about uncertainties introduced by deep learning models. Through thorough research and parameter tuning, the authors identified the most suitable model and utilized large-scale training data to enhance performance.
Article
Computer Science, Information Systems
Zhengru Ren, Bo Zhao, Dong Trong Nguyen
Proceedings Paper
Engineering, Ocean
Luca Pivano, Dong Nguyen, Olyvind Smogeli
PROCEEDINGS OF THE ASME 36TH INTERNATIONAL CONFERENCE ON OCEAN, OFFSHORE AND ARCTIC ENGINEERING, 2017, VOL 9
(2017)
Article
Automation & Control Systems
Trong Dong Nguyen, Asgeir J. Sorensen, Ser Tong Quek
Article
Engineering, Marine
Alba Ricondo, Laura Cagigal, Beatriz Perez-Diaz, Fernando J. Mendez
Summary: This research presents a site-specific metamodel based on the SWASH numerical model simulations, which can predict coastal hydrodynamic variables in a fast and efficient manner. The metamodel uses downscaled and dimensionality reduced synthetic database to accurately reproduce wave setup, wave heights associated with different frequency bands, and wave runup. This method has great potential in coastal risk assessments, early warning systems, and climate change projections.
Article
Engineering, Marine
Xiao Yu, Wangjun Ren, Bukui Zhou, Li Chen, Xiangyun Xu, Genmao Ren
Summary: This study investigated and compared the compression responses and energy absorption capacities of coral sand and silica sand at a strain rate of approximately 1000 s-1. The results showed that coral sand had significantly higher energy absorption capacity than silica sand due to its higher compressibility. The study findings suggest that using poorly graded coral sand can improve its energy absorption capacity.
Article
Engineering, Marine
Jingxi Zhang, Junmin Mou, Linying Chen, Pengfei Chen, Mengxia Li
Summary: This paper proposes a cooperative control scheme for ship formation tracking based on Model Predictive Control. A predictive observer is designed to estimate the current motion states of the leader ship using delayed motion information. Comparative simulations demonstrate the effectiveness and robustness of the proposed controller.
Article
Engineering, Marine
Yu Yao, Danni Zhong, Qijia Shi, Ji Wu, Jiangxia Li
Summary: This study proposes a 2DH numerical model based on Boussinesq equations to investigate the impact of dredging reef-flat sand on wave characteristics and wave-driven current. The model is verified through wave flume experiments and wave basin experiments, and the influences of incident wave conditions and pit morphological features on wave characteristics are examined.
Article
Engineering, Marine
Jayanta Shounda, Krishnendu Barman, Koustuv Debnath
Summary: This study investigates the double-average turbulence characteristics of combined wave-current flow over a rough bed with different spacing arrangements. The results show that a spacing ratio of p/r=4 offers the highest resistance to the flow, and the double-average Reynolds stress decreases throughout the flow depth. The advection of momentum-flux of normal stress shows an increase at the outer layer and a decrease near the bed region after wave imposition. Maximum turbulence kinetic energy production and diffusion occur at different layers. The turbulence structure is strongly anisotropic at the bottom region and near the outer layer, with a decrease in anisotropy observed with an increase in roughness spacing.
Article
Engineering, Marine
Meng Zhang, Lianghui Sun, Yaoguo Xie
Summary: The research proposes a method for online identification of wave bending and torsional moment in hull structures. For structures without large openings, the method optimizes sensor positions and establishes a mathematical model to improve accuracy. For structures with large openings, a joint dual-section monitoring method is proposed to simultaneously identify bending and torsional moments in multiple key cross sections.
Article
Engineering, Marine
Longming Chen, Shutao Li, Yeqing Chen, Dong Guo, Wanli Wei, Qiushi Yan
Summary: This study investigated the dynamic response characteristics and damage modes of pile wharves subjected to underwater explosions. The results showed that the main damaged components of the pile wharf were the piles, and inclined piles had a higher probability of moderate or more significant damage compared to vertical piles. The study also suggested that replacing inclined piles with alternative optimized structures benefits the blast resistance of pile wharves.
Article
Engineering, Marine
I. -C Kim, G. Ducrozet, V. Leroy, F. Bonnefoy, Y. Perignon, S. Bourguignon
Summary: Previous research focused on the accuracy and efficiency of short-term wave fields in specific prediction zones, while we developed algorithms for continuous wave prediction based on the practical prediction zone and discussed important time factors and strategies to reduce computational costs.
Article
Engineering, Marine
Hang Xie, Xianglin Dai, Fang Liu, Xinyu Liu
Summary: This study investigates the load characteristics of a three-dimensional stern model with pitch angle through a drop test, and reveals complex characteristics of pressure distribution near the stern shaft. The study also shows that the vibration characteristics of the load are influenced by the drop height and pitch angle, with the drop height having a greater effect on the high-frequency components.
Article
Engineering, Marine
Hangyuan Zhang, Wanli Yang, Dewen Liu, Xiaokun Geng, Wangyu Dai, Yuzhi Zhang
Summary: The deep-water bridge is more vulnerable to earthquake damage than the bridge standing in air. The larger blocking ratio has a significant impact on the added mass coefficient, which requires further comprehensive study. The generation mechanism of block effect is analyzed using numerical simulation software ANSYS Fluent. The results show that the recirculation zone with focus reduces the pressure on the back surface of the cylinder, resulting in the peak value of in-line force not occurring synchronously with the peak value of acceleration. The change in position and intensity of the recirculation zone with focus, as well as the change in water flow around the cylinder surface, are identified as the generation mechanism of the block effect, which has a 10% influence on the hydrodynamic force. The changing rule of the added mass coefficient with blocking ratio is discussed in detail, and a modification approach to the current added mass coefficient calculation method is suggested. Physical experiments are conducted to validate the modification approach, and the results show that it is accurate and can be used in further study and real practice.
Article
Engineering, Marine
Golnesa Karimi-Zindashti, Ozgur Kurc
Summary: This study examines the performance of an in-house code utilizing a deterministic vortex method on the rotation of circular and square cylinders. The results show that rotational motion reduces drag forces, suppresses fluctuating forces, and increases lift forces. The code accurately predicts vortex shedding suppression and identifies the emergence of near-field wakes in the flow over rotating square cylinders.
Article
Engineering, Marine
George Dafermos, George Zaraphonitis
Summary: The survivability of damaged ships is of great importance and the regulatory framework is constantly updated. The introduction of the probabilistic damage stability framework has rationalized the assessment procedure. Flooding simulation tools can be used to investigate the dynamic response of damaged ships.
Article
Engineering, Marine
Xuyue Chen, Xu Du, Chengkai Weng, Jin Yang, Deli Gao, Dongyu Su, Gan Wang
Summary: This paper proposes a real-time drilling parameters optimization method for offshore large-scale cluster extended reach drilling based on intelligent optimization algorithm and machine learning. By establishing a ROP model with long short-term memory neurons, and combining genetic algorithm, differential evolution algorithm, and particle swarm algorithm, the method achieves real-time optimization of drilling parameters and significantly improves the ROP.
Article
Engineering, Marine
Sung-Jae Kim, Chungkuk Jin, MooHyun Kim
Summary: This study investigates the dynamic behavior of a moored submerged floating tunnel (SFT) under tsunami-like waves through numerical simulations and sensitivity tests. The results show that design parameters significantly affect the dynamics of the SFT system and mooring tensions, with shorter-duration and higher-elevation tsunamis having a greater impact.
Article
Engineering, Marine
G. Clarindo, C. Guedes Soares
Summary: Environmental contours are constructed using the Inverse-First Order Reliability Method based on return periods. The paper proposes the use of the Burr distribution to model the marginal distribution of long-term significant wave heights. The newly implemented scheme results in different environmental contours compared to the reference approach.
