Article
Automation & Control Systems
Frantisek Blahoudek, Petr Novotny, Melkior Ornik, Pranay Thangeda, Ufuk Topcu
Summary: This paper discusses qualitative strategy synthesis for consumption Markov decision processes, a formalism for modeling agent dynamics under resource constraints in a stochastic environment. The algorithms presented in this paper work efficiently and synthesize strategies that ensure goal states are reached without resource exhaustion. The paper also presents heuristics to reduce the agent's expected time for mission fulfillment. The implemented algorithms and numerical examples demonstrate the effectiveness of the planning approach and the impact of the heuristics in a realistic scenario.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2023)
Article
Computer Science, Artificial Intelligence
George K. Atia, Andre Beckus, Ismail Alkhouri, Alvaro Velasquez
Summary: There is increased interest in the formal synthesis of decision-making policies in the planning domain, but challenges exist in deriving policies that satisfy constraints on an agent's steady-state behavior. The proposed linear programming solution for multichain Markov Decision Processes demonstrates rigorous guarantees of behavior in stationary policies.
JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH
(2021)
Article
Automation & Control Systems
Qinbo Bai, Vaneet Aggarwal, Ather Gattami
Summary: This paper investigates the optimization problem in peak Constrained Markov Decision Process (PCMDP) and proposes a model-free algorithm with Q-learning. The algorithm is proven to achieve maximum total reward while satisfying constraints, and performs well on energy harvesting and single machine scheduling problems.
JOURNAL OF MACHINE LEARNING RESEARCH
(2023)
Article
Automation & Control Systems
Melkior Ornik, Ufuk Topcu
Summary: This paper introduces a formal approach to online learning and planning for agents operating in unknown and time-varying environments. The proposed method can quickly and accurately identify changes in system dynamics, and introduces the concepts of exploration bonuses and uncertainty in learning algorithms.
JOURNAL OF MACHINE LEARNING RESEARCH
(2021)
Article
Automation & Control Systems
Mustafa O. Karabag, Melkior Ornik, Ufuk Topcu
Summary: The use of deceptive strategies is important for agents in adversarial environments. This study focuses on the synthesis of optimal deceptive policies and reference policies in a Markov decision process. The synthesis of optimal reference policies is proven to be NP-hard.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Computer Science, Theory & Methods
Richard Mayr, Eric Munday
Summary: This paper studies countably infinite Markov decision processes with real-valued transition rewards and examines the strategy complexity for different types of payoffs.
LOGICAL METHODS IN COMPUTER SCIENCE
(2022)
Article
Automation & Control Systems
Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani
Summary: The study introduces a method for constructing abstractions of Markov decision processes using approximate probabilistic relations to quantify the distance between interconnected gMDPs and their abstractions. By explicitly incorporating dependency between state transitions and allowing abstract models to have varying state spaces, the new approximate relation unifies compositionality results in the literature.
NONLINEAR ANALYSIS-HYBRID SYSTEMS
(2021)
Article
Engineering, Multidisciplinary
Maria Julia Blas, Silvio Gonnet
Summary: This paper introduces a method to improve the interaction between abstraction and concreteness in modeling and simulation using Model-Driven Engineering. By translating abstract definitions into formal simulation models, formalization and implementation times can be reduced. Additionally, a graphical software tool has been developed to support the development of simulation models.
ENGINEERING SCIENCE AND TECHNOLOGY-AN INTERNATIONAL JOURNAL-JESTECH
(2021)
Article
Engineering, Industrial
Lei Lei, Jian-Qiang Hu, Chenbo Zhu
Summary: This article presents a new method based on copulas to model correlated inputs in discrete-event stochastic systems. By defining Copula Correlated Processes (CCPs), the method enables the modeling of correlated inputs for such systems. It shows that systems with CCPs can be discretized and approximated by systems with discrete copula correlated processes, simplifying the analysis process. An illustrative queueing example is used to demonstrate the effectiveness of the method.
Review
Construction & Building Technology
Mohammad Fawaier, Balazs Bokor
Summary: Dynamic insulation is a method that allows the heat transmission rate through building envelopes to vary over time. This paper collects and assesses literature on different dynamic insulation structures and compares mathematical models, experimental studies, and numerical simulations. The results show that dynamic insulation can achieve low heat loss and significant energy savings.
ENERGY AND BUILDINGS
(2022)
Article
Management
Kyle E. Paret, Maria E. Mayorga, Emmett J. Lodree
Summary: This paper proposes a multi-server queuing model for assigning spontaneous volunteers to tasks in post-disaster areas. An optimal policy is generated using a Markov Decision Process, and simulations are used to compare it with heuristic policies.
OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE
(2021)
Article
Automation & Control Systems
Amber Srivastava, Srinivasa M. Salapaka
Summary: The framework presented addresses sequential decision-making problems with noisy data by quantifying exploration using Shannon entropy of trajectories and determining the optimal stochastic policy. This approach improves exploration quality early in the learning process, leading to faster convergence rates and robust solutions.
IEEE TRANSACTIONS ON CYBERNETICS
(2022)
Article
Management
Jiajun Dai, Na Geng, Xiaolan Xie
Summary: This paper introduces a dynamic approach to advance scheduling for outpatient medical appointments, using a finite-horizon Markov decision process model to maximize total expected net reward. Structural properties of the optimal value function and control policy are established, and two efficient heuristic policies are devised and validated through numerical experiments.
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH
(2021)
Article
Automation & Control Systems
Margaret P. Chapman, Riccardo Bonalli, Kevin M. Smith, Insoon Yang, Marco Pavone, Claire J. Tomlin
Summary: This article develops a safety analysis method that is sensitive to the possibility and severity of rare harmful outcomes. The method assesses the maximum cost of stochastic systems using Conditional Value-at-Risk and provides computationally tractable underapproximations to risk-sensitive safe sets. The article also proposes a second definition for risk-sensitive safe sets and provides a tractable method for their estimation.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Computer Science, Artificial Intelligence
Daniel A. Melo Moreira, Karina Valdivia Delgado, Leliane Nunes de Barros, Denis Deratani Maua
Summary: This study focuses on finding optimal policies for Markov Decision Processes by optimizing a risk-sensitive non-linear cumulative cost function. Two algorithms were developed to improve efficiency and solve large-scale problems without sacrificing optimality.
INTERNATIONAL JOURNAL OF APPROXIMATE REASONING
(2021)
Article
Computer Science, Artificial Intelligence
Gongming Wang, Junfei Qiao, Jing Bi, Qing-Shan Jia, MengChu Zhou
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2020)
Article
Computer Science, Artificial Intelligence
Gongming Wang, Qing-Shan Jia, MengChu Zhou, Jing Bi, Junfei Qiao
Summary: This paper proposes a Deep Belief Network with Event-triggered Learning (DBN-EL) to improve the efficiency and accuracy of soft-sensing model in WWTP. Through defining events, designing event-triggered learning strategy, and conducting convergence analysis, the effectiveness of this method in practical WWTP applications is demonstrated.
Article
Automation & Control Systems
Zhaoyu Jiang, Qing-Shan Jia, Xiaohong Guan
Summary: The idea of utilizing wind power to charge electric vehicles has gained attention for its potential in reducing air pollution. However, challenges arise due to uncertainties in wind power generation and EV charging demands. This study focuses on addressing the issues through simulation-based policy improvement, with a specific focus on computing budget allocation for decision-making in online applications. Research findings demonstrate the importance of addressing uncertainty in wind power forecasting for EV charging decisions while comparing different allocation methods through numerical experiments.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
(2021)
Review
Computer Science, Artificial Intelligence
Gongming Wang, Qing-Shan Jia, MengChu Zhou, Jing Bi, Junfei Qiao, Abdullah Abusorrah
Summary: This paper presents a comprehensive survey on water quality soft-sensing in wastewater treatment processes using artificial neural networks (ANNs). It covers problem formulation, common models, practical examples, and performance discussions. Various soft-sensing models are compared in terms of accuracy, efficiency, and complexity, with factors affecting the accuracy discussed as well. Challenges in soft-sensing models of WWTP are also pointed out for future exploration.
ARTIFICIAL INTELLIGENCE REVIEW
(2022)
Article
Computer Science, Information Systems
Zhaoyu Jiang, Qing-Shan Jia, Xiaohong Guan
Summary: This paper investigates the charging scheduling problem for electric vehicles (EVs), considering the uncertainty in renewable power generation and the randomness in EV charging demand. By exploring the structural property of the problem, an urgency index is developed to rank the EVs, and three methods are applied to search in the action space. Numerical demonstrations show that simulation-based policy improvement (SBPI) improves the performance of base policies in various cases compared to the CPLEX-based method.
SCIENCE CHINA-INFORMATION SCIENCES
(2022)
Article
Multidisciplinary Sciences
Hong Kang, Tong Lin, Xiaojin Xu, Qing-Shan Jia, Richard Lakerveld, Bryan Wei
Summary: The study introduces a dynamic switch scheme for DNA nanostructures, utilizing toehold-free strand displacement. Through simulations and experiments, the unique properties of toehold-free strand displacement in equilibrium control are demonstrated, along with showcasing the potential applications of controllable dynamics.
NATURE COMMUNICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Gongming Wang, Qing-Shan Jia, Junfei Qiao, Jing Bi, MengChu Zhou
Summary: This work introduces a deep learning-based model predictive control (DeepMPC) for modeling and controlling a continuous stirred-tank reactor (CSTR) system. DeepMPC achieves high performance in system identification and control through automatic determination of size and quadratic optimization.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
Article
Engineering, Electrical & Electronic
Teng Long, Qing-Shan Jia, Gongming Wang, Yu Yang
Summary: This paper presents an efficient and scalable real-time scheduling method for handling the charging demands of plug-in electric vehicles (PEV), demonstrating through simulations that the proposed method provides high computation efficiency and scalability while reducing operating costs for charging stations. Compared to existing methods, it outperforms in terms of charging policy search capabilities and performance guarantee.
IEEE TRANSACTIONS ON SMART GRID
(2021)
Article
Automation & Control Systems
Jiangliang Jin, Liangliang Hao, Yunjian Xu, Junjie Wu, Qing-Shan Jia
Summary: The study focused on the joint scheduling of deferrable demands and storage systems, proposing an optimal index-based priority rule and recommending the use of reinforcement learning methods for energy procurement decisions, achieving a significant improvement in system cost reduction compared to existing RL methods.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2021)
Article
Automation & Control Systems
Qing-Shan Jia, Junjie Wu
Summary: This work focuses on optimizing the charging scheduling policy for shared electric vehicles integrated with wind power generation. The research extends the least-laxity-longer-processing-time-first principle and demonstrates improved performance over existing algorithms. The new algorithm shows near-optimal results and significant speed improvement compared to CPLEX.
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY
(2021)
Article
Automation & Control Systems
Jie Chen, Yuqian Guo, Zhifeng Qiu, Bin Xin, Qing-Shan Jia, Weihua Gui
Summary: This article investigates multiagent dynamic task assignment for forest fires based on a point model, providing optimal solutions for static task assignment and proposing a dynamic task assignment scheme based on global information. The simulation on MATLAB platform verifies the performance of the proposed scheme when compared with a multistage global auction algorithm.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
(2022)
Article
Automation & Control Systems
Teng Long, Qing-Shan Jia
Summary: The article introduces a novel architecture consisting of hydrogen production stations, fast-charging stations, and commercial electric vehicles to optimize hydrogen energy dispatch and EV charging location selection. Case studies confirm the effectiveness of the architecture in reducing operating costs and improving performance by at least 13%.
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY
(2022)
Proceedings Paper
Automation & Control Systems
Junjie Wu, Kuo Li, Qing-Shan Jia
2020 59TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC)
(2020)
Proceedings Paper
Automation & Control Systems
Xinze Jin, Kuo Li, Qing-Shan Jia, Huaxia Xia, Yu Bai, Dongchun Ren
2020 CHINESE AUTOMATION CONGRESS (CAC 2020)
(2020)
Proceedings Paper
Automation & Control Systems
Jing-Xian Tang, Jin-Hong Du, Yiting Lin, Qing-Shan Jia