Article
Computer Science, Artificial Intelligence
Kensuke Nakamura, Stefano Soatto, Byung-Woo Hong
Summary: BCSC is a stochastic first-order optimization algorithm that adds a cyclic constraint to the selection of data and parameters, resulting in higher accuracy in image classification. It effectively limits the impact of outliers in the training set and provides better generalization performance within the same number of update iterations.
Article
Automation & Control Systems
Jing Dong, Xin T. Tong
Summary: The paper introduces a new algorithm that combines gradient descent and Langevin dynamics to achieve linear convergence to the global minimum in online settings, improving performance.
JOURNAL OF MACHINE LEARNING RESEARCH
(2021)
Article
Engineering, Industrial
Andre Gustavo Carlon, Henrique Machado Kroetz, Andre Jacomel Torii, Rafael Holdorf Lopez, Leandro Fleck Fadel Miguel
Summary: This paper proposes a stochastic gradient based method for Risk Optimization problems, approximating failure probabilities using the Chernoff bound and solving the problem with a Stochastic Gradient Descent algorithm. The approach efficiently avoids direct computation of failure probabilities and their gradients.
RELIABILITY ENGINEERING & SYSTEM SAFETY
(2022)
Article
Computer Science, Information Systems
Youngjoon Kim, Jiwon Yoon
Summary: In this paper, a new type of gradient-based fuzzer called MaxAFL is proposed to overcome the limitations of existing fuzzers. MaxAFL utilizes an objective function constructed through static analysis and a gradient-based optimization algorithm. It outperforms other fuzzers in terms of code coverage and bug discovery, showcasing its effectiveness in improving fuzzing techniques.
Article
Multidisciplinary Sciences
Austin Williams, Noah Walton, Austin Maryanski, Sandra Bogetic, Wes Hines, Vladimir Sobes
Summary: This analysis investigates the suitability of ADAM as a tool for optimizing k-eigenvalue nuclear systems, using challenge problems to verify its effectiveness. Despite the stochastic nature and uncertainty of k-eigenvalue problems, ADAM is able to successfully optimize nuclear systems. Furthermore, the results clearly demonstrate that low-compute time, high-variance estimates of the gradient lead to better performance in the optimization challenge problems tested here.
SCIENTIFIC REPORTS
(2023)
Article
Computer Science, Artificial Intelligence
Li Shen, Congliang Chen, Fangyu Zou, Zequn Jie, Ju Sun, Wei Liu
Summary: Integrating adaptive learning rate and momentum techniques into stochastic gradient descent has led to various efficient adaptive stochastic algorithms such as AdaGrad, RMSProp, Adam, and AccAdaGrad. This paper proposes a weighted AdaGrad algorithm called AdaUSM that incorporates a unified momentum scheme and a novel weighted adaptive learning rate. It shows that AdaUSM achieves $\mathcal{O}(\log(T)/\sqrt{T})$ convergence rate in the nonconvex stochastic setting with polynomially growing weights. Furthermore, it provides a new perspective for understanding Adam and RMSProp by showing that their adaptive learning rates correspond to exponentially growing weights in AdaUSM. Comparative experiments on deep learning models and datasets are also conducted.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Shi Pu, Alex Olshevsky, Ioannis Ch Paschalidis
Summary: This article focuses on minimizing the average of cost functions over a network, where agents can communicate and exchange information. The article studies the distributed stochastic gradient descent (DSGD) method when only noisy gradient information is available, and performs a nonasymptotic convergence analysis. The main contribution of the article is to characterize the transient time needed for DSGD to approach the asymptotic convergence rate, and construct a hard optimization problem to prove the sharpness of the obtained result. Numerical experiments demonstrate the tightness of the theoretical results.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Engineering, Electrical & Electronic
Srujan Teja Thomdapu, Harsh Vardhan, Ketan Rajawat
Summary: This work studies constrained stochastic optimization problems in the context of fair classification, fair regression, and queuing system design. The goal is to solve these problems in a large-scale setting with a minimal number of calls to the oracle. A quasi-gradient saddle point algorithm is proposed to construct approximate gradients and find the optimal and feasible solution almost surely. The proposed algorithm requires O(1/epsilon(4)) data samples to obtain an epsilon-approximate optimal point with zero constraint violation, outperforming state-of-the-art algorithms in terms of convergence rate in fair classification and fair regression problems.
IEEE TRANSACTIONS ON SIGNAL PROCESSING
(2023)
Article
Mathematics, Applied
Prateek Jain, Dheeraj M. Nagaraj, Praneeth Netrapalli
Summary: Stochastic gradient descent (SGD) is widely used for large-scale optimization, but the conventional step size sequences often lead to suboptimal convergence rates. This work presents new step size sequences that achieve information theoretically optimal bounds on the suboptimality, improving the final iterate significantly compared to standard sequences.
SIAM JOURNAL ON OPTIMIZATION
(2021)
Article
Computer Science, Artificial Intelligence
Yunwen Lei, Ke Tang
Summary: This paper develops novel learning rates of SGD for nonconvex learning by presenting high-probability bounds for both computational and statistical errors. It shows that the complexity of SGD iterates grows in a controllable manner with respect to the iteration number, shedding insights on implicit regularization. By also connecting the study to Rademacher chaos complexities, it slightly refines existing studies on the uniform convergence of gradients.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Mathematics, Applied
Jinda Yang, Haiming Song, Xinxin Li, Di Hou
Summary: This paper develops a block mirror stochastic gradient method for solving stochastic optimization problems involving both convex and nonconvex cases, treating the feasible set and variables as multiple blocks. The proposed method combines the features of classic mirror descent stochastic method and block coordinate gradient descent method. By acquiring stochastic gradient information through stochastic oracles, the method updates all variable blocks in a Gauss-Seidel type. Convergence is established for both convex and nonconvex cases, although the analysis of the method is challenging due to the failure of the typical unbiasedness assumption of stochastic gradients in the Gauss-Seidel renewal type, requiring more specific assumptions. The efficiency of the proposed algorithm is demonstrated through testing on the conditional value-at-risk problem and the stochastic LASSO problem.
JOURNAL OF SCIENTIFIC COMPUTING
(2023)
Article
Engineering, Electrical & Electronic
Yaxi Liu, Wei Huangfu, Huan Zhou, Haijun Zhang, Jiangchuan Liu, Keping Long
Summary: This paper investigates the optimization problem of UAV base station placement and proposes an accurate and efficient algorithm to solve the problem. The algorithm ensures fair coverage and energy efficiency while satisfying backhaul constraints. Experimental results demonstrate that the algorithm performs well in different scenarios.
IEEE TRANSACTIONS ON COMMUNICATIONS
(2022)
Article
Automation & Control Systems
Brian Swenson, Ryan Murray, H. Vincent Poor, Soummya Kar
Summary: This paper investigates the convergence and avoidance of saddle points of distributed stochastic gradient descent algorithms in nonconvex and nonsmooth scenarios.
JOURNAL OF MACHINE LEARNING RESEARCH
(2022)
Article
Computer Science, Artificial Intelligence
Anuraganand Sharma
Summary: The proposed guided SGD algorithm compensates for the deviation caused by delay and encourages consistent examples to steer the convergence of SGD, reducing the impact of delay on neural network models.
APPLIED SOFT COMPUTING
(2021)
Article
Operations Research & Management Science
Anastasiya Ivanova, Pavel Dvurechensky, Evgeniya Vorontsova, Dmitry Pasechnyuk, Alexander Gasnikov, Darina Dvinskikh, Alexander Tyurin
Summary: This paper presents a generic algorithmic framework for convex optimization problems, which separates oracle complexities for different functions and oracle types.
JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS
(2022)
Article
Computer Science, Information Systems
Chao Ren, Luchuan Liu, Haijun Zhang
Summary: In a low-carbon passive UAV network, the received signals can be used for energy supply, communication, and sensing. However, the different modalities may cause interference in the received signals, limiting the performance and flight flexibility of the UAVs. A location-aware approach is proposed to achieve multimodal interference compatibility and increase the feasible flight region of the UAVs. Additionally, a preinstalled localization method with fewer sources is proposed to ensure the implementation of the multimodal interference-compatible approaches.
IEEE WIRELESS COMMUNICATIONS LETTERS
(2023)
Article
Materials Science, Multidisciplinary
Yaxi Liu, Bin Xu, Wei Huangfu, Haiqing Yin
Summary: In this paper, a nickel-based polycrystalline superalloy composition design framework is established to efficiently and comprehensively explore novel alloys with superior properties. A model of creep resistance is established using Gaussian process regression, requiring limited data knowledge and computational effort. The constrained multi-objective nickel-based superalloy composition design problem is solved using meta-heuristic non-dominated sorting genetic algorithm II, effectively providing Pareto fronts that consider the trade-off between creep resistance and alloy cost. Well-performed alloys with high creep resistance and low alloy cost are obtained through filtering unqualified candidate alloys using screening indicators computed by JMatPro software.
COMPUTATIONAL MATERIALS SCIENCE
(2023)
Article
Engineering, Electrical & Electronic
Wenbo Du, Tao Tan, Haijun Zhang, Xianbin Cao, Gang Yan, Osvaldo Simeone
Summary: A set of low-cost sensors is used to infer the network topology of a self-organizing wireless network by extracting timing information from data packets and acknowledgment (ACK) packets. A new EM-based algorithm, called EM-CDA, is introduced to handle the impact of packet losses on causality metrics. Extensive experiments on the NS-3 simulation platform validate the effectiveness of the method.
IEEE TRANSACTIONS ON COMMUNICATIONS
(2023)
Article
Engineering, Electrical & Electronic
Wenyu Zhang, Sherali Zeadally, Huan Zhou, Haijun Zhang, Ning Wang, Victor C. M. Leung
Summary: Edge computing is a widely-used approach for providing low-latency computation services. This study proposes a novel Logistic function-based model for estimating service reliability probability in edge computing scenarios with stochastic resource demands. An alternative optimization algorithm is proposed to solve the average service reliability maximization problem by jointly optimizing service quality ratios and resource allocations. Simulation results show that the proposed method achieves similar performance as a convex optimization algorithm, but with lower complexity, and improves service reliability compared to a baseline weighted allocation method.
IEEE TRANSACTIONS ON COMMUNICATIONS
(2023)
Article
Engineering, Civil
Yingchao Yang, Zhiquan Bai, Ke Pang, Shuaishuai Guo, Haijun Zhang, Kyung Sup Kwak
Summary: This paper proposes a spatial-index modulation (SIM) based orthogonal time frequency space (OTFS) system, named SIM-OTFS, to enhance the performance of high mobility vehicular networks. The SIM-OTFS system utilizes a three dimensional index modulation (IM) technique to achieve higher transmission rate. The design, signal processing, and theoretical analysis of the SIM-OTFS system are presented, demonstrating its superiority over other systems in terms of average bit error rate (ABER) and diversity. Numerical results also show the influence of resolvable multipaths on the ABER performance of the SIM-OTFS system.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Engineering, Electrical & Electronic
Hui Ma, Haijun Zhang, Wenyu Zhang, Victor C. M. Leung
Summary: This article investigates the use of reconfigurable intelligent surface (RIS) as a promising technology for 6G networks. The study focuses on a power splitting aided broadcasting network where one access point (AP) transmits identical messages to multiple users. By controlling reflect amplitude coefficients, the RIS is able to assist the AP while achieving power self-sustainability through energy harvesting. An algorithm based on block coordinate descent, convex approximation, and alternating direction method of multipliers techniques is proposed to optimize the AP transmit beamforming vector and the RIS reflect beamforming matrix. Simulation results demonstrate the effectiveness of the algorithm.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2023)
Article
Engineering, Electrical & Electronic
Wenbo Du, Tao Wang, Haijun Zhang, Yunfan Dong, Yumeng Li
Summary: This research focuses on a UAV-aided communication scenario where a UAV with constrained onboard battery capacity communicates with multiple ground users during its flight. The trajectory and resource allocation of the UAV are investigated to minimize the time consumption for a specific task. To tackle the non-convex problem with numerous variables, a collection of line segments is used to discretize the real trajectory. The reformulated problem is then solved through the block coordinate descent algorithm by decoupling it into resource allocation and trajectory optimization subproblems, which are efficiently resolved using the successive convex approximation method. Simulation results demonstrate significant improvement in system performance compared to benchmark schemes.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2023)
Article
Engineering, Electrical & Electronic
Hui Ma, Haijun Zhang, Yongxu Zhu, Yi Qian
Summary: In this paper, we study an IRS assisted multiple-input single-output downlink broadcasting system and propose two transmission designs, a semidefinite relaxation (SDR) based design and a low complexity design, to maximize the transmission throughput. The simulation results show that the SDR based design can achieve near-optimal performance and the low complexity design performs close to the SDR based design with much lower complexity.
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS
(2023)
Article
Engineering, Electrical & Electronic
Hua Shao, Haijun Zhang, Huan Zhou, Jianquan Wang, Ning Wang, Arumgam Nallanathan
Summary: This paper investigates the performance issues of message-passing (MP) detectors based on factor graphs in the orthogonal time frequency space (OTFS) system. It is found that there may exist short girths (i.e. girth-4) in the Tanner graphs, which degrade the performance of MP detectors, especially with high modulation orders. By introducing interleavers, the vectorized channel matrix becomes a sparse upper block Heisenberg matrix, which benefits the computation of matrix QR decomposition (QRD).
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS
(2023)
Article
Engineering, Electrical & Electronic
Wei Huang, Yidi Shao, Kai Sun, Haijun Zhang, Victor C. M. Leung
Summary: This paper investigates user-centric clustering in cloud radio access networks, specifically the outage probability of the typical user considering void cell and composite fading. The locations of remote radio heads and users are modeled as the Poisson point process (PPP) and Matern hard-core point process of type II (MHCPP), respectively. Due to the complexity of MHCPP, a PPP-based approximation is adopted. The closed expression of Laplace transform of the probability density function of the interfering power under composite fading channels is derived using Gauss-Hermite quadrature. Based on the approximated PPP, the outage probability of the typical user in the presence of void cell with user-centric clustering is obtained. Simulation results show the importance of considering the effect of void cell on system performance, especially when the density of nodes or the size of the cluster is limited.
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS
(2023)
Article
Computer Science, Information Systems
Wenyu Zhang, Sherali Zeadally, Wei Li, Haijun Zhang, Jingyi Hou, Victor C. M. Leung
Summary: The breakthrough of AI techniques has accelerated their applications in various industries, including security protection, transportation, agriculture, and medical care. With the support of edge computing environments, providing AIaaS with latency guarantee can speed up the deployment of data-intensive and computation-intensive AI applications and reduce customers' investment cost. However, existing studies have not addressed the specific deployment architecture, working mechanism design, and performance optimization problems for AIaaS with configurable data quality and model complexity. To tackle this, we propose a configurable model deployment architecture (CMDA) for edge AIaaS and a flexible working mechanism that allows joint configuration of data quality ratios (DQRs) and model complexity ratios (MCRs) for AI tasks.
IEEE TRANSACTIONS ON CLOUD COMPUTING
(2023)
Article
Engineering, Multidisciplinary
Yanhang Shi, Siguang Chen, Haijun Zhang
Summary: This paper proposes a novel personalized semi-supervised learning paradigm that allows partially labeled or unlabeled clients to seek labeling assistance from data-related clients (helper agents) to enhance their perception of local data. By designing an uncertainty-based data-relation metric, the selected helpers can provide trustworthy pseudo labels and avoid misleading the local training. Additionally, a helper selection protocol is developed to mitigate network overload and achieve efficient communication.
IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING
(2023)
Article
Engineering, Multidisciplinary
Yupei Liu, Haijun Zhang, Huan Zhou, Keping Long, Victor C. M. Leung
Summary: This paper focuses on the resource allocation problem in the space-air-ground integrated vehicular networks (SAGVN). It proposes a user association and subchannel/power allocation scheme to optimize the connection and communication performance of small cells. Edge computing is also applied to offload local tasks to improve communication performance.
IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING
(2023)
Article
Engineering, Electrical & Electronic
Xiaonan Li, Haijun Zhang, Huan Zhou, Ning Wang, Keping Long, Saba Al-Rubaye, George K. Karagiannidis
Summary: This paper proposes a framework for resource allocation in the terrestrial-satellite network based on non-orthogonal multiple access (NOMA). A deployment method of local cache pools is also given to achieve lower time delay and maximize energy efficiency. The proposed method, which utilizes multi-agent deep deterministic policy gradient (MADDPG), shows better performance compared to traditional single-agent deep reinforcement learning algorithm in optimizing resource allocation and cache design in the integrated terrestrial-satellite network.
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS
(2023)
Article
Computer Science, Information Systems
Yun Wang, Shu Fu, Changhua Yao, Haijun Zhang, Fei Richard Yu
Summary: In this study, the caching placement problem of UAVs for enhancing service timeliness is investigated. A modified timeliness model called effective age of information (EAoI) is proposed to evaluate service timeliness comprehensively. Proximal policy optimization (PPO) algorithm is employed to build a deep reinforcement learning framework for adaptively finding the optimal caching strategy. Extensive simulation results demonstrate the superiority of the proposed scheme compared to conventional schemes.
IEEE WIRELESS COMMUNICATIONS LETTERS
(2023)