Article
Engineering, Electrical & Electronic
Huiyuan Yang, Tian Ding, Xiaojun Yuan
Summary: Recently, federated learning (FL) has emerged as an efficient and privacy-friendly machine learning (ML) paradigm. However, the communication cost for model aggregation remains a challenge in FL. This paper proposes a general framework for analyzing the performance of model aggregation based on the rate-distortion theory and establishes a connection between aggregation distortion and FL convergence performance. It also formulates an aggregation distortion minimization problem and develops algorithms for solving it.
IEEE TRANSACTIONS ON COMMUNICATIONS
(2023)
Article
Computer Science, Information Systems
Alexander Hamilton, Mohammed El-Hajjar, Robert G. Maunder
Summary: This paper proposes a novel Reordered Exponential Golomb Error Correction (RExpGEC) coding scheme, which is a flexible and practical JSCC technique for near-capacity performance. The flexibility of RExpGEC is demonstrated through the design of a novel trellis encoder and decoder. Simulation results show that the RExpGEC-URC-QPSK scheme outperforms other comparable JSCC and SSCC benchmarkers in providing reliable and efficient communications.
Article
Computer Science, Information Systems
Arun Padakandla
Summary: In this work, a Shannon-theoretic study of communicating correlated sources over multiple access and interference channels is presented. The fixed block-length coding technique is enhanced by incorporating the technique of inducing source correlation onto channel inputs. New single-letter characterizations for sufficient conditions are derived for both scenarios, and simple 'plug-in' approaches are proposed to further weaken the derived conditions. Subsuming Dueck's findings and going beyond in the example considered therein are made possible by the findings.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Recep Can Yavas, Victoria Kostina, Michelle Effros
Summary: In the random access communication scenario, a Random Access Channel (RAC) is introduced where neither the transmitters nor the receiver knows which transmitters are active. Feedback of a finite number of bits is used to synchronize the transmitters, and the decoder determines the number of active transmitters and their messages without knowing which transmitter sent which message. The decoding procedure achieves a rate that varies with the number of active transmitters, and a single threshold rule is used to achieve performance that is first-order optimal for each coding epoch.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Cheuk Ting Li, Venkat Anantharam
Summary: This paper introduces the Poisson matching lemma and uses it to prove one-shot achievability results for various settings, showing that the Poisson functional representation is a viable alternative in network information theory.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Neri Merhav
Summary: We prove the existence of codebooks that can be universally applied to both finite-alphabet memoryless sources and bounded additive rational distortion measures for d-semifaithful lossy compression. By utilizing independent random selection of codewords based on a mixture of all memoryless sources, we achieve redundancy rates that are within O(log n/n) close to the empirical rate-distortion function for every given source vector and bounded rational distortion measure.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2023)
Article
Computer Science, Information Systems
Vidyasagar Sadhu, Zhile Li, Zhuoran Qi, Dario Pompili
Summary: Reliable and persistent water monitoring in smart underwater IoT is challenging due to the harsh and unpredictable nature. Traditional digital sensors are unsuitable for high-resolution sensing in such environments. To address these challenges, we propose a novel architecture of dense underwater all-analog biodegradable sensors for persistent sensing and data transmission.
IEEE INTERNET OF THINGS JOURNAL
(2023)
Article
Chemistry, Analytical
Huan Deng, Dan Song, Zhiping Xu, Yanglong Sun, Lin Wang
Summary: This paper proposes a belief propagation algorithm based on the duality principle for lossy compression in the Internet of Things. By optimizing the algorithm, the impact of trapping sets on compression performance is weakened.
Article
Computer Science, Information Systems
Mengshu Song, Nan Ma, Chen Dong, Xiaodong Xu, Ping Zhang
Summary: The implementation of joint source-channel coding schemes using deep learning has advanced the development of semantic communication research. A novel semantic adaptive model called joint source-channel coding with adaptive models (AMJSCC) is proposed, which includes a semantic adaptive model selection module. The model adapts to real-time channel conditions and computational resources, and utilizes residual networks to improve information recovery accuracy.
Article
Computer Science, Information Systems
Ling Liu, Jinwen Shi, Cong Ling
Summary: The proposed polar lattices achieve the rate-distortion bound of a memoryless Gaussian source by integrating entropy coding into the lattice quantizer. The complexity of encoding and decoding is O(N log(2) N) for any target distortion and fixed rate larger than the rate-distortion bound. Additionally, the nesting structure of polar lattices provides solutions to various multi-terminal coding problems.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Shota Saito, Toshiyasu Matsushima
Summary: This paper investigates variable length source coding with criteria of normalized cumulant generating function of codeword lengths and excess distortion probability. It analyzes the non-asymptotic fundamental limit of normalized cumulant generating function of codeword lengths under the constraint that excess distortion probability is allowed up to e ? [0, 1). The non-asymptotic achievability and converse bounds are characterized by the quantity related to Renyi entropy.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2023)
Article
Computer Science, Hardware & Architecture
Tetsunao Matsuta, Tomohiko Uyematsu
Summary: This article examines the coding problem for lossy source coding with side information at the decoder, known as the Wyner-Ziv source coding problem. By utilizing the chromatic number and notions of covering of a set, an equivalent expression of the minimum rate is provided, allowing for analysis of the coding problem in relation to graph coloring and covering.
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES
(2022)
Article
Computer Science, Information Systems
Hao-Chung Cheng, Nilanjana Datta, Cambyse Rouze
Summary: This paper introduces a novel method for finding strong converse bounds in quantum network information theory, relying on recent results in the field of non-commutative functional inequalities. The technique is applied to tasks such as quantum source coding and distributed quantum hypothesis testing, as well as establishing strong converse bounds in broadcast communication scenarios. The method has potential applications in other important tasks of quantum network information theory.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Engineering, Electrical & Electronic
Li Deng, Xiaoxi Yu, Yixin Wang, Md. Noor-A-Rahim, Yong Liang Guan, Zhiping Shi, Zhongpei Zhang
Summary: This paper introduces a joint source channel anytime coding scheme and proposes an improved partial joint expanding window decoding algorithm. Through a comprehensive theoretical analysis, it is found that this scheme has the potential for high reliability and low latency communications, and can maximize its advantages in certain application scenarios.
IEEE TRANSACTIONS ON COMMUNICATIONS
(2023)
Article
Physics, Multidisciplinary
Naruki Shinohara, Hideki Yagi
Summary: This study discusses the important issue of protecting data privacy in the utilization of databases such as IoT. Building upon previous work, it introduces a measure of privacy for the encoder and examines the first-order rate analysis problem among coding rate, utility, decoder privacy, and encoder privacy. It also aims to establish the strong converse theorem for utility-privacy trade-offs.
Article
Computer Science, Information Systems
Victoria Kostina, Yuval Peres, Gireeja Ranade, Mark Sellke
Summary: This study examines the stability of a linear control system with unbounded random system gain, where the controller must act based on a rate-limited observation of the state. A time-varying achievable strategy is provided to stabilize the system in a second-moment sense with fixed, finite R. The strategy involves different actions based on the state value and operates in normal and emergency modes.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Recep Can Yavas, Victoria Kostina, Michelle Effros
Summary: In the random access communication scenario, a Random Access Channel (RAC) is introduced where neither the transmitters nor the receiver knows which transmitters are active. Feedback of a finite number of bits is used to synchronize the transmitters, and the decoder determines the number of active transmitters and their messages without knowing which transmitter sent which message. The decoding procedure achieves a rate that varies with the number of active transmitters, and a single threshold rule is used to achieve performance that is first-order optimal for each coding epoch.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Peida Tian, Victoria Kostina
Summary: This paper presents a precise error analysis for parameter estimation in a nonstationary Gauss-Markov process, showing a tight nonasymptotic exponentially decaying bound on the tail probability of the estimation error. The new estimation bound is applicable to lossy compression of nonstationary Gauss-Markov sources. New ideas include separately bounding eigenvalues and introducing new techniques in deriving the estimation error bound.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Parham Noorzad, Michelle Effros, Michael Langberg, Victoria Kostina
Summary: The research establishes a relationship between zero-error and classical information theory, introducing new requirements for codebook rate and expanding the classical birthday problem to the information theory domain.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Recep Can Yavas, Victoria Kostina, Michelle Effros
Summary: This paper presents finite-blocklength achievability bounds for the Gaussian multiple access channel (MAC) and random access channel (RAC) under average-error and maximal-power constraints, extending the improvement of performance beyond known results. The proposed rateless coding strategy and decoding method achieve the same performance as the best known result for the Gaussian MAC in operation, with improvements in terms of first-, second-, and third-order performance.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Nian Guo, Victoria Kostina
Summary: The study explores optimal encoding and decoding policies under a rate constraint to minimize estimation error. The optimal encoding policy transmits a 1-bit codeword once the process innovation passes one of two thresholds, and the optimal decoder recovers the last sample from the 1-bit codewords and time stamps to estimate the process in real time.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Computer Science, Information Systems
Victoria Kostina, Babak Hassibi
Summary: The article discusses the transmission of information from an n-dimensional source with memory between K isolated encoders and decoders, studying communication across different rounds and the loss of communication among communicators. By extending inner and outer bounds, it demonstrates limits for minimum distortion estimates and the minimum achievable total rate in Gaussian scenarios.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2021)
Article
Automation & Control Systems
Nian Guo, Victoria Kostina
Summary: This research investigates a communication scenario where an encoder observes and transmits information about a Wiener process in a causal manner, and a decoder estimates the process in real time. The study determines the optimal causal encoding and decoding policies that minimize estimation error while considering the communication rate constraint. Moreover, it shows that the optimal encoding policy involves sampling the Wiener process and compressing the sign of innovation using 1-bit codewords.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Automation & Control Systems
Victoria Kostina, Yuval Peres, Gireeja Ranade, Mark Sellke
Summary: This article investigates an unstable linear stochastic system and provides a new proof that the control input M must be greater than the system gain a for stability, given specific stability order and constraints on the moments of the random variables.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2022)
Article
Computer Science, Artificial Intelligence
Farshad Lahouti, Victoria Kostina, Babak Hassibi
Summary: This study focuses on querying an expert oracle for labeling a dataset in machine learning, proposing an efficient k-ary query scheme to improve query efficiency. Through analysis and simulation studies, the effectiveness of the proposed algorithms in handling datasets is demonstrated.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
Proceedings Paper
Computer Science, Information Systems
Recep Can Yavas, Victoria Kostina, Michelle Effros
Summary: This paper presents variable-length feedback codes for different communication scenarios, such as point-to-point, multiple access, and random access communication. The nested code structure is designed with flexibility, optimizing decoding time selection to minimize expected decoding time while meeting error probability constraints. Effective decoding and transmission control for different numbers of active transmitters are achieved through nested coding and single-bit feedback.
2021 IEEE INFORMATION THEORY WORKSHOP (ITW)
(2021)
Proceedings Paper
Computer Science, Theory & Methods
Recep Can Yavas, Victoria Kostina, Michelle Effros
Summary: Variable-length feedback (VLF) codes for the Gaussian point-to-point channel are investigated under maximal power, average error probability, and average decoding time constraints. A proposed strategy chooses decoding times and considers stop-feedback, with an achievability bound for VLF codes optimizing decoding times within the code architecture.
2021 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT)
(2021)
Proceedings Paper
Computer Science, Theory & Methods
Nian Guo, Victoria Kostina
Summary: This paper proposes a novel coding scheme for transmitting message bits in real time over a discrete-memoryless channel with noiseless feedback, using channel feedback to improve performance. The code applies the SED rule to the evolving message alphabet for better delay-reliability tradeoffs.
2021 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT)
(2021)
Proceedings Paper
Computer Science, Theory & Methods
Chung-Yi Lin, Victoria Kostina, Babak Hassibi
Summary: The study introduces a technique called differential quantization that enables quantized gradient descent algorithms to converge at the same rate as unquantized algorithms, with theoretical convergence guarantees. Experimental results validate the theoretical analysis.
2021 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT)
(2021)
Proceedings Paper
Computer Science, Theory & Methods
Oron Sabag, Victoria Kostina, Babak Hassibi
Summary: In this paper, the computable expression for the feedback capacity of channels with non-white Gaussian noise is solved in the scenario of MIMO channels and a state-space noise model. The solution involves a finite-dimensional convex optimization problem and unveils the relation between the capacity formulae and Riccati equations. The problem is viewed as maximizing the measurements' entropy rate subject to a power constraint, leading to a tight lower bound by optimizing over a family of time-invariant policies.
2021 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT)
(2021)
