We study the vector Gaussian CEO problem under logarithmic loss distortion measure. Specifically, @math agents observe independent noisy versions of a remote vector Gaussian source, and communicate independently with a decoder over rate-constrained noise-free links. The CEO also has its own Gaussian noisy observation of the source and wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. We find an explicit characterization of the rate-distortion region of this model. For the proof of this result, we first extend Courtade-Weissman's result on the rate-distortion region of the DM @math -encoder CEO problem to the case in which the CEO has access to a correlated side information stream which is such that the agents' observations are independent conditionally given the side information and remote source. Next, we obtain an outer bound on the region of the vector Gaussian CEO problem by evaluating the outer bound of the DM model by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, application of our results allows us to find the complete solutions of three related problems: the vector Gaussian distributed hypothesis testing against conditional independence problem, a quadratic vector Gaussian CEO problem with determinant constraint, and the vector Gaussian distributed Information Bottleneck problem.
In this paper, we study the problem of two-encoder multiterminal source coding with side information of Figure in the case of logarithmic loss distortion measure. That is, our model generalizes that of @cite_20 to the setting in which the decoder observes side information sequence that is statistically dependent on the sources to be compressed. We develop a single-letter characterization of the rate-distortion region of this model in the discrete memoryless case. In doing so, we show that a slight generalization of the Gastpar inner bound [Theorem 2] G04 for general distortion measures that accounts for time-sharing is optimal. The proof of the converse follows that of [Theorem 12] CW14 , and extends it to the case of correlated side information at decoder. It also involves a redefinition of the required auxiliary random variables appropriately. Furthermore, specializing the results to the case in which only Encoder 2 communicates with the decoder, i.e., @math , and the decoder is interested in reproducing 'soft' estimates of only the source @math , we characterize the trade-off among the complexity and accuracy of the system, i.e., a generalization of the so-called Information Bottleneck Method @cite_50 @cite_23 to the setting with side information at decoder.
@cite_50 @cite_23 @cite_20
1686946872 2070945723 2173267959
We define the relevant information in a signal @math as being the information that this signal provides about another signal @math . Examples include the information that face images provide about the names of the people portrayed, or the information that speech sounds provide about the words spoken. Understanding the signal @math requires more than just predicting @math , it also requires specifying which features of @math play a role in the prediction. We formalize this problem as that of finding a short code for @math that preserves the maximum information about @math . That is, we squeeze the information that @math provides about @math through a bottleneck' formed by a limited set of codewords @math . This constrained optimization problem can be seen as a generalization of rate distortion theory in which the distortion measure @math emerges from the joint statistics of @math and @math . This approach yields an exact set of self consistent equations for the coding rules @math and @math . Solutions to these equations can be found by a convergent re-estimation method that generalizes the Blahut-Arimoto algorithm. Our variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere. It is well-known that the information bottleneck method and rate distortion theory are related. Here it is described how the information bottleneck can be considered as rate distortion theory for a family of probability measures where information divergence is used as distortion measure. It is shown that the information bottleneck method has some properties that are not shared with rate distortion theory based on any other divergence measure. In this sense the information bottleneck method is unique. We consider the classical two-encoder multiterminal source coding problem where distortion is measured under logarithmic loss. We provide a single-letter description of the achievable rate distortion region for all discrete memoryless sources with finite alphabets. By doing so, we also give the rate distortion region for the m-encoder CEO problem (also under logarithmic loss). Several applications and examples are given.