$\mathbf{H}_{n}\in\mathbb{R}^{M_{n}\times N}$ is a sparsifying (to be clarified soon) sensing matrix, $\{\mathbf{y}_{n}(t)\}$ is a $\mathbb{R}^{M_{n}}$-valued observation sequence for the $n$-th agent and for each $n$ where possibly $M_{n} \ll  N$, $\left\{\mathbf{\gamma}_{n}(t)\right\}$ is a zero-mean temporally independent and identically distributed (i.i.d.)  noise sequence with nonsingular covariance matrix $\mathbf{R}_{n}$. It is to be noted that the assumption that the dimension of the parameter $\boldsymbol{\theta}^{\ast}$ is equal to the number of agents, $N$, is simply made for clarity of presentation. In particular, all our proofs and assertions will continue to hold with appropriate modifications if the dimension of the global parameter is different from $N$.

[A1]{} \[as:1\] *There exists $\epsilon_{1}>0$, such that, for all $n$, $\mathbb{E}_{\btheta}\left[\left\|\gamma_{n}(t)\right\|^{2+\epsilon_{1}}\right]<\infty$.*

The above assumption encompasses a broad class of noise distributions in the setup. The heterogeneity of the setup is exhibited in terms of the sensing matrix and the noise covariances at the agents. We now formalize an assumption on global model observability.\

[A2]{} \[as:2\] *The matrix $\mathbf{G}=\sum_{n=1}^{N}\mathbf{H}_{n}^{\top}\mathbf{R}_{n}^{-1}\mathbf{H}_{n}$ is full rank*.

Assumption \[as:2\] is crucial for our distributed setup. It is to be noted that such an assumption is needed for even a setup with a centralized node which has access to all the data samples at each 