Title: HyperFeel: An Efficient Federated Learning Framework Using Hyperdimensional Computing. Abstract: Federated Learning (FL) aims to establish a shared model across decentralized clients under the privacy-preserving constraint. Each client learns an independent model with local data, and only model updates are communicated. However, as the FL model typically employs computation-intensive neural networks, major issues in Federated Learning are (i) significant computation overhead for local training; (ii) the massive communication overhead that arises from sending around the model updates; (iii) notable performance degradation resulting from the non-IID scenario.

In this work, we propose HyperFeel, an efficient learning framework for federated learning based on Hyper-Dimensional Computing (HDC), that can significantly improve communication/storage efficiency over existing works with nearly no performance degradation. Unlike current solutions that employ neural networks as the learned model, HyperFeel introduces a simple yet effective computing paradigm that encodes and represents data using hyperdimensional vectors. Then, it performs concise and highly parallel operations for encryption, computation, and communication, taking advantage of the lightweight feature representation of hyperdimensional vectors. For further enhance HyperFeel performance, we propose a two-fold optimization scheme combining the characteristics of encoding and updating in hyper-dimensional computing. On the one hand, we design the personalization update based on hyperdimensional computing with a client-specific model, which achieves better accuracy to the non-IID data. On the other hand, we extend the framework from horizontal FL to vertical FL based on a shared encoding mechanism.
Comprehensive experimental results demonstrate our method consistently outperforms the state-of-the-art FL models. Typically, we achieves $26\times$ storage reduction and up to $81\times$ communication reduction over FedAvg, with minimal accuracy drops on FEMNIST and Synthetic. \emph{Code will be open-source in the camera-ready version.}