TT-TFHE: a Torus Fully Homomorphic Encryption-Friendly Neural Network
  Architecture
This paper presents TT-TFHE, a deep neural network Fully Homomorphic
Encryption (FHE) framework that effectively scales Torus FHE (TFHE) usage to
tabular and image datasets using a recent family of convolutional neural
networks called Truth-Table Neural Networks (TTnet). The proposed framework
provides an easy-to-implement, automated TTnet-based design toolbox with an
underlying (python-based) open-source Concrete implementation (CPU-based and
implementing lookup tables) for inference over encrypted data. Experimental
evaluation shows that TT-TFHE greatly outperforms in terms of time and accuracy
all Homomorphic Encryption (HE) set-ups on three tabular datasets, all other
features being equal. On image datasets such as MNIST and CIFAR-10, we show
that TT-TFHE consistently and largely outperforms other TFHE set-ups and is
competitive against other HE variants such as BFV or CKKS (while maintaining
the same level of 128-bit encryption security guarantees). In addition, our
solutions present a very low memory footprint (down to dozens of MBs for
MNIST), which is in sharp contrast with other HE set-ups that typically require
tens to hundreds of GBs of memory per user (in addition to their communication
overheads). This is the first work presenting a fully practical solution of
private inference (i.e. a few seconds for inference time and a few dozens MBs
of memory) on both tabular datasets and MNIST, that can easily scale to
multiple threads and users on server side.
Adrien Benamira, Tristan Guérand, Thomas Peyrin, Sayandeep Saha
# TT-TFE: a Torus Fully Homomorphic Encryption-Friendly Neural Network Architecture

###### Abstract

This paper presents TT-TFHE, a deep neural network Fully Homomorphic Encryption (FHE) framework that effectively scales Torus FHE (TFHE) usage to tabular and image datasets using a recent family of convolutional neural networks called Truth-Table Neural Networks (TTnet). The proposed framework provides an easy-to-implement, automated TTnet-based design toolbox with an underlying (python-based) open-source Concrete implementation (CPU-based and implementing lookup tables) for inference over encrypted data. Experimental evaluation shows that TT-TFHE greatly outperforms in terms of time and accuracy all Homomorphic Encryption (HE) set-ups on three tabular datasets, all other features being equal. On image datasets such as MNIST and CIFAR-10, we show that TT-TFHE consistently and largely outperforms other TFHE set-ups and is competitive against other HE variants such as BFV or CKKS (while maintaining the same level of 128-bit encryption security guarantees). In addition, our solutions present a very low memory footprint (down to dozens of MBs for MNIST), which is in sharp contrast with other HE set-ups that typically require tens to hundreds of GBs of memory per user (in addition to their communication overheads). This is the first work presenting a fully practical solution of private inference (i.e. a few seconds for inference time and a few dozens MBs of memory) on both tabular datasets and MNIST, that can easily scale to multiple threads and users on server side.

Machine Learning, ICML

## 1 Introduction

Deep Neural Networks (DNNs) have achieved remarkable results in various fields, including image recognition, natural language processing or medical diagnostics. "Machine Learning as a Service" (MLaaS) is a recent and popular DNN-based business use-case (Philipp et al., 2020; Li et al., 2017; Ribeiro et al., 2015), where clients pay for predictions from a service provider. However, this approach requires trust between the client and the service provider. In cases where the data is sensitive, such as military, financial, or health information, clients may be hesitant (or are simply not allowed) to share their data with the service provider for privacy reasons. On the service provider's side, training DNNs requires large amounts of data, technical expertise, and computer resources, which can be expensive and time-consuming. As a result, service providers may hesitate to give the model directly to the client, as it may be easily reverse-engineered (or at least make the attacker's task much easier), hindering the growth of MLaaS activity. Allowing the clients to perform the inference locally is also not very practical as any model update would have to be pushed to all clients, not to mention the complex support of the various client hardware/software configurations, etc.

HE/FHE (Gentry, 2009) is an ideal technology to address Privacy-Preserving in Machine Learning (PPML) as it allows the computations to be performed directly on encrypted data. By encrypting its data before sharing it with the service provider, the client ensures that it remains private while the service provider can still provide accurate predictions. This solves the trust issue and also gives a competitive advantage in regions where data regulations are stricter, such as Europe's General Data Protection Regulation (GDPR) (Regulation, 2016). The most popular HE schemes are BGV/BFV (Brakerski et al., 2014; Brakerski, 2012; Fan and Vercauteren, 2012), CKKS (Cheon et al., 2017) and Torus-FHE (TFHE) (Chillotti et al., 2016, 2020) and this paper presents our proposed solution that utilizes TFHE to provide a privacy-preserving MLaaS framework for DNNs. TFHE enables very fast gate
bootstrapping as well as circuit bootstrapping and operations over Boolean gates. Moreover, extended versions of TFHE, such as Concrete (Chillotti et al., 2020) allows programmable bootstrapping enabling evaluation of certain functions during the bootstrapping step itself.

The security and flexibility provided by HE come at a cost, as the computation, communication, and memory overheads are significant, especially for complex functions such as DNNs:

* Time overhead: on server side, an FHE logic gate computation within a TFHE scheme takes milliseconds (Chillotti et al., 2016), compared to nanoseconds for a standard logic gate. On client side the overhead for encryption/decryption of the data is usually not an issue, as milliseconds are enough for those operations.
* Communication overhead: an MNIST image sent in clear represents a few kBs, while encrypted within any HE scheme it is of the order of a few MBs (excluding the public key which typically requires hundred(s) of MBs) (Clet et al., 2021). Moreover, PPML schemes not based on TFHE, such as those based on CKKS, require a significant communication overhead due to the need for multiple exchanges between the client and the cloud (Clet et al., 2021).
* Memory overhead: TFHE-based schemes typically require a few of MBs of RAM on server side, while those based on CKKS need several GBs per image (Gilad-Bachrach et al., 2016; Brutzkus et al., 2019), even up to hundreds of GBs for the most time-efficient solutions (Lee et al., 2022) (384GB of RAM usage to infer a single CIFAR-10 image using ResNet).
* Technical difficulties and associated portability issues: mastering an HE/FHE framework requires highly technical and rare expertise.

These challenges can be approached from different directions, but only a few works have considered designing DNNs models that are compatible/efficient with state-of-the-art HE frameworks in a flexible and portable manner. This paper focuses on the integration of the recent FHE scheme, Torus-FHE (TFHE), with DNNs.

Our contributions.To address the above issue, we propose a DNN design framework called TT-TFHE that effectively scales TFHE usage to tabular and large datasets using a new family of Convolutional Neural Networks (CNNs) called Truth-Table Neural Networks (TTnet). Our proposed framework provides an easy-to-implement, automated TTnet-based design toolbox that utilizes the Pytorch and Concrete (python-based) open-source libraries for state-of-the-art deployment of DNN models on CPU, leading to fast and accurate inference over encrypted data. The TTnet architecture, being lightweight and differentiable, allows for the implementation of CNNs with direct expressions as truth tables, making it easy to use in conjunction with the TFHE open-source library (specifically the Concrete implementation) for automated operation on lookup tables. Therefore, in this paper, we try to tackle the TFHE efficiency overhead as well as the technical/portability issue.

Our experimental results.Evaluation on three tabular datasets (Cancer, Diabetes, Adult) shows that our proposed TT-TFHE framework outperforms in terms of accuracy (by up to +3%) and time (by a factor 7x to 1200x) any state-of-the-art DNN & HE set-up. For all these datasets, our inference time runs in a few seconds, with very small memory and communication requirements, enabling for the first time a fully practical deployment in industrial/real-world scenarios, where tabular datasets are prevalent (Cartella et al., 2021; Buczak & Guven, 2015; Clements et al., 2020; Ulmer et al., 2020; Evans, 2009).

For MNIST and CIFAR-10 image benchmarks, we further explore an approach for private inference proposed by LoLa (Brutzkus et al., 2019), in which the user/client side is able to compute a first layer and send the encrypted results to the cloud. In this real-world scenario, the user/client performs the computation of a standard public layer (such as the first block of the open-source VGG16 model) and sends the encrypted results to the cloud for further computation using HE. Through experimental evaluation, we demonstrate that our proposed framework greatly outperforms all previous TFHE set-ups (Sanyal et al., 2018; Fu et al., 2021; Chillotti et al., 2021) in terms of inference time. Specifically, we show that TT-TFHE can infer one MNIST image in 7 seconds with an accuracy of 98.1% or one CIFAR-10 image in 570 seconds with an accuracy of 74%, which is from one to several orders of magnitude faster than previous TFHE schemes, and even comparable to the fastest state-of-the-art HE set-ups (that do not benefit from the TFHE advantages) while maintaining the same 128-bit security level.

Our solutions represent a significant step towards practical privacy-preserving inference, as they offer fast inference with limited requirements in terms of memory on server side (only a few MBs, in contrary to other non-TFHE-based schemes), and thus can easily be scaled to multiple users. In addition, they benefit from lower communication overhead. In other words, this is the first work presenting a fully practical solution
of private inference (_i.e._ a few seconds for inference time and a few MBs of memory/communication) on both tabular datasets and MNIST.

Outline.In Section 2, we present related works in the field of HE and HE-friendly neural networks. In Section 3, we introduce our proposed TT-TFHE framework, while in Section 4 we provide an evaluation of the performance of our framework on various datasets and various privacy settings. Finally, in Section 5, we discuss the limitations of the proposed framework and present our conclusions.

## 2 Related Works

A lot of interest has been observed in PPML in the recent past, especially for implementing DNNs. Most of these efforts assume that the (unencrypted) model is deployed in the cloud, and the encrypted inputs are sent from the client side for processing. Inference timing of HE-enabled DNN models is the key parameter, but other factors, such as ease of automation and simplicity of such transformation have also attracted some attention (Boemer et al., 2019; Dathathri et al., 2019; Carpov et al., 2015).

Broadly, the problem can be approached from four complementary directions: 1) Optimizing the implementation of some DNN building blocks, such as the activation layers, using HE operations (Jovanovic et al., 2022; Lee et al., 2022); 2) Parallelizing the computation and batching of images (Gilad-Bachrach et al., 2016; Chou et al., 2018; Brutzkus et al., 2019) (this is aided by the ring encoding of HE in certain cases). Such efforts also include implementing a hybrid client-server protocol for computation (Juvekar et al., 2018; Mishra et al., 2020); 3) Optimizing the underlying HE operations (Chillotti et al., 2016, 2021; Ducas and Micciancio, 2015); 4) Designing a HE-friendly DNN (Sanyal et al., 2018; Lou and Jiang, 2019; Fu et al., 2021). This fourth category has been relatively less explored and is the main focus of this work.

To the best of our knowledge, the FHE-DiNN paper (Bourse et al., 2018) was the first to propose a quantified DNN to facilitate FHE operations. Then, the TAPAS framework (Sanyal et al., 2018) pushed this strategy further by identifying Binary Neural Networks (BNNs) as effective DNN modeling techniques for HE-enabled inference. This direction has been enhanced later by GateNet (Fu et al., 2021), which optimizes the BNN models by grouping the channels to reduce the number of gates.

Yet, none of these works actually explored the automation perspectives for optimizing a model itself for HE inference, as they still heavily leverage some manual optimizations concerning the underlying FHE library. TT-TFHE, however, is fully automated. Moreover, compared to previously proposed automated approaches, the translation from non-HE model to HE-enabled model is much simpler as all optimizations are handled during the design phase of the model, making TT-TFHE much more amenable for typical machine learning experts with little knowledge of FHE.

## 3 Tt-TFHE Framework

### Threat model

PPML methods are designed to protect against a variety of adversaries, including malicious insiders and external attackers who may have access to the neural network's inputs, outputs, or internal parameters. The level of secrecy required depends on the specific application and the potential impact of a successful attack. Common secrecy goals include protecting the input to the inference, ensuring that only authorized parties know the result of the inference, and keeping the weights and biases of the neural network secret from unauthorized parties. Some PPML approaches also aim to keep the architecture of the neural network confidential from unauthorized parties. However, most PPML methods do not address this last point, and some interactive approaches assume that the architecture of the neural network is known to all parties. Moreover, attacks on MLaaS settings exist and are very tricky to defend against (Tramer et al., 2016; Juuti et al., 2019). Thus, in this paper, we assume that an attacker can access the neural network and only the client's data privacy matters.

### Fhe general set-up

In this paper, the client \(\mathcal{C}\) will encrypt its data locally and send it along with its public key to the server \(\mathcal{S}\) (there is no need to send it again once it is pre-shared). The server will compute its algorithm on the encrypted data and send the encrypted result to \(\mathcal{C}\), who will decrypt its result locally. The server will have no access to the data in clear.

The classical configuration is that the entire model is computed privately on server side and we call this configuration Fully Private (Full-Pr). However, as it is anyway quite hard to defend against model weight-stealing attacks (our threat model does not include model privacy) and as already proposed by the LoLa team (Brutzkus et al., 2019), we also consider situations where the client can perform some local pre-processing (_i.e._ a first layer or block of the neural
network) to speed up the server computation without compromising its data security. This setting helps the user to perform its inference faster than in a Full-Pr situation with little memory cost on their side. Such pre-computation would usually be from a public architecture such as VGG, AlexNet or a ResNet(Simonyan and Zisserman, 2014; Krizhevsky et al., 2017; He et al., 2016) which are fully available online. In our case, this layer will then typically be followed by a Tnet model and a linear regression that will be computed through FHE on server side. This is a common approach in deep learning, where most models are fine-tuned from one of these three, or with fixed first layers followed by a shallow network.

We will denote \(N_{1}/N_{2}\) a configuration where the client performs the computation of the neural network \(N_{1}\) locally and the remaining part \(N_{2}\) is performed privately on server side. The final linear regression after \(N_{2}\) is always performed privately on server side, but some of the last few computations can be done by the client (for example a part of the sum and the final \(ArgMax\)(Podschwadt et al., 2022)). The general setup \(N_{1}/\texttt{TT}\) (where the part performed on server is a Tnet) is depicted in Figure 1. \(\emptyset\)/TT will represent one extreme case where the entire Tnet neural network is performed privately (Full-Pr) and \(N/\emptyset\) will represent the other extreme case where the entire neural network \(N\) is performed on client side, except the final linear regression. We will denote VGG\({}_{1L}\) the first layer of VGG16 (the first convolution), and VGG\({}_{1B}\) its first block (the first two convolutions and the pooling layer afterwards).

### Truth-table DCNN (TInet)

Truth Table Deep Convolutional Neural Networks (TInet) were proposed in (Benamira et al., 2022) as DCNNs convertible into truth tables by design, with security applications (a companion video is provided by the authors [https://youtu.be/loGlpVcy0AI](https://youtu.be/loGlpVcy0AI)). While recent developments in DNN architecture have focused on improving performance, the resulting models have become increasingly complex and difficult to verify, interpret and implement. Thus, the authors focused on CNNs, which are widely used in the field, and tried to transform them into tractable Boolean functions. A depiction of the general architecture of TInet can be found in Figure 3 of Appendix A.

**CNN filter as a tractable Boolean function.** Their proposed method converts CNN filters into binary truth tables by 1) decreasing the input size (noted as \(n\) in the rest of the paper) which reduces the complexity of the CNN filter function, 2) using the Heaviside step function denoted as \(bin_{act}=(1+sgn(x))/2\) to transform the inputs and outputs into binary values. This results in a tractable Boolean function (for \(n\) not too large) that can be stored as a truth table, as seen in Figure 2a. To achieve high accuracy, the CNN filter must also be non-linear before the step function. Then, the CNN filter becomes a tractable non-linear truth table, which is referred to as a Learning Truth Table (LTT) block.

**Description of an LTT block.** Among all the families of LTT blocks possible - we represent in Figure 2b an Expanding Auto-Encoder LTT block (E-AE LTT). An E-AE LTT block is composed of two layers of grouped CNN with an expanding factor. Figure 2a

Figure 1: The \(N_{1}/\texttt{TT}\) setting. The client computes locally a layer \(N_{1}\), encrypts the obtained output, and sends it to the server/cloud with the public key (which can be pre-shared). The server will compute through FHE the TInet layer with the linear regression and send the result to the client. The client can decrypt this output and make a few last computations to obtain the result of the inference. When \(N_{1}\) is identity (\(\emptyset\)/TT or more generally \(\emptyset/N_{2}\) for some neural network \(N_{2}\)), we denote the setting as Fully Private (Full-Pr).
shows a computation of E-AE 1D LTT block. We can observe that the input size is small (\(n=6\)), the input/output values are binary and SeLU is an activation function. Note that the inputs are binary but the weights and the intermediate values are real. We integrated LTT blocks into \(\mathcal{T}\mathcal{T}\)net as CNN filters are integrated into DCNNs: each LTT layer is composed of multiple LTT blocks and there are multiple LTT layers in total.

### Challenges and optimizations for integrating TTnet with TFHE-Concrete

The Concrete library (Chillotti et al., 2020) is a software implementation of TFHE. It is designed to provide a highly efficient and secure platform for performing mathematical operations on Boolean encrypted data. The library utilizes automated operations on lookup tables (concrete-numpy) to achieve high performance while maintaining a high level of security. The library is also designed to be very user-friendly, with simple and intuitive interfaces for performing encryption and decryption operations. The Concrete library has been shown to provide significant improvements in terms of memory and communication overheads compared to other HE schemes. Furthermore, it is open source, which allows researchers and practitioners to easily integrate it into their projects and benefit from its advanced features.

The use of TTnet architecture in combination with TFHE (and more specifically with Concrete) naturally provides a number of advantages. Firstly, the lightweight and differentiable nature of TTnet allows for the implementation of CNNs with direct expressions as truth tables, which is well-suited for Concrete. Additionally, the reduced complexity of the TTnet architecture leads to reduced computations and good scalability. Yet, the integration of TTnet into the TFHE framework presents several challenges that need to be addressed in order to achieve high performance. We detail below the constraints imposed by FHE libraries and the optimizations implemented to overcome these limitations and achieve state-of-the-art performance on various datasets.

Constraints imposed by Concrete.The Concrete implementation of the TFHE library, which utilizes automated operations on lookup tables, imposes a maximum limit of 16 on the input bit size \(n\) of the truth table. However, \(n\) has a strong impact on efficiency (cf Appendix B) and our tests show that \(n=4\) or \(n=6\) seem to offer the best trade-offs. Indeed, since we learn kernels of convolutional layers of size \((3,2)\) or \((2,3)\), it is convenient to use \(n\) a multiple of 2 and/or 3. Moreover, input sizes larger than \(n=8\) lead to a prohibitive average time per call.

Concrete proposes a built-in linear regression feature, but we tested that it performs not as well as an ad-hoc linear regression with multiplications and additions.

Finally, a very important limitation is that Concrete does not support multi-precision table lookups: the call time of each table lookup (even the small ones)

Figure 2: A Learning Truth Table (LTT) block. The intermediate values and weights are floating points, input/output values are binary.
will be equal to the call time of the largest bitwidth operations in the whole circuit. For example, if one of the computations operates on 16 bits, all lookup tables, even small, will require the call time of a 16-bit table lookup. This can be problematic when handling the final linear regression as we will have to sum many Boolean values and the sum result bitwidth might be larger than our planned table lookup size, therefore slowing down our entire implementation. If the sum result of our linear regression requires 16 bits, all our 4-bit lookup tables call time will be on par with 16-bit table lookups, going from 75ms to almost 5s per call. For now, the computation time is too slow for results that require a sum on more than 4 bits. To overcome this limitation, we implemented several optimizations to improve the accuracy of the model, particularly for image datasets (see below).

Limitations imposed by TInet.The original TInet paper proposed a training method that is not suitable for high accuracy performance as it was trained to resist PGD attacks (Madry et al., 2017), which reduces accuracy. Additionally, the pre-processing and final sparse layers in TInet being binary, this also leads to a significant decrease in accuracy. To address these limitations, we replace the final sparse binary linear regression with a linear layer with floating point weights (later quantified on 4 bits not to deteriorate too much the performances on Concrete) and propose a new training method that emphasizes accuracy (see below). We also propose to use a setting \(N_{1}/\texttt{TT}\), where a first layer \(N_{1}\) (a layer or block of a general open-source model) is applied to overcome the loss of information due to the first binarization. This is for example a standard method used in BNNs.

Training optimizations.To improve the accuracy of our model, we took several steps to optimize the training process. First, we removed the use of PGD attacks during training, as they have been shown to reduce accuracy. Next, we employed the DoReFa-Net method (Zhou et al., 2016) for CIFAR-10, a technique for training convolutional neural networks with low-bitwidth activations and gradients. Finally, to overcome the limitations of the TInet grouping method, we extended the training to 500 epochs, resulting in a more accurate model.

Architectural optimizations.For tabular datasets, the straightforward enhancements proposed in the previous sections are sufficient to achieve high accuracy. However, this is not the case for image datasets such as MNIST, CIFAR-10, and ImageNet. Therefore, we modify the general architecture of our model. Specifically, we use an architecture similar to the one presented for ImageNet in the original TInet paper. The limitation of our model is that it can see far in the spatial dimension but has limited representation in the channel dimension. To balance this property, we use three techniques: multi-headers, residual connections (He et al., 2016), and channel shuffling (Zhang et al., 2018). We have a single layer composed of four different functions in parallel: one LTT block with a kernel size of (3,2) and group 1 (to see far in space and low in channel), one LTT block with a kernel size of (2,3) and group 1, one LTT block with a kernel size of 1 and 6 groups (to see low in space but high in channel representation), and an identity function (as a residual connection for stability). We tried with a second layer to increase accuracy, but with the current version of Concrete (without the support of multi-precision table lookups), this led to sub-optimal performance/accuracy trade-offs with a drastic increase in FHE inference time.

Client server usage as optimizations.We describe here a solution to the issue of Concrete not supporting the results of linear regression that would be on more than 4 bits: we utilize the client's computing resources not only to prepare the input to the server, but also to post-process the server's output. Namely, the client will compute a small part of the final linear regression. Indeed, we quantify the weights of the final linear regression of TInet to 4 bits, which we divide into 4 binary matrices. Then, the server performs partial sums on each of these 4 matrices. Since the outputs of TInet are binary, the weights are binary, and the maximum number input bitwidth for our lookup tables is 4 bits. For optimal performance, we perform sub-sums of size 16 to ensure that the result of each sub-sum is always lesser than \(2^{4}\) as proposed by Zama's team1. It is important to note that by doing this, we maintain the privacy of the weights of the linear regression as they remain unknown to the client. Additionally, the function computation by the client is very light, for example in the case of the MNIST dataset in the VGG\({}_{1B}/\texttt{TT}\) setting, it will represent at most \(\frac{N_{\texttt{features}}}{N_{\texttt{bits}}}=\frac{576}{16}=12\) sums of 4-bit integers to be performed for each 4 weight matrices. The client will eventually add these four outputs to obtain its final result, _i.e._ computing \(\sum_{i=0}^{3}2^{i}*w_{i}*\text{output}_{i}\). Also, note that this client computation is fixed and will remain the same even if the model needs to be updated. Finally, this limitation is due to Concrete and likely temporary as Concrete identified lookup tables multi-precision as an "important feature to come"2.

Footnote 1: [https://community.zama.ai/t/load-model-complex-circuit/369/4](https://community.zama.ai/t/load-model-complex-circuit/369/4)

Footnote 2: [https://community.zama.ai/t/load-model-complex-circuit/369/4](https://community.zama.ai/t/load-model-complex-circuit/369/4)
Patching optimizations.Due to its highly decoupled nature, our model performance is dependent on the number of patches. The classification of an image can be viewed as \(N\) independent computations, where \(N\) is the number of patches of the original image. This is a unique property and is highly useful for addressing the issue of Concrete requiring a lot of RAM when compiling complex circuits, especially when one will have to scale to larger datasets. For example on MNIST, we have a patch of size (8x8x1) instead of an entire image of size (28x28x1).

## 4 Results

The project implementation was done in Python, with the PyTorch library (Paszke et al., 2019) for training, in numpy for testing in clear, and the Concrete-numpy library for FHE inference. Our workstation consists of 4 Nvidia GeForce 3090 GPUs (only for training) with 24576 MiB memory and eight cores Intel(R) Core(TM) i7-8650U CPU clocked at 1.90 GHz, 16 GB RAM. For all experiments, the CPU Turbo Boost is deactivated and the processes were limited to using four cores.

The details of all our models' architecture can be found in Appendix C.

### Tabular datasets

Our experimental results shown in Table 1 (in Table 6 of Appendix F for results normalized to a single CPU) demonstrate the superior performance of the proposed TT-TFHE framework on three tabular datasets (Cancer, Diabetes, Adult) in terms of both accuracy and computational efficiency. The framework achieved an increase in accuracy of up to \(+3\%\) compared to state-of-the-art DNN & HE set-ups based on TFHE such as TAPAS (Sanyal et al., 2018) or on CKKS such as the recent work from the ETHZ team (Jovanovic et al., 2022). More impressively, the inference time per CPU was significantly reduced by a factor 1205x, 7x, and 825x on Adult, Cancer, and Diabetes datasets respectively.

This enables the practical deployment of our framework in industrial and real-world scenarios where tabular datasets are prevalent (Cartella et al., 2021; Buczak and Guven, 2015; Clements et al., 2020; Ulmer et al., 2020; Evans, 2009), with low memory and communication overhead (see Table 5 in Appendix D). Note that these results are in the Full-Pr setting, as the binarization process in TT-TFHE does not lead to any particular issue for tabular datasets.

### Image datasets

Our experimental results and comparisons for image datasets are given in Table 2 (in Table 7 of Appendix F for all results normalized to a single CPU).

#### 4.2.1 Fully private (Full-Pr) setting

In the Full-Pr configuration, we focused on the performance of our method on the MNIST dataset, as the binarization process in TT-TFHE resulted in a significant loss of accuracy for CIFAR-10 or an increase in inference time. TT-TFHE offers a competitive trade-off compared to other TFHE-based methods, such as TAPAS (Sanyal et al., 2018), GateNet (Fu et al., 2021) or Zama (Chillotti et al., 2021). It is three orders of magnitude faster than TAPAS or GateNet, while showing only a slightly lower accuracy of \(1.4\%\). In comparison to Zama, our method is 2x faster per CPU for the same level of accuracy. Additionally, we highlight that for our single layer LTT block of size \(n=6\), we require 1600 calls to 6-bit lookup tables, which leads to an inference time of only 83 seconds on four CPU cores.

TT-TFHE is even competitive in terms of inference time with some non-TFHE-based schemes such as (Fast) CryptoNets (Gilad-Bachrach et al., 2016), but can be one order of magnitude slower with slightly lower accuracy. Therefore, one can observe that the Full-Pr setting of TFHE implemented in our framework generally underperforms compared to the very latest Full-Pr CKKS or BFV setups. This is explained by the first binarization process in TT-TFHE, which compresses too much information embedded in the input image. Yet,

\begin{table}
\begin{tabular}{|l|c|c|c|c|c|c|c|} \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**FHE**} & \multicolumn{2}{c|}{**Adult**} & \multicolumn{2}{c|}{**Cancer**} & \multicolumn{2}{c|}{**Diabetes**} \\ \cline{2-9} \multicolumn{1}{c|}{} & **family** & & & Acc. & Time & Acc. & Time & Acc. & Time \\ \hline ETHZ/CCS2 & CKKS & 64 & 81.6\% & 420s & - & - & - & - \\ \hline TAPAS & \multirow{2}{*}{THHE} & 16 & - & - & 97.1\% & 3.8\% & 54.9\% & 250s \\ \cline{2-9}
**Ours** & & 4 & 85.3\% & 5.6\% & 97.1\% & 1.8\% & 57.0\% & 1.2\% \\ \hline \end{tabular}
\end{table}
Table 1: Tabular dataset results for TT-TFHE and competitors. All results are in Full-Pr setting. All our models use a table lookup bitwidth of \(n=5\), except for Diabetes where we use \(n=6\).

\begin{table}
\begin{tabular}{|l|c|c|c|c|c|c|c|} \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**P1-TFHE**} & \multicolumn{2}{c|}{**IFHE**} & \multicolumn{2}{c|}{**IFHE**} & \multicolumn{2}{c|}{**IFHE**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**TFHE**} & \multicolumn{2}{c|}{**Cartella**} & \multicolumn{2}{c|}{**Ours**} & \multicolumn{2}{c|}{**Ours**} & \multicolumn{2}{c|}{**Zama**} & \multicolumn{2}{c|}{**Ours**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**Ours**} & \multicolumn{2}{c|}{**Ours**} & \multicolumn{2}{c|}{**Zama**} & \multicolumn{2}{c|}{**Ours**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} \\ \cline{2-9} \multicolumn{1}{c|}{} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**GFUCE**} & \multicolumn{2}{c|}{**
we again emphasize the many advantages of TFHE-based solutions compared to non-TFHE-based ones: little memory required allowing easy/efficient multiclient inference, low communication overhead, no security warning on TFHE while CKKS secret key can be recovered in polynomial time (Li & Micciancio, 2021) (a fix was proposed afterward (Li et al., 2022) but not yet implemented in SEAL for example), etc. We will see in the next sub-section that the performance situation is very different in the setting where the client can perform some pre-computation layer.

#### 4.2.2 Other settings

We first observe that when allowing the client to apply a simple pre-processing layer, the performance increases drastically for TT-TFHE. We have implemented and benchmarked both VGG\({}_{1L}\)/TT and VGG\({}_{1B}\)/TT settings, both with TTnet models with \(n=6\) and \(n=4\) and we obtained a 10x performance improvement, with an increase in accuracy. For reference, we have also tested the VGG\({}_{1B}\)/\(\emptyset\) setting where only a linear regression is computed privately on server: we remark that adding a TTnet in the server computation indeed improves accuracy by about 4%.

One could argue that more VGG blocks could be computed on client side to further increase the accuracy, but this would reduce the generality of the first layers and lead to PPML solutions that would not adapt very well to multiple use cases. We have tried blocks of other more recent CNNs than VGG, such as DenseNet(Huang et al., 2017), but the results remain very similar.

We can compare the TT-TFHE results to some TFHE-based competitors, as Zama proposed a similar setting 3 with VGG\({}_{1L}\) pre-computed by the client for CIFAR-10, and against which we infer 100x faster per CPU and with a 10% increase in accuracy.

Footnote 3: [https://github.com/zama-ai/concrete-ml/tree/release/0.6.x/use_case_examples/cifar_10_with_model_splitting](https://github.com/zama-ai/concrete-ml/tree/release/0.6.x/use_case_examples/cifar_10_with_model_splitting)

Our TT-TFHE results are now even competitive against non-TFHE-based solutions (even though they miss many of TFHE advantages), being faster than (Fast) CryptoNets (Gilad-Bachrach et al., 2016) and SHE (Lou & Jiang, 2019), and on par with Lola (Brutzkus et al., 2019) and Lee _et. al._(Lee et al., 2022) per CPU. We note that SHE and Lee _et. al._ have better accuracy than our model.

### Memory/communication cost of TT-TFHE

In Table 5 from Appendix D, we compare the memory and communication needs for all our settings. We can observe that the deeper the representation, the smaller the communication needs: the inputs to the server become smaller as we go deeper into the neural network (there are also fewer computations to do in the server). Some settings do not need public keys as only a linear regression is done, and thus no programmable bootstrapping is involved (Chillotti et al., 2021). Then, the largest the lookup tables (in terms of the number of features and size), the larger will be the public keys as there will be more bootstrapping. Also, the optimization proposed in Section 3.4 to ease the linear regression comes with a cost: it increases the size of the outputs. Indeed, only the VGG\({}_{1B}\)/\(\emptyset\) setting does not use this optimization as there is no lookup-table involved and thus no bootstrapping. Finally, the encrypted inputs size increases with the number of features.

Comparison with other methods.As stated in (Clet et al., 2021), CKKS is more memory greedy than TFHE for communication. In Table 4 of Appendix D, we compare between each method the amount of RAM needed on server side for CIFAR-10 dataset. The best accuracy is the ResNet proposed by (Lee et al., 2022), but it also leads to the highest consumption in RAM. With LoLa setting, the accuracy is indeed lower but it requires 32x lesser RAM. Then, our method reduces again the memory on server by almost a factor of 15x for the same accuracy. RAM size on server matters for cloud computing as pricing increases along with memory needs4, thus low-memory solutions help the scalability of the MLaaS.

Footnote 4: [https://azure.microsoft.com/en-us/pricing/](https://azure.microsoft.com/en-us/pricing/)

## 5 Limitations and Conclusion

Limitations.Our proposed framework is still not as performant as the latest non-TFHE-based solutions with regard to running time and accuracy. Moreover, CIFAR-10 and larger datasets remain out of reach for industrial use. Finally, TT-TFHE is for the moment limited by the constraints of the Concrete library.

Conclusion.In this paper, we presented a new framework, TT-TFHE, which greatly outperforms all TFHE-based PPML solutions in terms of inference time, while maintaining acceptable accuracy. Our proposed framework is a practical solution for real-world applications, particularly for tabular data and small image datasets like MNIST, as it requires minimal memory/communication cost, provides strong security for the client's data, and is easy to deploy. We believe that this research will spark further investigations into the utilization of truth tables for privacy-preserving data usage, emphasized by the recently NIST Artificial Intelligence Risk Management Framework (AI, 2023).