Activation Compression of Graph Neural Networks using Block-wise
  Quantization with Improved Variance Minimization
Efficient training of large-scale graph neural networks (GNNs) has been
studied with a specific focus on reducing their memory consumption. Work by Liu
et al. (2022) proposed extreme activation compression (EXACT) which
demonstrated drastic reduction in memory consumption by performing quantization
of the intermediate activation maps down to using INT2 precision. They showed
little to no reduction in performance while achieving large reductions in GPU
memory consumption. In this work, we present an improvement to the EXACT
strategy by using block-wise quantization of the intermediate activation maps.
We experimentally analyze different block sizes and show further reduction in
memory consumption (>15%), and runtime speedup per epoch (about 5%) even when
performing extreme extents of quantization with similar performance trade-offs
as with the original EXACT. Further, we present a correction to the assumptions
on the distribution of intermediate activation maps in EXACT (assumed to be
uniform) and show improved variance estimations of the quantization and
dequantization steps.
Sebastian Eliassen, Raghavendra Selvan
Activation Compression of Graph Neural Networks Using Block-Wise Quantization With Improved Variance Minimization

###### Abstract

Efficient training of large-scale graph neural networks (GNNs) has been studied with a specific focus on reducing their memory consumption. Work by Liu et al. (2022) proposed extreme activation compression (EXACT) which demonstrated drastic reduction in memory consumption by performing quantization of the intermediate activation maps down to using INT2 precision. They showed little to no reduction in performance while achieving large reductions in GPU memory consumption. In this work, we present an improvement to the EXACT strategy by using block-wise quantization of the intermediate activations. We experimentally analyze different block sizes and show further reduction in memory consumption (\(>15\%\)), and runtime speedup per epoch ( \(\approx 5\%\)) even when performing extreme extents of quantization with similar performance trade-offs as with the original EXACT. Further, we present a correction to the assumptions on the distribution of intermediate activation maps in EXACT (assumed to be uniform) and show improved variance estimations of the quantization and dequantization steps. 1

Footnote 1: Source code will be available at the official paper repository [https://github.com/saintslab/i-Exact](https://github.com/saintslab/i-Exact).

Sebastian Eliassen\({}^{\star}\) _Raghavendra Selvan\({}^{\star}\)_

\({}^{\star}\) Department of Computer Science, University of Copenhagen graph neural networks, quantization, activation compression, efficient machine learning, deep learning

## 1 Introduction

Graph neural networks (GNNs) are a class of deep learning (DL) models most useful when dealing with graph structured data [1, 2]. They have shown widespread applications in a range of diverse applications [3, 4, 5, 6]. GNNs are known to scale poorly with the number of nodes in the graph data primarily due to the memory requirements for storing the adjacency matrices and intermediate activation maps [7]. The increase in memory consumption necessitates use of more computational resources. This is, unfortunately, in line with the growing resource consumption of recent classes of deep learning methods [8, 9]. A common approach to reducing resource consumption of DL methods is to explore different efficiency strategies [10, 11] such as training neural networks with quantized weights [12] or quantized activation maps [13].

The main focus of efficiency improvements in GNNs has been either by operating on subgraphs to use smaller adjacency matrices [14] or to store compressed node embeddings or activation maps for computing gradients [15]. In this work, we are interested in the latter, specifically following the method introduced in [15] that proposed extreme activation compression (EXACT) using a combination of stochastic rounding-based quantization and random projections.

In this work we make two contributions, starting from EXACT, that further improve the memory consumption and yield training runtime speedup. Firstly, we introduce block-wise quantization [16] of the activation maps which quantizes large groups of tensors instead of individual tensors with support down to INT2 precision. Secondly, the quantization variance estimation in EXACT is performed using assumption that the activation maps uniformly distributed. We show that the activation maps do not follow a uniform distribution but instead follow a type of clipped normal distribution with empirical evidence. Using this insight, we present an improvement to the variance minimization strategy when performing the quantization of activation maps. Experimental evaluation on multiple graph datasets shows a consistent reduction in memory consumption and speedup in training runtime compared to EXACT.

## 2 Notations and Background

We describe a graph with \(N\) nodes as \(\mathcal{G}=(\mathbf{X},\mathbf{A})\), with node feature matrix \(\mathbf{X}\in\mathbb{R}^{N\times F}\) containing \(F\)-dimensional features for each of the \(N\) nodes, and binary adjacency matrix \(\mathbf{A}\in\{0,1\}^{N\times N}\) with the relations between each of the nodes. Specifically \(\mathbf{A}_{i,j}=1\) if there is an edge between node \(i\) and \(j\) and \(\mathbf{A}_{i,j}=0\) otherwise.

The GNN from [2] with \(L\) layers can be compactly written as the recursion:

\[\mathbf{H}^{(\ell+1)}=\sigma\left(\hat{\mathbf{A}}\mathbf{H}^{(\ell)}\mathbf{ \Theta}^{(\ell)}\right) \tag{1}\]

where the symmetric normalized adjacency matrix is \(\hat{\mathbf{A}}=\tilde{\mathbf{D}}^{-\frac{1}{2}}\mathbf{A}\tilde{\mathbf{D} }^{-\frac{1}{2}}\) with \(\tilde{\mathbf{D}}\) as the degree matrix of \(\mathbf{A}+\mathbf{I}\), \(\mathbf{H}^{(0)}:=\mathbf{X}\), the trainable parameters at layer-\(\ell\) are \(\mathbf{\Theta}^{(\ell)}\) and a suitable non-linearity \(\sigma(\cdot)\).
Since the activation maps, specifically the intermediate results \(\left(\mathbf{H}^{(\ell)}\mathbf{\Theta}^{(\ell)}\right)\) and the node embedding matrix \(\mathbf{H}^{(\ell)}\), are the biggest users of memory, EXACT [15] focused on reducing the size of the activation maps from FLOAT32 to lower precision using two methods:

**Stochastic Rounding**: For a given node \(i\) its embedding vector \(\mathbf{h}_{i}^{(\ell)}\) is quantized and stored using \(b\)-bit integers as:

\[\mathbf{h}_{i_{\texttt{int}}}^{(\ell)}=\mathrm{Quant}\left(\mathbf{h}_{i}^{( \ell)}\right)=\left\lfloor\frac{\mathbf{h}_{i}^{(\ell)}-Z_{i}^{(\ell)}}{r_{i} ^{(\ell)}}B\right\rceil=\left\lfloor\bar{\mathbf{h}}\right\rfloor \tag{2}\]

where \(B=2^{b}-1\), \(Z_{i}^{(\ell)}=\min(\mathbf{h}_{i}^{(\ell)})\) is the zero-point, \(r_{i}^{(\ell)}=\max(\mathbf{h}_{i}^{(\ell)})-\min(\mathbf{h}_{i}^{(\ell)})\) is the range for \(\mathbf{h}_{i}^{(\ell)}\), \(\bar{\mathbf{h}}\) is the normalized activation map, and \(\lfloor\cdot\rceil\) is the stochastic rounding operation [17]. Stochastic rounding is a rounding method that rounds a number to its nearest integer with a probability inversely proportional to the distance from the quantization boundaries. 2

Footnote 2: For any scalar activation map, \(h\), stochastic rounding is given by:

\[\left\lfloor h\right\rceil=\begin{cases}\left\lceil h\right\rceil,\text{with probability }h-\left\lfloor h\right\rfloor\\ \left\lfloor h\right\rfloor,\text{with probability }1-\left(h-\left\lfloor h\right\rfloor\right) \end{cases}\]

 where \(\lceil\cdot\rceil\), \(\lfloor\cdot\rfloor\) are the ceil and floor operators, respectively.

Due to its stochastic nature, which is determined by the distance to the quantization boundaries, the operator itself is an unbiased operator. Figure 1-A) illustrates stochastic rounding with uniform bin widths.

The inverse process of dequantization is defined as:

\[\hat{\mathbf{h}}_{i}^{(\ell)}=\mathrm{Dequant}\left(\mathbf{h}_{i_{\texttt{ int}}}^{(\ell)}\right)=r_{i}^{(\ell)}\mathbf{h}_{i_{\texttt{int}}}^{( \ell)}/B+Z_{i}^{(\ell)} \tag{3}\]

which linearly transforms the quantized values from \([0,B]\) back to their original ranges. Note that we still have some information-loss, since \(\mathbf{h}_{i_{\texttt{int}}}^{(\ell)}\) is only an estimate of \(\mathbf{h}_{i}^{(\ell)}\).3

Footnote 3: Note that quantization followed by dequantization is unbiased due to stochastic rounding, i.e., \(\mathbb{E}[\hat{\mathbf{h}}_{i}^{(\ell)}]=\mathbb{E}[\mathrm{Dequant}(\mathrm{ Quant}(\mathbf{h}_{i}^{(\ell)})]]=\mathbf{h}_{i}^{(\ell)}\).

**Random Projection**: Another way of minimizing memory footprint of activation maps is to perform dimensionality reduction on them. This is done via random projection in EXACT as:

\[\mathbf{h}_{i_{\texttt{true}}}^{(\ell)}=\mathrm{RP}(\mathbf{h}_{i}^{(\ell)}) =\mathbf{h}_{i}^{(\ell)}\mathbf{R} \tag{4}\]

where \(\mathbf{R}\in\mathbb{R}^{D\times R}\) with \(R<D\) is the normalized Rademacher random matrix [18] that satisfies \(\mathbb{E}[\mathbf{R}\mathbf{R}^{\top}]=\mathbf{I}\).

The random projected node embeddings are inversely transformed by

\[\hat{\mathbf{h}}_{i}^{(\ell)}=\mathrm{IRP}\left(\mathbf{h}_{i_{\texttt{true}} }^{(\ell)}\right)=\mathbf{h}_{i_{\texttt{true}}}^{(\ell)}\mathbf{R}^{T}. \tag{5}\]

The matrix containing all projected and recovered activation maps are defined as \(\mathbf{H}_{\mathrm{proj}}^{(\ell)}\) and \(\hat{\mathbf{H}}^{(\ell)}\), respectively.4

Footnote 4: Also note that the RP and IRP operations are also unbiased. i.e., \(\mathbb{E}[\hat{\mathbf{H}}^{(\ell)}]=\mathbb{E}[\mathrm{IRP}(\mathbf{R}^{( \ell)})]=\mathbf{H}^{(\ell)}\).

EXACT method combines random projection and quantization to obtain compounding reductions in memory consumption. Specifically, node embeddings are compressed as \(\tilde{\mathbf{h}}_{i}^{(\ell)}=\mathrm{Quant}\left(\mathrm{RP}\left(\mathbf{ h}_{i}^{(\ell)}\right)\right)\) are stored in memory during the forward pass, and during the backward pass the they are recovered as \(\hat{\mathbf{h}}_{i}^{(\ell)}=\mathrm{IRP}\left(\mathrm{Dequant}\left(\tilde{ \mathbf{h}}_{i}^{(\ell)}\right)\right)\).

## 3 Methods

Quantizing activation maps of GNNs reduces the memory consumption when training GNNs but does introduce an additional overhead in the computation time due to the quantization/dequantization steps. We propose to perform large block-wise quantization [16] in place of quantizing individual tensors in order to recover some of the slowdown and to further reduce the memory consumption.

### Block-wise Quantization of Activation maps

The quantization in Eq. (1) is performed over each node embedding, which is a tensor \(\mathbf{h}_{i}^{(\ell)}\in\mathbb{R}^{D}\) resulting in a sequence of \(b\)-bit integers i.e., \(\mathbf{h}_{i\texttt{true}}^{(\ell)}\in\left\{0,\dots,B-1\right\}^{D}\). Instead of quantizing each node embedding, block-wise quantization takes a larger chunk of tensors and performs the quantization on them which further reduces the memory footprint and yields speedup. Block-wise quantization has been shown to be effective in reducing the memory footprint as demonstrated in [16] where optimizer states are block-wise quantized to \(8\)-bits (INT8)[19].

Consider the complete node embedding matrix after random projection, \(\mathbf{H}_{\texttt{proj}}^{(\ell)}\in\mathbb{R}^{N\times R}\). To perform block-wise quantization first the node embedding matrix is reshaped into a stack of tensor blocks of length \(G\):

\[\mathbf{H}_{\texttt{block}}^{(\ell)}\in\mathbb{R}^{\frac{N_{i}R}{G}\times G}:= \mathrm{reshape}\left(\mathbf{H}_{\texttt{proj}}^{(\ell)},G\right). \tag{6}\]

Figure 1: Demonstration of stochastic rounding for \(b=2\) i.e., \(2^{b}=4\) quantization bins for 128 points uniformly sampled datapoints. Here the sampled points can be quantize to any of the four levels. The closer the color of the sample is to the color of the vertical bar, the larger the probability that it quantizes to said vertical bar. A) Quantization bins when using uniform bin widths are showed. B) The effect of using non-linear bin widths when performing variance optimization introduced in Sec 3.2 is visualized..
The sequence of random projection and quantization as described in Section 2 are performed on each block in \(\mathbf{h}_{i_{\text{block}}}^{(\ell)}\in\mathbb{R}^{G}\ \forall\ i=[1,\ldots,(N\cdot R/G)]\). Performing quantization using larger blocks of tensors is shown to improve training stability, as block-wise quantization localizes the effects of outliers to within its own block [16]. In this work, we experiment different block sizes to study the impact on memory consumption and test performance.

### Improved Variance Minimization

Starting from the observation that \(\mathbf{h}_{i_{\text{inter}}}^{(\ell)}\) is an unbiased estimate, we want to find the quantization boundaries such that its variance, \(\mathrm{Var}(\mathbf{h}_{i_{\text{inter}}}^{(\ell)})\), is minimized to further reduce the effect of quantization. To achieve this we need three components: 1) distribution of activation maps, 2) variance as a function of the activation maps, and 3) minimization of the expected variance as a function of quantization boundaries.

In the EXACT [15], the quantization boundaries are always set to integer values i.e., bins are of constant width. This stems from the assumption that the underlying distribution of activation maps are _uniformly_ distributed [15] (Figure 2-center). In this work we show, on multiple datasets, that the activation maps are more accurately distributed as a variation of the normal distribution which we call the clipped normal.

Letting \(B=2^{b}-1\) define the number of quantization bins, and \(\Phi^{-1}\) the Probability Point Function, we describe the clipped normal distribution as

\[\mathcal{CN}_{[1/D]}(\mu,\sigma)=\min\left(\max\left(0,\mathcal{ N}(\mu,\sigma)\right),B\right), \tag{7}\] \[\text{where }\mu=B/2\text{ and }\sigma=-\mu/\Phi^{-1}(1/D).\]

The similarity between the observed and the modelled activation maps are visualized in Figure 2, where we can see that the clipped normal distribution is better at approximating the activation maps compared to the uniform distribution.

We next expand stochastic rounding to use irregular bin widths. Consider the normalized activation, \(h\in\hat{\mathbf{h}}\) within the bin-\(i\), stochastic rounding when using irregular bin widths, \(\delta_{i}\ \forall\ i=[1,\ldots,B]\), is given by:

\[\lfloor h\rceil=\begin{cases}\lceil h\rceil,\text{with probability }(h-\lfloor h \rfloor)/\delta_{i}\\ \lfloor h\rfloor,\text{with probability }1-((h-\lfloor h\rfloor)/\delta_{i}). \end{cases} \tag{8}\]

Following the variance estimation from [20]5 and assuming a normalized activation \(h\), we calculate its stochastic rounding variance as

Footnote 5: Check Eq. 4.4 onwards in [20] for detailed derivation.

\[\mathrm{Var}(\lfloor h\rceil)=\sum_{i=1}^{i=B}\left(\delta_{i}(h-\alpha_{i-1} )-(h-\alpha_{i-1})^{2}\right), \tag{9}\]

where \(\delta_{i}\) is the width of the bin containing \(h\), and \(\alpha_{i}\) is the starting position of the bin.

Assuming INT2 quantization i.e., with \(B=3\) bins, the expected variance of the stochastic rounding operation under the clipped normal distribution is obtained from Eq. (9) and Eq. (7):

\[\mathbb{E}[\mathrm{Var}(\lfloor h\rceil)]=\int_{0}^{\alpha}( \alpha\cdot h-h^{2})\mathcal{CN}(h;\mu,\sigma)\,dh\] \[+\int_{\alpha}^{\beta}\left((\beta-\alpha)(h-\alpha)-(h-\alpha)^{ 2}\right)\mathcal{CN}(h;\mu,\sigma)\,dh\] \[+\int_{\beta}^{B}\left((B-\beta)(h-\beta)-(h-\beta)^{2}\right) \mathcal{CN}(h;\mu,\sigma)\,dh \tag{10}\]

where \([\alpha,\beta]\) are the tunable edges of the central bin (see Figure 1-B). Given this expected variance in Eq. (10), we optimize the boundaries \([\alpha,\beta]\) that minimize the variance due to stochastic rounding. This can be done using standard numerical solvers implemented in Python.

## 4 Experiments and Results

**Data**: Experiments are performed on two large-scale graph benchmark datasets for inductive learning tasks. The open graph benchmark (OGB) Arxiv dataset [21] consisting of graph with \(\approx 170k\) nodes and \(>1M\) edges, and the Flickr dataset [22] consisting of \(\approx 90k\) nodes and \(\approx 900k\) edges.

**Experimental Set-up**: The GNN used in our experiments are the popular GraphSAGE architecture [14] implemented in Pytorch [23], which is also the baseline model with no activation compression i.e., operating in FP32 precision. EXACT is used in INT2 precision and \(D/R=8\) as the second baseline which uses extreme compression. We experiment our proposed compression methods in INT2 precision and different group sizes \(G/R=[2,4,8,16,32,64]\) to demonstrate the influence of block-wise quantization. To keep the dimensionality proportion between the GNN layers, we scale

Figure 3: _Variance of stochastic rounding for INT2 quantization with different quantization boundaries \([\alpha,\beta]\) based on Eq. (9). When \([\alpha=1.0,\beta=2.0]\) uniform bin width is obtained._

Figure 2: _The observed normalized activations in a GNN model on the OGB-Arxiv data (left) compared to different modelled distributions: uniform (center), and clipped normal (right). Notice the clipped normal is able to model the observed distribution more accurately, including the edges where the spikes are caused due to clipping at the boundaries._
the dimensionality of each layer equally when performing grouping, hence the block size is presented using the \(G/R\). The influence of variance minimization (VM) on the test performance is also reported.

**Results**: Performance of the baseline methods and different configurations of the method presented in this work for two datasets are reported in Table 1. The most astonishing trend is that there is no noticeable difference in test performance on both datasets, across all models, even with extreme quantization (INT2) and any of the reported block sizes. With our proposed method there is a further improvement in memory consumption compared with EXACT by about 15% (97% with baseline FP32) and about 8% (97% with baseline FP32) for the Arxiv and Flickr datasets, respectively, when using the largest block size (G/R=64). We also gain a small speedup in training time per epoch: 5% for Arxiv, and 2.5% for Flickr, compared to EXACT.

Use of clipped normal distribution in Eq. (7) to model the activation maps is better than uniform distribution. This is captured using the Jensen-Shannon divergence measure, reported in Table 2 where we observe that for all layers, in both datasets, the distance to the observed distribution is smaller for clipped normal distribution.

Variance minimization when performed for EXACT (reported as INT2+VM in Table 1) does not show any further improvement or degradation in performance.

## 5 Discussion and Conclusion

Based on the experiments and the results in Table 1, we notice that block-wise quantization of activation maps on top of random projection and stochastic rounding yields a further reduction in memory consumption and small speedup in runtime. Increasing block size does not hamper the test performance but progressively yields further reduction in memory consumption.

Activation maps in GNNs are not uniformly distributed; we demonstrated this using empirical visualizations in Figure 2. We quantified this using the clipped normal distribution which had a smaller Jensen-Shannon divergence to the observed distribution, as seen in Table 2. This implies that using uniform quantization bin width could be sub-optimal. We presented an extension to stochastic rounding that accounts for variable bin widths in Eq. (8). The influence on quantization variance using Eq. (9) visualized in Figure 3 clearly demonstrates the value of using non-uniform bin widths.

**Limitations**: The compute overhead even with the proposed modifications do not fully recover the reduction in speedup compared to the baseline i.e., when using FP32. While the variance estimation improvement introduced by modelling the activation maps with clipped normal distribution better models the activation maps, minimizing the variance of stochastic rounding under this distribution does not yield a noticeable improvement in test performance. This could simply be due to the fact that the overall drop in performance even with block-wise quantization is small, and there is no room for further improvement. The software implementations of the quantization and variance minimization strategies are not highly optimized and there is room for further fine-tuning.

**Conclusion**: Improving efficiency of training GNNs is an important step towards reducing their resource consumption. We have demonstrated that combining block-wise quantization with extreme compression (down to INT2) can be achieved with a small drop performance. The reduction in memory consumption from baseline (FP32) is \(>95\)%, and compared to EXACT we gain a further \(>15\)% in memory reduction and up to \(\approx 5\)% training runtime speedup per epoch. We have empirically shown that the activation maps for common GNN architectures do not follow uniform distribution. We proposed an improved modelling of these activation maps using a variation of the normal distribution (the clipped normal) and show that tighter variance minimization of the quantization noise was achievable.

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Quant.** & **G/R** & **Accuracy \(\uparrow\)** & **S** (e/s) \(\uparrow\) & **M**(MB) \(\downarrow\) \\ \hline \multirow{8}{*}{Arxiv} & FP32[14] & – & 71.95 \(\pm\) 0.16 & 13.07 & 786.22 \\  & INT2[15] & – & 71.16 \(\pm\) 0.21 & 10.03 & 30.47 \\ \cline{2-6}  & & 2 & 71.16 \(\pm\) 0.34 & 10.23 & 27.89 \\  & & 4 & 71.77 \(\pm\) 0.22 & 10.46 & 26.60 \\  & & 8 & 71.21 \(\pm\) 0.39 & 10.54 & 25.95 \\  & INT2 & 16 & 71.01 \(\pm\) 0.19 & 10.55 & 25.72 \\  & & 32 & 70.87 \(\pm\) 0.29 & 10.54 & 25.60 \\  & & 64 & 71.28 \(\pm\) 0.25 & 10.54 & 25.56 \\ \cline{2-6}  & INT2+VM & – & 71.20 \(\pm\) 0.19 & 9.16 & 30.47 \\ \hline \hline \multirow{8}{*}{Flickr} & FP32[14] & – & 51.81 \(\pm\) 0.16 & 17.95 & 546.92 \\  & INT2[15] & – & 51.65 \(\pm\) 0.23 & 11.26 & 20.39 \\ \cline{2-6}  & & 2 & 51.58 \(\pm\) 0.24 & 11.38 & 19.54 \\ \cline{1-1}  & & 4 & 51.57 \(\pm\) 0.29 & 11.50 & 19.12 \\ \cline{1-1}  & & 8 & 51.60 \(\pm\) 0.25 & 11.55 & 18.95 \\ \cline{1-1}  & INT2 & 16 & 51.65 \(\pm\) 0.21 & 11.54 & 18.86 \\ \cline{1-1}  & & 32 & 51.61 \(\pm\) 0.19 & 11.53 & 18.84 \\ \cline{1-1}  & & 64 & 51.72 \(\pm\) 0.24 & 11.53 & 18.84 \\ \cline{1-1} \cline{2-6}  & INT2+VM & – & 51.71 \(\pm\) 0.18 & 10.78 & 20.39 \\ \hline \hline \end{tabular}
\end{table}
Table 1: _Performance of block-wise quantization with \(D/R=8\), different quantization precision (FP32, INT2), block size (G), and with variance minimization (VM). We report the following metrics on the Arxiv [21] and Flickr [22] datasets: accuracy (%), speed (S) reported as epochs/second and memory (M) consumption in MB. Standard deviations of test accuracy is computed over 10 runs._

\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Layer** & **R** & \(\mathcal{U}\) & \(\mathcal{CN}_{\left[1/D\right]}\) \\ \hline \multirow{2}{*}{Arxiv} & layer 1 & 16 & 0.0495 & 0.0213 \\  & layer 2 & 16 & 0.0446 & 0.0016 \\  & layer 3 & 16 & 0.0451 & 0.0041 \\ \hline \multirow{2}{*}{Flickr} & layer 1 & 63 & 0.0674 & 0.0017 \\  & layer 2 & 32 & 0.0504 & 0.0033 \\ \hline \hline \end{tabular}
\end{table}
Table 2: _Jensen-Shannon divergence measure for Uniform and Clipped Normal distributions compared to the normalized activations \(\bar{\mathbf{h}}\) at each layer of the GNN for Arxiv and Flickr datasets. In all cases we see a smaller divergence measure between the clipped normal and the empirical distribution of activation maps._
**Acknowledgements**: The authors acknowledge funding received under European Union's Horizon Europe Research and Innovation programme under grant agreements No. 101070284 and No. 101070408.
