Cross Spectral Image Reconstruction Using a Deep Guided Neural Network
Cross spectral camera arrays, where each camera records different spectral
content, are becoming increasingly popular for RGB, multispectral and
hyperspectral imaging, since they are capable of a high resolution in every
dimension using off-the-shelf hardware. For these, it is necessary to build an
image processing pipeline to calculate a consistent image data cube, i.e., it
should look like as if every camera records the scene from the center camera.
Since the cameras record the scene from a different angle, this pipeline needs
a reconstruction component for pixels that are not visible to peripheral
cameras. For that, a novel deep guided neural network (DGNet) is presented.
Since only little cross spectral data is available for training, this neural
network is highly regularized. Furthermore, a new data augmentation process is
introduced to generate the cross spectral content. On synthetic and real
multispectral camera array data, the proposed network outperforms the state of
the art by up to 2 dB in terms of PSNR on average. Besides, DGNet also tops its
best competitor in terms of SSIM as well as in runtime by a factor of nearly
12. Moreover, a qualitative evaluation reveals visually more appealing results
for real camera array data.
Frank Sippel, Jürgen Seiler, André Kaup
# Cross Spectral Image Reconstruction Using a Deep Guided Neural Network

###### Abstract

Cross spectral camera arrays, where each camera records different spectral content, are becoming increasingly popular for RGB, multispectral and hyperspectral imaging, since they are capable of a high resolution in every dimension using off-the-shelf hardware. For these, it is necessary to build an image processing pipeline to calculate a consistent image data cube, i.e., it should look like as if every camera records the scene from the center camera. Since the cameras record the scene from a different angle, this pipeline needs a reconstruction component for pixels that are not visible to peripheral cameras. For that, a novel deep guided neural network (DGNet) is presented. Since only little cross spectral data is available for training, this neural network is highly regularized. Furthermore, a new data augmentation process is introduced to generate the cross spectral content. On synthetic and real multispectral camera array data, the proposed network outperforms the state of the art by up to 2 dB in terms of PSNR on average. Besides, DGNet also tops its best competitor in terms of SSIM as well as in runtime by a factor of nearly 12. Moreover, a qualitative evaluation reveals visually more appealing results for real camera array data.

Frank Sippel, Jurgen Seiler, and Andre Kaup+Friedrich-Alexander-Universitat Erlangen-Nurnberg

Multimedia Communications and Signal Processing

Cauerstrasse 7, 91058 Erlangen, Germany Multispectral Imaging, Image Reconstruction, Deep Learning

Footnote †: Source code: [https://github.com/FAU-LMS/dgnet](https://github.com/FAU-LMS/dgnet).

## 1 Introduction

Multispectral camera arrays[1, 2] and hyperspectral camera arrays[3] are capable of recording different spectral areas by employing multiple cameras. In contrast to other multispectral and hyperspectral imaging devices, this approach has the advantage of yielding a high resolution in spatial and temporal dimension as well as being cost-efficient and flexible regarding the used filters. The recorded channels do not necessarily have to lie in the visible wavelength area, but can also contain information about areas in the infrared or ultraviolet area of the spectrum. These areas of the spectrum are particularly interesting for classification problems. Multispectral and hyperspectral cameras can be used in medicine to classify the degree of burn [4], in recycling to sort materials [4], in forensics by determining the age of blood [5], or in agriculture by discriminating between parts of fields that need water and fertilizer and those that are healthy [6].

Genser et al. [7] introduced a multispectral camera array consisting of nine cameras. However, a reconstruction pipeline to reconstruct a consistent multispectral datacube is necessary. The goal of this processing pipeline is to reconstruct a consistent multispectral center view by mapping all peripheral cameras to the center perspective, see Fig. 1. Then, each pixel of the multispectral datacube shows exactly the same object in different spectral bands.

This reconstruction pipeline consists of multiple separate

Figure 1: A multispectral camera array (CAMSI) with three corresponding recorded images of the center row (top). Additionally, the registered multispectral image with missing pixels (bottom left) and the reconstructed image (bottom right) are depicted.
-62023 IEEE. Published in 2023 IEEE International Conference on Image Processing (ICIP), scheduled for 8-11 October 2023 in Kuala Lumpur, Malaysia. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE.

DOI: tba

###### Abstract

We present a novel method for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating network for generating the generated network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating network for generating the proposed network for generating the proposed network for generating the proposed network for generating network for
-62023 IEEE. Published in 2023 IEEE International Conference on Image Processing (ICIP), scheduled for 8-11 October 2023 in Kuala Lumpur, Malaysia. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE.

DOI: tba

The resulting multispectral channel \(I_{c}[x,y]\) is calculated by only using the reconstructed image for masked pixels

\[I_{c}[x,y]=(1-M_{c}[x,y])\cdot D_{c}[x,y]+M_{c}[x,y]\cdot O_{c}[x,y]. \tag{3}\]

### Data Augmentation

A big challenge to train this network is that there is not enough multispectral data available to yield diverse scenes. Thus, we suggest a smart data augmentation using RGB data, since there are many large size RGB databases with thousands of different scenes. Furthermore, realistic masks needs to be generated.

#### 2.2.1 Spectral Image Generation

The main goal here is not to perfectly generate spectral data out of these images, but rather imitate that the objects have different textures for different spectral bands. To be consistent across the whole object, the idea of this data augmentation is to map random grayscale values to a certain number of colors in the image. This is done by transforming the RGB image to an HSV image and use the hue value to assign colors. Between these colors, a linear interpolation is performed for consistency reasons. Moreover, random intensities for white and black are generated to also use the saturation and value of the HSV image. Again, linear interpolation is used. Subsequently, the grayscale image is normalized to the full intensity range and a random exposure between 0.2 and 1.5 is applied to also cover over- and underexposed cases. Finally, synthetic white Gaussian noise is added and the image is clipped to the intensity range.

#### 2.2.2 Mask Generation

Realistic masks also need to be generated for training. To that end, five different patterns are implemented. First, the classic stroke mask known from inpainting papers such as [11] is used. Moreover, a random pixel loss mask and a random block loss mask are used to simulate cases, where the depth map is noisy. Furthermore, a border mask is provided, which just cuts off random portions of random borders of the image. This also happens in reality, since the cameras are spatially distributed and thus cannot see all borders of the center camera. Finally, an edge mask can be applied. Typically, missing pixels occur on edges of objects, because often there is a bigger disparity difference between foreground and background. Therefore, an edge mask is calculated by extracting edges from the RGB image using a Canny edge detector. These thin edges are extended to a random width in a random direction by calculating starting and ending points of lines using the gradient map and setting all pixels on these lines to be reconstructed. In general, more diverse masks will make the network generalize better as long as enough image content is shown to estimate the coefficients. Examples for generated images and masks are shown in Fig. 3.

### Configuration

Since only convolutional layers are used, the network can handle any image size. The input images need to be padded such that all cube bins cover full image areas. Otherwise, the border linear regression coefficients will not be properly estimated. The desired spatial bin size is set to \(16\times 16\), while there are \(G_{l}=32\) luma bins. A weighted \(l_{1}\)-loss is used as loss function

\[L(\hat{I}_{c}[x,y],O_{c}[x,y],M_{c}[x,y])= \tag{4}\] \[\frac{1}{WH}\sum_{x,y}\begin{cases}\alpha\cdot|O_{c}[x,y]-\hat{I }_{c}[x,y]|,&\text{if }M_{c}[x,y]=1\\ |O_{c}[x,y]-\hat{I}_{c}[x,y],&\text{else},\end{cases}\]

where \(\alpha\) is the weight factor for missing pixel positions. This weight factor is set to 10 to put more importance to missing regions. Moreover, the cells in the grid should be steered towards reconstructing missing parts of the image rather than already existing parts. However, it is still important that the whole image is reconstructed such that the network learns how images look like. As optimizer, Adam with parameters \(\beta_{1}=0.5\) and \(\beta_{2}=0.999\) is picked. The network is trained for 64 epochs using the Places2 database [17], which contains approximately 1.8 million RGB images. The learning rate starts at 0.0001 and is halved at epochs 20, 32, 40, 48, and 56.

Figure 3: In the first row, two generated spectral grayscale images from an RGB image are shown. In the bottom row, three masks are depicted. Note that the edge mask is generated using the RGB image of the first row.
62023 IEEE. Published in 2023 IEEE International Conference on Image Processing (ICIP), scheduled for 8-11 October 2023 in Kuala Lumpur, Malaysia. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. DOI: tba

## 3 Evaluation

The evaluation is done quantitatively as well as qualitatively. Our novel DGNet is compared against the inpainting methods Free Form Inpainting (FF) [12], EdgeConnect (EC) [11], Frequency Selective Reconstruction (FSR) [10], and the guided reconstruction methods Cross-Spectral Reconstruction (CSR) from [7] and Non-local Cross-Spectral Reconstruction (NOCS) [14].

### Synthetic Data

To evaluate DGNet, the HyViD database [18] is used, which contains 7 scenes, each with 30 frames. The images have 31 hyperspectral channels and are rendered from a camera array very similar to the one in Fig. 1. Moreover, ground-truth depth data is available. Hence, the database can be used to evaluate realistic cross spectral reconstruction problems. For that, the 31 hyperspectral channels are synthetically filtered to nine different bandpasses, including a red, green and blue component.

The ground-truth depth is used for the reconstruction pipeline to warp the images accordingly. In Table I, the results are summarized in terms of average PSNR and SSIM, which indicates that our novel neural network-based method outperforms the reference algorithms. Moreover, the best of three runtimes using a single thread CPU implementation is shown in Table II, which are evaluated on the first frame of the scene _family house_. DGNet outperforms the reference algorithms by a factor of at least 11.8. When executed on an RTX 3090 GPU, the runtime is decreased to 340 ms, corresponding to a factor of 287 in comparison to the second fastest method NOCS.

### Real Data

To evaluate real-world performance, an evaluation based on the Middlebury 2006 database [19], which contains 21 scenes, as well as on the database of [7], which are both much smaller than the synthetic database, is shown in Table I. Note that the Middlebury database only contains RGB images. Hence, the red channel of the left camera, the green channel of the center camera and the blue channel of the right camera are chosen to simulate a linear camera array. Again, the proposed DGNet outperforms all reference methods. Additionally, another image was recorded using CAMSI [7] for qualitative evaluation. In Fig. 4, false color images of the cross spectral methods and our novel network are depicted. As indicated, the novel DGNet produces fewer artefacts than its competitors.

## 4 Conclusion

This paper introduced a neural network-based method for reconstructing occluded pixels using a guide image, e.g., when employing a multispectral camera array. For that, a cube of grid coefficients for linear regression is estimated in spatial as well as intensity direction. Afterwards, this cube is sliced into two high-resolution linear regression parameter images using the guide image. Finally, the linear regression coefficients are applied to this guide image to yield a final estimate of the peripheral view. It was shown that this method outperforms the state of the art by up to 2 dB in terms of PSNR on multispectral camera array data as well as in terms of SSIM. The novel neural network also provides a significant speedup and yields visually more accurate images.

\begin{table}
\begin{tabular}{c c c c c c c}  & Database & FF[12] & EC[11] & FSR[10] & NOCS[14] & CSR[7] & DGNet \\ \hline \multirow{4}{*}{**Image**} & HyViD[18] & 33.72 & 29.40 & 34.32 & 35.29 & 38.98 & **40.95** \\  & MiBu[19] & 28.85 & 27.38 & 30.12 & 30.94 & 33.64 & **35.43** \\  & CAMSI[7] & 28.57 & 28.40 & 28.21 & 29.78 & 29.26 & **30.57** \\ \cline{2-8} \cline{1-1} \cline{2-8}  & HyViD[18] & 0.942 & 0.869 & 0.951 & 0.962 & 0.974 & **0.989** \\  & MiBu[19] & 0.923 & 0.895 & 0.931 & 0.934 & 0.949 & **0.952** \\  & CAMSI[7] & 0.896 & 0.896 & 0.901 & 0.906 & 0.907 & **0.913** \\ \end{tabular}
\end{table}
Table I: Average PSNR in dB and SSIM of different reconstruction methods on different databases [18, 19, 7].

Figure 4: Reconstruction results of spectrally guided methods as (false) color images combining three spectral bands. Left column: center row of camera array (red, green, blue). Right column: top row of camera array (infrared filters at 750 nm, 850 nm and 950 nm). Zoomed areas are depicted using red rectangles.

\begin{table}
\begin{tabular}{c c c c c c}  & & DGNet \\  & EC[11] & FSR[10] & NOCS[14] & CSR[7] & DGNet \\ \hline
2960 & 391 & 855 & 97.8 & 162 & **8.27** & **0.34** \\ \end{tabular}
\end{table}
Table II: Single-thread CPU runtime of all methods in seconds.
IEEE. Published in 2023 IEEE International Conference on Image Processing (ICIP), scheduled for 8-11 October 2023 in Kuala Lumpur, Malaysia. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. DOI: tba
