KAN See In the Dark
Aoxiang Ning Minglong Xue Jinhong He Chengyun Song


















































KAN See In the Dark
    Aoxiang Ning,
Minglong Xue1, Jinhong He and Chengyun Song












This work is supported by Chongqing Postgraduate Research and Innovation Project Funding (No. CYS23677 and No. CYS240680), Youth Project of Science and Technology Research Program of Chongqing Education Commission of China (No. KJQN202401106).(Corresponding author: Minglong Xue)
Aoxiang Ning, Minglong Xue, Jinhong He and Chengyun Song are with Chongqing University of Technology, Chongqing, 400054, China. (e-mail: ningax@stu.cqut.edu, xueml@cqut.edu.cn, hejh@stu.cqut.edu.cn, scyer123@cqut.edu.cn)





    September 9, 2024
===================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================





§ ABSTRACT 
Existing low-light image enhancement methods are difficult to fit the complex nonlinear relationship between normal and low-light images due to uneven illumination and noise effects. The recently proposed Kolmogorov-Arnold networks (KANs) feature spline-based convolutional layers and learnable activation functions, which can effectively capture nonlinear dependencies. In this paper, we design a KAN-Block based on KANs and innovatively apply it to low-light image enhancement. This method effectively alleviates the limitations of current methods constrained by linear network structures and lack of interpretability, further demonstrating the potential of KANs in low-level vision tasks. Given the poor perception of current low-light image enhancement methods and the stochastic nature of the inverse diffusion process, we further introduce frequency-domain perception for visually oriented enhancement. Extensive experiments demonstrate the competitive performance of our method on benchmark datasets. The code will be available at: https://github.com/AXNing/KSIDhttps://github.com/AXNing/KSID.




Low-light image enhancement; Kolmogorov-Arnold networks; Frequency-domain perception; Diffusion model






































§ INTRODUCTION

Low-light image enhancement (LLIE) is a critical task in computer vision and is essential for various applications ranging from surveillance to autonomous driving<cit.>. Images captured in low-light environments often suffer from low contrast and loss of detail, making downstream tasks such as object or text detection, semantic segmentation, and others highly challenging<cit.>. Therefore, to further enhance various visual applications in poor environments, low-light image enhancement tasks have received extensive attention from researchers<cit.>.Traditional methods utilize retinex theory<cit.> and gamma correction<cit.> to correct image illumination. With the development of deep learning, some methods<cit.> significantly improve the performance of low-light image enhancement by learning the mapping between low-light and normal images in a data-driven way. Recently, the diffusion model <cit.> has received much attention for its remarkable performance in generative tasks. <cit.> introduced the diffusion model into the low-light image enhancement task to improve the recovery of image details and textures in low-light conditions. However, there are nonlinear degradation factors in low-light enhancement tasks, such as uneven illumination and varying degrees of noise in low-light images, and it is difficult for existing methods to model complex nonlinear relationships on limited data. On the other hand, although current state-of-the-art technologies have achieved remarkable breakthroughs in the field of LLIE, its inner workings are often viewed as a black box that is difficult to decipher, limiting the development of the model in specific domains. The recent introduction of Kolmogorov-Arnold Networks<cit.> has raised hopes of opening the black box of traditional networks<cit.>. It enables networks to efficiently represent complex multivariate functions by employing the Kolmogorov-Arnold representation theorem<cit.>. Unlike MLPs, which have fixed activation functions at nodes, KANs use fixed activation functions at edges. KANs make each part of the model easier to understand by decomposing a complex function into a series of simpler combinations of bivariate functions. This decomposition helps to reveal the decision-making process and output of the model, which enhances the interpretability of the model. <cit.> was the first to introduce KANs into visual tasks, reformulating U-Net as U-KAN to improve medical image segmentation and generation. <cit.> utilized KANs' local plasticity for human activity recognition. Despite these initial explorations, the potential of KANs for low-level visual tasks such as low-light image enhancement has not yet been demonstrated.
In this paper, we propose a novel LLIE method (KSID) that introduces KANs to low-level visual tasks for the first time to learn better the nonlinear dependencies between the normal and low-light domains and enhance the interpretability of the model. Specifically, we design the KAN-Block and embed it into the U-Net used for denoising by the diffusion model. KAN-Block consists of the KAN-Layer and DwConv. Since the KAN-Layer has spline-based convolutional layers and the learnable activation function, it can capture nonlinear dependencies more efficiently, thus significantly improving the image generated by the diffusion model. In addition, to improve the stability and visualization of the generation process, we reconstruct the image at each step in the denoising process and introduce frequency domain perception using the Fast Fourier Transform (FFT) to further refine the image details by learning the spectrum of the normal image. There is evidence that our method not only has good interpretability but also significantly improves the performance of the model, as shown in Fig. <ref>. We performed extensive experiments on benchmark datasets to demonstrate the effectiveness of our method. Our contribution can be summarized as follows:


  * To the best of our knowledge, we are the first to successfully introduce KANs into the LLIE task. While enhancing the performance of the model, it also further improves the interpretability of the model.


  * We introduce frequency-domain perception for visual orientation enhancement by learning the spectrum of a normal image through the Fast Fourier Transform.

  * We performed extensive experiments on the low-light image enhancement benchmark datasets and achieved impressive performance.





§ METHODS



 §.§ Kolmogorov-Arnold theorem Preliminaries

The Kolmogorov-Arnold theorem states that any continuous function can be represented as a composition of a finite number of continuous univariate functions. Specifically, for any continuous function f(x) defined in 
n-dimensional real space, where x=(x_1,x_2,...,x_n),  it can be expressed as a composition of a univariate continuous function h and a series of continuous bivariate functions x_i and g_q, i. Specifically, the theorem shows that there exists such a representation:

    f(x_1,x_2,...,x_n)=∑_q=1^2n+1h(∑_i=1^ng_q,i(x_i) )

 This representation indicates that even complex functions in high-dimensional spaces can be reconstructed through a series of lower-dimensional function operations.



 §.§ Overall Network Architecture

The structure of our proposed (KSID) is shown in Fig. <ref> (a). Our training is divided into two phases: In the first phase, inspired by <cit.>, we introduced uncertainty-guided regularization into the diffusion process to enhance the recovery performance in challenging areas; in the second phase, we froze the weights of the uncertainty network to guide the network's learning. In both phases, we utilized the KAN-Block to strengthen the learning of nonlinear dependencies, and in the second phase, we incorporated a frequency-domain perception module to achieve visually-guided enhancement.


 §.§ KAN-Block


We aim to embed KANs into low-light image enhancement networks to enhance the interpretability of the model and its ability to learn from nonlinear dependencies. As shown in Fig. <ref> (c), <cit.> proposed KANs based on the  Kolmogorov-Arnold representation theorem, similar to multilayer perception (MLP), KANs with N KAN-Layers can be represented as:

    KAN(Z)=(Φ _N-1∘Φ _N-2∘···∘Φ _1∘Φ _0)


where Z is the input feature; Φ_i signifies the i-th layer of the entire KAN network. Each KAN-Layer, with n_in-dimensional input and n_out-dimensional output, Φ comprises n_in× n_out learnable activation functions ϕ:

    Φ={ϕ_q,p}, p=1,2,...,n_in, q=1,2,...,n_out

where ϕ_q,p is the parameter that can be learned. Unlike the traditional MLP, the weight of each connection in KAN is not a simple numerical value but is parameterized as a learnable spline function, which is used to parameterize these variable functions in the formula. The results of the computation of KANs from the l-th layer to the l+1-th layer can be expressed in the form of a matrix:

    Z_l+1=  
    [       ϕ_l,1,1(· )       ϕ_l,1,2(· )                 ⋯     ϕ_l,1,n_l(· );       ϕ_l,2,1(· )       ϕ_l,2,2(· )                 ⋯     ϕ_l,2,n_l(· );                 ⋮                 ⋮                 ⋱                 ⋮;   ϕ_l,n_l+1,1(· )   ϕ_l,n_l+1,2(· )                 ⋯ ϕ_l,n_l+1,n_l(· ) ]_Φ _l Z_l

where Φ _l is the function matrix corresponding to the l-th KAN-Layer.As shown in Fig. <ref>(b), we designed KAN-Block to introduce KANs into the U-Net structure in the low-light enhancement task. The U-Net extracts high-level features through a stepwise downsampling operation and recovers low-level details using jump connections. To avoid interference from low-level information, we replace the middle layer in the U-Net with the KAN-Block, with no change in the sampling stage. Specifically, our network first takes the image X_t∈ R^H× W with added random noise ϵ∈ R^H× W and extracts the high-frequency features through downsampling operations and residual learning, where the height and width become 1/8 of the original, and we change its shape and input it into the first KAN-Block. As shown in Fig. <ref>(b), KAN-Block consists of KAN-Layer and DwConv. We take the input features by combining multiple KAN-Layer and DwConv to learn the nonlinear dependencies and fine-grained information further. The operation is defined as follows:

    X_l =DwConv(KAN(X_l-1))

where X_l is the feature map output from the l-th KAN-Layer. To learn global nonlinear dependencies, we designed a loop structure. 


 §.§ Frequency Domain Perception Module

Although the current low-light enhancement methods based on the diffusion model have made good progress, due to the stochastic nature of their inverse diffusion process and unsatisfactory visual effects, we have introduced frequency-domain perception to make the whole process more stable and achieve visually oriented enhancement.The training of the diffusion denoising probabilistic model<cit.> (DDPM) starts with obtaining a closed form X_t at any time step t; in the subsequent process, the model uses a learnable function ϵ _θ(Y, X_t,α̅_t ) to learn the underlying noise distribution<cit.>. Since the network can successfully learn this noise, we construct a learnable X_t-1 for frequency-domain perception, thereby constraining the entire training process to be more stable. It is defined as follows:

    X_t-1=1/√(α _t)(X_t-1-α _t/√(1-α̅)ϵ _θ(Y,X_t,α̅_t) )

where ϵ _θ∈ R^H× W is the predicted noise distribution. We introduce a frequency domain perceptual loss to learn the spectrum of the normal image. Specifically, we perform the FFT of X_0 and X_t-1, respectively, which can be expressed as follows:

    amp_high,pha_high=F(X_0)
        amp_low,pha_low=F(X_t-1)

where amp denotes amplitude and pha denotes phase. F stands for Fast Fourier Transform. To align X_t-1 with X_0 in high-frequency details, we construct the frequency domain loss L_f, which is defined as follows:

    L_f=    γ _1 amp_low-amp_high + 
       γ _2 pha_low-pha_high

where γ _1 and γ _2 are weighting parameters for amplitude loss and phase loss.





§ EXPERIMENTS



 §.§ Experimental Settings



  §.§.§ Datasets and Metrics

We used four common low-light image enhancement benchmark datasets for evaluation: LOLv1, LOLv2, and LSRW. For evaluation metrics, we use the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) as two full-reference distortion metrics to evaluate the performance of the proposed method. In addition, we use Learned Perceptual Image Block Similarity (LPIPS) and FréchetInceptionDistance (FID) as two perceptual metrics to measure the visual quality of the enhanced results.



  §.§.§ Implementation Details

We implemented KSID on a NVIDIA RTX 3090 GPU with PyTorch, setting the batch size to 8 and patch size to 96x96. The learning rate was set to 1e-4, with Adam as the optimizer. The training process is divided into two phases: the first phase focuses on optimising the uncertainty network, with the number of epochs set to 1e6; the second phase extends the training by setting the number of epochs to 2e6.


 §.§ Comparisons With State-of-the-Art Methods

We qualitatively and quantitatively compare the proposed KSID with state-of-the-art low-light image enhancement methods. We train on the LOLv1 dataset and test on all datasets.


  §.§.§ Quantitative Comparisons

Table <ref> presents the quantitative results of various LLIE methods, indicating that our approach is competitive across PSNR, SSIM, LPIPS, and FID metrics. Notably, our method attains the best SSIM scores on the LOLv1 dataset and remains competitive in other metrics. Furthermore, on the LOLv2-real dataset, our method excels in both SSIM and PSNR, which are full-reference metrics, as well as in FID and LPIPS, which are perceptual metrics. These results underscore the ability of our model to effectively learn and emulate complex nonlinear degradation processes in image restoration, while also adeptly recovering image details and markedly enhancing their perceptual quality. Additionally, our method achieves the best results on the large-scale LSRW dataset in SSIM, LPIPS, and FID metrics, further confirming its strong generalization and robustness.


  §.§.§ Qualitative Comparisons

We performed a qualitative comparison with different LLIE methods.
As shown in Fig. <ref>, in the LOLV2-real test dataset, we observed that the images restored by other methods suffered from colour distortion and could not effectively handle uneven illumination. However, our method has effectively addressed these issues with restored images closer to the reference images' colour distribution and better at recovering detailed information.



 §.§ Ablation Study

To evaluate the effectiveness of our model for low-light image enhancement tasks, we performed ablation studies on different modules on the LOLv2 test set. Table <ref> demonstrates that the KAN-Block significantly enhances the model's ability to learn the nonlinear degradation relationship between low-light and normal images, resulting in restored images that closely align with the true distribution. The frequency-domain perceptual module effectively enhances the detail information in the restored images, thereby improving perceptual quality.



§ CONCLUSION

In this paper, we propose a novel low-light image enhancement method, KSID, which introduces KANs into the LLIE task for the first time, improves the model's ability to learn nonlinear dependencies and achieves high-quality mapping of the degradation parameters. In addition, we introduce the Frequency Domain Perception Module to refine the image details further and make the inverse diffusion process more stable. Extensive experiments validate the effectiveness and robustness of our method. Overall, we provide an initial exploration of the potential of KANs in the field of LLIE and argue that this non-traditional linear network structure is important for processing low-level visual tasks.





































  














































plain



