Enhancing User-Centric Privacy Protection: An Interactive Framework through Diffusion Models and Machine Unlearning
Huaxi Huang Xin Yuan Qiyu Liao Dadong Wang Tongliang Liu


Rethinking Improved Privacy-Utility Trade-off with Pre-existing  Knowledge for DP Training
    
        Yu Zheng, 
        Wenchao Zhang, 
        Yonggang Zhang^∗, 
        Wei Song,  
        Kai Zhou, 
        Bo Han 
    
    ∗ Yonggang Zhang is the corresponding author. 
    Yonggang Zhang is with the Department of Computer Science, University of Technology Sydney, Australia. E-mails: yonggang.zhang-1@uts.edu.auYu Zheng is with the Department of Information Engineering, Chinese University of Hong Kong, Shatin, Hong Kong SAR. E-mail: yuzheng404@link.cuhk.edu.hkWenchao Zhang is with the College of Computer Science and Technology, Xi’an University of Science and Technology, China. E-mail: zhangwenchao@xust.edu.cn; Wei Song is with the School of Computer Science and Engineering, Northeastern University, China.  E-mail: songwei@mail.neu.edu.cn; Kai Zhou is with the Department of Computing, Hong Kong Polytechnic University, Kowloon, China. E-mail: kaizhou@polyu.edu.hk; Bo Han is with the Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR. E-mails: bhanml@comp.hkbu.edu.hk
    

    MOX - Dipartimento di Matematica “F. Brioschi”, Politecnico di
Milano, via Bonardi 9, 20133 Milan, Italy

    ^1 arashandrea.roknian@mail.polimi.it

    ^2 anna.scotti@polimi.it

    ^3 alessio.fumagalli@polimi.it
    September 9, 2024
==================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
In the realm of multimedia data analysis, the extensive use of image datasets has escalated concerns over privacy protection within such data. Current research predominantly focuses on privacy protection either in data sharing or upon the release of trained machine learning models. Our study pioneers a comprehensive privacy protection framework that safeguards image data privacy concurrently during data sharing and model publication. We propose an interactive image privacy protection framework that utilizes generative machine learning models to modify image information at the attribute level and employs machine unlearning algorithms for the privacy preservation of model parameters. This user-interactive framework allows for adjustments in privacy protection intensity based on user feedback on generated images, striking a balance between maximal privacy safeguarding and maintaining model performance. Within this framework, we instantiate two modules: a differential privacy diffusion model for protecting attribute information in images and a feature unlearning algorithm for efficient updates of the trained model on the revised image dataset. Our approach demonstrated superiority over existing methods on facial datasets across various attribute classifications.









§ INTRODUCTION

Images inherently contain a wealth of private information, such as gender, race, and age in facial datasets to license plate numbers and vehicle types in car photographs. The extensive use of such images across social networks, government databases, and industrial applications has significantly raised privacy risks, leading to profound public concerns <cit.>. This growing reliance on image data brings privacy issues to the forefront of global discussions and legislation. Countries around the world have responded by implementing stringent privacy laws, with the European General Data Protection Regulation (GDPR) <cit.> and the Australian Privacy Act 1988 <cit.> being prime examples. These laws emphasize the safeguarding of “personal data,” which includes any information linked to a specific or identifiable individual. Given this definition, images fall under the category of personal data due to their frequent inclusion of personal sensitive attributes such as faces, textual content, and license plates. This situation highlights the urgent need for effective methods to protect image data privacy.










Contemporary multimedia research has focused on dataset publication and model design, with a key challenge being privacy protection. Traditional methods like pixel-level image obfuscation <cit.> are ineffective against advanced deep learning techniques, while recent approaches using generative adversarial networks (GANs) <cit.> suffer from instability and poor image quality. Moreover, despite progress in preventing unauthorized access to sensitive model information, a unified solution addressing privacy in both dataset sharing and model deployment is lacking.
This challenge is particularly relevant for companies providing facial image applications, such as facial recognition and emotion detection services, which require comprehensive privacy solutions to manage user requests for the removal of private data, requiring adjustments to both datasets and models.

Recognizing these theoretical and practical challenges, this paper 
proposes an integrated framework that bridges the gap by effectively managing privacy concerns in image data sharing and secure machine learning model release. 

Developed from published datasets, these models are designed to prioritize privacy protection and can efficiently unlearn specific private information upon user requests. This approach ensures a balance between privacy safeguarding and the utility and performance of the data and models. Fig. <ref> provides a schematic overview of this framework, illustrating the interactive, user-centric process from customer input to model refinement, underscoring our strategy for dynamic privacy management in both data and models.

Specifically, our framework addresses the privacy preservation challenge in image sharing by introducing differential privacy (DP). To overcome the instability in GAN training and issues with image quality, we have drawn inspiration from diffusion models <cit.> and developed a novel approach to differential image privacy. This method employs diffusion models for both the extraction of intermediate features and the generation of new images. The fundamental advantage of diffusion models lies in their process, which primarily involves adding noise to images and optimizing the subsequent denoising steps, circumventing the adversarial training intrinsic to GAN models and leading to more stable model training. Existing research <cit.> indicates that diffusion models can achieve superior image quality compared to GANs. 
We design the Diffusion Differential Privacy model (Diffusion-DP). By utilizing a diffusion autoencoder model <cit.>, we train an auxiliary classifier to disentangle intermediate features, allowing precise control over target face images by injecting DP noises. This enhances the privacy protection of the shared images while maintaining their utility.

To address the challenge of updating models in response to dataset changes, especially in erasing privacy information, our framework incorporates an advanced machine unlearning (AMU) module. This module is designed for scenarios requiring the removal of specific private information from trained models due to dataset updates. Unlike traditional methods of retraining models from scratch, characterized by inefficiency and high costs, our method utilizes cutting-edge machine unlearning techniques <cit.>. We have adapted the machine feature unlearning algorithm <cit.> specifically for our task, allowing for rapid fine-tuning of models with minimal data input. 
This enables the models to efficiently unlearn specific information, significantly reducing the resources and time required while ensuring high performance. Integrated with the Diffusion-DP module and the AMU module, our framework offers an efficient solution for adapting models to dataset changes, rigorously safeguarding user privacy, and maintaining the utility and effectiveness of the models.

In summary, this paper makes the following contributions:

    
  * We propose a user-centric interactive
image privacy protection framework designed to safeguard user privacy concurrently during image data sharing and model release phases. This framework is interactive, enabling user-driven adjustments for personalized privacy considerations.
    
  * We instantiate two specific modules within our broader framework to separately address the processing of private information in data sharing and model updates. The Diffusion-DP module focuses on editing and generating image data with an emphasis on privacy preservation, while the AMU module enables efficient and rapid model adjustments in response to dataset updates, ensuring privacy compliance.
    
  * Comprehensive experiments are conducted to evaluate the effectiveness of our proposed framework. The results demonstrate that our approach not only achieves significant improvements in privacy protection but also maintains high utility and performance.


The rest of this paper is organized as follows. Section <ref> introduces related works. Section <ref> presents the proposed framework. In Section <ref>, we evaluate the proposed method on widely-used facial datasets. The conclusion is discussed in Section <ref>.





§ RELATED WORK
 


 §.§ Privacy Protection Methods for Image Data



Traditional privacy-preserving methods like pixel-level image obfuscation are often ineffective against advanced deep learning techniques <cit.>, as they can significantly degrade the utility of the image and fail to protect against sophisticated attacks <cit.>. Recent research has proposed using deep GANs for image privacy protection, where DP noise is infused into intermediate features extracted from the encoder, which are then used by the GAN's image generator to produce new images <cit.>. While this approach offers a more robust solution, it faces challenges such as GAN training instability, susceptibility to mode collapse, and the need for further improvement in the quality of generated images <cit.>. 

This challenge extends to contemporary multimedia research, which focuses on both dataset publication <cit.> and specific model design <cit.>. Privacy protection remains a critical concern, particularly in the publication and sharing of datasets. Although substantial progress has been made in developing privacy-preserving techniques for both datasets and models, a unified approach that addresses privacy concerns in both domains is still lacking. This gap is particularly evident in practical scenarios faced by companies specializing in facial image applications <cit.>, where there is a growing demand for solutions that safeguard privacy while maintaining the functionality of AI services based on facial data, like facial recognition <cit.>, age estimation <cit.>, gender identification <cit.>, and emotion detection <cit.>. 
When users request the retraction of their private data to prevent privacy violations, these companies must carefully adjust both datasets and corresponding AI models, underscoring the complexity and importance of a comprehensive privacy-preserving strategy.





 §.§ Machine Unlearning

In the field of machine learning, both exact and approximate machine unlearning methods have been explored to address the challenge of completely removing the influence of certain data segments from trained models. Exact machine unlearning aims to fully eliminate data influence, often necessitating some degree of model retraining. Early work by Cao et al. <cit.> introduced a heuristic approach to convert machine learning algorithms into a summation format, facilitating the removal of data lineage. Subsequent methods, such as Sharded, Isolated, Sliced, and Aggregated training (SISA) by Bourtoule et al. <cit.>, proposed a sharded, isolated, sliced, and aggregated training approach to enhance retraining efficiency. DeltaGrad <cit.>, another exact method, accelerates retraining by counteracting the data designated for deletion, though it is limited to specific algorithms and cannot manage mini-batch sizes.

On the other hand, approximate machine unlearning focuses on minimizing the differences between models before and after data removal, aiming to preserve model performance. Notable methods include the certified-removal approach by Guo et al. <cit.>, which uses a Newton step to erase the influence of data points in L_2-regularized linear models, and a scrubbing method for deep neural networks proposed by Golatkar et al. <cit.>. These approaches often incorporate differential privacy mechanisms to obscure residual information. Additionally, specialized methods, such as those by Ginart et al. <cit.> for K-means, highlight the development of model-specific unlearning techniques. While approximate methods offer privacy safeguards and performance preservation, they face challenges in verifying implementation and aligning with legal requirements like the “right to be forgotten.”


§ USER-CENTRIC INTERACTIVE
IMAGE PRIVACY PROTECTION FRAMEWORK








In this paper, we propose a user-centric interactive
image privacy protection framework that converts a potentially risky model into a safe one by unlearning specific attribute features, as depicted in Fig. <ref>. 
The framework begins with a training dataset labeled as “risky” due to its inclusion of sensitive information. This dataset is used to train a risky model. 
The process involves identifying a sensitive subset within the training data, guided by a resource specification step, which isolates sensitive attributes or individuals. Concurrently, a transferring dataset, deemed “safe”, undergoes distribution matching to ensure alignment with the sensitive subset. This allows for the safe transfer of attribute features, producing an “attribute transferred sensitive subset” that mitigates privacy risks. 
The core of this framework is the advanced machine unlearning (AMU) module, which employs a refined machine feature unlearning algorithm to efficiently remove sensitive attributes from the model without the need for
costly retraining.

To ensure privacy preservation, the framework integrates a Diffusion-DP module, providing DP guarantees by training on the transformed sensitive subset.
The result is a “safe model” that can perform accurate inferences while safeguarding sensitive information. 
This approach not only mitigates the risks associated with sensitive data exposure but also maintains high model performance and utility, making it an efficient solution for adapting to dataset changes while rigorously protecting user privacy.

In what follows, we will provide detailed insights into each module of the framework.



 §.§ Differential Privacy

For an (ϵ, δ)-DP mechanism, ϵ > 0 denotes the distinguishable bound of all outputs on adjacent datasets 𝒟 and 𝒟' in a dataset[Two datasets, 𝒟 and 𝒟', are adjacent if 𝒟' can be built by adding or removing a single training example from 𝒟.], and δ denotes the probability that the ratio of the probabilities of two adjacent datasets 𝒟 and 𝒟' cannot be bounded by exp(ϵ) after adding a privacy-preserving mechanism <cit.>. The specific definition of DP is provided as follows.

A randomized mechanism ℳ: 𝒳→ ℛ with domain 𝒳 and range ℛ satisfies ( ϵ, δ)-DP, if 

    [ ℳ( 𝒟) ∈ 𝒮] ≤ e^ϵ[ ℳ(𝒟')  ∈ 𝒮] + δ,

for all measurable sets 𝒮⊆ ℛ and for any two adjacent datasets 𝒟, 𝒟' ∈ 𝒳.	



A randomized mechanism ℳ: 𝒳→ ℛ with domain 𝒳 and range ℛ satisfies (α, ϵ)-RDP, if 

    D_α( ℳ(𝒟) ℳ(𝒟')) ≤ϵ,

for all measurable sets 𝒮⊆ ℛ and for any two adjacent datasets 𝒟, 𝒟' ∈ 𝒳.	



    If a mechanism ℳ: 𝒳→ ℛ satisfy ( α, δ)-RDP, it also satisfies ( ϵ + log(1/δ)/α -1, δ)-DP for any 0 < δ < 1. Moreover, ℳ satisfies pure ϵ-DP.


We analyze the sensitivity and privacy performance of the proposed time-varying DP noise perturbation mechanism. We use the ℓ_2-norm sensitivity, as given by

    Δ s = max_𝒟,𝒟' s(𝒟) - s(𝒟'),

where s(·) is a general function in 𝒟.	




 §.§ Diffusion Differential Privacy Model

To implement DP for images, we consider a diffusion mechanism that generates images while incorporating noise to ensure privacy.

    Assume 𝐗,𝐗' ∈ℝ^d, let ℳ(𝐗) be the mechanism that is a random function taking X as the input, and returning ℳ(𝐗) at the (t+1)-the iteration:
    
    X_t+1(i) = s_t+1(X_t(i)) + w^t_c(i) · n_t+1(i),

    
    X_t+1 = Decoder(Encoder(X_t) + w^t_c(i) · n_t+1(i)),

    where s_t+1(·) is the contractive map, 𝐰^t_c =[w^t_c(1),⋯,w^t_c(i),⋯,w^t_c(d)] is the weighted vector obtained from a N-class classifier, and 𝐧_t+1∼ OU (θ, ρ)  at the (t+1)-th iteration, i.e., 𝐧_t =  𝒩 (e^-θ tx, ρ^2/θ(1-e^-2θ t)𝕀_d). Let ℳ(𝐗') be obtained from 𝐗' under the same mapping. For α≥ 1, ℳ(𝐗) satisfies
    
    D_α( ℳ(𝐗) ℳ(𝐗')) ≤α𝐗 - 𝐗' ^2/2T (w^t_c(i)/∑_i=1^N w^t_c(i)·ρ^2/θ(e^2θ t-1)).





Suppose that an input image 𝐗_0, and its corresponding output image 𝐗_T. s(·) maps the image into its latent space (or, feature space), and s'(·) is a 1-Lipschitz. 
Let s(·) have global L_2-sensitivity Δ_s and 𝐏 = (P_t)_t ≥ 0 be the Ornstein-Uhlenbeck (OU) process with parameters θ and ρ. For any α >1 and t >0 the OU mechanism ℳ_t^s(𝐗_t) = P_t (s(𝐗_t)) satisfies (α, αθΔ_s^2/2T (w^t_c(i)/∑_i=1^N w^t_c(i)ρ^2 (e^2 θ t -1 )))-RDP.



According to Definition <ref>, we have ℳ(s(𝐗)) for α≥ 1 satisfies


    D_α( ℳ(s(𝐗)) ℳ(s(𝐗'))) 
          ≤αs(𝐗) - s(𝐗') ^2/2T(w^t_c(i)/∑_i=1^N w^t_c(i)·ρ^2/θ(e^2θ t-1))
       ≤αmax_{X,X'}s(𝐗) - s(𝐗') _2^2/2T (w^t_c(i)/∑_i=1^N w^t_c(i)·ρ^2/θ(e^2θ t-1))
        = αΔ^2_s/2T (w^t_c(i)/∑_i=1^N w^t_c(i)·ρ^2/θ(e^2θ t-1)),


where (<ref>) is obtained based on (<ref>).

Based on the definition of (α,ϵ)-RDP, we have D_α( ℳ(𝐗) ℳ(𝐗')) ≤ϵ. Therefore, the OU mechanism ℳ_t^s(𝐗_t) = P_t (s(𝐗_t)) satisfies (α, αθΔ_s^2/2T (w^t_c(i)/∑_i=1^N w^t_c(i)ρ^2 (e^2 θ t -1 )))-RDP.











 §.§ Advanced Machine Unlearning


This section presents our methodology for unlearning specific attributes from input images using first-order and second-order optimization techniques, aiming to remove the influence of specific attributes from the model while preserving the overall performance of other features.





































  §.§.§ First-Order Update

To unlearn data, we aim to find an update Δ(𝐗, 𝐗̃) for our model w^*,
where 𝐗 and 𝐗̃ are the original data and its perturbed version. If the loss ℒ is differentiable, we compute the first-order update as:

    Δ(𝐗, 𝐗̃) = -η( ∑_x̃∈𝐗̃∇_w ℒ(x̃, w^*) - ∑_x ∈𝐗∇_w ℒ(x, w^*) ),

where η is the unlearning rate. This update shifts the model parameters to minimize the loss on x̃ while removing the information in x. It differs from gradient descent by moving the model based on the gradient difference between original and perturbed data. The gradients can be computed in 𝒪(p), with p being the number of parameters in the learning model <cit.>. 




  §.§.§ Second-Order Update

The unlearning rate η can be eliminated if ℒ is twice differentiable and strictly convex. The influence of a single data point is approximated by:

    . ∂ w^*_ϕ, x →x̃/∂ϕ|_ϕ=0 = -H_w^*^-1( ∇_w ℒ(x̃, w^*) - ∇_w ℒ(x, w^*) ),

where H_w^*^-1 is the inverse Hessian of the loss at w^*. This leads to a linear approximation, as given by

    w^*_x →x̃≈ w^* - H_w^*^-1( ∇_w ℒ(x̃, w^*) - ∇_w ℒ(x, w^*) ).

Extending this to multiple data points gives the second-order update:

    Δ(𝐗, 𝐗̃) = -H_w^*^-1( ∑_x̃∈X̃∇_w ℒ(x̃, w^*) - ∑_x ∈ X∇_w ℒ(x, w^*) ).


This update does not require parameter calibration, as it derives from the inverse Hessian of the loss function. The second-order update is preferred for unlearning in models with strongly convex and twice differentiable loss functions, and can be easily calculated with common machine learning frameworks.













































§ EXPERIMENT



We verify the effectiveness of our proposed method on the two manually corrupted datasets: CelebA-HQ dataset <cit.> and CelebA dataset <cit.>.
CelebA-HQ contains 30k high-resolution celebrity images with 40 attributes. We use this CelebA-HQ dataset to train our diffusion model. 
CelebA has 202,599 images with 40 same attributes as CelebA-HQ. We use this dataset to test our proposed method. 




 §.§ Comparison Methods

We compare our method against several SOTA methods. The image-manipulated methods involve processing the input images (224x224 pixels) using different techniques: (i) Random central removal of 100x100 pixels, which involves randomly removing a central portion of the image; (ii) Whole image mosaic, which applies a mosaic filter to the entire image, blurring detailed features; and (iii) our proposed diffusion-based method, which leverages diffusion processes to selectively alter specific attributes in the images. For fine-tuning methods, we implement three unlearning approaches: (i) Retraining, which involves training the model from scratch on a modified dataset; (ii) First-order unlearning, which adjusts the model parameters using gradient descent based on the attribute-specific loss; and (iii) Second-order unlearning, our proposed method, which employs both gradient and curvature information to more effectively remove specific attributes from the model. 



 §.§ Evaluation Metrics


In our experiments, conducted on the CelebA dataset, we employed several evaluation metrics to assess the performance of attribute modification and image realism. Specifically, we focused on two primary metrics: Attribute Classification Accuracy (ACC) and Structural Similarity Index (SSIM).

For the assessment of attribute modification, we utilized the ACC. In this approach, we trained a linear binary classifier on the training set for each of the 40 attributes. This allowed us to evaluate the accuracy with which each attribute was correctly modified or maintained. The effectiveness of the attribute modifications across the entire image was further quantified by calculating the average ACC across all 40 attributes. This average ACC provides a comprehensive measure of the system's performance in handling multiple attributes simultaneously, ensuring that modifications are not only accurate but also consistent across the dataset.

In addition to ACC, we also assessed the realism of the generated images using the SSIM. Following the methodologies presented in <cit.>, SSIM was chosen due to its ability to measure the perceptual quality of images. SSIM evaluates the similarity between two images based on luminance, contrast, and structural information, producing a similarity score that ranges from -1 to 1. Higher SSIM values indicate a greater degree of similarity between the images, reflecting the extent to which the modified images retain their visual fidelity and align with human visual perception. Conversely, lower SSIM values suggest a decline in image quality or realism.

Together, these metrics provide a robust framework for evaluating both the technical accuracy of attribute modifications and the visual quality of the resulting images. The combination of ACC and SSIM ensures that our approach is not only effective in modifying attributes but also capable of generating realistic and visually pleasing images, which is critical for applications that require high levels of visual authenticity.



 §.§ Experimental Results


















































































Table <ref> provides an extensive comparison of various image manipulation methods and their impact on attribute classification performance under different levels of data change (i.e., 1%, 3%, and 5%). The methods compared include `Mosaic', `Random', and `Ours', with each method's performance evaluated across multiple attributes such as , , , , and .

Table <ref> presents the results for attributes categorized under different percentages of data changes. 
Notably, in the case of 1% data change, 
our method achieves the highest average accuracy of 89.82% across 40 attributes, outperforming both the Mosaic' (82.38%) and `Random' (84.32%) methods. 
Our method also shows strong performance in specific attributes, such as  and , indicating its effectiveness in handling minimal data changes.
As the data change increases to 3% and 5%, the `Ours' method consistently demonstrates superior performance, maintaining high accuracy in most attributes. 
For example, under 3% data change, our method reaches an accuracy of 89.07%, compared to 88.96% and 89.28% for `Mosaic' and `Random', respectively.
This trend continues under the 5% data change scenario, where our method achieves an accuracy of 89.05%, again outperforming Mosaic (88.96%) and Random (88.91%).



Overall, the results indicate that our method is particularly robust and adaptable, offering high accuracy even as the data change percentage increases. This consistent performance across different attributes and levels of data manipulation highlights the method's potential for robust attribute classification in dynamic datasets. Such reliability is crucial for real-world applications where data may vary significantly, and accurate attribute classification is essential. 
Our proposed method's superior results in most categories, especially under higher data changes, highlight its effectiveness and utility in improving attribute classification performance.



























































































Table <ref> provides a detailed comparison of various unlearning methods and their impact on attribute classification performance under different levels of data change (3%, 5%, and 10%). The methods evaluated include `Retraining', `First_order', and `Second_order', with  their performance assessed across multiple attributes such as , , , , and . The differences from the `clean' baseline are highlighted, with all selected attributes having an initial accuracy of less than 90%.


In the clean data scenario, the `Retraining' method method achieves the highest average accuracy of 89.75% across 40 attributes, followed closely by `First_order' (89.03%) and `Second_order' (89.01%). As the data change increases to 3%, 5%, and 10%, the `First_order' and `Second_order' methods exhibit competitive performance, often surpassing `Retraining' in specific attributes. 
For instance, under a 3% data change, `Second_order' attains notable accuracies in attributes like  (98.29%) and  (94.00%), with values close to those of the clean scenario, demonstrating robustness against minor data alterations.
As the data change reaches 5% and 10%, the `First_order' and `Second_order' methods continue to demonstrate robust performance. 
Specifically, at a 10% data change, `Second_order' maintains an average accuracy of 88.99%, excelling in challenging attributes like  (98.04%) and  (91.15%). 
The `First_order' method, although slightly lower in accuracy, still achieves a respectable average accuracy of 88.64%, showing reasonable adaptability in high data change scenarios.
The method's ability to maintain high accuracy under significant data changes underscores its robustness and effectiveness.

Overall, these results demonstrate the efficacy of the `First_order' and `Second_order' unlearning methods in maintaining high attribute classification performance across varying levels of data change. The `Second_order' method, in particular, shows superior adaptability and accuracy in more challenging scenarios, suggesting its potential as a reliable unlearning technique for robust attribute classification in dynamic and diverse datasets. These insights are valuable for selecting appropriate unlearning methods to enhance attribute classification performance in practical applications.


Table <ref> compares the performance of various image generation models on the CelebA-HQ dataset, measured by the SSIM metric.


Diff-AE leads with a near-perfect SSIM of 0.991, indicating exceptional retention of image details. Our method, with an SSIM of 0.951, ranks second, showcasing its strong capability to preserve image realism, highlighting its competitiveness and effectiveness in generating high-quality images after attribute modifications.




Table <ref> summarizes the training times for different unlearning methods. 
The `Clean' method takes 13,322 seconds, while the `Retraining' method, which likely involves comprehensive retraining, required the most time at 15,185 seconds. In contrast, the `First_order' and `Second_order' methods drastically reduce the training times to 174.41 seconds and 506.99 seconds, respectively. 
Notably, the `Second_order' method, while not as fast as `First_order', still offers a substantial reduction in time compared to `Clean' and `Retraining'. This additional computational time for the `Second_order' method suggests it may involve a more detailed unlearning process, potentially leading to better accuracy, as indicated in Table <ref>. 
This efficiency, coupled with the possibility of improved accuracy, makes the `Second_order' method particularly valuable for applications where balancing computational resources and maintaining high performance is critical. The sharp decrease in training times with both `First_order' and `Second_order' methods demonstrates their potential utility in scenarios necessitating quick updates or modifications, while the slight increase in computational time for the `Second_order' method could indicate a trade-off that favors accuracy without significantly compromising efficiency.



§ CONCLUSION

This paper introduces a privacy protection framework for image data, designed to address challenges in both data sharing and model publication. By integrating a diffusion model for attribute modification with machine unlearning algorithms to secure model parameters, our approach successfully balances the dual objectives of privacy protection and model performance. The framework’s user-interactive design enables customized privacy settings, ensuring that high-quality image generation and precise attribute modifications are maintained. Our results demonstrate that this framework not only preserves image realism but also enhances accuracy across facial datasets, representing a significant advancement in privacy-preserving machine learning.












