Creating a Microstructure Latent Space with Rich Material Information for Multiphase Alloy Design
Xudong Ma Yuqi Zhang Chenchong Wang Ming Wang Mingxin Huang Wei Xu




SoK: Bitcoin Layer Two (L2)
    
Minfeng Qi^1,These authors contributed equally to the work.  , Qin Wang^2,*,  Zhipeng Wang^3, Manvir Schneider^4, 

Tianqing Zhu^1, Shiping Chen^2, William Knottenbelt^3, Thomas Hardjono^5  

^1City University of Macau, China | 
^2CSIRO Data61, Australia | 
^3Imperial College London, UK   
 
^4Cardano Foundation, Switzerland | 
^5Massachusetts Institute of Technology, US 

    
============================================================================================================================================================================================================================================================================================================================================================================================





§ ABSTRACT 
The intricate microstructure serves as the cornerstone for the 
composition/processing-structure-property (CPSP) connection in multiphase alloys. Traditional alloy design methods often overlook microstructural details, which diminishes the reliability and effectiveness of the outcomes.  This study introduces an improved alloy design algorithm that integrates authentic microstructural information to establish precise CPSP relationships. The approach utilizes a deep-learning framework based on a variational autoencoder to map real microstructural data to a latent space, enabling the prediction of composition, processing steps, and material properties from the latent space vector. By integrating this deep learning model with a specific sampling strategy in the latent space, a novel, microstructure-centered algorithm for multiphase alloy design is developed. This algorithm is demonstrated through the design of a unified dual-phase steel, and the results are assessed at three performance levels. Moreover, an exploration into the latent vector space of the model highlights its seamless interpolation ability and its rich material information content. Notably, the current configuration of the latent space is particularly advantageous for alloy design, offering an exhaustive representation of microstructure, composition, processing, and property variations essential for multiphase alloys. 



Keywords: Microstructure; Latent space; Alloy design; Variational autoencoder.








Graphical abstract:



    < g r a p h i c s >







§ INTRODUCTION



In the realm of alloy design, the paramount goal is to engineer novel materials suitable for targeted applications. Central to this is the elucidation of composition/processing–structure–property (CPSP) relationships. These relationships, traditionally derived from experimental data, such as the Hall–Petch relationship linking grain size to yield strength <cit.>, or the influence of finely dispersed second phases on strength augmentation <cit.>, underscore the critical role of microstructure. The complexity and heterogeneity of multiphase alloys, however, necessitate a refined understanding of these microstructural features. In such cases, conventional CPSP relationships often fall short, as they may not adequately reflect the nuanced and often nonlinear interactions within the microstructure of these alloys.

In contrast to conventional methods, machine learning (ML) techniques can be used to model complex and nonlinear relationships, which is fundamental for their applications in alloy design <cit.>. Previous studies <cit.> in this area have demonstrated that ML models that use the composition and process parameters of alloys as model inputs, can predict mechanical properties. These models were then combined with heuristic optimization algorithms to search for optimal solutions and develop novel alloys. Although these ML-based methods are fast and relatively simple <cit.>, they overlook the key role of the microstructure. To address this limitation, the incorporation of microstructural information is crucial. Some microstructural metrics, such as the volume fraction of a certain phase <cit.>, can be used as additional input to the ML model to improve the prediction accuracy of the mechanical properties. However, relying solely on simple metrics might be insufficient for alloy systems with complex microstructures.

Complex microstructures can often be represented as images using scanning electron microscopy. Hence, the use of deep learning methods to process microstructural images and extract key features is important for alloy design. Generative models, such as the generative adversarial network (GAN) <cit.> and variational autoencoder (VAE) <cit.>, offer new solutions for alloy design <cit.>. These models typically perform unsupervised learning on training images to output low-dimensional representations and link them to the compositions, processes, or properties relevant to alloy design. For instance, Cao et al. <cit.> applied a conditional GAN to extract the microstructural features of Ti–6Al–4V and established a relationship between the process and microstructure. However, their model required process parameters and random vectors as inputs to generate the microstructure, making it impossible to achieve an inverse processing design based on the microstructure. In addition, GAN models are associated with challenges during training, including pattern collapse <cit.>. Therefore, building a generative model with a stable training process and determining suitable input and output features are critical aspects of alloy design. Kusampudi et al. <cit.> applied the VAE to extract descriptors from the microstructure of synthetic dual-phase (DP) steels, built relationships between the descriptors and properties, and used Bayesian optimization to determine the best combination of descriptors. Similarly, Kim et al. <cit.> employed the VAE and Gaussian process regression to determine the optimal microstructure. These methods use microstructural images as inputs to avoid subjectivity when selecting the microstructural features. However, the relationship between the composition/process and the microstructure is missing in these approaches, as they were developed using synthetic microstructures. Consequently, designed microstructures with ideal properties may not be realized experimentally.

A unified DP (UniDP) steel is a type of unified steel <cit.> that aims to meet different performance requirements with a single-alloy composition and different process parameters. The use of this type of steel can significantly simplify the systematic issues encountered in automotive steel production and recycling. The microstructures of UniDP steels are mainly ferrite and martensite, and an appropriate balance between them is crucial for the success of the UniDP steel design. However, the complexity and diversity of microstructural morphology pose challenges in this regard. Hence, a microstructure-centered UniDP steel design is required.

 In this study, we proposed a novel deep-learning algorithm specifically tailored for the design of multiphase alloys, emphasizing the pivotal role of microstructural characteristics in the alloy design process. Through VAE, we transformed microstructural images of dual-phase steels into a compact representation within a latent space, capturing the complex microstructural features of alloys. This latent space is then correlated with the alloy's composition, processing parameters, and mechanical properties to establish complete CPSP relationships. To emphasize the usefulness of the algorithm in the field of the design of multiphase alloy, we proposed a microstructure-centered design method for UniDP steels based on the principles of physical metallurgy (PM) and a specific sampling strategy within the latent space. Distinct from previous methods, it leverages experimental microstructures and complete CPSP connections, accelerating the design of UniDP steels. In addition, a visualization of the latent space demonstrated that integrating authentic microstructural details with precise CPSP linkages results in a continuously interpolated and information-rich mapping space. This space provides a robust foundation for the effective design and discovery of novel multiphase alloys, emphasizing the pivotal role of microstructural features in advancing the frontier of alloy design.




§ METHODOLOGY





 §.§ Microstructure-centered alloy design framework

Our framework for alloy design is fundamentally rooted in the microstructure and reveals three distinct phases, as shown in Fig. <ref>. First, we obtained microstructural images from prior studies <cit.>. These images were subjected to binarization and data augmentation to enhance their quality and variability, providing the basis for accurate modeling. The core of our framework is a deep-learning architecture designed to establish robust CPSP connections. This architecture comprises a VAE network <cit.> with an encoder and a decoder supplemented by dual multilayer perceptrons (MLPs). Together they work to predict the composition, processing parameters, and properties of the alloy directly from an initial microstructural image. The final phase involved applying the CPSP relationships derived from our modeling for alloy design. We explored the latent vector space created by our deep learning model to identify promising alloy candidates. Our selection process was guided by the foundational principles of PM, which ensured that our choices were grounded in scientific reasoning. To validate our approach, experiments were conducted to test the properties of the designed UniDP steels.



In the initial phase, the creation of a dataset was crucial. Given the high costs associated with alloy production and testing, creating a dataset that accurately mirrors the structural and performance characteristics of DP steels is challenging. To overcome this limitation, we collected data on DP steels from highly cited papers <cit.>, including the chemical composition, heat treatment parameters, scanning electron microscope (SEM) images, and mechanical properties. We collected and preprocessed 22 samples (Supplementary Table <ref> and Supplementary Fig. <ref>). The heat treatment processes were divided into three distinct groups for ease of prediction. The microstructural images were converted into a binary format (with black representing ferrite and white representing martensite) based on the martensite volume fraction (MVF) reported in literature. Then, these images were normalized, cropped, and expanded to facilitate the incorporation of microstructural data into the model training phase and prevent model overfitting. Details of image preprocessing are shown in Section 2.2.1.

The second phase involved the development of a deep learning model. To overcome the lack of generalization in microstructural feature selection across various alloy systems, we devised a VAE-centric deep learning model (VAE–DLM) to establish a robust and comprehensive CPSP linkage. The key innovation of the model is the ability of the VAE to autonomously extract authentic microstructural features through iterative optimization, forming a latent space <cit.>. Meanwhile, the two MLPs, namely the MCP and MP models, bolster the representational capacity of the VAE by imposing regularization constraints derived from the composition, process, and property information of the material. The VAE–DLM was then trained using the DP steel dataset constructed in the initial phase.

In the final stage, the design process leveraged the CPSP relationships established in the second phase by probing the latent vector space of the VAE–DLM. This exploration yielded an array of latent vectors, each of which could be utilized by the VAE–DLM to predict the corresponding compositions, processes, properties, and microstructural images. The fundamental principles of PM provided guidance for the identification of potential UniDP steel candidates. Experiments were conducted to validate the design outcomes and confirm the feasibility of the microstructure-centered alloy design approach.





 §.§ Details of the design framework



  §.§.§ Preprocessing of microstructural images of literature data


The raw image data were composed of microstructural images of 22 steels with different chemical compositions. Initially, we cropped the images to eliminate extraneous parts (e.g., markers) and binarized them based on the MVF reported in literature. Then, each image was divided into four equally sized sub-images and all the data were partitioned into four groups, with each group containing one sub-image from each original image. To mitigate the overfitting problem, we applied an offline data augmentation technique: we randomly sampled each sub-image eight times at a pixel size of 224 × 224. Ultimately, we obtained 176 sub-images for each group, totaling 704 sub-images. 



  §.§.§ Ultimate tensile strength and uniform elongation criteria of UniDP


In general, the commercial performance standards for DP780, DP980, and DP1180 require ultimate tensile strength (UTS) values exceeding 780, 980, and 1180 MPa, respectively, along with corresponding total elongation (TEL) values exceeding 12%, 8%, and 5% <cit.>. However, the tensile plate specimen size used in the commercial performance standard (No. 5, JIS Z 2241 standard) differed from the plate specimen size used in this study, which are A25 (25 mm in length and 6 mm in width). Additionally, since we used UTS and uniform elongation (UE) for evaluation during the design process of UniDP steel, the commercial performance standard lacked information about UE. Therefore, we needed to perform a standard conversion to obtain the TEL standard under A25 conditions. Then, we collected complete data for commercial DP780, DP980, and DP1180, adjusted their TEL values to equivalent values under A25 conditions, and calculated the adjusted TEL values and corresponding UE ratios. We averaged the ratios for the three steel grades. Finally, we divided the TEL criteria under A25 conditions by this average to determine the final UE criteria. 

The TEL criteria for DP780, DP980, and DP1180 were 11.90%, 7.93%, and 4.96%, respectively, under A25 conditions (calculated using equations from the literature <cit.>). The ratios of the adjusted TEL values (computed using the same formulas <cit.>) to the corresponding UE values for the collected commercial DP780, DP980, and DP1180 were 1.63, 1.91, and 1.42, respectively, resulting in an average ratio of 1.65. Finally, the UE criteria for DP780, DP980, and DP1180 were 7.21%, 4.81%, and 3.01%, respectively.



  §.§.§ Evaluation metrics for the prediction


We used two metrics to assess the predictive ability of the model: squared correlation coefficient (R^2) and mean absolute error (MAE). These metrics can be calculated as follows:


    R^2 =  ( n∑_n^i=1 f (x_i ) y_i- ∑_n^i=1 f (x_i ) ∑_n^i=1 y_i)^2/ ( n∑_n^i=1 f (x_i )^2- (  ∑_n^i=1 f (x_i ) )^2 )  (  n∑_n^i=1 y_i^2- (  ∑_n^i=1 y_i )^2 )



    MAE = 1/n∑_n^i=1 | f (x_i ) - y_i |


where n is the number of samples and f (x_i ) and y_i  represent the predicted and experimental values of the i_th sample, respectively.

Given the limited data, we employed 4-fold cross-validation to evaluate the predictive performance of the model, which entailed using one group for testing and the remainder for training; this process was repeated four times, each with a different test group.



 §.§ Experimental validation



  §.§.§ Experimental validation of designed steel


The alloy with the designed chemical composition was smelted and forged into a steel ingot weighing approximately 50 kg, hot-rolled into slabs with a thickness of 3 mm, and then cooled in a furnace. Following the removal of the oxide layer by pickling, the slabs were cold-rolled to produce sheets with a thickness of 1.4 mm. The sheets were then heated to 900 ^∘ C for a duration of 5 min and quenched. Intercritical annealing regimes with varying temperatures and durations were subsequently employed to achieve the desired properties and microstructures (see Supplementary Table <ref>). Finally, the specimens were subjected to SEM characterization and performance testing using a JSM-7800F field-emission scanning electron microscope and a tensile testing machine, respectively. The tensile specimens, fabricated in alignment with the rolling direction of the plates, were prepared in accordance with the ASTM E8 standard with a gauge length of 25 mm and a width of 6 mm, termed A25.



  §.§.§ Preprocessing of microstructural images from experimental validation


Each heat treatment process yielded 4 SEM images, totaling 24. We selected four experimental images, obtained at an annealing temperature of 815 ^∘ C and an annealing time of 3 min, as the validation results for DP1180. In addition, experimental images obtained at annealing temperatures of 765 ^∘ C and 715 ^∘ C, and an annealing time of 13 min, were chosen as the validation results for DP980 and DP780, respectively.

To facilitate comparison with the design results, the microstructural images of the experimental steels required preprocessing. Initially, the experimental images were binarized using the ImageJ software <cit.>. Based on the average scale of all the images in the dataset, the experimental images with a resolution of 1024×768 pixels were resized to 833×720 pixels to ensure the scale of experimental images was close to that of literature-collected images. Subsequently, all the experimental images were sequentially sampled with a sampling size of 224×224 pixels, resulting in 108 sub-images (36 sub-images each for DP780, DP980, and DP1180). Upon feeding these images into the VAE model, the corresponding latent vectors and generated images were computed.



§ RESULTS AND DISCUSSION



 §.§ Construction of CPSP relationship based on VAE–DLM


The fundamental principle of the deep learning model applied to build the CPSP relationship, termed VAE–DLM (Fig. <ref>a), is to condense microstructural images into a low-dimensional representation in the latent space, which is then used for information fusion and alloy design. Specifically, the model integrates three key elements: a VAE and two MLP models.



The VAE, which is a generative model renowned for its latent representation learning, comprises an encoder and a decoder. The encoder employs a modified ResNet-18 <cit.> architecture, tailored to process grayscale images through a single-channel first layer. It maps the input data into a latent space, generating a mean and standard deviation that define a probability distribution. The decoder, which is a combination of transposed convolution, batch normalization, activation, and convolution layers, reconstructs the original input by sampling from the probability distribution of the encoder. The transposed convolutional layers of the decoder utilize a kernel size of 3×3, a stride of 2×2, and the same padding to preserve the spatial dimensions of the input. Both the MCP and MP models were composed of fully connected, batch normalized, activation and dropout layers. Each hidden layer has 512 neurons. Although the inputs of both models are vectors in the latent space, their outputs were distinctly different. The MCP model predicts the parameters related to the material composition and processing: carbon (C), manganese (Mn), intercritical annealing temperature, annealing time, and process type. In contrast, the MP model focuses on predicting the material properties: the UTS, UE, and MVF.

The predictions of VAE–DLM are shown in Fig. <ref>b-d. R^2 for most of the test set outputs surpassed 80%, suggesting that the VAE–DLM had robust generalization capabilities and a high predictive accuracy. Considering that the VAE–DLM will be subsequently used for the alloy design of DP steels and that predicting the mechanical properties is a crucial step in performance-oriented alloy design, we further examined the prediction effects of UTS and UE, as shown in Fig. <ref>c, d. For the training set, nearly all the results aligned precisely on the line with a slope of 1 and showed small error bars. For the test set, most of the points were near the diagonal, although a few points in the high-UE region deviated. The MAEs for the UTS and UE were only 33.9 MPa and 1.35%, respectively, suggesting that the models adequately learned the relationship between the microstructure and performance.

In addition, we also compared our methodology with two different series of ML models: the first series of ML models takes the composition and process as inputs <cit.>, whereas the second takes composition, process, and MVF <cit.>. The comparative analysis demonstrated that our approach, which uses microstructural images as input, outperforms these models in accurately predicting the material properties (Supplementary Figs. <ref> and <ref>). This highlights the importance of incorporating actual microstructural data and establishing complete CPSP relationships to enhance the prediction accuracy of the mechanical properties.



 §.§ UniDP steel design based on probability distribution sampling


The CPSP relationship, as clarified by the VAE–DLM, offers novel perspectives on the alloy composition and process design of UniDP steels. By sampling the latent space, we can derive the composition, process parameters, and properties of the new DP alloys. In the context of the VAE model, the feature vector 𝑍 in the latent space is contingent upon two important variables: the mean (μ) and the standard deviation (σ):


    Z = μ +  σ×ε


Where ε adheres to a standard normal distribution during model training, denoted by ε ∼ 𝒩(0, 1). The 704 sub-images corresponded to the 704 𝑍 vectors. To investigate the impact of varying sampling ranges on the sampling results (refer to Supplementary Fig. <ref>), the standard deviation of ε was incrementally increased from 1 to 15. To ensure the validity of the results, each range was independently sampled 14,080 times, equating to 20 samples for each 𝑍 vector. The findings indicate that a higher standard deviation yields more design outcomes that satisfy the DP980 performance criteria (UE and UTS) but also leads to an increase in the unreasonable design results (i.e., negative values of the composition, process, or properties, or predicted martensite volume fractions exceeding 100%). Considering that a standard deviation of 10 for ε preserves an accuracy of over 90% (reasonable sample proportion) and covers a wide range of properties (as depicted in Fig. <ref>a and Supplementary Fig. <ref>), we chose this sampling result for this study and excluded any new unreasonable alloys.

In Fig. <ref>a, the design alloys that meet the performance requirements of DP780, DP980, and DP1180, hereinafter referred to as the initially screened alloys, are denoted by blue boxes. These performance criteria require UTS values exceeding 780, 980, and 1180 MPa, and corresponding UE values surpassing 7.21%, 4.81%, and 3.01%, respectively. Figure <ref>b shows the compositional distribution of the initially screened alloys. The compositions of these alloys are primarily concentrated around C = 0.09 wt.% and Mn = 2.0 wt.%, suggesting this composition as a reference for the composition of our UniDP steel. It is widely recognized that DP steels typically belong to the C–Mn–silicon (Si) system (referring to the GB/T 20564.2-2017 standard). Hence, based on the distribution characteristics of Si in the training dataset collected from literature, we adopted a mode value of Si = 0.42 wt.%. For the heat treatment, we opted for quenching and intercritical annealing. This decision was informed by its frequent occurrence within the dataset and relative cost-effectiveness. The initially screened alloys (called secondary-screened alloys) that did not meet the above criteria for composition and process type were excluded. The initially screened alloys that meet the criteria (called candidate alloys) are used to determine the annealing temperature range for UniDP steels (refer to Supplementary Table <ref>). The annealing time was not designed because of the limited diversity within the training dataset.



Figure  <ref>c presents the experimental mechanical properties of the designed UniDP steels. All the designed UniDP steels had the same chemical composition with C = 0.09 wt.%, Mn = 2.0 wt.%, and Si = 0.42 wt.%. Quenching and intercritical annealing were performed. The properties of DP780 and DP980 were attained at an annealing time of 13 min and annealing temperatures of 715 ^∘ C and 765 ^∘ C, respectively. The properties of DP1180 were attained at an annealing temperature of 815 ^∘ C and annealing times of 3, 6, 9, and 13 min. An alteration in the strength level corresponds to microstructural transformation, particularly in the MVF (refer to Supplementary Figs. <ref> and <ref>). Figure <ref>d shows a cost comparison of the elemental constituents of the experimental steels and commercial steels. Because of the reduced Mn content, the designed UniDP steels exhibited lower elemental costs than commercial steels. Simultaneously, the singular heat treatment process flow of the experimental steels can help reduce the production expenses of enterprises. This highlights the significant potential inherent in current methodologies for designing new high-quality alloys.



 §.§ Comparison between candidate and experimental alloys


To assess the rationality of our alloy design process, we compared the latent vectors, microstructural images generated, and mechanical properties of the experimental alloys and candidate alloys. The experimental images were preprocessed (refer to Section 2.3.2) and forwarded into the encoder. The Manhattan distances were then calculated between the latent vectors of the experimental alloys and those of the candidate/secondary-screened alloys. We used the mean value, denoted by μ, of the encoder output as the latent vector for alloys. As depicted in Fig. <ref>a, compared with the secondary-screened alloys, the average distances between the vectors of the candidate alloys and those of the experimental alloys were generally smaller. This suggests that the experimental alloys were more similar to the candidate alloys. Subsequently, we compared the generated images and properties of the candidate and experimental alloys with the smallest vector distances. As shown in Fig. <ref>b–d, the generated images of both the sets of alloys exhibit numerous morphological similarities and comparable properties, with the average relative errors for the UTS and UE being 5% and 19%, respectively. The experimental UniDP alloys resembled the candidate alloys closely, thereby providing a robust reference for UniDP steel design. This study integrated the actual microstructure, composition, and processing into the design, thereby effectively excluding unreasonable regions within the composition and processing space.





 §.§ Exploratory data analysis of latent space


The latent vector space is formed by the probability distributions associated with all the images in the dataset. This space provided a qualitative framework for investigating the relationship between the attributes of alloys. The logic behind its construction directly influences the design results of the compositions, processes, and microstructures. Hence, it is necessary to visualize the latent space to reveal hidden correlations between attributes. In the following, we describe our methodologies and findings from the visualization.

First, we utilized the mean, denoted by μ, of the encoder output as the sampling result to mitigate randomness. The unsupervised t-SNE method <cit.> is then used to project the sampling vectors of the dataset images into a 2D space for visualization. Figure <ref>a shows the distribution of the microstructures following the t-SNE dimensionality reduction, with distinct morphologies being clearly delineated. From top left to bottom right, the microstructural images depict a progression. Initially, they present a combination of fibrous martensite and ferrite, followed by a mixture of blocky martensite and ferrite, and ultimately, they exhibit a structure composed mainly of martensite. These distinctly separated clusters suggest a robust correlation between the points in the t-SNE space and the microstructural images. This correlation was established within the constraints of the MP and MCP models, which encompass composition, process, and property information.



The t-SNE space illustrates the distribution of the Mn content, intercritical annealing temperature, and mechanical properties. As shown in Fig. <ref>b, c, most Mn elements and annealing temperature intervals showed a significant enrichment. The potential link between the two factors clearly demonstrates the characteristics of the dataset. However, the link between a single factor and the microstructure is more ambiguous. This could be due to the interplay of other factors, such as the annealing time and C content, highlighting the intricate relationships between the composition/process and microstructure and the challenges associated with adjusting the microstructure of DP steels using the conventional trial-and-error method. The UTS and UE distributions of the DP steels is shown in Fig. <ref>d, e. The clusters demonstrated a distinct pattern, with the UTS and UE progressively increasing and decreasing, respectively, from the upper left to the lower right. This pattern aligns with the previously mentioned microstructural variations and is in accordance with material science principles, indicating that the model could effectively comprehend the microstructural and property characteristics of DP steels.



 §.§ Continuous latent space


A continuous interpolation of the latent vector space is crucial to material design frameworks based on the VAE–DLM. This is primarily because of its ability to generate a wide array of plausible new data through interpolation or out-of-domain sampling within a certain range. Hence, it is essential to investigate the continuity of latent vector spaces. We selected a sample image and fed it into the encoder to obtain the mean (denoted by μ) and standard deviation (denoted by σ). Subsequently, we defined 𝑍 = μ + τσ and gradually increased the value of τ. Finally, the latent vector 𝑍 was fed into the decoder, MP, and MCP models to attain the generated image, UTS, and MVF, respectively.

Figure <ref> shows the alterations in the generated images, UTS, and MVF. The martensite phase (represented in white) within the generated image progressively expands toward the interior of the ferrite phase (depicted in black), transitioning from a fibrous to a blocky structure, and eventually coalescing into a structure predominantly composed of martensite. The corresponding MVF and UTS values increased incrementally. However, the growth rate of the MVF decelerated, while the growth rate of the UTS remained relatively constant, which might be attributed to an increase in the elemental content. Consequently, in the alloy design process, the design results corresponding to larger τ values might not satisfy the actual requirements when considering the issue of elemental cost. This further illustrates the significance of an appropriate sampling range for the successful design of UniDP steel. In conclusion, the latent vector space is well constructed, continuously interpolated, and can comprehensively consider the DP steel composition, process, properties, and microstructural information, thereby contributing to the UniDP steel design.





§ CONCLUSION

The design of alloys, particularly for multiphase alloys, presents formidable challenges in establishing accurate and complete CPSP links. To address this issue, we introduced a specialized deep learning model, namely the VAE–DLM, specifically for multiphase alloy systems and developed a novel alloy design framework centered on the microstructure. The framework merges advanced deep learning techniques with PM knowledge to create precise and robust CPSP connections from limited datasets. Moreover, this framework facilitates the rapid development of multiphase alloys using specific sampling strategies and PM knowledge. 

The effectiveness of this design approach was demonstrated by the development of a new UniDP steel. We compared the experimental results of the UniDP steel with its design results in terms of latent vectors, generated microstructures, and performance, justifying the design results. Moreover, through visualization analysis, we founded that the latent space generated by the VAE–DLM is both continuous and richly informative, which can aid in comprehending the intricate relationships between the material parameters and in providing robust guidance for the design of multiphase alloys.

This study represents a breakthrough in multiphase alloy design, stressing the critical incorporation of actual microstructural details into the CPSP framework to achieve high-fidelity correlations. In the future, we plan to extend our dataset and refine deep learning model to address more complex compositions of multiphase alloys, including quenching and partitioning steels. Moreover, ongoing research into the interpretative aspects of the latent space will improve the credibility of the framework in the design of multiphase alloys.




 §.§ CRediT authorship contribution statement


Xudong Ma: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – original draft, Writing – review and editing. Yuqi Zhang: Conceptualization, Formal analysis, Investigation, Methodology, Software, Writing – review & editing. Chenchong Wang: Conceptualization, Formal analysis, Project administration, Supervision, Validation, Visualization. Ming Wang: Formal analysis, Resources, Supervision, Writing – review & editing. Mingxin Huang: Formal analysis, Funding acquisition, Project administration, Visualization. Wei Xu: Formal analysis, Funding acquisition, Project administration, Supervision.



 §.§ Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.



 §.§ Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.



 §.§ Code availability

The codes are available from the corresponding author upon reasonable request.



 §.§ Acknowledgements


The research was supported by the National Key Research and Development Program of China (No. 2022YFB3707501), the National Natural Science Foundation of China (No. U22A20106 and No. 52304392). M.X. Huang acknowledged the support from Mainland-Hong Kong Joint Funding Scheme, Platform (MHP/064/20).





 §.§ Appendix. Supplementary materials

See supplementary materials for more details.

10

RN282
E.O. Hall,
The Deformation and Ageing of Mild Steel: III Discussion of Results,
Proc. Phys. Soc. London, Sect. B 64 (1951) 747. 10.1088/0370-1301/64/9/303.

RN406
N.J. Petch,
The cleavage strength of polycrystals,
J. Iron Steel Inst. 174 (1953) 25-28. 

RN288
Z. LU, S. JIANG, J. HE, J. ZHOU, W. SONG, Y. WU, H. WANG, X. LIU,
Second phase strengthening in advanced metal materials,
Acta Metall. Sin. 52 (2016) 1183-1198. 

RN310
X. Geng, F. Wang, H.H. Wu, S. Wang, G. Wu, J. Gao, H. Zhao, C. Zhang, X. Mao,
Data‐driven and artificial intelligence accelerated steel material research and intelligent manufacturing technology,
MGE Advances 1 (2023) e10. 10.1002/mgea.10.

RN344
S. Han, C. Wang, Y. Zhang, W. Xu, H. Di,
Employing deep learning in non-parametric inverse visualization of elastic–plastic mechanisms in dual-phase steels,
MGE Advances 2 (2024) e29. https://doi.org/10.1002/mgea.29.

RN247
Q. Wei, B. Cao, H. Yuan, Y. Chen, K. You, S. Yu, T. Yang, Z. Dong, T.-Y. Zhang,
Divide and conquer: Machine learning accelerated design of lead-free solder alloys with high strength and high ductility,
npj Comput. Mater. 9 (2023) 201. 10.1038/s41524-023-01150-0.

RN248
L. Jiang, Z. Zhang, H. Hu, X. He, H. Fu, J. Xie,
A rapid and effective method for alloy materials design via sample data transfer machine learning,
npj Comput. Mater. 9 (2023) 26. 10.1038/s41524-023-00979-9.

RN249
H. Fu, H. Zhang, C. Wang, W. Yong, J. Xie,
Recent progress in the machine learning-assisted rational design of alloys,
Int. J. Miner. Metall. Mater. 29 (2022) 635-644. 10.1007/s12613-022-2458-8.

RN175
A. Molkeri, D. Khatamsaz, R. Couperthwaite, J. James, R. Arróyave, D. Allaire, A. Srivastava,
On the importance of microstructure information in materials design: PSP vs PP,
Acta Mater. 223 (2022) 117471. https://doi.org/10.1016/j.actamat.2021.117471.

RN173
C. Shen, C. Wang, X. Wei, Y. Li, S. van der Zwaag, W. Xu,
Physical metallurgy-guided machine learning and artificial intelligent design of ultrahigh-strength stainless steel,
Acta Mater. 179 (2019) 201-214. https://doi.org/10.1016/j.actamat.2019.08.033.

RN304
I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio,
Generative adversarial nets,
Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, Montreal, Canada, 2014, pp. 2672–2680.

RN302
D.P. Kingma, M. Welling,
Auto-Encoding Variational Bayes,
International Conference on Learning Representations, 2013.

RN187
Z. Li, W.T. Nash, S.P. O'Brien, Y. Qiu, R.K. Gupta, N. Birbilis,
cardiGAN: A generative adversarial network model for design and discovery of multi principal element alloys,
J. Mater. Sci. Technol. 125 (2022) 81-96. https://doi.org/10.1016/j.jmst.2022.03.008.

RN188
Z. Pei, K.A. Rozman, Ö.N. Doğan, Y. Wen, N. Gao, E.A. Holm, J.A. Hawk, D.E. Alman, M.C. Gao,
Machine-Learning Microstructure for Inverse Material Design,
Adv. Sci. 8 (2021) 2101207. https://doi.org/10.1002/advs.202101207.

RN217
Z. Rao, P.Y. Tung, R. Xie, Y. Wei, H. Zhang, A. Ferrari, T.P.C. Klaver, F. Körmann, P.T. Sukumar, A. Kwiatkowski da Silva, Y. Chen, Z. Li, D. Ponge, J. Neugebauer, O. Gutfleisch, S. Bauer, D. Raabe,
Machine learning-enabled high-entropy alloy discovery,
Science 378 (2022) 78-85. 10.1126/science.abo4940.

RN177
Z. Cao, Q. Liu, Q. Liu, X. Yu, J.J. Kruzic, X. Li,
A machine learning method to quantitatively predict alpha phase morphology in additively manufactured Ti-6Al-4V,
npj Comput. Mater. 9 (2023) 195. 10.1038/s41524-023-01152-y.

RN178
N. Kusampudi, M. Diehl,
Inverse design of dual-phase steel microstructures using generative machine learning model and Bayesian optimization,
Int. J. Plast. 171 (2023) 103776. https://doi.org/10.1016/j.ijplas.2023.103776.

RN179
Y. Kim, H.K. Park, J. Jung, P. Asghari-Rad, S. Lee, J.Y. Kim, H.G. Jung, H.S. Kim,
Exploration of optimal microstructure and mechanical properties in continuous microstructure space using a variational autoencoder,
Mater. Des. 202 (2021) 109544. https://doi.org/10.1016/j.matdes.2021.109544.

RN243
T. Che, Y. Li, A.P. Jacob, Y. Bengio, W.J.a.p.a. Li,
Mode regularized generative adversarial networks,
5th International Conference on Learning Representations, ICLR 2017, 2016.

RN224
Q. Lu, Q. Lai, Z. Chai, X. Wei, X. Xiong, H. Yi, M. Huang, W. Xu, J. Wang,
Revolutionizing car body manufacturing using a unified steel metallurgy concept,
Sci. Adv. 7 (2021). 10.1126/sciadv.abk0176.

RN191
J. Zhang, H. Di, Y. Deng, R.D.K. Misra,
Effect of martensite morphology and volume fraction on strain hardening and fracture behavior of martensite–ferrite dual phase steel,
Mater. Sci. Eng., A 627 (2015) 230-240. https://doi.org/10.1016/j.msea.2015.01.006.

RN192
M. Balbi, I. Alvarez-Armas, A. Armas,
Effect of holding time at an intercritical temperature on the microstructure and tensile properties of a ferrite-martensite dual phase steel,
Mater. Sci. Eng., A 733 (2018) 1-8. https://doi.org/10.1016/j.msea.2018.07.029.

RN193
Y.-G. Deng, H.-S. Di, J.-C. Zhang,
Effect of Heat-Treatment Schedule on the Microstructure and Mechanical Properties of Cold-Rolled Dual-Phase Steels,
Acta Metall. Sin. Engl. Lett. 28 (2015) 1141-1148. 10.1007/s40195-015-0305-x.

RN194
S.K. Paul, N. Stanford, T. Hilditch,
Effect of martensite volume fraction on low cycle fatigue behaviour of dual phase steels: Experimental and microstructural investigation,
Mater. Sci. Eng., A 638 (2015) 296-304. https://doi.org/10.1016/j.msea.2015.04.059.

RN259
S. Lin, R. Clark, N. Trigoni, S. Roberts,
Uncertainty Estimation with a VAE-Classifier Hybrid Model,
ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 3548-3552.

RN260
P.D. Alfano, M. Rando, M. Letizia, F. Odone, L. Rosasco, V.P. Pastore,
Efficient Unsupervised Learning for Plankton Images,
2022 26th International Conference on Pattern Recognition (ICPR), 2022, pp. 1314-1321.

RN265
BaoSteel,
Family of ultra-high strength steels for Baosteel's automotive plates,
https://ecommerce.ibaosteel.com/portal/download/manual/X-GPa.pdf, 2023 (accessed 16 June 2023).

RN264
Z.W. Tone Jiena,
Analysis on conversion of elongation in different standard distance of metallic material,
J. Shanghai Coll. Metall 20 (1999) 40-44. 

RN262
J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J.-Y. Tinevez, D.J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, A. Cardona,
Fiji: an open-source platform for biological-image analysis,
Nat. Methods 9 (2012) 676-682. 10.1038/nmeth.2019.

RN200
K. He, X. Zhang, S. Ren, J. Sun,
Deep Residual Learning for Image Recognition,
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.

RN292
L. van der Maaten, G. Hinton,
Viualizing data using t-SNE,
J. Mach. Learn. Res. 9 (2008) 2579-2605. 









	Supplemental Materials for

	
	Creating a Microstructure Latent Space with Rich Material Information for Multiphase Alloy Design

	


Xudong Ma^a, Yuqi Zhang^a, Chenchong Wang^a, *, Ming Wang^b, *, Mingxin Huang^b, *, Wei Xu^a, *

^a State Key Laboratory of Rolling and Automation, Northeastern University, Shenyang, Liaoning 110819, China

^b Department of Mechanical Engineering, The University of Hong Kong, Hong Kong, China

* Corresponding Author


This supplementary material includes:

Supplementary Figure <ref>. Details of data on dual-phase (DP) steels gathered from literature.

Supplementary Figure <ref>. Mean absolute error for two series of machine learning models on ultimate tensile strength (UTS) and uniform elongation (UE).

Supplementary Figure <ref>. Comparison of mechanical property prediction capabilities among existing methods, and the CPP and CPMP methods.

Supplementary Figure <ref>. Trends in the number of alloys meeting DP980 performance requirements and the percentage of reasonable samples (accuracy), with an increasing standard deviation of the normal distribution ε.

Supplementary Figure <ref>. Tensile curves of alloys under varying intercritical annealing process parameters.

Supplementary Figure <ref>. SEM micrographs of samples of DP steels inter-critically annealed at different annealing temperatures and times.

Supplementary Table <ref>. Output distribution in the current dataset.

Supplementary Table <ref>.  Composition and intercritical annealing process details for various designed DP steels.

Supplementary Table <ref>. Composition and intercritical annealing process details for the experimental alloys.

Supplementary Note 1. Training strategy.

Supplementary Reference





























Supplementary Note 1. Training strategy.


The VAE–DLM has a total loss that can be expressed as follows: 




    L = L_KL + L_CE + 300 ( L_CPP + L_CP )


where L_KL represents the Kullback–Leibler divergence, and L_CE denotes the cross-entropy loss. L_CPP encompasses the regression losses of C, Mn, annealing temperature, annealing time, and classification loss of the process category, whereas L_CP includes the regression losses of the UTS, UE, and MVF. The model was optimized over 3500 epochs, using a batch size of 64 within the PyTorch framework [6]. To achieve an optimal balance between the prediction accuracy and image reconstruction quality, we employed a step learning rate decay strategy with an initial learning rate of 0.001, coupled with the Adam optimization algorithm [7]. The learning rate decay strategy entailed a specific reduction in the learning rate to 0.9 times its current value after every 50 epochs.

Supplementary Reference



  * J. Platt, Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods, Advances in large margin classifiers. 10 (1999) 61-74. 

  * L. Breiman, Random Forests, Mach. Learn. 45 (2001) 5-32. 10.1023/A:1010933404324.

  * G.E. Hinton, Connectionist learning procedures, Artif. Intell. 40 (1989) 185-234. https://doi.org/10.1016/0004-3702(89)90049-0.

  * J.H. Friedman, Greedy Function Approximation: A Gradient Boosting Machine, Ann. Stat. 29 (2001) 1189-1232. 10.2307/2699986.

  * T. Chen, C. Guestrin, XGBoost: A Scalable Tree Boosting System,  Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, 2016, pp. 785–794.

  * A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, S. Chintala, PyTorch: An Imperative Style, High-Performance Deep Learning Library, 2019.

  * D. Kingma, J. Ba, Adam: A Method for Stochastic Optimization,  International Conference on Learning Representations, 2014.


