Thames & Kosmos Physics Pro (V 2.0) Science Kit | 96 Page Color Manual | 31 Experiments | Advanced Physics Education Kit | Parents' Choice Silver Award Winner
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. Quality control (QC) in mass spectrometry (MS)-based proteomics is mainly based on data-dependent acquisition (DDA) analysis of standard samples. Here, we collect 2754 files acquired by data independent acquisition (DIA) and paired 2638 DDA files from mouse liver digests using 21 mass spectrometers across nine laboratories over 31 months. Our data demonstrate that DIA-based LC-MS/MS-related consensus QC metrics exhibit higher sensitivity compared to DDA-based QC metrics in detecting changes in LC-MS status. We then prioritize 15 metrics and invite 21 experts to manually assess the quality of 2754 DIA files based on those metrics. We develop an AI model for DIA-based QC using 2110 training files. It achieves AUCs of 0.91 (LC) and 0.97 (MS) in the first validation dataset ( n = 528), and 0.78 (LC) and 0.94 (MS) in an independent validation dataset ( n = 116). Finally, we develop an offline software called iDIA-QC for convenient adoption of this methodology. The rapid advancement of mass spectrometry (MS) fosters the evolution of high-throughput, multi-batch, and large-scale quantitative proteomics 1 (#ref-CR1) , 2 (#ref-CR2) , 3 (/articles/s41467-024-54871-1#ref-CR3) . A high-performing and stable liquid chromatography-tandem mass spectrometry (LC-MS/MS) workflow is essential for the precise identification and quantification of proteins 4 (#ref-CR4) , 5 (#ref-CR5) , 6 (/articles/s41467-024-54871-1#ref-CR6) . Currently, the assessment of instrument performance relies primarily on the periodic analysis of a quality control (QC) sample. The stability of the instrument is evaluated based on variations of the QC data. In a typical proteomics experiment, the optimal condition results in a satisfactory outcome as reflected by the number of peptide and protein identifications. Therefore, in most cases, proteomics practitioners adopt the number of identified peptides and proteins as a simple and direct measure for quality control of LC-MS. Nonetheless, pinpointing the cause is usually challenging, time-consuming and highly empirical when the figures fall below the anticipated range. In quantitative experiments, it is not uncommon for the quantitative accuracy and reproducibility to be compromised without reducing the peptide and protein identifications. Therefore, the development of more sophisticated and elegant parameters involving the LC and MS systems has been proposed. Most QC studies focused on data-dependent acquisition (DDA) MS experiments using a specific LC-MS system. Rudnick et al. 7 (/articles/s41467-024-54871-1#ref-CR7) published the milestone paper in the field, introducing 46 metrics related to the QC of DDA data. The 46 metrics concern chromatography, ion source, MS1, dynamic sampling, MS2, and peptide identification. The Tabb team further refined these parameters and developed QuaMeter for DDA QC of multivendor LC-MS instruments 8 (/articles/s41467-024-54871-1#ref-CR8) , 9 (/articles/s41467-024-54871-1#ref-CR9) . Matzke 10 (/articles/s41467-024-54871-1#ref-CR10) et al. reported a multivariate statistical strategy to identify outlier LC-MS runs. The Payne team manually curated 1150 multivendor DDA-MS raw files and developed machine learning models for QC 11 (/articles/s41467-024-54871-1#ref-CR11) . The Laukens team and the Martens team have developed new file format and software tools to describe and evaluate the quality of DDA-MS files 12 (#ref-CR12) , 13 (#ref-CR13) , 14 (/articles/s41467-024-54871-1#ref-CR14) . The MacCoss team have reported Skyline-based QC software tools SProCoP 15 (/articles/s41467-024-54871-1#ref-CR15) and Panorama AutoQC webserver 16 (/articles/s41467-024-54871-1#ref-CR16) for targeted proteomics experiments, i.e . selected/multiple reaction monitoring (SRM/MRM). Nevertheless, these QC studies and software tools are exclusively for DDA-MS and/or SRM-based targeted proteomics. Currently, there has been no systematic investigation of data-independent acquisition (DIA) QC. This lack of investigation is not surprising, considering that the generation of large DIA data sets only emerged recently thanks to the rapid progresses of LC and MS systems 2 (/articles/s41467-024-54871-1#ref-CR2) , 17 (#ref-CR17) , 18 (#ref-CR18) , 19 (/articles/s41467-024-54871-1#ref-CR19) . Nowadays, DIA-MS has become arguably the method of choice for large-scale label-free quantitative proteomics experiments, not only for body fluid 20 (/articles/s41467-024-54871-1#ref-CR20) , 21 (/articles/s41467-024-54871-1#ref-CR21) , bulk cell and tissue specimens 22 (#ref-CR22) , 23 (#ref-CR23) , 24 (/articles/s41467-024-54871-1#ref-CR24) , but also for single cell proteomics 25 (/articles/s41467-024-54871-1#ref-CR25) , 26 (/articles/s41467-024-54871-1#ref-CR26) . The success of DIA experiments requires robust LC and MS systems, therefore QC is crucial. However, the existing QC pipelines mentioned above cannot be directly applied to analyze large-scale DIA experiments. In this work, we establish a computational pipeline demonstrating that DIA-based quality control outperforms DDA-based approaches in detecting changes in LC-MS status across multivendor systems, validated through data from 21 mass spectrometers over 31 months and further supported by an AI model built from expert-inspected QC metrics. Nine laboratories dedicate 21 high-resolution mass spectrometers of eight models, including Orbitrap, TripleTOF, and timsTOF Pro series instruments, for this comprehensive QC study. The same mouse liver digests are injected into these mass spectrometers for both DDA and DIA analyses. The MS operators monitor the performance of these instruments for up to 31 months, and all the maintenance data are recorded. This allows an unbiased comparison of DDA and DIA for LC-MS QC. Our data show that the DIA-based QC outperforms the DDA-based QC in detecting changes in LC-MS status. Twenty-one experts are recruited from 10 laboratories for manual inspection of the 2754 DIA QC files based on 15 prioritized consensus QC metrics. These original manually inspected data are then used to build an AI model, which is further implemented as a standalone software tool for community applications. Firstly, we generated paired DDA and DIA files as QC using a homogenous mouse liver digest samples from six mice, called Westlake Mouse Liver Digests (WMLD), employing pressure cycling technology (PCT)-assisted sample preparation 27 (/articles/s41467-024-54871-1#ref-CR27) , 28 (/articles/s41467-024-54871-1#ref-CR28) (Fig. 1 (/articles/s41467-024-54871-1#Fig1) ). The WMLD aliquots, reconstituted using a standardized protocol, were distributed to nine different laboratories. Altogether, 21 mass spectrometers of eight different types were included in this study, including Orbitrap, TripleTOF, and timsTOF Pro series mass spectrometers (Fig. 1 (/articles/s41467-024-54871-1#Fig1) ; Supplementary Fig. 1 (/articles/s41467-024-54871-1#MOESM1) ). Each instrument was operated to generate a pair of DDA and DIA runs using a 250 to 1000 ng WMLD sample over an effective LC gradient of 30 min, with the corresponding run-to-run time ranging from 40 to 55 min with different LC systems (Supplementary Data 1 (/articles/s41467-024-54871-1#MOESM4) ). The 21 LC-MS systems from nine laboratories were monitored for a period ranging from 91 to 873 days, with a median duration of 321 days (Fig. 2A (/articles/s41467-024-54871-1#Fig2) ). The entire project spans over 125 weeks. Detailed data acquisition parameters are provided in Supplementary Data 1 (/articles/s41467-024-54871-1#MOESM4) . A Generation of 2754 annotated DIA files using 21 mass spectrometers of eight types from nine platforms. B Peptide precursors pre-selection from retrospective datasets of 221 DIA files. C Establishment of a machine learning model for quality control of DIA-based proteomics. FWHM full width at half maximum, RT retention time, PIC precursor ion chromatogram. A The landscape of 2638 DIA files and 785 LC-MS maintenance events for 21 MS instruments over 873 days of data acquisition. Each MS instrument is represented by two circles of the same color, with the acquired DIA files shown as triangles positioned at the outermost part of the circle. The circles in different colors represent different instruments. The varied sizes of rings in the innermost circle represent the number of maintenance events performed on each day. The icons, distinguished by their colors and shapes, represent different types of maintenance. D, timsTOF Pro; W, TripleTOF series; R, Orbitrap series instruments. Bar chart ( B ) shows the frequency of LC-MS maintenance events for each instrument. Box plot ( C ) illustrates the distribution of identified proteins across the 2638 DIA files for the 21 MS instruments. The boxes, displayed in various colors, represent different types of instruments. Boxes are first and third quartiles, the center line is median, whiskers are ± 1.5 interquartile range, and dots are individual data points. Source data are provided with this paper. A total of 2638 DIA raw files and 2638 paired DDA raw files were obtained, accompanied by a detailed maintenance log (Supplementary Data 2 (/articles/s41467-024-54871-1#MOESM5) ). Throughout this period, we conducted a total of 785 LC-MS maintenance procedures, which could be categorized into three primary groups: LC maintenance, MS front-end maintenance, and MS back-end maintenance (Fig. 2 (/articles/s41467-024-54871-1#Fig2) ). Additionally, these maintenance procedures were further categorized into 27 subtypes, as listed in Fig. 2 (/articles/s41467-024-54871-1#Fig2) . The five most frequently performed subtypes included changing analytical column, changing trap column, calibrating MS accuracy, cleaning quadrupole, and cleaning S-lens (specific to certain Orbitrap instruments). On average, maintenance was carried out 0.6 to 5 times per month across various laboratories. The number of proteins identified varied widely across different instruments. The timsTOF Pro instrument had the highest number of protein identifications, followed by the Orbitrap series, with the TripleTOF series ranking third (Fig. 2C (/articles/s41467-024-54871-1#Fig2) ). The quality of these QC files was examined by 21 MS experts from 10 different participating laboratories using 15 LC-MS-related consensus metrics (Fig. 1A (/articles/s41467-024-54871-1#Fig1) ). We first selected commonly used metrics for both DDA and DIA, encompassing both quantitative and qualitative measures such as peptide number, protein number, and MS signals. For MS signals, we used the MS1 area of targeted peptide precursors for DDA data, while the MS2 intensities of targeted peptide precursors were used for DIA files. We monitored a QE HF-X instrument for 280 days, selecting four representative time periods of 30 to 40 days each. During these periods, we analyzed various maintenance activities (Fig. 3 (/articles/s41467-024-54871-1#Fig3) ). Each figure in Fig. 3A–D (/articles/s41467-024-54871-1#Fig3) represents a specific time frame. Our data showed that while DDA and DIA metrics were generally consistent, the DIA-based QC were more sensitive in detecting various system faults. Additionally, we observed that MS1 signals from DDA were more sensitive indicators of instrument performance compared to peptide and protein identifications in DDA QC files. Similarly, MS signals in DIA files were more sensitive than peptide and protein identifications in those files. This is not unexpected, as most MS signals are not interpreted by current software. Overall, our results demonstrate that metrics from DIA files provide superior sensitivity in monitoring fluctuations across multiple LC-MS systems. A – D Illustrates four distinct time periods, each randomly selected by the QE-HF X instrument’s longitudinal monitoring system, spanning 0 to 280 days. Each period lasts between 30 and 40 days and highlights various maintenance activities conducted during these intervals. For each time period, we selected three metrics to characterize the differences between DDA and DIA: peptide number, protein number, and MS signal. The MS signal is represented by the MS1 area from the DDA files and the MS2 intensities from the DIA files. The y-axis of each figure represents the ratio of metric values between selected raw files collected at time points n and 1. The ratio of change is calculated as follows: Ratio of change = (Y n - Y 1 )/Y n . The green vertical lines represent the types of instrument maintenance, which include the following four categories: a) Clean ion funnel, b) Clean quadrupole, c) Change pre-column, and d) Change analytical column. The red line indicates the performance of the current metric in DIA, while the black line represents its performance in DDA. Detailed information about the metric values is provided in the titles of each figure. For instance, in the first row of three figures, the red line represents the number of peptide identifications for DIA in each time period. Source data are provided with this paper. Despite numerous proposed performance metrics in prior articles 8 (/articles/s41467-024-54871-1#ref-CR8) , 9 (/articles/s41467-024-54871-1#ref-CR9) , 29 (/articles/s41467-024-54871-1#ref-CR29) , relatively few studies have delved into DIA-based QC 30 (/articles/s41467-024-54871-1#ref-CR30) . To assemble a comprehensive set of performance metrics for evaluating DIA files, we adopted six metrics from previous publications 30 (/articles/s41467-024-54871-1#ref-CR30) and assembled 135 additional candidate metrics from papers related to DDA and selected reaction monitoring (SRM) (Fig. 4A (/articles/s41467-024-54871-1#Fig4) ). More details are provided in Supplementary Data 3 (/articles/s41467-024-54871-1#MOESM6) . First, we categorized those 141 metrics into eight main groups based on the concerned sample issue and LC-MS system components: sample preparation, LC system, ion source, MS1 scan, MS2 scan, peptide and protein identification, and peptide and protein quantification, statistical analysis (Supplementary Data 3 (/articles/s41467-024-54871-1#MOESM6) ). We first removed metrics used for evaluating sample preparation that did not align with the quality control criteria of our study. Then we eliminated redundant metrics 7 (/articles/s41467-024-54871-1#ref-CR7) , including those unsuitable for DIA files, such as ion injection time, TopN, and identified PSM. We also removed 14 multivariate statistics-related metrics, including MBR alignment 29 (/articles/s41467-024-54871-1#ref-CR29) that are specific to MaxQuant results, PCA analysis 9 (/articles/s41467-024-54871-1#ref-CR9) , and correlation coefficient analysis 9 (/articles/s41467-024-54871-1#ref-CR9) , 31 (/articles/s41467-024-54871-1#ref-CR31) . We reassigned the remaining metrics to the five categories (Fig. 4B (/articles/s41467-024-54871-1#Fig4) ) and conducted independent evaluation for each category to ensure both the completeness and simplicity of the metrics. Since most metrics reported in the literature are specific to DDA, we also selected a few metrics from SRM-MS for evaluating DIA data. Flowchart ( A ) depicts the selection of 15 metrics used in this study. B Description of the 15 metrics used in the annotationg DIA files. F15*: F15 reserved exclusively for evaluation of timsTOF Pro instruments. After the above-mentioned filtering, we focused on 13 metrics. To further refine the selection, we sought input from 21 MS experts, including instrument operators, application engineers, and service engineers from analytical instrument companies including SCIEX, Thermo Fisher Scientific, and Bruker Daltonics. After discussion, the experts proposed to add two additional metrics, namely precursor ion chromatogram (PIC) 32 (/articles/s41467-024-54871-1#ref-CR32) and ion mobility accuracy to evaluate the chromatography and ion mobility specifically for diaPASEF acquired by timsTOF Pro instruments. Thereby, we assembled 15 metrics corresponding to five characteristics of the LC-MS system, including chromatography, ions source, MS1 scan, MS2 scan, identification and quantification. More details are provided in Supplementary Data 3 (/articles/s41467-024-54871-1#MOESM6) and Fig. 4B (/articles/s41467-024-54871-1#Fig4) . To evaluate whether these 15 metrics can capture common LC-MS system faults during maintenance, we constructed a Sankey diagram illustrating the relationship between these metrics and common instrument faults. These faults include the decline in MS2 signal and good MS1 signal with poor identification, good MS1 and MS2 signals with poor identification, similar proportional decline in MS1 and MS2 signals with poor identification, unstable spray, loss of hydrophilic peptides, loss of hydrophobic peptides, retention time shift with poor peptide identification, and poor peak shape as shown in the 1st node of Fig. 5A (/articles/s41467-024-54871-1#Fig5) . Our data suggested that combinations of the 15 metrics (2nd node, Fig. 5A (/articles/s41467-024-54871-1#Fig5) ) can effectively detect all these LC-MS faults. For instance, the combination of F4 (charge state distribution) and F5 (intensity variation of adjacent MS1 scans) could be utilized to detect a raw file acquired with an unstable ion source spray, typically stemming from anomalies in the spray needle and fluctuations in the LC gradient flowrate (4th node, Fig. 5A (/articles/s41467-024-54871-1#Fig5) ). It’s worth noting that this Sankey diagram is concluded for the 20 LC instruments (Supplementary Data 1 (/articles/s41467-024-54871-1#MOESM4) ) evaluated in this study. Other types of LC instruments, such as Evosep 33 (/articles/s41467-024-54871-1#ref-CR33) , are not included. We have uploaded the 15 metrics to the mzQC 34 (/articles/s41467-024-54871-1#ref-CR34) Format Specification Group from the Human Proteome Organization−Proteomics Standards Initiative (HUPO-PSI) Quality Control working group to ensure that more people are aware of and can use these metrics for quality assessment. The controlled vocabulary terms of those metrics are provided in Supplementary Data 3 (/articles/s41467-024-54871-1#MOESM6) . A Sankey diagram illustrates the relationship between instrument configuration issues and the metrics. Different flows are associated with various issues that contribute to a decline in instrument performance. B displays a heatmap illustrating the distribution of the 21 raters across the 21 instruments. The letters in panel ( B ) represent abbreviations for the experts who annotated the raw files. Panel ( C ) shows the observed agreement values among 11 technical replicates. Panels ( D ) and ( E ) depict the frequency of agreement among raters on 17 metrics, categorized by four and five raters’ instruments, respectively. In ( D and E ), the differently colored bar charts represent the proportions of the same labels for various sample sizes (2, 3, 4 or 5 people). Source data are provided with this paper. The 15 metrics were utilized to assess the performance of the 2638 DIA raw files. For reliable annotation results, 21 expert raters were recruited from 10 different institutions, including instrument operators and application engineers from manufacturers. Each DIA file underwent an independent review by four or five experts possessing extensive hands-on experience with their respective MS instruments. More details are provided in Fig. 5B (/articles/s41467-024-54871-1#Fig5) . The DIA raw files were evaluated by raters and categorized as either qualified or unqualified based on the 15 metrics.