In the task of anomaly detection in modern timedomain photometric surveys the primary goal is to identify astrophysically interesting rare and unusual objects among a large volume of data. Unfortunately artifacts  such as plane or satellite tracks bad columns on CCDs and ghosts  often constitute significant contaminants in results from anomaly detection analysis. In such contexts the Active Anomaly Discovery AAD algorithm allows tailoring the output of anomaly detection pipelines according to what the expert judges to be scientifically interesting. We demonstrate how the introduction realbogus scores obtained from a machine learning classifier improves the results from AAD. Using labeled data from the SNAD ZTF knowledge database we train four realbogus classifiers XGBoost CatBoost Random Forest and Extremely Randomized Trees. All the models perform realbogus classification with similar effectiveness achieving ROCAUC scores ranging from 0.93 to 0.95. Consequently we select the Random Forest model as the main model due to its simplicity and interpretability. The Random Forest classifier is applied to 67 million light curves from ZTF DR17. The output realbogus score is used as an additional feature for two anomaly detection algorithms static Isolation Forest and AAD. While results from Isolation Forest remained unchanged the number of artifacts detected by the active approach decreases significantly with the inclusion of the realbogus score from 27 to 3 out of 100. We conclude that incorporating the realbogus classifier result as an additional feature in the active anomaly detection pipeline significantly reduces the number of artifacts in the outputs thereby increasing the incidence of astrophysically interesting objects presented to human experts.
['Astrophysics - Instrumentation and Methods for Astrophysics']
Realbogus scores for active anomaly detection
{'Real-bogus scores for active anomaly detection': "T. A. Semenikhin a, ∗ , M. V. Kornilov a,b , M. V. Pruzhinskaya c , A. D. Lavrukhina a , E. Russeil c , E. Gangler c , E. E. O. Ishida c , V. S. Korolev d , K. L. Malanchev e,f , A. A. Volnova g,a , S. Sreejith h , (The SNAD team)  \na Lomonosov Moscow State University, Sternberg Astronomical Institute, Universitetsky pr. 13, Moscow, 119234, Russia  \nb National Research University Higher School of Economics, 21 / 4 Staraya Basmannaya Ulitsa, Moscow, 105066, Russia  \nc Universit'e Clermont Auvergne, CNRS / IN2P3, LPCA, Clermont-Ferrand, 63000, France  \nd Independent researcher,  \ne McWilliams Center for Cosmology and Astrophysics, Department of Physics, Carnegie Mellon University, Pittsburgh, PA 15213, USA  \nf Department of Astronomy, University of Illinois at Urbana-Champaign, 1002 West Green Street, Urbana, IL 61801, USA  \ng Space Research Institute of the Russian Academy of Sciences (IKI), 84 / 32 Profsoyuznaya Street, Moscow, 117997, Russia  \nh Physics Department, University of Surrey, Stag Hill Campus, GU2 7XH, Guildford, UK", 'Abstract': 'In the task of anomaly detection in modern time-domain photometric surveys, the primary goal is to identify astrophysically interesting, rare, and unusual objects among a large volume of data. Unfortunately, artifacts - such as plane or satellite tracks, bad columns on CCDs, and ghosts - often constitute significant contaminants in results from anomaly detection analysis. In such contexts, the Active Anomaly Discovery (AAD) algorithm allows tailoring the output of anomaly detection pipelines according to what the expert judges to be scientifically interesting. We demonstrate how the introduction real-bogus scores, obtained from a machine learning classifier, improves the results from AAD. Using labeled data from the SNAD ZTF knowledge database, we train four real-bogus classifiers: XGBoost, CatBoost, Random Forest, and Extremely Randomized Trees. All the models perform real-bogus classification with similar e ff ectiveness, achieving ROC-AUC scores ranging from 0.93 to 0.95. Consequently, we select the Random Forest model as the main model due to its simplicity and interpretability. The Random Forest classifier is applied to 67 million light curves from ZTF DR17. The output real-bogus score is used as an additional feature for two anomaly detection algorithms: static Isolation Forest and AAD. While results from Isolation Forest remained unchanged, the number of artifacts detected by the active approach decreases significantly with the inclusion of the real-bogus score, from 27 to 3 out of 100. We conclude that incorporating the real-bogus classifier result as an additional feature in the active anomaly detection pipeline significantly reduces the number of artifacts in the outputs, thereby increasing the incidence of astrophysically interesting objects presented to human experts.  \nKeywords: Astronomy data analysis, Classification, Outlier detection, Sky surveys', '1. Introduction': "Modern sky surveys o ff er the opportunity to automatically observe a vast number of astrophysical objects across different classes. For instance, the Zwicky Transient Facility (ZTF; Bellm et al. 2019) generates about 2 terabytes of data per night, and the upcoming Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST; LSST Science Collaboration et al. 2009) is expected to increase this amount by an order of magnitude. When dealing with such large volumes of data, machine learning (ML) methods are unavoidable, especially in tasks such as classification and anomaly detection (e. g., Baron 2019; Ishida 2019; Chen et al. 2020; Malik et al. 2022; Pruzhinskaya et al. 2023). However, the task of anomaly detection is particularly challenging due to the inherent nature of searching for statistical deviations in the data, which often results in a significant number of outliers being artifacts of automatic image processing, such as defocusing, bad columns, bright star  \nEmail address: semenikhintimofey@gmail.com (T. A. Semenikhin)  \nspikes, ghosts, etc. For example, in Malanchev et al. (2021), 68 percent of outliers found by anomaly detection algorithms in the third ZTF data release (DR) were identified as bogus light curves. Around 80 instances of data artifacts were discovered as unwanted contaminants during the search for gravitational self-lensing binaries with ZTF (Crossland et al., 2023). Similarly, S'anchez-S'aez et al. (2021) encountered an overwhelming amount of bogus among the 8809 anomalies selected while searching for Changing-state AGNs in the ZTF DR5.  \nGiven limited human resources, it is crucial to avoid spending time on non-astrophysical candidates. Active anomaly detection algorithms partly address this issue by incorporating feedback from human experts, which helps fine-tuning the algorithm and reduces the number of astrophysically uninteresting objects (Ishida et al., 2021). However, artifacts, due to their diversity, can occupy di ff erent regions of the parameter space, and even active algorithms cannot guarantee their elimination from outliers after several iterations.  \nOne way to approach this problem is to make a real-bogus classification and then reject all objects that the classifier con-  \nsidered as artifacts. For example, this was done for Nearby Supernova Factory (Aldering et al., 2002) by Bailey et al. (2007) and other time domain surveys such as The Palomar Transient Factory (PTF; Brink et al. 2013), the Dark Energy Survey (DES; Goldstein et al. 2015), the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS; Wright et al. 2015). Nowadays, neural network approaches are actively used for this purpose. For example, Weston et al. (2024) used di ff erence imaging and convolutional neural networks for real-bogus classification in the Asteroid Terrestrial-impact Last Alert System (ATLAS; Tonry et al. 2018). Acero-Cuellar et al. (2023) used data from DES and investigated the performance of convolutional neural networks without using template subtraction. Deep learning methods are also used on ZTF survey data in real-bogus classification tasks (e. g., Duev et al. 2019; CarrascoDavis et al. 2021; Semenikhin 2024).  \nThe SNAD team 1 has been working on the development and adaptation of anomaly detection techniques for astronomical data since 2018. In this work, we present and integrate into the SNAD pipeline a novel approach designed to reduce even further the number of artifacts presented to the expert in sequential iterations of the adaptive learning cycle. The idea involves the development of a real-bogus classifier whose predictions are used as an additional feature for each data instance. These enhanced data are then fed into the Active Anomaly Discovery (AAD; Das et al. 2016; Ishida et al. 2021) algorithm. It has been demonstrated that with this approach, AAD learns significantly faster to avoid returning artifacts to the expert, thus saving the expert's time.  \nThe article is structured as follows: Section 2 describes the data used for training and testing. Section 3 is dedicated to the construction of real-bogus classifiers. Section 4 presents a comparison of the performance of anomaly detection methods with and without real-bogus scores. Finally, Sections 5 and 6 contain the discussion and conclusions, respectively.", '2. Data': 'The classifier is developed to increase the e ffi ciency of anomaly detection in the data from the Zwicky Transient Facility 2 , an automated wide-field sky survey. ZTF is conducted at the Palomar Observatory in California, USA, using a 1.26-m Samuel Oschin telescope with a field of view of approximately 47 square degrees. The survey operates in three photometric passbands ( zg , zr , zi ). Data from ZTF are provided in two formats: alerts and data releases. Alerts are issued in real-time, while data releases contain light curves of variable objects over the entire observation period. In this work, we use the zr -band light curves from the data releases only.', '2.1. Real-bogus dataset': "The anomaly knowledge base (AKB; see Section 4 in Malanchev et al. 2023) was created by experts from the  \nGhostOID 721206200037497 HMJD 58218.41947  \n<!-- image -->  \nSN  \n<!-- image -->  \nOID 722213400026577 HMJD 58216.41375  \n<!-- image -->  \nOID 634205100045765 HMJD 58249.42073  \n<!-- image -->  \nGalaxy  \nOID 714207400011552 HMJD 59375.21720  \nTrack  \nOID 587211200012088 HMJD 58226.44181  \n<!-- image -->  \nStar  \nOID 721213100007378 HMJD 59425.20617Figure 1: Object examples of di ff erent classes from ZTF data. Each image has size of 28 px × 28 px, 1 px = 1 . 012 '' .  \n<!-- image -->  \nSNAD team during the scrutiny of the outputs of anomaly detection algorithms run on various ZTF DRs. Each ZTF object, identified by the object ID (OID), is classified according to the SNAD classification scheme (see examples in Fig. 1) based on extensive analysis, including visual screening, literature review, photometric model fitting, and follow-up observations. As of July 2023, the database comprised 3311 objects: 1646 artifacts and 1665 astrophysical objects of di ff erent types. We use this dataset, referred to as the real-bogus dataset , to train and test real-bogus classifiers.", '2.2. Target dataset': 'For the anomaly detection task we used light curves from ZTF DR17 within the MJD interval ∈ [59184; 59524] subjected to the following cuts: at least 100 observations and | b | > 15 · , where b is the galactic latitude. This resulted in approximately 67 millions OIDs that make up the target dataset . We ran the Isolation Forest and the Active Anomaly Discovery algorithm on this dataset.', '2.3. Light curve features': 'For each light curve we extracted 54 features using the lightcurve package described at Malanchev et al. (2021). Some of them have a simple form, such as flux amplitude or mean value. Others have a more complex dependency with the light curve shape, such as skewness or optimized parameters of the Bazin function from Bazin et al. (2009). We name this feature set initial .', '3. Real-bogus classifiers': "As classifiers, we consider 4 of the most popular models for working with tabular data. Two of them are based on gradient boosting (Friedman, 2001): XGBoost (Chen and Guestrin, 2016) and CatBoost (Dorogush et al., 2018). The others have simpler architectures and interpretability: Random  \nTable 1: Parameters for real-bogus classifiers optimized with Optuna . † This parameter was not varied and set equal to 1.  \nForest (Breiman, 2001) and Extremely Randomized Trees (ExtraTrees, Geurts et al. 2006). The models' parameters were tuned using Optuna 3 framework (see Table 1). When selecting parameters, the algorithm aimed to maximize accuracy. We choose this metric so it does not coincide with our main validation metric (Receiver Operating Characteristic - Area Under the Curve (ROC-AUC)).  \nWeuse objects and corresponding labels (0 stands for an artefact, 1 - astrophysical object) from the real-bogus dataset for training classifiers. The real-bogus classifier works in the following way: we feed light curve features to one of the classical machine learning method listed above, which returns a realbogus score in the range [0 , 1]. The closer the number is to 0, the more confident the model is that the light curve represents an artefact.  \nSince the real-bogus dataset is relatively small (3311 objects), we use a k-fold ( k = 5) cross-validation when training the classifiers. In this method, we divide the available data into k subsets. Then, a model is trained on union of k -1 subsets and evaluated on the remaining one, called the test set. This process is repeated k times, each time using a di ff erent subset  \nFigure 2: ROC-curve examples of trained real-bogus classifiers for one of the cross-validation splits.  \n<!-- image -->  \nTable 2: Real-bogus classifiers results. The metric values are averaged over 5 test sets, as well as the standard deviation.  \nas the test set. As a result, k model quality metric estimates are obtained, which are then averaged to compute the final estimate. This approach allows us to avoid overfitting and to obtain a more reliable e ffi ciency estimate.", '3.1. Validation of classifiers': 'In order to determine the predicted class of an object, it is necessary to choose a threshold for the real-bogus score. However, in this work, the prediction of the classifier serves as an additional source of information for the Active Anomaly Discovery algorithm (see Section 4). In this case, the determination of a specific threshold is the prerogative of the AAD algorithm based on expert feedback. Moreover, technically, such a threshold may vary for di ff erent regions of the feature space. Therefore, we chose ROC-AUC as a main quality metric for the real-bogus classifiers. As additional quality metrics, we also considered Accuracy and F1-score with the standard threshold set to 0.5 as well as ROC-curves (see Fig. 2).  \nFrom Table 2, which presents the final values of the quality metrics averaged over the test sets, and Fig. 2, it can be seen that the 4 considered models achieve similar performances. Therefore, we select the Random Forest as the main model because it is the simplest and most interpretable algorithm. All the realbogus scores presented below were obtained using this model.', '3.2. Inference': 'Fig. 3 demonstrates the distributions of real-bogus score based on the Random Forest model. The histogram of predictions for objects from the real-bogus dataset, as expected, has a bimodal distribution, while the distribution for target dataset  \nresembles a bell curve. From this, we conclude that our classifier, trained on objects identified by SNAD experts as outliers in ZTF DRs, is capable of classifying such outliers. However, when the classifier see features of an object with a distribution not represented in the real-bogus dataset, it outputs a real-bogus score close to 0.5. This can be interpreted as the model being unsure about the class to which this object belongs. This reduces the likelihood of the classifier labeling an interesting anomaly as an artefact.', '4. Anomaly detection with real-bogus scores': "In the SNAD pipeline, anomaly detection is carried out using algorithms based on random trees. The input data consist of light curve features, based on which the algorithm ranks objects according to their anomaly score. Then, a reasonable sample of the most anomalous objects is provided to the expert for manual verification. The sample size (budget) is a hyperparameter, which is defined by expert.  \nIntegration of the real-bogus score into the anomaly detection algorithm is achieved by adding the classifier's prediction to the initial set of features (described in Section 2.3). We refer to the resulting set matrix as the augmented feature set , i.e., with realbogus score. Next, we feed this matrix to the anomaly detection algorithm.  \nBelow we test two anomaly detection algorithms and feature sets: Isolation Forest and Active Anomaly Detection, are applied on the initial feature set and on the augmented. Our goal is to understand how the addition of a new feature a ff ects the e ffi ciency of anomaly detection.", '4.1. Isolation forest': "Isolation Forest (IF; Liu et al. 2008) is an ensemble of random trees that allows for identifying anomalies within the data. The main assumption of this algorithm is that anomalies are isolated in the feature space from normal objects. The fewer random splits a tree needs to separate an object from the rest, the more anomalous it is considered. The anomaly score of the IF is inversely proportional to the average number of such splits across all trees in the ensemble.  \nIn this work, we use an IF with 100 trees, each using 256 objects for construction.  \nWe independently run the IF from the coniferest 4 Python library on both feature sets. Fig. 4 shows histograms of classifier prediction distributions for objects with the highest anomaly scores (for the first hundred most anomalous objects and for 5000). The blue histogram represents the result for the IF run on the initial feature set, while the red one shows outcomes from the augmented set. It can be seen that the histograms of the 100 most anomalous objects in both cases have a similar shape, with a prevalence of objects with high realbogus scores. However, the distributions for 5000 objects show  \nsignificant di ff erences: for the initial feature set, there are significantly more objects with low real-bogus scores, and conversely, for the augmented set, there are significantly more objects with high real-bogus scores. As a result, it can be concluded that the real-bogus classifier's predictions help decision trees in the IF to group artefacts together, so they become less anomalous. However, this e ff ect is noticeable when considering 5000 anomaly candidates, making this method ine ffi cient for practical applications.  \nWe manually verified 100 objects with the highest anomaly scores for both sets of features. The result is the same - in both cases, out of 100 objects, 13 are artefacts.", '4.2. Active Anomaly Discovery': "Active Anomaly Discovery (Das et al., 2016, 2017) is an algorithm based on the IF that adapts to expert actions on the fly. During the model initialization stage, an IF is trained. Then, the object with the highest anomaly score is sent for review to an expert who provides feedback on its classification (anomalous or nominal). The obtained feedback is used to update the base model by changing its parameters. Subsequently, the original data is fed into the modified IF. These steps are repeated until a certain number of reviewed objects (budget) is reached. This algorithm is flexible to changes in the expert requirements, as the user chooses what to consider as an anomaly and what to consider as a normal object (see example usage in Ishida et al. 2021; Pruzhinskaya et al. 2023).  \nAs with IF, AAD 5 was independently applied to the data represented by the initial feature set and the augmented. During the operation, the expert classified artefacts with the label 'NO' (0), which corresponds to nominal samples, and with the label 'YES' (1) - to indicate real astrophysical events. For both feature sets, 100 objects were checked. The ratio of artefacts to the total number of scrutinized objects is 27 / 100 for the initial feature set, and 3 / 100 for the augmented feature set.  \nThe dependency of the fraction of anomalies on the number of candidates verified by the expert is presented in Fig. 5. It can be seen that AAD, receiving input data represented by the initial feature set (blue line), returns significantly more artefacts for expert verification compared to AAD trained on data with real-bogus score (red line). Therefore, it can be concluded that the augmented feature set accelerates the training of the AAD model compared to the initial set.", '5. Discussion on the forest structure': 'To analyze the change in the structure of the IF when adding a new feature, we constructed IF leaf vectors z (Das et al., 2017). For instance x there is a vector z ∈ { 0 , 1 } N , where N is the number of leaves in the IF ensemble. Each component of z stands for particular leaf in IF, zi = 1 means that instance x fell into i-th leaf, while zi = 0 means the opposite. By construction P N i = 1 zi equals to number of trees in the IF.  \nFigure 3: Histograms of real-bogus classifier (Random Forest model) predictions. Left: objects from real-bogus dataset (3311 objects). Right: objects from target dataset (approximately 67 millions objects).  \n<!-- image -->', 'Isolation forest budget=100': 'initial feature set  \naugmented feature set  \nFigure 4: Distributions of real-bogus classifier (Random Forest model) predictions for two feature sets. Left: histogram for 100 objects with highest anomaly scores. Right: histogram for 5000 objects with highest anomaly scores.  \n<!-- image -->  \n8  \n7  \n6  \n5  \n4  \n3  \n2  \n1  \n0  \n0.0  \n0.2  \n0.4  \n0.6  \n0.8  \n1.0  \nReal-bogus score  \nDensity  \nFigure 5: Results of running AAD on two feature sets: the dependence of the anomalies fraction on the number of verified candidates.  \n<!-- image -->  \nWe computed z for each object from the real-bogus dataset represented by the initial and augmented feature sets. Then the confusion matrix can be estimated for every zi as if it was a real-bogus classifier prediction:  \nCM i = " TN i FP i FN i TP i # , (1)  \nwhere TN i is the number of real objects with zi = 0, FP i is the number of real objects with zi = 1, FN i is the number of artifact objects with zi = 0, TP i is the number of artifact objects with zi = 1. We use accuracy score:  \nAccuracy score i = TN i + TP i M , (2)  \n(where M - number of objects in the real-bogus dataset) to explore artifact sensibility of every single leaf. Essentially, this means that the higher the value of the accuracy score for a leaf, the more often artefacts are encountered in that leaf and the less frequently astrophysical objects are found there. For the majority of leafs, the score is expected be 0 . 5. Fig. 6 shows the accuracy scores for each leaf in the IF. According to this figure, it can be observed that with additional information from the real-bogus classifier, the IF occasionally creates leaves where artefacts are encountered much more frequently (Average score > 0 . 8). Conversely, the IF without classifier predictions does not have such leaves as it was expected.  \nAs discussed in Section 4.1, there is no di ff erence between two feature sets when running IF with budget = 100. This is due to the fact that IF calculates anomaly score as average random splits needed to separate an object from the rest (across the number of trees in the ensemble), so several leaves cannot significantly alter the result.  \nHowever, the presence of such leaves in the AAD structure significantly a ff ects the final result. This is because the anomaly score in AAD is not calculated as the average number of splits needed to isolate an object, but as a weighted sum where the weights are the optimized parameters of the leaves. Therefore, AAD can e ff ectively incorporate additional information from the classifier, which is confirmed in practice (see Fig. 5).', '6. Conclusions': 'In this work, we focus on reducing the fraction of artefacts presented to the expert by the AAD algorithm. We started from the SNAD knowledge database, which consists of 3311 objects represented by light curve features (Malanchev et al., 2023) and whose labels were assigned by experts after visual inspection. This data set was used as a training set for 4 di ff erent real-bogus classifiers. Comparing their performance metrics, we selected the Random Forest model, for which the ROCAUC = 0 . 94 ± 0 . 01. The classification scores obtained from this classifier were then used as an additional feature in the data input to two anomaly detection methods: static Isolation Forest and Active Anomaly Discovery. We found that additional information from the real-bogus classifier allows IF to identify feature subspaces where artefacts group together. However, the desired reduction in the fraction of artefacts is only observed with large budgets. Considering only the 100 objects with highest anomaly score, the artifact ratios are the same for both feature sets (with and without the real-bogus classification scores). Meanwhile, the rate to which AAD presents real objects to the expert is significantly accelerated by using a feature set containing the real-bogus score. The e ff ectiveness of this approach has been demonstrated in practice: 27 / 100 artefact ratio for initial feature set against 3 / 100 for augmented feature set.  \nThe approach proposed in this work can be applied to other classes of objects (e. g., supernovae vs. non-supernovae). We can train several binary classifiers targeting di ff erent classes and then add the resulting predictions as new features to the input data. The results shown in this work indicate that one can expect significant speed up the rate to which AAD adapts to the feedback from the user.  \nThe relevance of this work is highlighted by the forthcoming projects like LSST, which will generate an order of magnitude more data than ZTF. Filtering artefacts in such a data stream will become critically important.', 'Acknowledgements': "T. Semenikhin, M. Kornilov, A. Lavrukhina, and A. Volnova acknowledges support from a Russian Science Foundation grant 24-22-00233, https://rscf.ru/en/project/ 24-22-00233/ . Support was provided by Schmidt Sciences, LLC. for K. Malanchev. E. E. O. Ishida received support from Universit'e Clermont Auvergne through the grant AAP DRIF 2024 vague 1, 'Mobilit'es internationales sortantes'.", 'References': "Acero-Cuellar, T., Bianco, F., Dobler, G., Sako, M., Qu, H., Collaboration, T.L.D.E.S., 2023. What's the di ff erence? the potential for convolutional neural networks for transient detection without template subtraction. The Astronomical Journal 166, 115. URL: https://dx.doi.org/10.3847/ 1538-3881/ace9d8 , doi: 10.3847/1538-3881/ace9d8 .  \nAldering, G., Adam, G., Antilogus, P., Astier, P., Bacon, R., Bongard, S., Bonnaud, C., Copin, Y., Hardin, D., Henault, F., Howell, D.A., Lemonnier, J.P., Levy, J.M., Loken, S.C., Nugent, P.E., Pain, R., Pecontal, A., Pecontal, E., Perlmutter, S., Quimby, R.M., Schahmaneche, K., Smadja, G., Wood-Vasey, W.M., 2002. Overview of the Nearby Supernova Factory, in: Tyson, J.A.,  \n<!-- image -->  \nFigure 6: Accuracy score for each leaf in IF. Left: Leafs from IF trained on initial feature set. Right: Leafs from IF trained on augmented feature set. Both IFs have 1000 trees. Although for the majority of leafs accuracy score stay the same when adding new feature, a few of them become significantly better discriminator - larger number of extreme behavior in the right hand plot.  \n<!-- image -->  \nWol ff , S. (Eds.), Survey and Other Telescope Technologies and Discoveries, pp. 61-72. doi: 10.1117/12.458107 .  \nBailey, S., Aragon, C., Romano, R., Thomas, R.C., Weaver, B.A., Wong, D., 2007. How to Find More Supernovae with Less Work: Object Classification Techniques for Di ff erence Imaging. ApJ 665, 1246-1253. doi: 10.1086/ 519832 , arXiv:0705.0493 .  \nBaron, D., 2019. Machine Learning in Astronomy: a practical overview. doi: 10.48550/arXiv.1904.07248 , arXiv:1904.07248 .  \nBazin, G., Palanque-Delabrouille, N., Rich, J., Ruhlmann-Kleider, V., Aubourg, E., Le Guillou, L., Astier, P., Balland, C., Basa, S., Carlberg, R.G., Conley, A., Fouchez, D., Guy, J., Hardin, D., Hook, I.M., Howell, D.A., Pain, R., Perrett, K., Pritchet, C.J., Regnault, N., Sullivan, M., Antilogus, P., Arsenijevic, V., Baumont, S., Fabbro, S., Le Du, J., Lidman, C., Mouchet, M., Mour˜ao, A., Walker, E.S., 2009. The corecollapse rate from the Supernova Legacy Survey. A&A 499, 653-660. doi: 10.1051/0004-6361/200911847 , arXiv:0904.1066 .  \nBellm, E.C., Kulkarni, S.R., Graham, M.J., Dekany, R., Smith, R.M., Riddle, R., Masci, F.J., Helou, G., Prince, T.A., Adams, S.M., Barbarino, C., Barlow, T., Bauer, J., Beck, R., Belicki, J., Biswas, R., Blagorodnova, N., Bodewits, D., Bolin, B., Brinnel, V., Brooke, T., Bue, B., Bulla, M., Burruss, R., Cenko, S.B., Chang, C.K., Connolly, A., Coughlin, M., Cromer, J., Cunningham, V., De, K., Delacroix, A., Desai, V., Duev, D.A., Eadie, G., Farnham, T.L., Feeney, M., Feindt, U., Flynn, D., Franckowiak, A., Frederick, S., Fremling, C., Gal-Yam, A., Gezari, S., Giomi, M., Goldstein, D.A., Golkhou, V.Z., Goobar, A., Groom, S., Hacopians, E., Hale, D., Henning, J., Ho, A.Y.Q., Hover, D., Howell, J., Hung, T., Huppenkothen, D., Imel, D., Ip, W.H., Ivezi'c, ˇ Z., Jackson, E., Jones, L., Juric, M., Kasliwal, M.M., Kaspi, S., Kaye, S., Kelley, M.S.P., Kowalski, M., Kramer, E., Kupfer, T., Landry, W., Laher, R.R., Lee, C.D., Lin, H.W., Lin, Z.Y., Lunnan, R., Giomi, M., Mahabal, A., Mao, P., Miller, A.A., Monkewitz, S., Murphy, P., Ngeow, C.C., Nordin, J., Nugent, P., Ofek, E., Patterson, M.T., Penprase, B., Porter, M., Rauch, L., Rebbapragada, U., Reiley, D., Rigault, M., Rodriguez, H., van Roestel, J., Rusholme, B., van Santen, J., Schulze, S., Shupe, D.L., Singer, L.P., Soumagnac, M.T., Stein, R., Surace, J., Sollerman, J., Szkody, P., Taddia, F., Terek, S., Van Sistine, A., van Velzen, S., Vestrand, W.T., Walters, R., Ward, C., Ye, Q.Z., Yu, P.C., Yan, L., Zolkower, J., 2019. The Zwicky Transient Facility: System Overview, Performance, and First Results. PASP 131, 018002. doi: 10.1088/1538-3873/aaecbe , arXiv:1902.01932 .  \nBreiman, L., 2001. Random forests. Machine Learning 45, 5-32. URL: https://doi.org/10.1023/A:1010933404324 , doi: 10.1023/ A:1010933404324 .  \nBrink, H., Richards, J.W., Poznanski, D., Bloom, J.S., Rice, J., Negahban, S., Wainwright, M., 2013. Using machine learning for discovery in synoptic survey imaging data. MNRAS 435, 1047-1060. doi: 10.1093/mnras/ stt1306 , arXiv:1209.3775 .  \nCarrasco-Davis, R., Reyes, E., Valenzuela, C., Forster, F., Est'evez, P.A., Pignata, G., Bauer, F.E., Reyes, I., S'anchez-S'aez, P., Cabrera-Vives, G., Eyher-a  \ny, S., Catelan, M., Arredondo, J., Castillo-Navarrete, E., Rodr'ıguezMancini, D., Ruz-Mieres, D., Moya, A., Sabatini-Gacit'ua, L., Sep'ulvedaCobo, C., Mahabal, A.A., Silva-Farf'an, J., Camacho-I˜niguez, E., Galbany, L., 2021. Alert Classification for the ALeRCE Broker System: The Realtime Stamp Classifier. AJ 162, 231. doi: 10.3847/1538-3881/ac0ef1 , arXiv:2008.03309 .  \nChen, T., Guestrin, C., 2016. Xgboost: A scalable tree boosting system, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM. pp. 785794. URL: http://dx.doi.org/10.1145/2939672.2939785 , doi: 10. 1145/2939672.2939785 .  \nChen, X., Wang, S., Deng, L., de Grijs, R., Yang, M., Tian, H., 2020. The Zwicky Transient Facility Catalog of Periodic Variable Stars. ApJS 249, 18. doi: 10.3847/1538-4365/ab9cae , arXiv:2005.08662 .  \nCrossland, A., Bellm, E.C., Klein, C., Davenport, J.R.A., Kupfer, T., Groom, S.L., Laher, R.R., Riddle, R., 2023. A Pilot Search for Gravitational Self-Lensing Binaries with the Zwicky Transient Facility. doi: 10.48550/ arXiv.2311.17862 , arXiv:2311.17862 .  \nDas, S., Wong, W.K., Dietterich, T.G., Fern, A., Emmott, A., 2016. Incorporating expert feedback into active anomaly discovery, in: Proceedings of the IEEE International Conference on Data Mining, pp. 853-858.  \nDas, S., Wong, W.K., Fern, A., Dietterich, T.G., Siddiqui, M.A., 2017. Incorporating feedback into tree-based anomaly detection. arXiv:1708.09441 . Dorogush, A.V., Ershov, V., Gulin, A., 2018. Catboost: gradient boosting with categorical features support. arXiv:1810.11363 .  \nDuev, D.A., Mahabal, A., Masci, F.J., Graham, M.J., Rusholme, B., Walters,  \nR., Karmarkar, I., Frederick, S., Kasliwal, M.M., Rebbapragada, U., Ward, C., 2019. Real-bogus classification for the Zwicky Transient Facility using deep learning. MNRAS 489, 3582-3590. doi: 10.1093/mnras/stz2357 , arXiv:1907.11259 .  \nFriedman, J.H., 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics , 1189-1232.  \nGeurts, P., Ernst, D., Wehenkel, L., 2006. Extremely randomized trees. Machine Learning 63, 3-42. doi: 10.1007/s10994-006-6226-1 .  \nGoldstein, D.A., D'Andrea, C.B., Fischer, J.A., Foley, R.J., Gupta, R.R., Kessler, R., Kim, A.G., Nichol, R.C., Nugent, P.E., Papadopoulos, A., Sako, M., Smith, M., Sullivan, M., Thomas, R.C., Wester, W., Wolf, R.C., Abdalla, F.B., Banerji, M., Benoit-L'evy, A., Bertin, E., Brooks, D., Carnero Rosell, A., Castander, F.J., da Costa, L.N., Covarrubias, R., DePoy, D.L., Desai, S., Diehl, H.T., Doel, P., Eifler, T.F., Fausti Neto, A., Finley, D.A., Flaugher, B., Fosalba, P., Frieman, J., Gerdes, D., Gruen, D., Gruendl, R.A., James, D., Kuehn, K., Kuropatkin, N., Lahav, O., Li, T.S., Maia, M.A.G., Makler, M., March, M., Marshall, J.L., Martini, P., Merritt, K.W., Miquel, R., Nord, B., Ogando, R., Plazas, A.A., Romer, A.K., Roodman, A., Sanchez, E., Scarpine, V., Schubnell, M., Sevilla-Noarbe, I., Smith, R.C., Soares-Santos, M., Sobreira, F., Suchyta, E., Swanson, M.E.C., Tarle, G., Thaler, J., Walker, A.R., 2015. Automated Transient Identification in the Dark Energy Survey.  \nAJ 150, 82. doi: 10.1088/0004-6256/150/3/82 , arXiv:1504.02936 . Ishida, E.E.O., 2019. Machine learning and the future of supernova cosmology. Nature Astronomy 3, 680-682. doi: 10.1038/s41550-019-0860-6 , arXiv:1908.02315 .  \nIshida, E.E.O., Kornilov, M.V., Malanchev, K.L., Pruzhinskaya, M.V., Volnova, A.A., Korolev, V.S., Mondon, F., Sreejith, S., Malancheva, A.A., Das, S., 2021. Active anomaly detection for time-domain discoveries. A&A 650, A195. doi: 10.1051/0004-6361/202037709 , arXiv:1909.13260 .  \nLiu, F.T., Ting, K.M., Zhou, Z.H., 2008. Isolation forest, in: 2008 Eighth IEEE International Conference on Data Mining, pp. 413-422. doi: 10.1109/ ICDM.2008.17 .  \nLSST Science Collaboration, Abell, P.A., Allison, J., Anderson, S.F., Andrew, J.R., Angel, J.R.P., Armus, L., Arnett, D., Asztalos, S.J., Axelrod, T.S., Bailey, S., Ballantyne, D.R., Bankert, J.R., Barkhouse, W.A., Barr, J.D., Barrientos, L.F., Barth, A.J., Bartlett, J.G., Becker, A.C., Becla, J., Beers, T.C., Bernstein, J.P., Biswas, R., Blanton, M.R., Bloom, J.S., Bochanski, J.J., Boeshaar, P., Borne, K.D., Bradac, M., Brandt, W.N., Bridge, C.R., Brown, M.E., Brunner, R.J., Bullock, J.S., Burgasser, A.J., Burge, J.H., Burke, D.L., Cargile, P.A., Chandrasekharan, S., Chartas, G., Chesley, S.R., Chu, Y.H., Cinabro, D., Claire, M.W., Claver, C.F., Clowe, D., Connolly, A.J., Cook, K.H., Cooke, J., Cooray, A., Covey, K.R., Culliton, C.S., de Jong, R., de Vries, W.H., Debattista, V.P., Delgado, F., Dell'Antonio, I.P., Dhital, S., Di Stefano, R., Dickinson, M., Dilday, B., Djorgovski, S.G., Dobler, G., Donalek, C., Dubois-Felsmann, G., Durech, J., Eliasdottir, A., Eracleous, M., Eyer, L., Falco, E.E., Fan, X., Fassnacht, C.D., Ferguson, H.C., Fernandez, Y.R., Fields, B.D., Finkbeiner, D., Figueroa, E.E., Fox, D.B., Francke, H., Frank, J.S., Frieman, J., Fromenteau, S., Furqan, M., Galaz, G., GalYam, A., Garnavich, P., Gawiser, E., Geary, J., Gee, P., Gibson, R.R., Gilmore, K., Grace, E.A., Green, R.F., Gressler, W.J., Grillmair, C.J., Habib, S., Haggerty, J.S., Hamuy, M., Harris, A.W., Hawley, S.L., Heavens, A.F., Hebb, L., Henry, T.J., Hileman, E., Hilton, E.J., Hoadley, K., Holberg, J.B., Holman, M.J., Howell, S.B., Infante, L., Ivezic, Z., Jacoby, S.H., Jain, B., R, Jedicke, Jee, M.J., Garrett Jernigan, J., Jha, S.W., Johnston, K.V., Jones, R.L., Juric, M., Kaasalainen, M., Styliani, Kafka, Kahn, S.M., Kaib, N.A., Kalirai, J., Kantor, J., Kasliwal, M.M., Keeton, C.R., Kessler, R., Knezevic, Z., Kowalski, A., Krabbendam, V.L., Krugho ff , K.S., Kulkarni, S., Kuhlman, S., Lacy, M., Lepine, S., Liang, M., Lien, A., Lira, P., Long, K.S., Lorenz, S., Lotz, J.M., Lupton, R.H., Lutz, J., Macri, L.M., Mahabal, A.A., Mandelbaum, R., Marshall, P., May, M., McGehee, P.M., Meadows, B.T., Meert, A., Milani, A., Miller, C.J., Miller, M., Mills, D., Minniti, D., Monet, D., Mukadam, A.S., Nakar, E., Neill, D.R., Newman, J.A., Nikolaev, S., Nordby, M., O'Connor, P., Oguri, M., Oliver, J., Olivier, S.S., Olsen, J.K., Olsen, K., Olszewski, E.W., Oluseyi, H., Padilla, N.D., Parker, A., Pepper, J., Peterson, J.R., Petry, C., Pinto, P.A., Pizagno, J.L., Popescu, B., Prsa, A., Radcka, V., Raddick, M.J., Rasmussen, A., Rau, A., Rho, J., Rhoads, J.E., Richards, G.T., Ridgway, S.T., Robertson, B.E., Roskar, R., Saha, A., Sarajedini, A., Scannapieco, E., Schalk, T., Schindler, R., Schmidt, S., Schmidt, S., Schneider, D.P., Schumacher, G., Scranton, R., Sebag, J., Seppala, L.G., Shemmer, O., Simon, J.D., Sivertz, M., Smith, H.A., Allyn Smith, J., Smith, N., Spitz, A.H., Stanford, A., Stassun, K.G., Strader, J., Strauss, M.A., Stubbs, C.W., Sweeney, D.W., Szalay, A., Szkody, P., Takada, M., Thorman, P., Trilling, D.E., Trimble, V., Tyson, A., Van Berg, R., Vanden Berk, D., VanderPlas, J., Verde, L., Vrsnak, B., Walkowicz, L.M., Wandelt, B.D., Wang, S., Wang, Y., Warner, M., Wechsler, R.H., West, A.A., Wiecha, O., Williams, B.F., Willman, B., Wittman, D., Wol ff , S.C., Wood-Vasey, W.M., Wozniak, P., Young, P., Zentner, A., Zhan, H., 2009. LSST Science Book, Version 2.0. doi: 10.48550/arXiv.0912.0201 , arXiv:0912.0201 .  \nMalanchev, K., Kornilov, M.V., Pruzhinskaya, M.V., Ishida, E.E.O., Aleo, P.D., Korolev, V.S., Lavrukhina, A., Russeil, E., Sreejith, S., Volnova, A.A., Voloshina, A., Krone-Martins, A., 2023. The SNAD Viewer: Everything You Want to Know about Your Favorite ZTF Object. PASP 135, 024503. doi: 10.1088/1538-3873/acb292 , arXiv:2211.07605 .  \nMalanchev, K.L., Pruzhinskaya, M.V., Korolev, V.S., Aleo, P.D., Kornilov, M.V., Ishida, E.E.O., Krushinsky, V.V., Mondon, F., Sreejith, S., Volnova, A.A., Belinski, A.A., Dodin, A.V., Tatarnikov, A.M., Zheltoukhov, S.G., (The SNAD Team), 2021. Anomaly detection in the Zwicky Transient Facility DR3. MNRAS 502, 5147-5175. doi: 10.1093/mnras/stab316 , arXiv:2012.01419 .  \nMalik, A., Moster, B.P., Obermeier, C., 2022. Exoplanet detection using machine learning. MNRAS 513, 5505-5516. doi: 10.1093/mnras/ stab3692 , arXiv:2011.14135 .  \nPruzhinskaya, M.V., Ishida, E.E.O., Novinskaya, A.K., Russeil, E., Volnova, A.A., Malanchev, K.L., Kornilov, M.V., Aleo, P.D., Korolev, V.S., Krushinsky, V.V., Sreejith, S., Gangler, E., 2023. Supernova search with active learning in ZTF DR3. A&A 672, A111. doi: 10.1051/0004-6361/202245172 , arXiv:2208.09053 .  \nS'anchez-S'aez, P., Lira, H., Mart'ı, L., S'anchez-Pi, N., Arredondo, J., Bauer, F.E., Bayo, A., Cabrera-Vives, G., Donoso-Oliva, C., Est'evez, P.A., Eyheramendy, S., Forster, F., Hern'andez-Garc'ıa, L., Arancibia, A.M.M., P'erezCarrasco, M., Sep'ulveda, M., Vergara, J.R., 2021. Searching for Changingstate AGNs in Massive Data Sets. I. Applying Deep Learning and Anomalydetection Techniques to Find AGNs with Anomalous Variability Behaviors. AJ 162, 206. doi: 10.3847/1538-3881/ac1426 , arXiv:2106.07660 .  \nSemenikhin, T.A., 2024. Neural network architecture for artifacts detection in ZTF survey. Systems and Means of Informatics 34, 7079. URL: http://dx.doi.org/10.14357/08696527240106 , doi: 10. 14357/08696527240106 .  \nTonry, J.L., Denneau, L., Heinze, A.N., Stalder, B., Smith, K.W., Smartt, S.J., Stubbs, C.W., Weiland, H.J., Rest, A., 2018. ATLAS: A High-cadence All-sky Survey System. PASP 130, 064505. doi: 10.1088/1538-3873/ aabadf , arXiv:1802.00879 .  \nWeston, J.G., Smith, K.W., Smartt, S.J., Tonry, J.L., Stevance, H.F., 2024. Training a convolutional neural network for real-bogus classification in the ATLAS survey. RAS Techniques and Instruments 3, 385-399. doi: 10. 1093/rasti/rzae027 .  \nWright, D.E., Smartt, S.J., Smith, K.W., Miller, P., Kotak, R., Rest, A., Burgett, W.S., Chambers, K.C., Flewelling, H., Hodapp, K.W., Huber, M., Jedicke, R., Kaiser, N., Metcalfe, N., Price, P.A., Tonry, J.L., Wainscoat, R.J., Waters, C., 2015. Machine learning for transient discovery in Pan-STARRS1 di ff erence imaging. MNRAS 449, 451-466. doi: 10.1093/mnras/stv292 , arXiv:1501.05470 ."}