Panopticon: a novel deep learning model to detect single transit events with no prior data filtering in PLATO light curves
H. G. Vivien M. Deleuil N. Jannsen J. De Ridder D. Seynaeve M. -A. Carpine Y. Zerah








Transit detection in unfiltered PLATO light curves






Aix Marseille Univ, CNRS, CNES, Institut Origines, LAM, Marseille, France
 hugo.vivien@lam.fr Institute for Astronomy, KU Leuven, Celestijnenlaan 200D bus 2401, 3001 Leuven, Belgium AIM, CEA, CNRS, Université Paris-Saclay, Université Paris Diderot, Sorbonne Paris Cité, 91191 Gif-sur-Yvette, France


Vivien et al.





 





To prepare for the analyses of the future PLATO light curves, we develop a deep learning model, , to detect transits in high precision photometric light curves. Since PLATO's main objective is the detection of temperate Earth-size planets around solar-type stars, the code is designed to detect individual transit events. The filtering step, required by conventional detection methods, can affect the transit, which could be an issue for long and shallow transits. To protect transit shape and depth, the code is also designed to work on unfiltered light curves.

The  model is based upon the Unet family architectures, able to more efficiently extract and combine features of various scale length, leading to a more robust detection scheme. We trained the model on a set of simulated PLATO light curves in which we injected, at pixel level, either planetary, eclipsing binary, or background eclipsing binary signals. We also include a variety of noises in our data, such as granulation, stellar spots or cosmic rays. We then assessed its capacity to detect transits in a separate dataset.

The approach is able to recover 90% of our test population, including more than 25% of the Earth-analogs, even in the unfiltered light curves. The model also recovers the transits irrespective of the orbital period, and is able to retrieve transits on a unique event basis. These figures are obtained when accepting a false alarm rate of 1%. When keeping the false alarm rate low (<0.01%), it is still able to recover more than 85% of the transit signals. Any transit deeper than ∼180ppm is essentially guaranteed to be recovered.

This method is able to recover transits on a unique event basis, and does so with a low false alarm rate. Due to the nature of machine learning, the inference time is minimal; around 0.2 s per light curve of 126 720 points. Thanks to light curves being one-dimensional, model training is also fast, on the order of a few hours per model. This speed in training and inference, coupled to the recovery effectiveness and precision of the model make it an ideal tool to complement, or be used ahead of, classical approaches.



: a novel deep learning model to detect single transit events with no prior data filtering in PLATO light curves
    
    H. G. Vivien1 ^https://orcid.org/0000-0001-7239-6700
    < g r a p h i c s > M. Deleuil1 ^https://orcid.org/0000-0001-6036-0225
    < g r a p h i c s > N. Jannsen2 ^https://orcid.org/0000-0003-4670-9616
    < g r a p h i c s > J. De Ridder2 ^https://orcid.org/0000-0001-6726-2863
    < g r a p h i c s > D. Seynaeve2 ^https://orcid.org/0000-0002-0731-8893
    < g r a p h i c s > M.-A. Carpine3 Y. Zerah1 ^https://orcid.org/0000-0003-1786-7367
    < g r a p h i c s >

    Received Month dd, yyyy; accepted Month dd, yyyy
=============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================







§ INTRODUCTION

sections/introduction.tex



§ DEEP LEARNING MODEL


In this paper, we present a DL model able to identify transit signals in a light curve. This is done by localizing the position of transit events. We do this in the context of the forthcoming ESA's PLATO mission, designed to determine the frequency of Earth-sized planets orbiting Sun-like stars. We opt for a classifier approach, rating the probability that a transit is occurring at each point of a light curve. This allows our approach to retrieve an arbitrary number of transits in a light curve, including the case of mono-transits. Additionally, we rely on the DL ability to extract and classify features to properly identify the events to bypass the filtering process.



 §.§ Architectures


We implement a custom 1-dimensional version of the Unet family: Unet, Unet++, Unet3+ <cit.>. This architecture is a type of fully convolutional neural network, that adds successive upsampling layers to the usual contracting networks. By combining the features extracted from the contracting during the upsamling process, the model can yield a high resolution output. For our light curves, the output generated acts as a one-to-one map of the input, where each point is classified individually based on neighboring context.

A Unet model can be seen as an auto-encoder with skip connections between the layers of the encoder part and the decoder part. The encoder extracts contextual information from the input, while the decoder builds the output point by point. During the encoding process the input is iteratively down-sampled, allowing a fixed-size kernel to extract information over a larger window at each step. Then, the decoder iteratively upsamples the output of the encoder, combining it with the features previously extracted at various timescales. The output of the decoder is therefore a segmentation map covering the input point-to-point, allowing for precise localization of the object of interest in the input light curve. Besides, a DL model with skip connections is beneficial, as they have been shown to increase training speed, and also allow for deeper network <cit.>.

Because this approach is a point-wise detection, it presents a few advantages. First and foremost, it allows any transit in a light curve to be detected individually, as points are classified based on local context. Second, we can extract the T_0 and duration for an arbitrary number of transits in a given light curve. Third, because the output yields a probability, it is possible to define the confidence level depending on the required certainty to extract the TCEs. Finally, because DL builds an internal noise model, there is no need for prior filtering of the light curves. Bypassing this step ensures that no shallow signal will be removed by mistake, and simplifies the detection process significantly. The theoretical detection of a transit in a light curve by our model is illustrated in Fig. <ref>, and the models architectures are given in Figs. <ref>, <ref> and <ref>.





 §.§ Implementation


graphs/conv_block

The three variants, Unet, Unet++ and Unet3+ are illustrated in Figs. <ref>, <ref> and <ref>, respectively. The encoder and decoder can be seen as a series of nodes, each either extracting features or combining them, respectively. Each node is built upon a basic convolution block, illustrated in Fig. <ref>. This block consists of three base operations: a convolution, a batch normalization and a rectified linear unit. An additional, optional, drop-out layer can be included at the beginning of the block. The node of the backbone make use of either two consecutive blocks in the cases for Unet and Unet++, or a single block for Unet3+.





































We identify the nodes of the model as x^i,j. The index i corresponds to the depth of the node in the network. It increases with each downsample operation, and decreases with each upsample. Conversely, index j tracks number of upsampling steps to reach a given node. We can therefore easily identify every node in the models. The backbone of the models corresponds to nodes x^i,0. Reciprocally, the decoder is made up of the nodes x^i,j>0. The backbone is common to all model and can be computed using:


    x^i,0 = 𝒩( x^𝑖𝑛𝑝𝑢𝑡),     i=0 
    
            𝒩(𝒟( x^i-1,0)),     i>0


where 𝒩 is the operation assigned to the node using the default block, described above. 𝒟 corresponds to the downsampling operation (Max Pooling; selecting the maximum value within a certain kernel). In this case, the kernel is set to a length of 2, and results in halving the resolution of the input at each level. Additionally, the number of feature channels also increases at each level. The decoders of each model can then be computed as follow:


    x^i,j>0 = 𝒩([ x^i,0, 𝒰(x^i+1,j-1) ]),    Unet
    𝒩([ [x^i,k]_k=0^j-1, 𝒰(x^i+1,j-1) ]),    Unet++
    𝒩([ [𝒩(𝒟(x^k,0))]_k=0^i-1, 
    𝒩(x^i,0), 
    [𝒩(𝒰(x^0,k))]_k=0^j-1]),    Unet3+


where 𝒟 remains the downsampling operation, and 𝒰 is the upsampling operation (Transpose Convolution for Unet and Unet++ or Upsample for Unet3+). The upsampling operation is setup so that it upsamples the data to the same length as the target node. Anything contained within [] is concatenated feature-wise.



The kernel size of the convolution operation is key on two aspects; (i) the number of trainable parameters and (ii) the coverage of the signal it offers. A larger kernel can increase the quality of the features identification, at the cost of longer training time. Additionally, the kernel must be large enough to encompass recognizable features within the signal. To achieve a good balance between feature quality, training time and feature coverage, we make substantial use of kernel dilation:


    k_s = k_t + (k_t - 1)(d - 1)


where k_s is the total length of the kernel, k_t is the number of active parameters in the kernel, and d is the dilation factor, namely, the spacing between active points in the kernel. For a default kernel where all active points are next to each other, the dilation factor is 1. This allows us to increase the size of the kernel for a fixed number of trainable parameters, at the cost of a lower resolution per feature. Typical kernel size for each architecture is shown in Table <ref>, for a constant depth of four and the number of initial feature maps set to eight. The number of learnable kernel parameters has a strong impact, while each model appears fairly similar. The Unet3+ version displays the smallest number of parameters for a given number of parameters, showing the advantage of the full skip connection over the nested skip connections of Unet++.

As described above, the goal of the model is to identify transit events directly within the stellar noise. We therefore first limit the model to a binary classification scheme, distinguishing two classes: "continuum" and "event". Each point in the light curve is assigned a likelihood score to belong to either the continuum (0) or an event (1). To retrieve the classification, we need to define a threshold to separate the two classes. Given the intrinsic class imbalance present within our data, due to the short nature of the transits compared with that of the light curve, it is not guaranteed that setting the threshold to 0.5 will yield the best results <cit.>. To more effectively constrain the best threshold value, we evaluate the performance of the model for values ranging from 0.05 to 0.95.

Finally, to compare the performances of the model, we train multiple models over a wide range of parameters, leading to the comparison of multiple versions of the model. This allows to compare the aforementioned kernel sizes and coverage of the features. We limit the model to use a binary cross entropy (BCE) loss function, which has proven effective. We use the AdamW optimizer, with γ=0.001, β_1=0.9 and β_2=0.999. We set the feature dropout rate to 10%.



§ DATASET AND ENVIRONMENT



 
To prepare a realistic dataset, tailored for the PLATO instrument, while controlling the astrophysics content of the light curves, we took advantage of the mission end-to-end camera simulator  <cit.>.  is developed to generates accurate and realistic simulated images to be received from the PLATO satellite. It includes a wide range of instrumental noise sources at different levels: platform, camera, with realistic PSFs, and detector. With stars that can be observed by a variable number of cameras and the complexity of the instrument, which accommodates 26 cameras on a single optical bench, to control the observation conditions for a given star, we used 's toolkit, called , which, based on the PLATO input catalog <cit.>, enables the simulator to be programmed in a friendly way. In addition to a realistic representation of the instrument, the other advantage of using , is that the signal is injected at pixel level. While this is not fundamental to test the mere detection, this is central for later identify our capacity to sort false positives generated by background eclipsing binaries, and bona fide planets.

To build our simulated dataset, we chose stars identified in the PIC as potential targets for the prime sample (P1; main sequence stars with V_mag<11), as the signal-to-noise ratio (SNR) of these stars enables a detection of an Earth-like planet. We then simulate those stars, including various astrophysical signals:


    
  - stellar activity effects that include granulation, stochastic oscillations and stellar spots
    
  - exoplanet transits, simulated with  <cit.> 
    
  - eclipsing binaries, simulated with  <cit.>


We can thus simulate a target combining all of these effects to produce either a transiting planet in front of an active star, or an eclipsing binary on the target, or an eclipsing binary on a nearby contaminant. All the physical characteristics used to generate the signal (masses, radii, effective temperature, orbital period, ephemerides, eccentricity, rotation period of the star, pulsation frequency, etc.) are also documented and saved.

The prime sample stars can potentially be observed by up to 24 cameras, the simulator generates the resulting signals and light curve, for each camera individually. We include all main effects that are currently implemented in , including the photometry module. At the time when the dataset was generated, only on-board algorithms were implemented by this module. This means that once the full processing chain is complete, the flux is extracted at pixel level using optimal aperture photometry <cit.>. We underline that the photometry for bright stars will eventually be derived by PSF fitting. This prevents us from making use of centroids for the targets. However, since we have control over the simulation, this is a great baseline to evaluate the performance of the method. To reduce the computational cost, the simulations were not performed on a complete PLATO field of view (i.e. simulating full-frame CCD images) but star per star, on a CCD subfield of 10×10 pixels. We also choose to reduce the cadence from the nominal 25 sec to 1 min and limit the simulations to the first four quarters, to cover a one year time span. We underline our objective was not to assess the performances of the instrument but to test the ability of our software to detect transit-like events. Depending on the type of simulation (planetary transit, eclipsing binary, number of contaminants...), and the version of the simulator, the computation time for a single target, on one quarter and for one camera, takes on the order of ≃ 12 minutes for version 3.6 of , while previous versions took around ≃ 7 minutes. Finally, still in an effort to save computational resource, we decided to adapt the simulation to the orbital period of the transiting body, and did not generate light curves on a given quarter when no transit is expected to occur. As a result, the number of quarters for a given star and a given astrophysical signal is not constant, but is tailored to the orbital period of the transit signal.

Table <ref> gives the summary of the number of the different astrophysical signals that were used in this study. Taking into account the fact that simulations cover one year time span, and that we forego quarters where no transit is present, we end up with a total of 16 094 quarters that were treated as independent light curves. This is all the more relevant as the periodic nature of transits does not come into play in our approach, and that the light curves were not corrected from any trend, such as instrument aging, or even cosmic impacts. We show the resulting distribution of radii and period of the transiting bodies simulated in Fig. <ref>. 



We further filter the dataset by removing edge cases where, due to numerical errors, the transits were not visible in the quarters. We also truncate the dataset to remove cases where non physical parameters were used to generate light curves. For instance, we remove cases where the stellar radius >2.5 , or where the transit depth is <50 ppm. This leaves us with 14 594 light curves, that we randomly split into two datasets; 85% training and 15% validation, that is 12 405 and 2 189 quarters, respectively. The final counts of signals in the dataset used is shown in the right column of Table <ref>.



§ PERFORMANCES


Evaluation of the performance of the model can be done in two ways. First, directly evaluating the raw output compared to the desired label. Second, assessing the ability of the model to detect transit events, or lack thereof. The former is achieved by computing conventional metrics, such as precision, recall, average precision, F_1 score and Jaccard score (or intersection over union; IOU). The later is done by comparing the positions of the ground truths of the events to the predicted positions by the model. While the direct approach allows a straight-forward evaluation of the model, it also doesn't reflect its actual ability to detect transits. We therefore focus our estimates on the ability of the model to recover transits, as well as its false positive rate (FAR). We deem a transit to be successfully recovered if an overlap between the prediction and the ground truth exists. The FAR is defined as the fraction of false positives to the total number of predicted event.

Models are trained on A40 GPUs in the in-house cluster of the laboratory. We trained a total of 16 models using the Unet3+ architecture, expected to perform the best. We test multiple initial kernel lengths and trainable parameters, using 4 or 8 initial features, and did this up to 70 training epochs, using batches of 40 light curves per training pass. To find the best performing versions of the models, we check the recovery and FAR performance on the last 20 epochs. We explored two options: a conservative approach that takes the model that has the lowest FAR at the 0.95 confidence threshold, and finding the version of the models that retrieve the most planets for a constant FAR of 1%. We show the summary of the parameters, and their associated results, in Table <ref>.

When taking the models in their conservative regime, we find that our models are able to retrieve more than 80% of our test population, and that with a FAR under 0.1%, less than 1 false positive for 1 000 predictions. When fixing the FAR to 1%, we find that we are able to retrieve 90% of the planets in our test dataset. These performances demonstrate that this approach is not only viable, but beneficial. The inference mechanism is very fast (∼0.2 seconds per light curve on a CPU), allowing for processing large amounts of data, which will be the case of PLATO. In this case, keeping the number of false positives small is key to enabling rapid and accurate processing of the vast amount of light curves.



As highlighted in Table <ref>, we consider three models that offer the best performances. Model A retrieves the largest fraction of the test population, model B yields a FAR of less than 0.01% while still recovering more than 85% of the planets, and finally model C provides a solid compromise between recovery and FAR. We use these models as an illustration for our approach on the population of our dataset. We show in Fig. <ref> the effectiveness of model C at 1% FAR at recovering the planets in our test population. The limiting factor for detection that emerges is the depth of the transits, and their associated SNR. We here compute the SNR after <cit.>:


    SNR = δ/σ_CDPP√(n_tr· t_dur/3hr)




where δ is the depth of the transit, σ_CDPP is the combined differential photometric precision, n_tr is the number of observed transits and t_dur the transit duration. While the recovery rate noticeably drops for depths lower than ∼150 ppm (SNR of ∼15), Earth-analogs are detectable by the model. Fig. <ref> (c) shows the depths of the events where the expected Earth depth is highlighted as a black vertical line, and neighboring planets are recovered at a rate between 25–33%. Additionally, the duration of transits are found to have little impact on the recovery rate (panel d), and crucially, the orbital period also has no impact on transit recovery (panel e). This holds true even for planets with orbital periods longer than a single quarter, indicating that transits are indeed identified on a unique event basis. We therefore find that our approach should be be able to identify at least 25% of the Earth-analogs robustly.

We also subsequently train the Unet and Unet++ architectures to compare their performances relative to the best Unet3+ versions. We find that these alternative models perform slightly worse. Namely, we find that the recovery is lower at equal FAR, especially in the small planets regime. We therefore limit our analysis to the pest-performing Unet3+ models.



To better highlight the performances of the models, we illustrate in the top panel of Fig. <ref> the trade-off between the recovery rate and the FAR in a receiver operating characteristic (ROC). We show the three selected models and their compromise, identifying the FAR selected in Table <ref>. We see for each case that the number of planets recovered increases with the FAR. Importantly, we find that even for the lowest possible FAR, here model B with <0.01%, a sizable 85.81% of the population is successfully recovered. The lower panel of Fig. <ref> illustrates the recovery for various depths of transits. It illustrates clearly that the recovery rate rapidly rises above ∼180 ppm, essentially reaching 100% (as also visible in panel c of Fig. <ref>, for model C).

We illustrate the detection of an Earth-analog signal (R_P=1.11 R_⊕, R_S=1.23 R_⊙, δ=83.16 ppm) in Fig. <ref>, using model C. This unique event is recovered with a confidence level of more than 0.65, making this planet detected at a corresponding FAR of less than 0.3%. While not strictly equivalent to the false alarm probability, it sufficiently analogous to give insight on the likelihood that this event is a true positive.





§ DISCUSSION


sections/discussion.tex




      HV and MD acknowledge funding from the Institut Universitaire de France (IUF) that made this work possible. This research made use of the computing facilities operated by the CeSAM data center at the LAM, Marseille, France.













aa









§ MODEL ARCHITECTURES



graphs/unet


graphs/unet++


graphs/unet3+


