Forecast of Daily Major Flare Probability Using Relationships between Vector Magnetic Properties and Flaring Rates
Daye Lim Yong-Jae Moon Jongyeob Park Eunsu Park Kangjin Lee Jin-Yi Lee Soojeong Jang
astro-ph.SR









§ INTRODUCTION


Solar flares rapidly release a tremendous amount of energy as the form of electromagnetic radiation, high energy particles and shock waves, which result in space weather hazards. In space age, precautions against economic risk by predicting solar flare occurrence are very essential <cit.>.

Most solar energetic events such as solar flares explode out of active regions (ARs), which are areas of complex and intense photospheric magnetic fields. Thus, characteristics of solar ARs are considered to closely be related to solar flares. Several different types of AR information have been used for solar flare forecasts. Many studies have considered morphological characteristics of ARs such as size, shape and complexity using Mount Wilson magnetic classification and McIntosh classification <cit.>. In addition, various magnetic parameters characterizing distribution and non-potentiality of ARs from magnetograms have been studied for flare forecasting <cit.>. The magnetic parameters have been also calculated by magnetohydrodynamic (MHD) simulations and used for predicting flares <cit.>.

There have been many studies on flare probability depending on AR's characteristics and flare probability forecasting models. <cit.> examined probabilities of an solar eruption in relation to sunspot group's size, type, and development. <cit.> developed a flare prediction system using McIntosh classification, which gives daily flare probabilities based on Poisson statistics. <cit.> considered magnetic parameters such as the vertical current, the current helicity, the twist parameter α, and the magnetic shear angles for discriminating flaring and flare-quiet ARs, which had been examined in a series of papers <cit.>. In order to parameterize these magnetic parameters, they first used vector magnetograms from Imaging Vector Magnetograph (IVM) at the University of Hawai'i Mees Solar Observatory. As a forecast result of their studies, <cit.> provided flare probabilities using Bayes's theorem. <cit.> found an empirical relationship between flare event rates and a proxy of magnetic free energy based on line-of-sight magnetograms from Solar and Heliospheric Observatory (SOHO; )/Michelson Doppler Imager (MDI; ). They predicted flare probabilities from a power law relationship between M- and X-class flare occurrence rates and the free magnetic energy proxy in order to forecast flares. <cit.> determined the Poisson flare probabilities from McIntosh sunspot classes. Among a variety of forecast verification scores, they proposed true skill statistic (TSS) as a standard score for comparing between flare forecasts. They also presented optimum thresholds to convert probabilities into binary predictions and forecast verification measures using these thresholds. <cit.> also reported flare occurrence Poisson probabilities of McIntosh classification. They found that these flare probabilities tend to increase with AR's area and flaring probabilities for increasing AR's area are higher than those for steadying and decreasing area.

Recently, Solar Dynamics Observatory (SDO; ) was launched in 2010 and Helioseismic and Magnetic Imager (HMI), which is one of three instruments on the SDO, provides full-disk photospheric vector magnetic fields with 12 minutes cadence <cit.>.

The HMI team developed a set of derivative data called Space-weather HMI Active Region Patches (SHARPs) data <cit.>. These data contain automatically identified HMI Active Region Patches (HARPs) and magnetic parameters which summarize the size, distribution, and non-potentiality of vector magnetic fields in each HARP, and these parameters have been adapted from numerous studies <cit.>. SHARP data have been applied to machine learning algorithms for binary flare forecasting <cit.>. The probabilistic forecasting have been performed by <cit.> and <cit.> using Bayesian probabilities. Although there have been many categorical forecasts using vector magnetic fields, probabilistic forecasts have been rarely considered. SHARP magnetic parameters are also ranked by performance of discriminating between flaring events and non-flaring events based on machine learning algorithm <cit.>.

In this paper, we study empirical relationships between photospheric SHARP magnetic parameters and daily major flaring rates. Furthermore, we will develop forecast models of daily probability of major flares (M- and X-class) based on these empirical relationships. We will also identify the rank of SHARP magnetic parameters examining the performance of the models developed from the parameters.
In general, the occurrence probability could present more continuous information on potentiality of flare occurrence than the binary (flaring/non-flaring) forecasts. Furthermore, the probability forecasts can be converted into binary forecasts using proper threshold values <cit.>.

The paper is organized as follows. Section <ref> deal with detailed description of the data. The empirical relationships using our model is explained in detail in Section <ref>.
Forecasting models and their results are given in Section <ref>. A summary and discussion are presented in Section <ref>.




§ DATA AND ANALYSIS
 


 §.§ Geostationary Operational Environmental Satellites (GOES) X-ray flares


GOES have measured solar X-rays in the passbands of 1 - 8 Å and 0.5 - 4 Å. Solar X-ray flares are classified according to the peak flux of X-rays with wavelength bands 1 to 8 Å as measured by GOES.
We use GOES major (M- and X-class) X-ray flare data from May 2010 to April 2018 and their locations are identified by the Lockheed Martin Solar and Astrophysics Laboratory (LMSAL)[<https://www.lmsal.com/solarsoft/latest_events_archive.html>]. Our data include 448 M-class flares and 27 X-class flares.




 §.§ SDO/HMI and magnetic parameters


SHARP magnetic parameters have been used for flare forecasting <cit.> based on machine learning algorithm. Among the SHARP parameters, we just consider ten parameters which have linear Pearson correlation coefficients (CCs) between these and flaring rates higher than 0.85 described in Section <ref>. The ten parameters are classified into three types: the total unsigned parameters (TOTUSJH H_C_total, TOTUSJZ J_z,total, TOTPOT ρ_tot, and USFLUX Φ), the total signed parameters (SAVNCPP J_z,sum and ABSNJZH H_C,abs), and the mean parameters (MEANPOT ρ, SHRGT45 A_shear, MEANSHR Γ, and MEANGAM γ). The description and formula of these parameters are listed in Table <ref>.



We use 00:00 TAI definitive HARPs in cylindrical equal area (CEA) coordinates (hmi.sharp_720s_cea data series) when their longitudes are within ± 60 degrees of the central meridian and corresponding ten magnetic parameters from the Joint Science Operations Center (JSOC)[<http://jsoc.stanford.edu/>]. According to <cit.>, the number of high-confidence pixels in SHARP data decreases significantly beyond ± 60 degrees of the central meridian. In this study, a HARP is regarded as an unit of area to occur flares.





§ EMPIRICAL RELATIONSHIPS BETWEEN MAGNETIC PARAMETERS AND SOLAR MAJOR FLARE OCCURRENCE RATES
 

To develop a solar flare occurrence probability forecasting model, we need relationships between ten magnetic parameters and major flare occurrence rates. For identifying these relationships, we need a data set of major flare occurrence history and corresponding ten parameter values for each HARP.



 §.§ Flare identification


We identify each HARP at 00:00 TAI that produced one or more major flares within a day by using the flare locations which are corrected for differential rotation rates. When a flare event is located in a HARP's box, the flare event is considered to occur from that HARP. We assume that all flare events are independent of one another.



 §.§ Data set


Our data sets are divided into two sets (training and test) in chronological order. 70% of the data, HARPs from 1 May 2010 to 20 April 2015 including the ascending and maximum phase of the solar cycle (SC) 24, are used for finding a relationship between parameters and flaring rates.  And 30% of the data, HARPs from 21 April 2015 to 30 April 2018 including the part of the descending phase of SC 24, are used for testing it. Accordingly, the training data consist of 11040 samples (different 1889 HARPs) and the test data consist of 4724 samples (different 898 HARPs). The training data sample consist of 224 event samples and 10816 non-event samples. The test data sample consist of 38 event samples and 4686 non-event samples. The sample ratio is unbalanced as well as previous major flare forecasting models, because major flares are rare events and the solar cycle 24 is unprecedented quiet.




 §.§ Major flare occurrence rates as a function of magnetic parameters


We want to identify relationships between ten magnetic parameters and major flare occurrence rates. For each parameter, we divide our data into 50 subgroups in which there is the equal number of HARPs. Then, we determine the average parameter value and the number of major flares within a day from each daily HARP in each subgroup. Each mean major flare occurrence rate (R_i) of i-th group (G_i) is given by

    R_i = # of major flares of G_i/# of HARPs of G_i.

In order to find relationships, these rates are plotted as a function of each parameter in log-log scales. As <cit.> presented a power law function of major flare occurrence rates and their parameter from line-of-sight magnetic fields, our models also considers the power law function of each parameter as shown in Figure <ref>. The occurrence rates range from 0.001 to about 1 for all parameters.
The flaring rates are only considered above 0.01 for obtaining more accurate fitting functions as <cit.> did. The fitting function is given by the following

    log (R) =  alog (x) + b,

where R is a mean flare occurrence rate, x is a mean parameter value of a group, a is a power law slope, and b is a fitting constant. To examine the dependence of the binning size, we also consider three cases of binning size =10, 20, 100. Power law functions are well fitted with data for all four cases and their differences are very small. Thus, we use the power law function with binning size =50 and its fitting coefficients and uncertainties of ten SHARP parameters are shown in Table <ref>.



In Table <ref>, the CCs between flaring rates and ten SHARP parameters are high (above 0.85), implying that they are well correlated with each other. Among the ten parameters, the total unsigned vertical current has the minimum root mean square error (RMSE) between the power law fitting function and its original value (Figure <ref>a and <ref>b).







§ FORECAST MODELS AND THEIR VERIFICATION
 



 §.§ Forecast models based on empirical relationships between parameters and flaring rates


We develop forecasting models of daily probability of major flares using the relationships between ten magnetic parameters and daily flaring rates. For a given x, we can predict a major flare occurrence rate from the empirical fitting functions. From these predicted rates, we calculate flare probabilities using the Poisson distribution <cit.>. The probability (P) of occurring at least one flare in a day is given by

    P=1-exp(-R).




 §.§ Verification measures 




  §.§.§ Verification measures of probability


Forecast models represent verification measures which are a single number measuring forecast performance. We consider the mean squared error (MSE), the Brier skill score (BSS), and the reliability plots. These measures have been used for verifying the performance of probabilistic forecast models <cit.>. The MSE, which is a measure of accuracy, is given by

    MSE=1/N∑_i=1^N (P_i-O_i)^2,

where P_i is the predicted probability and O_i is the observation that events occurred (O_i = 1) or did not occur (O_i = 0). When the perfect forecast occur, the MSE is 0. The BSS, which represents the relative skill compared to the model using the climatological event rate during the testing interval, is given by
 
    BSS=1/N∑_i=1^N (P_i-O_i)^2-1/N∑_i=1^N (O_i-O_i)^2/0-1/N∑_i=1^N (O_i-O_i)^2.

When the perfect forecast occur, BSS is 1, and "no-skill" as compared to the climatological forecast results in 0. The reliability plots are observed occurrence rates against predicted occurrence rates. When the perfect forecast occur, the all points in the reliability plot lie on the diagonal line.

From the test samples, we calculate the MSE and BSS of ten forecast models, which are listed in Table <ref>. The total unsigned current helicity has the best performance in terms of both MSE and BSS. In view of the BSS, the total unsigned parameters have higher value than the signed and mean parameters; more specifically, H_C,total > H_C,abs, J_z,total > J_z,sum, and ρ_tot > ρ.



Figure <ref> show that reliability plots and their RMSEs (i.e., standard deviation of the residuals from the y=x line) between observed rates and predicted ones. Most of the data points for the total unsigned and signed parameters are relatively adjacent to the diagonal line compared to the mean parameters. The total signed parameters can predict a wider range of probabilities than the other types of parameters. The Φ gives the best reliability (RMSE = 0.03), but the reliability plots and their RMSEs can depend on the selection of probability bins.



As an example, we present the probabilities of HARP 7115 (NOAA 12673) predicted by our models with top six BSS at the cadence of 1 hour from 2017 September 2 to 6 in Figure <ref>. Our forecast models present their own probabilities  of major flare occurrence within a day after the observation time. Our predicted probabilities from six models are compared with GOES-15 X-ray fluxes (5 minute data). During this period, several major flares occurred in HARP 7115 (NOAA 12673). On September 2, probabilities that major flares occur are mostly lower than 20%. Actually, there is no major flare during that period.
The models of the total signed parameters (H_C,abs and J_z,sum) show a rapid and significant increase of forecast probabilities around September 4 when the first major flare occurred as well as persistence of high probabilities (greater than 50%) over the next three days. On the other hand, the models of the total unsigned parameters show relatively low probabilities (mostly less than 50%) with a gradually increasing trend. This suggests that the total signed parameters may help the empirical models in this study to better perform in some cases such as HARP 7115.



Our forecast models are to predict daily flare probability at 00:00 TAI using SDO/HMI observations. However, the models may have an issue on operational forecasting since there is a delay of getting magnetic parameters after the observations. At present, the data that we can get the fastest are available in less than three hours after the observation time. The calculation of the probability from our models needs only a few minutes. Thus, the delay time from observations to forecasts will not exceed 3 hours. For operational forecasting, we can predict daily flaring probability at 00:00 TAI using the observation at 21:00 TAI of the previous day. In order to examine how much this operational model is different with the original model, we investigate the relationship between J_z,total at 21:00 TAI, which has the highest CC between parameter values and flaring rates in Table <ref>, and flaring rate for a day (00:00 - 23:59 TAI). In this case, the fitting coefficients, a and b, are 1.53 and -21.83, respectively, which corresponds to about 2% difference with the fitting coefficients of the original model with J_z,total in Table <ref>. The predicted probabilities from the operational model are underestimated by 2 - 6.3% than those from the original model. If we provide the operational forecasting with this model, the forecast has to have this uncertainty together with its original uncertainties.





  §.§.§ Verification measures of Binary


The probabilistic forecast results have been converted into contingency tables by using probability thresholds <cit.>. A contingency table consists of four components: the number of TP (flare event predicted and occurred), FN (no flare event predicted and flare occurred), FP (flare event predicted and did not occur), and TN (no flare event predicted and did not occur) as shown in Table <ref>. Verification measures for binary forecasts are obtained by combining these four components of the contingency table. We build contingency tables using probability thresholds and calculate six measures.




In this study, we consider the following verification measures: the proportion correct (PC), the probability of detection (POD), the false alarm ratio (FAR), the Heidke skill score (HSS), the true skill statistic (TSS), and the symmetric extremal dependence index (SEDI). The first five measures have been widely used for the evaluation of flare forecasts, and the last one has been rarely used but is meaningful for validating forecast of rare events such as major flares <cit.>. These six performance measures are described in Table <ref>.



We determine the values of six measures and their thresholds to achieve optimum TSS and HSS in contingency tables, which are summarized with their uncertainties in Table <ref> and <ref>, respectively. In view of the optimized TSS, the total unsigned parameters (H_C,total, ρ_tot, J_z,total, and Φ) achieved higher values of verification measures than the total signed (H_C,abs and J_z,sum) and mean parameters (ρ, A_shear, Γ, and γ). H_C,total is one of the parameters having the highest TSS, which is consistent with the highest F-score in the classification between flaring events and non-flaring events <cit.>. The rank order of TSS in view of the optimized TSS is also consistent with that of F-score from <cit.>. When the probability threshold is small, TSS values are very high but FAR values are also high. In view of the optimized HSS, most of total unsigned parameters also have higher value than the signed and mean parameters (H_C,total > H_C,abs, J_z,total ≈ J_z,sum, and ρ_tot > ρ). <cit.> suggested that TSS is optimized when FN/FP is similar to the ratio of flare event number to non-flare event number, and the HSS is optimized when FN/FP ≈ 1. The thresholds of the optimum TSS and HSS proposed by <cit.> are slightly different (from 0 to 0.03) with the thresholds of the optimum TSS and HSS in our models.






Another interesting measure is a relative operating characteristic (ROC) curve in which POD is plotted as a function of POFD and a ROC area which is a area under the ROC curve. When the perfect forecast occurs, the curve travels from bottom left to top left of diagram, then across to top right of diagram. In this case, the ROC area is 1. ROC areas under the curve of ten parameters with their uncertainties are as follows: 0.98 ± 0.001 from H_C,total, 0.98 ± 0.001 from J_z,total, 0.97 ± 0.001 from ρ_tot, 0.97 ± 0.001 from Φ, 0.95 ± 0.002 from J_z,sum, 0.96 ± 0.001 from H_C,abs, 0.93 ± 0.001 from ρ, 0.91 ± 0.002 from A_shear, 0.9 ± 0.002 from Γ, and 0.88 ± 0.002 from γ. As shown in Figure <ref>, the total unsigned parameters have higher ROC areas than the total signed parameters and the mean parameters, which gives the better performance in terms of the ROC curve measure. The difference in ROC areas is very small between all of the total unsigned parameters.





  §.§.§ Uncertainties of verification measures


Several forecast studies have mentioned the necessity of uncertainties for forecast verification measures <cit.> because the verification measures are calculated from finite number of samples. To estimate uncertainties of verification measures, we use a bootstrap method, which accounts for random errors using resampling. We consider the size of resampled data which is the difference between the number of non-events and events. We make a resampled data with size =4648 by picking randomly. Then, we calculate the verification measures from the resampled data. This process iterates 1000 times and the uncertainties are estimated by the standard deviation of the resampled values of verification measures in Table <ref> and <ref>.





§ SUMMARY AND DISCUSSION
 

We have presented the forecasting models of major flare probability based on the power law relationships between ten magnetic parameters and major flare occurrence rates. The magnetic parameters calculated from SDO/HMI vector magnetic fields are used. The data are taken from May 2010 to April 2018 and divided into two sets (training of 11,040 HARPs and test of 4,724 HARPs) in chronological order, which is proper for forecast purpose. All values of the magnetic parameters are divided into 50 subgroups to estimate corresponding flare occurrence rates. From this, we considered the power law relationships between magnetic parameters and flaring rates.

The major results of this study are summarized as follows. First, major flare occurrence rates are well correlated with ten magnetic parameters (CC ≥ 0.86). Second, the logarithmic values of flaring rates are well approximated by the linear equations.
Third, the total unsigned parameters achieved relatively higher values of the optimized TSS and HSS than the total signed and mean parameters. Among the total unsigned parameters, H_C,total, ρ_tot, and J_z,total are also the highest ranked of univariate F-scores in <cit.>. Our results are well consistent with <cit.> who found that H_C,total, ρ_tot, and J_z,total are stronger proportional to magnetic free energy, which is calculated from 3D reconstructed magnetic fields, than the other parameters.

Although we calculate various verification measures of ten magnetic parameters, it is difficult to conclude that what parameter is outperformed than the others. Most of parameters have high values of measures with only small differences. The result also depends on verification, where the total unsigned parameters have slightly better performance than the total signed parameters in terms of optimized TSS and HSS, but the total signed parameters can predict higher probabilities than the total unsigned parameters in practical forecasts for given AR. The performances of converted binary forecasts depend on the probability thresholds. When the probability threshold is small, TSS are very high but FAR are also high. On the other hand, all verification measure values are moderate in case of optimized HSS. Thus, the decision-maker or user may select a proper threshold or model for their own purpose.

Our parameters that give a relatively good performance are conventionally considered to non-potential parameters. This result supports the importance of non-potential magnetic fields in ARs (e.g., ).
<cit.> have found that the time variations of current helicity in the highly flaring active regions are more significant than those of the poorly flaring active regions. The magnetic free energy plays an important role in producing major flares, which has been already well known <cit.>. <cit.> revealed a quite high correlation between vertical current and flares.
<cit.> suggested that the total unsigned vertical current and the photospheric magnetic free energy should be responsible for flare productivity.
Our results together with the previous results demonstrate an importance of total quantity of non-potential magnetic properties for flare forecasting <cit.>.






We acknowledge the referees for giving us useful comments. This work was supported by the BK21 plus program through the National Research Foundation (NRF) funded by the Ministry of Education of Korea, the Basic Science Research Program through the NRF funded by the Ministry of Education (NRF-2016R1A2B4013131, NRF-2016R1A6A3A11932534, NRF-2019R1A2C1002634), NRF of Korea Grant funded by the Korean Government (NRF-2013M1A3A3A02042232), the Korea Astronomy and Space Science Institute under the R&D program supervised by the Ministry of Science, ICT and Future Planning, the Korea Astronomy and Space Science Institute under the R&D program ‘Development of a Solar Coronagraph on International Space Station (Project No. 2019-1-850-02)’ supervised by the Ministry of Science, ICT and Future Planning, and Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (2018-0-01422, Study on analysis and prediction technique of solar flares). The data used here are courtesy of NASA/SDO and the HMI science team, as well as the GOES team. 







[Ahmed et al. (2013)]Ahmed13 Ahmed, O. W., Qahwaji, R., Colak, T., et al. 2013, Solar Flare Prediction Using Advanced Feature Extraction, Machine Learning, and Feature Selection, SoPh, 283, 157
[Aulanier et al. (2013)]Aulanier13 Aulanier, G., Démoulin, P., Schrijver, C. J., Janvier, M., Pariat, E., & Schmieder, B. 2013, The standard flare model in three dimensions II. Upper limit on solar flare energy, A&A, 549, A66
[Bao et al. (1999)]Bao99 Bao, S. D., Zhang, H. Q., Ai, G. X., & Zhang, M. 1999, A Survey of Flares and Current Helicity in Active Regions, A&AS, 139, 311
[Barnes et al. (2016)]Barnes16 Barnes, G., Leka, K. D., Schrijver, C. J., et al. 2016, A Comparison of Flare Forecasting Methods. 1. Results from the "All-Clear" Workshop, ApJ, 829, 89
[Barnes & Leka (2006)]Barnes06 Barnes, G., & Leka, K. D. 2006, Photospheric Magnetic Field Properties of Flaring Versus Flare-quiet Active Regions. 3. Magnetic Charge Topology Models, ApJ, 646, 1303
[Barnes et al. (2007)]Barnes07 Barnes, G., Leka, K. D., Schumer, E. A., & Della-Rose, D. J. 2007, Probabilistic Forecasting of Solar Flares from Vector Magnetogram Data, SpWea, 5, S09002
[Bloomfield et al. (2012)]Bloomfield12 Bloomfield, D. S., Higgins, P. A., McAteer, R. T. J., & Gallagher, P. T. 2012, Toward Reliable Benchmarking of Solar Flare Forecasting Methods, ApJL, 747, L41
[Bobra & Couvidat (2015)]Bobra15 Bobra, M. G., & Couvidat, S. 2015, Solar Flare Prediction Using SDO/HMI Vector Magnetic Field Data with a Machine-learning Algorithm, ApJ, 798, 135
[Bobra et al. (2014)]Bobra14 Bobra, M. G., Sun, X., Hoeksema, J. T., et al. 2014, The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: SHARPs - Space-Weather HMI Active REgion Patches, SoPh, 289, 3549
[Bocchialini et al. (2018)]Bocchialini18 Bocchialini, K., Grison, B., Menvielle, M., et al. 2018, Statistical Analysis of Solar Events Associated with Storm Sudden Commencements over One Year of Solar Maximum During Cycle 23: Propagation from the Sun to the Earth and Effects, SoPh, 293, 75
[Bornmann & Shaw (1994)]Bornmann94 Bornmann, P. L., & Shaw, D. 1994, Flare Rates and the McIntosh Active-region Classifications, SoPh, 150, 127
[Canfield et al. (1999)]Canfield99 Canfield, R. C., Hudson, H. S., & McKenzie, D. E. 1999, Sigmoidal Morphology and Eruptive Solar Activity, GeoRL, 26, 627
[Colak & Qahwaji (2009)]Colak09 Colak, T., & Qahwaji, R. 2009, Automated Solar Activity Prediction: A Hybrid Computer Platform Using Machine Learning and Solar Imaging for Automated Prediction of Solar Flares, SpWea, 7, S06001
[Cui et al. (2006)]Cui06 Cui, Y., Li, R., Zhang, L., He, Y., & Wang, H. 2006, Correlation between Solar Flare Productivity and Photospheric Magnetic Field Properties, SoPh, 237, 45
[Crown (2012)]Crown12 Crown, M. D. 2012, Validation of the NOAA Space Weather Prediction Center's Solar Flare Forecasting Look-up Table and Forecaster-issued Probabilities, SpWea, 10, S06006
[Domingo (1995)]Domingo95 Domingo, V., Fleck, B., & Poland, A. I. 1995, The SOHO Mission: An Overview, SoPh, 162, 1
[Falconer et al. (2011)]Falconer11 Falconer, D. A., Barghouty, A. F., Khazanov, I., & Moore, R. L. 2011, A Tool for Empirical Forecasting of Major Flares, Coronal Mass Ejections, and Solar Particle Events from a Proxy of Active-region Free Magnetic Energy, SpWea, 9, S04003
[Falconer et al. (2014)]Falconer14 Falconer, D. A., Moore, R. L., Barghouty, A. F., & Khazanov, I. 2014, MAG4 Versus Alternative Techniques for Forecasting Active Region Flare Productivity, SpWea, 12, 306
[Ferro & Stephenson (2011)]Ferro11 Ferro, C. A T., & Stephenson, D. B. 2011, Extremal Dependence Indices: Improved Verification Measures for Deterministic Forecasts of Rare Binary Events, Wea. Forecasting, 26, 699
[Fisher et al. (2012)]Fisher12 Fisher, G. H., Bercik, D. J., Welsch, B. T., & Hudson, H. S. 2012, Global Forces in Eruptive Solar Flares: The Lorentz Force Acting on the Solar Interior, SoPh, 277, 59
[Gallagher et al. (2002)]Gallagher02 Gallagher, P. T., Moon, Y.-J., & Wang, H. 2002, Active-region Monitoring and Flare Forecasting. 1. Data Processing and First Results, SoPh, 209, 171
[Giovanelli (1939)]Giovanelli39 Giovanelli, R. G. 1939, The Relationships between Eruptions and Sunspots, ApJ, 89, 555
[Guennou et al. (2017)]Guennou17 Guennou, C., Pariat, E., Leake, J. E., & Vilmer, N. 2017, Testing predictors of eruptivity using parametric flux emergence simulations, JSWSC, 7, A17
[Hale et al. (1919)]Hale19 Hale, G. E., Ellerman, F., Nicholson, S. B., & Joy, A. H. 1919, The Magnetic Polarity of Sun-spots, ApJ, 49, 153
[Hoeksema et al. (2014)]Hoeksema14 Hoeksema, J. T., Liu, Y., Hayashi, K., et al. 2014, The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: Overview and Performance, SoPh, 289, 3483
[Huang & Wang (2013)]Huang13 Huang, X., & Wang, H.-N. 2013, Solar Flare Prediction Using Highly Stressed Longitudinal Magnetic Field Parameters, RAA, 13, 351
[Huang et al. (2018)]Huang18 Huang, X., Wang, H., Xu, L., et al. 2018, Deep Learning Based Solar Flare Forecasting Model. 1. Results for Line-of-sight Magnetograms, ApJ, 856, 7
[Ji et al. (2003)]Ji03 Ji, H. S., Song, M. T., Zhang, Y. A., & Song, S. M. 2003, The Horizontal and Vertical Electric Currents in Three Solar Active Regions and Their Relations with Flares, Chin. Astron. Astrophys., 27, 79
[Jing et al. (2012)]Jing12 Jing, J., Park, S.-H., Liu, C., et al. 2012, Evolution of Relative Magnetic Helicity and Current Helicity in NOAA Active Region 11158, ApJL, 752, L9
[Kontogiannis et al. (2017)]Kontogiannis17 Kontogiannis, I., Georgoulis, M. K., Park, S.-H., & Guerra, J. A. 2017, Non-neutralized Electric Currents in Solar Active Regions and Flare Productivity, SoPh, 292, 159
[Kubo et al. (2017)]Kubo17 Kubo, Y., Den, M., & Ishii, M. 2017, Verification of Operational Solar Flare Forecast: Case of Regional Warning Center Japan, JSWSC, 7, A20
[Kusano et al. (2012)]Kusano12 Kusano, K., Bamba, Y., Yamamoto, T. T., Toriumi, S., & Asai, A. 2012, Magnetic Field Structures Triggering Solar Flares And Coronal Mass Ejections, ApJ, 760, 31
[Lee et al. (2012)]Lee12 Lee, K., Moon, Y.-J., Lee, J.-Y., et al. 2012, Solar Flare Occurrence Rate and Probability in Terms of the Sunspot Classification Supplemented with Sunspot Area and Its Changes, SoPh, 281, 639
[Lee et al. (2016)]Lee16 Lee, K., Moon, Y.-J., & Nakariakov, V. M. 2016, Dependence of Occurrence Rates of Solar Flares and Coronal Mass Ejections on the Solar Cycle Phase and the Importance of Large-scale Connectivity, ApJ, 831, 131
[Leka & Barnes (2003a)]Leka03a Leka, K. D., & Barnes, G. 2003a, Photospheric Magnetic Field Properties of Flaring Versus Flare-quiet Active Regions. 1. Data, General Approach, and Sample Results, ApJ, 595, 1277
[Leka & Barnes (2003b)]Leka03b Leka, K. D., & Barnes, G. 2003b, Photospheric Magnetic Field Properties of Flaring Versus Flare-quiet Active Regions. 2. Discriminant Analysis, ApJ, 595, 1296
[Leka & Barnes (2007)]Leka07 Leka, K. D., & Barnes, G. 2007, Photospheric Magnetic Field Properties of Flaring Versus Flare-quiet Active Regions. 4. A Statistically Significant Sample, ApJ, 656, 1173
[Leka et al. (2018)]Leka18 Leka, K. D., Barnes, G., & Wagner, E. 2018, The NWRA Classification Infrastructure: Description and Extension to the Discriminant Analysis Flare Forecasting System (DAFFS), JSWSC, 8, A25
[Li et al. (2008)]Li08 Li, R., Cui, Y., He, H., & Wang, H. 2008, Application of Support Vector Machine Combined with K-nearest Neighbors in Solar Flare and Solar Proton Events Forecasting, AdSpR, 42, 1469
[Li & Zhu (2013)]Li13 Li, R., & Zhu, J. 2013, Solar Flare Forecasting Based on Sequential Sunspot Data, RAA, 13, 1118
[Liu et al. (2017)]Liu17 Liu, C., Deng, N., Wang, J. T. L., & Wang, H. 2017, Predicting Solar Flares Using SDO/HMI Vector Magnetic Data Products and the Random Forest Algorithm, ApJ, 843, 104
[Liu et al. (2016)]Liu16 Liu, L., Wang, Y., Wang, J., Shen, C., Ye, P., Liu, R., Che, J., Zhang, Q., & Wang, S. 2016, Why Is a Flare-rich Active Region CME-poor?, ApJ, 826, 119
[Low (1994)]Low94 Low, B. C. 1994, Magnetohydrodynamic Processes in the Solar Corona: Flares, Coronal Mass Ejections, and Magnetic Helicity, PhPl, 1, 1684
[McAteer et al. (2005)]McAteer05 McAteer, R. T. J., Gallagher, P. T., & Ireland, J. 2005, Statistics of Active Region Complexity: A Large-scale Fractal Dimension Survey, ApJ, 631, 628
[McCloskey et al. (2016)]McCloskey16 McCloskey, A. E., Gallagher, P. T., & Bloomfield, D. S. 2016, Flaring Rates and the Evolution of Sunspot Group McIntosh Classifications, SoPh, 291, 1711
[McIntosh (1990)]McIntosh90 McIntosh, P. S. 1990, The Classification of Sunspot Groups, SoPh, 125, 251
[Moon et al. (2001)]Moon01 Moon, Y.-J., Choe, G. S., Yun, H. S., & Park, Y. D. 2001, Flaring Time Interval Distribution and Spatial Correlation of Major X-ray Solar Flares, JGR, 106, 29951
[Moore et al. (2001)]Moore01 Moore, R. L., Sterling, A. C., Hudson, H. S., & Lemen, J. R. 2001, Onset of the Magnetic Explosion in Solar Flares and Coronal Mass Ejections, ApJ, 552, 833
[Murray et al. (2017)]Murray17 Murray, S. A., Bingham, S., Sharpe, M., & Jackson, D. R. 2017, Flare Forecasting at the Met Office Space Weather Operations Centre, SpWea, 15, 577
[Nishizuka et al. (2017)]Nishizuka17 Nishizuka, N., Sugiura, K., Kubo, Y., et al. 2017, Solar Flare Prediction Model with Three Machine-learning Algorithms Using Ultraviolet Brightening and Vector Magnetograms, ApJ, 835, 156
[Nishizuka et al. (2018)]Nishizuka18 Nishizuka, N., Sugiura, K., Kubo, Y., et al. 2018, Deep Flare Net (DeFN) Model for Solar Flare Prediction, ApJ, 858, 113
[Park et al. (2018)]Park18 Park, E., Moon, Y.-J., Shin, S., et al. 2018, Application of the Deep Convolutional Neural Network to the Forecast of Solar Flare Occurrence Using Full-disk Solar Magnetograms, ApJ, 869, 91
[Park et al. (2017)]Park17 Park, J., Moon, Y.-J., Choi, S., et al. 2017, Application of Decision-making to a Solar Flare Forecast in the Cost-loss Ratio Situation, SpWea, 15, 704
[Pesnell et al. (2012)]Pesnell12 Pesnell, W. D., Thompson, B. J., & Chamberlin, P. C. 2012, The Solar Dynamics Observatory (SDO), SoPh, 275, 3
[Qahwaji & Colak (2007)]Qahwaji07 Qahwaji, R., & Colak, T. 2007, Automatic Short-term Solar Flare Prediction Using Machine Learning and Sunspot Associations, SoPh, 241, 195
[Raboonik et al. (2017)]Raboonik17 Raboonik, A., Safari, H., Alipour, N., & Wheatland, M. S. 2017, Prediction of Solar Flares Using Unique Signatures of Magnetic Field Images, ApJ, 834, 11
[Sammis et al. (2000)]Sammis00 Sammis, I., Tang, F., & Zirin, H. 2000, The Dependence of Large Flare Occurrence on the Magnetic Structure of Sunspots, ApJ, 540, 583
[Scherrer et al. (1995)]Scherrer95 Scherrer, P. H., Bogart, R. S., Bush, R. I., et al. 1995, The Solar Oscillations Investigation - Michelson Doppler Imager, SoPh, 162, 129
[Scherrer et al. (2012)]Scherrer12 Scherrer, P. H., Schou, J., Bush, R. I., et al. 2012, The Helioseismic and Magnetic Imager (HMI) Investigation for the Solar Dynamics Observatory (SDO), SoPh, 275, 207
[Schou et al. (2012)]Schou12 Schou, J., Scherrer, P. H., Bush, R. I., et al. 2012, Design and Ground Calibration of the Helioseismic and Magnetic Imager (HMI) Instrument on the Solar Dynamics Observatory (SDO), SoPh, 275, 229
[Schrijver (2007)]Schrijver07 Schrijver, C. J. 2007, A Characteristic Magnetic Field Pattern Associated with All Major Solar Flares and Its Use in Flare Forecasting, ApJL, 655, L117
[Schrijver (2009)]Schrijver09 Schrijver, C. J. 2009, Driving Major Solar Flares and Eruptions: A Review, AdSpR, 43, 739
[Schwenn (2006)]Schwenn06 Schwenn, R. 2006, Space Weather: The Solar Perspective, LRSP, 3, 2
[Shin et al. (2016)]Shin16 Shin, S., Lee, J.-Y., Moon, Y.-J., et al. 2016, Development of Daily Maximum Flare-Flux Forecast Models for Strong Solar Flares, SoPh, 291, 897
[Toriumi & Takasao (2017)]Toriumi17 Toriumi, S., & Takasao, S. 2017, Numerical Simulations of Flare-productive Active Regions: δ-sunspots, Sheared Polarity Inversion Lines, Energy Storage, and Predictions, ApJ, 850, 39
[Tsurutani et al. (2005)]Tsurutani05 Tsurutani, B. T., Judge, D. L., Guarnieri, F. L., Gangopadhyay, P., Jones, A. R., et al. 2005, The October 28, 2003 extreme EUV solar flares and resultant extreme ionospheric effect: Comparison to other Halloween events and the Bastille Day event, GRL, 32, L03S09
[Welsch et al. (2009)]Welsch09 Welsch, B. T., Li, Y., Schuck, P. W., & Fisher, G. H. 2009, What Is the Relationship between Photospheric Flow Fields and Solar Flares?, ApJ, 705, 821
[Wheatland (2000)]Wheatland00 Wheatland, M. S. 2000, The Origin of the Solar Flare Waiting-time Distribution, ApJL, 536, L109
[Wheatland (2005)]Wheatland05 Wheatland, M. S. 2005, A Statistical Solar Flare Forecast Method, SpWea, 3, S07003
[Yu et al. (2009)]Yu09 Yu, D., Huang, X., Wang, H., & Cui, Y. 2009, Short-term Solar Flare Prediction Using a Sequential Supervised Learning Method, SoPh, 255, 91
[Yuan et al. (2010)]Yuan10 Yuan, Y., Shih, F. Y., Jing, J., & Wang, H.-M. 2010, Automated Flare Forecasting Using a Statistical Learning Technique, RAA, 10, 785
[Zhang (2016)]Zhang16 Zhang, H. 2016, Photospheric Magnetic Free Energy Density of Solar Active Regions, SoPh, 291, 3501







