Time-only Molchan test for the reference model (time-invariant)
Molchan test is a standard test, used by the Collaboratory for the Study of Earthquake Predictability (CSEP), for the evaluation of earthquake forecasts. Molchan test is alarm-based and evaluates whether a given alarm function has a skill, that is if it chooses better alarm sets than a random function.
The Molchan diagram shows the fraction of space-time occupied by alarms (x-axis) versus the miss-rate (y-axis). In the time-only Molchan test, the x-axis shows the fraction of time occupied by alarms.
The a above figure illustrates time-only test for the time-invariant reference model, which has no temporal forecasting skill and does not pass the time-only Molchan test. In this case, the area-skill-score (af) is very close to random guessing (af = 0.5). The spatial forecasting skill of the reference model was eliminated by applying the test to the regions having the same value of the seismic potential (time-only Molchan test).
According to the Omega-Theory, the reference model for the Quantectum forecasts is the so-called average seismic potential, which is based on the maximum-magnitude law and the Gutenberg-Richter’s law: Ps = A (log N + Mmax). Here, A is the normalization parameter, N is the long-range average rate of events at a certain point, and Mmax is the maximum magnitude of the past earthquakes in the catalog. We use the USGS seismic catalog (https://earthquake.usgs.gov/earthquakes/) since 1950 for the M5.5+ earthquakes. We calculate the smoothed average seismic potential, where the smoothing function w = 1/((r/Lc)^3 + 1) is defined based on the Green’s near field extended to the Cosserat continuum. Here, r is the distance from the earthquake epicenter and Lc is the Cosserat characteristic length. We use Lc = 250 km.
The following figure illustrates our reference model.
The normalized seismic potential used here has a spatial resolution of 1 degree. The range of possible values of the normalized seismic potential is between 0 and 1.
Meaning of the reference model
- The choice of the reference model is crucial if the test result is to be useful.
- The reference model should be the best possible model that one is not yet proud of. A quantity carefully derived from the observed long-term seismicity by means of the seismic potential would be a good choice.
- It is good enough (getting anything better is amazing) and bad enough (it is not an amazing result in itself) at the same time.
- The reference model is a much stricter concept than an alarm function.
- The value of the reference model (seismic potential) in a location (bin) is some quantitative measure of the assumed probability density (frequency) of earthquakes.
- The reference model should fall into the same class of physical models as the alarm function.
- Failure to produce a good reference model is the easiest way to obtain a skilled alarm function
- After the reference model is found to be suitable, the testing is straightforward: the area skill score for the tested alarm function is to be computed and checked if it is above the significance limit. If it is, the function passes the test, if not, it fails.
- Usually, the significance limit in literature is set to 1 %.
- Molchan, G.M., 1990. Strategies in strong earthquake prediction. Physics of the Earth and Planetary Interiors. 61, 84-98.
- Molchan, G.M., 1991. Structure of optimal strategies in earthquake prediction. Tectonophysics. 193, 267-276.
- Molchan, G.M., Kagan, Y.Y., 1992. Earthquake Prediction and Its Optimization. Journal of Geophysical Research. 97/B4, 4823-4838.
- Molchan G.M., 2012. On the testing of seismicity models. Acta Geophysica. 60, 624-637.
- Zechar, D.J., Jordan, T.H., 2007. Testing alarm-based earthquake predictions. Geophysical Journal International. 172(2), 715-724.