Conditional Warning Probabilities
Within the realm of National Weather Service (NWS) products, there are three main categories of products (ordered from longest lead time to shortest lead time): outlooks, watches, and warnings. Generally speaking, "outlooks" provide a broad overview of what areas are at greatest risk for hazardous weather at lead times of 12+ hours. Depending on the hazard, watches can be issued 1-3 days in advance or a few hours in advance, and these alerts are intended to highlight more specific areas where conditions appear favorable for hazardous weather. In the case of severe thunderstorm watches and tornado watches (the so-called "convective watches"), the lead times and durations are usually 3-9 hours. Ultimately, the alerts that get the most attention are the "warnings", which indicate that life-threatening and/or destructive conditions are already occurring or about to occur.
While it is generally accepted that areas under a watch are more likely to experience hazardous weather than areas not under a watch, very little (if any) research has been conducted on how frequently a watch becomes a warning for any given location within the watch. The same could also be said about NWS outlooks; while locations in higher risk areas are supposedly more likely to experience hazardous weather than lower risk areas, the exact probabilities of witnessing a warning have not been established.
Since tornadoes, severe thunderstorms, and flash floods typically cause most weather-related casualties in the United States, the main focus of the proceeding analysis will be for these three hazards and their attendant products.
Methodology
For convective (tornado and severe thunderstorm) watches, the probability of a warning was determined by (1) rendering the watch polygon to a discrete 0.1° latitude × 0.1° longitude grid and (2) calculating the fraction of grid points that featured at least one warning being issued between the watch's issuance time and the watch's expiration time. The polygon (a parallelogram) was chosen because a forecaster has more control over the shape of the polygon than the counties that are included (counties are fixed areas and have a wide variety of sizes and shapes). Though, it is acknowledged that the polygon can include portions of counties that are not actually included in the watch and vice versa.
If a warning was issued without a watch being active, the warning is classified as an "unwatched warning". To estimate the probability of witnessing an "unwatched warning", the total number of days with unwatched warnings within each calendar year was then calculated and divided by the total number of days in a year (365 days for a normal year, 366 days for a leap year).
Estimating warning probabilities for outlook areas involved the same fraction of area calculation used for watches. Unlike watches, outlook areas are more clearly defined, so the estimated probabilities will be more accurate. Also like the watches, the outlook areas were rendered to a discrete 0.1° latitude × 0.1° longitude grid when calculating the fractional coverage of warnings.
Part I: Warning probability as a function of watch
The first result (presented in the below table) shows the estimated probability that a location will experience a warning based on what kind of watch (if any) that location is under.
| Warning Type | No Watch | Severe Thunderstorm Watch | Tornado Watch | Particularly Dangerous Situation (PDS) Severe Thunderstorm Watch | Particularly Dangerous Situation (PDS) Tornado Watch |
|---|---|---|---|---|---|
| Tornado Warning | 0.038% | 14.9% | 23.4% | 23.4% | 36.9% |
| Severe Thunderstorm Warning | 0.43% | 72.3% | 69.6% | 83.7% | 74.8% |
To read this table, find the watch type that your location is currently under. Within that column, you'll find the probability that your location will see a tornado warning before the watch expires (first row below the header) and the probability that your location will see a severe thunderstorm warning before the watch expires (second row). As an example, if your location is under a normal tornado watch, there is a 23.4% chance that your location will see a tornado warning before the watch expires.
Flash flood warnings were excluded from this part of the analysis for the following reasons: (1) flash flood warnings frequently get issued after a convective watch has already expired or expire after a convective watch is issued and (2) the duration of flash flood warnings can actually be longer than the duration of a convective watch.
For those that aren't familiar, the Storm Prediction Center (SPC) can designate a watch as a "particularly dangerous situation (PDS)" watch if there is high confidence in a major severe weather event occurring within the watch area. For tornado watches, this means a relatively high likelihood of destructive (EF3+) tornadoes. For severe thunderstorm watches, this means a relatively high likelihood of destructive (90+ mph) wind gusts.
It should be noted that the column for "no watch" is unique in that it is the probability for a 24-hour timeframe and not the timeframe of a watch. So, if no watch is in effect, the probability that your location will see a severe thunderstorm warning on any given day is about 0.43% (about 1-2 instances per year), and the probability that your location will see a tornado warning on any given day is about 0.038% (about 1 instance every 8 years). It should be noted that this is a national average, and these particular probabilities are also influenced by what region of the country someone lives in. For example, a lot of severe thunderstorm warnings are issued in the southeast United States during the summer, but the SPC rarely issues watches for these storms because they're usually short-lived and disorganized.
One important conclusion is that the existence of a watch increases the probability that you'll see a warning by a factor of about 150-400. That's a huge increase, and it suggests that the issuance of a watch is objectively a big deal. If a watch is issued for your location, there's a greater than 50% chance that you will see a warning of some kind before the watch expires.
There are a couple of other interesting details in this finding that are worth discussing. One is that you're less likely to see a severe thunderstorm warning in a tornado watch vs a severe thunderstorm watch. This is likely due to the fact that tornado watches are usually issued when there is potential for isolated supercells to form within the watch area. Since these storms are isolated (low coverage), they tend to be more hit-or-miss. This is in contrast to a large line of thunderstorms (such as a quasi-linear convective system, QLCS), which impacts a much larger area and is more likely to prompt a severe thunderstorm warning at any given location.
Another important detail is the probability of a tornado warning within a normal severe thunderstorm watch. Usually, a severe thunderstorm watch is issued if the threats for straight-line winds and hail are expected to be more impactful than the threat of tornadoes, but the potential for tornadoes is not always 0. This is clearly reflected in the ~15% probability of seeing a tornado warning within a severe thunderstorm watch.
Finally, the issuance of a PDS watch is associated with a significantly higher probability of experiencing a warning before the watch expires. In fact, for any given location within a PDS tornado watch, there is a roughly 1 in 3 chance that a tornado warning will be issued for that location, which is significantly higher than the same outcome within a normal tornado watch.
Part II: Warning probability as a function of SPC categorical risk
The SPC can issue 7 different levels of risk on any given day: no thunderstorms, general thunderstorms (meaning thunderstorms are expected to occur, but they won't be "severe"), marginal risk, slight risk, enhanced risk, moderate risk, and high risk. Generally speaking, the higher level risks should correspond to higher probabilities of experiencing a warning. Sure enough, this expectation is confirmed in the below table:
| Warning Type | General Thunderstorms | Marginal Risk | Slight Risk | Enhanced Risk | Moderate Risk | High Risk |
|---|---|---|---|---|---|---|
| Tornado Warning | 0.0074% | 0.18% | 1.2% | 5.4% | 16.7% | 29.1% |
| Severe Thunderstorm Warning | 0.26% | 4.2% | 18.2% | 42.5% | 63.8% | 52.3% |
| Flash Flood Warning | 0.16% | 1.1% | 2.5% | 5.0% | 6.8% | 19.9% |
One pattern that is evident is the tendency to have higher warning probabilities when the SPC issues a higher risk. The one exception to this pattern is "severe thunderstorm warnings" within "high risks". I believe this is an artifact caused by manpower constraints within NWS offices on big risk days. Forecasters only have so much time to analyze a storm and issue a warning for it, and, in the case of a full-fledged tornado outbreak, it is not realistically possible to issue a severe thunderstorm warning on every single severe storm. Accordingly, some NWS offices will stop issuing severe thunderstorm warnings during a tornado outbreak, because the tornado threat is the highest priority. Since major tornado outbreaks are more likely on high risk days, this would explain why the probability of a severe thunderstorm warning actually decreases on high risk days.
Otherwise, this table is read in the same way as the table for watches. If your location is in a moderate risk, there is a roughly 1 in 6 chance that you will have to deal with a tornado warning at some point in the day. Or, put another way, about one-sixth of the moderate risk area will see tornado warnings. Such information is useful to emergency managers as it communicates approximately how much of their area of responsibility will see warnings. This can also be useful to the general public as it gives people a rough idea of what the probability of a warning is long before the severe weather event begins.
A concerning pattern is the tendency for the probability of flash flood warnings to suddenly spike on high risk days. Here's why this is concerning: If a tornado warning is issued for your location, you should take cover in a small interior room on the lowest floor of a well-built structure. If a flash flood warning is issued for your location, you should move to higher ground. What do you do if you're under both a tornado warning and a flash flood warning at the same time? The recommended protective actions contradict each other, which makes for a major public messaging problem.
Part III: Warning probability as a function of SPC hazard specific probabilities
The SPC's categorical risk is largely determined by 25 mile (40 km) neighborhood probabilities for each of the three severe weather hazards (tornadoes, 58+ mph straight-line winds, and large hail). Therefore, it would be reasonable to expect a pattern between the SPC's tornado probability forecast and the probability of being included in a tornado warning. A similar pattern should also exist between the SPC's wind and hail probabilities and the probability of being included in a severe thunderstorm warning. Sure enough, this supposition is confirmed by the following three tables:
| Warning Type | 2% Tornado Risk | 5% Tornado Risk | 10% Tornado Risk | 15% Tornado Risk | 30% Tornado Risk | 45% Tornado Risk | 60% Tornado Risk |
|---|---|---|---|---|---|---|---|
| Tornado Warning | 0.71% | 3.5% | 12.3% | 23.2% | 28.5% | -- | -- |
| Severe Thunderstorm Warning | 12.6% | 25.5% | 40.0% | 54.7% | 52.7% | -- | -- |
| Flash Flood Warning | 2.2% | 4.8% | 8.4% | 9.3% | 20.2% | -- | -- |
| Warning Type | 5% Wind Risk | 15% Wind Risk | 30% Wind Risk | 45% Wind Risk | 60% Wind Risk |
|---|---|---|---|---|---|
| Tornado Warning | 0.30% | 1.1% | 4.6% | 17.3% | -- |
| Severe Thunderstorm Warning | 4.7% | 19.9% | 46.0% | 70.9% | -- |
| Flash Flood Warning | 1.1% | 2.0% | 4.9% | 7.8% | -- |
| Warning Type | 5% Hail Risk | 15% Hail Risk | 30% Hail Risk | 45% Hail Risk | 60% Hail Risk |
|---|---|---|---|---|---|
| Tornado Warning | 0.39% | 1.4% | 5.5% | 16.5% | -- |
| Severe Thunderstorm Warning | 7.3% | 22.2% | 45.3% | 69.7% | -- |
| Flash Flood Warning | 0.89% | 1.9% | 4.7% | 10.5% | -- |
The reason why some cells are blanked out in the above tables is due to small sample size. Tornado probabilities at or above 45% are extremely rare (issued about once every 5 years). This statement also applies to 60% wind probabilities (issued about once every 5 years) and 60% hail probabilities (issued about once every 10 years). Still, there is enough of a pattern present in the more common probabilities to extrapolate what the warning probabilities should be for these rare issuances.
Generally speaking, the tornado probability issued by the SPC is roughly equivalent to the probability of experiencing a tornado warning. The main exception to this rule is the 15% tornado risk. This is likely due to the large difference between 15% and 30% tornado probabilities. Since tornadoes are the rarest of the three severe weather phenomena, a 15% change in probability is more significant than having the same probability change for more common hazards like wind and hail. There are likely several days where the actual tornado probability is 20% - 25% and therefore not quite high enough to justify a 30% tornado probability area (which itself would prompt the rare and attention-getting "high risk").
Once again, we see the same concerning pattern noted above (flash flood warnings becoming significantly more likely within the highest tornado risks). Based on this finding, NWS forecasters probably should be especially conscious about what warnings they issue during major tornado outbreaks. For instance, should a tornado warning be issued on a weak looking velocity couplet in an area extremely susceptible to flooding? Probably not, because that might encourage people to seek shelter underground where they might drown. Should a flash flood warning for low-end flooding be issued while an EF4+ tornado is approaching a major city? Probably not, because that might discourage people from seeking out an underground shelter. This is, unfortunately, a conundrum with no simple solution. Ultimately, the forecasters issuing warnings may have to decide which threat is more likely to kill people and act accordingly.
Part IV: Warning probability as a function of SPC significant severe areas
Within the tornado, wind, and hail outlooks issued by the SPC, it is possible for SPC forecasters to issue a "hatched area". On the tornado outlook, this means there is a 10% or greater probability of an EF2+ tornado within 25 miles of any given location. On the wind outlook, this means there is a 10% or greater probability of a 65+ kt (74+ mph) wind gust within 25 miles of any given location. On the hail outlook, this means there is a 10% or greater probability of a 2.00"+ diameter hail event within 25 miles of any given location.
Since the hatched area is itself a 10% probability for a "significant event", it is unreasonable to issue a hatched area if the probability of the hazard itself is less than 10%. That is, it makes little sense to say, "there's a 10% chance you'll see 2.00"+ hail today, and there's a 5% chance you'll see 1.00"+ hail today". Accordingly, hatched areas are only issued if the hazard's normal probability is itself at least 10%.
If a hatched area is present, that implies the overall potential for severe weather is higher when contrasted to an outlook with no hatched area. Accordingly, it would be reasonable to expect higher warning probabilities when examining hatched areas.
When examining probabilistic tornado forecasts, there is, as a practical matter, only one potential point of comparison (10% unhatched areas vs 10% hatched areas). It is rare for the SPC to issue a 15% tornado probability without hatching since having lots of tornadoes increases the likelihood of an EF2+ tornado occurring (by the law of averages). There are only two possible scenarios when this might occur: a tropical cyclone prolifically producing weak tornadoes, and a line of thunderstorms producing numerous brief spin-ups along the leading edge. The SPC has never issued a 30% unhatched tornado probability, and the only way this could theoretically be issued is via one of the two aforementioned scenarios hosting a particularly extreme case.
Wind and hail forecasts have more practical points of comparison. The SPC has historically issued hatched and unhatched risks for all possible hail/wind probabilities of at least 15%, although the rarest combination is a 60% unhatched hail risk (issued only once on the 0100Z outlook for February 28, 2007) followed by a 45% unhatched hail risk (which has only ever been issued 6 times and hasn't been issued since 2009). Since 60% probabilities are rare regardless of hatching, it is difficult to draw any definitive conclusions regarding the effect of hatching on warning probabilities when a 60% probability is in effect.
With all that in mind, here are three tables showing the different warning probabilities based on the existence (SIG) or absence (non-SIG) of a hatched area:
| Warning Type | 10% (non-SIG) Tornado Risk | 10% (SIG) Tornado Risk | 15% (non-SIG) Tornado Risk | 15% (SIG) Tornado Risk | 30% (non-SIG) Tornado Risk | 30% (SIG) Tornado Risk | 45% (non-SIG) Tornado Risk | 45% (SIG) Tornado Risk | 60% (non-SIG) Tornado Risk | 60% (SIG) Tornado Risk |
|---|---|---|---|---|---|---|---|---|---|---|
| Tornado Warning | 9.2% | 11.2% | -- | 23.2% | -- | 28.5% | -- | -- | -- | -- |
| Severe Thunderstorm Warning | 38.5% | 41.3% | -- | 54.7% | -- | 52.7% | -- | -- | -- | -- |
| Flash Flood Warning | 7.9% | 8.2% | -- | 9.3% | -- | 20.2% | -- | -- | -- | -- |
| Warning Type | 15% (non-SIG) Wind Risk | 15% (SIG) Wind Risk | 30% (non-SIG) Wind Risk | 30% (SIG) Wind Risk | 45% (non-SIG) Wind Risk | 45% (SIG) Wind Risk | 60% (non-SIG) Wind Risk | 60% (SIG) Wind Risk |
|---|---|---|---|---|---|---|---|---|
| Tornado Warning | 1.1% | 2.3% | 4.3% | 5.0% | 16.0% | 17.9% | -- | -- |
| Severe Thunderstorm Warning | 19.3% | 40.1% | 41.9% | 52.5% | 50.3% | 71.7% | -- | -- |
| Flash Flood Warning | 2.0% | 2.1% | 4.8% | 5.0% | 8.8% | 7.8% | -- | -- |
| Warning Type | 15% (non-SIG) Hail Risk | 15% (SIG) Hail Risk | 30% (non-SIG) Hail Risk | 30% (SIG) Hail Risk | 45% (non-SIG) Hail Risk | 45% (SIG) Hail Risk | 60% (non-SIG) Hail Risk | 60% (SIG) Hail Risk |
|---|---|---|---|---|---|---|---|---|
| Tornado Warning | 1.3% | 1.9% | 5.8% | 5.6% | -- | 16.5% | -- | -- |
| Severe Thunderstorm Warning | 21.0% | 29.2% | 38.2% | 46.2% | -- | 69.7% | -- | -- |
| Flash Flood Warning | 1.8% | 2.4% | 4.8% | 4.2% | -- | 10.5% | -- | -- |
The table with SPC tornado probabilities show the expected pattern; tornado warnings are more likely to be issued in locations where the SPC has highlighted a significant tornado risk. The same could be said for severe thunderstorm warning probabilities, but the difference between flash flood warning probabilities is insignificant.
The same general pattern shows up on the table with SPC wind probabilities, but there are a few unexpected results that are worth discussing. One is the significant difference in severe thunderstorm warning probabilities between a normal 15% wind risk and a 15% wind risk with hatching. This could partially be explained by the fact that significant wind areas tend to be small, so the fractional coverage could be inflated because of that. Still, when the wind probability is 15%, there is a two-fold increase in severe thunderstorm warning probability if a hatched area is also present for wind.
Another interesting deviation is how the probability of a severe thunderstorm warning drops a bit when comparing a 30% significant wind risk to an unhatched 45% wind risk. It is not entirely clear why such a drop might occur, but it does imply that unhatched 45% wind risks are potentially being issued too frequently.
The other detail worth highlighting on the wind probability table is the slight decrease in flash flood warning probability when comparing 45% unhatched wind risks to 45% hatched wind risks. This is likely due to the fact that higher-end wind events typically correlate with faster-moving storms, which limits the potential for rapid heavy rainfall.
Within the hail probabilities, the only discrepancy is with the tornado warning probability and flash flood warning probability between 30% unhatched hail risks and 30% hatched hail risks. However, the differences are so small that it cannot be considered significant.
Conclusions
This analysis has revealed a discernible pattern between warning probabilities and longer-lead time products (watches and outlooks). This information can help provide some perspective on what exactly is expected to happen within a watch or an SPC risk area. How to present this information to the general public in the most effective way possible is a potential avenue of future work for social scientists. However, NOAA offices, emergency management agencies, venue managers, and politicians could theoretically benefit from having the information provided in this analysis.
Aside from the potential messaging improvements, this analysis also establishes that there is an appreciable amount of forecast skill within watches and outlooks. This is especially impressive when one considers that individual NWS offices have different philosophies when it comes to issuing warnings, so the existence of such a clear pattern (Pearson correlation coefficients generally 0.90-0.95) despite these potential inconsistencies is a testament to the quality of these watches and outlooks. In other words, if these watches and outlooks were unskilled forecasts, the warning probabilities would show a more chaotic and unexplainable pattern.
This is an active research project, so additional results may be posted to this webpage in the near future.