What makes a GOOD water supply forecast?... a BAD forecast?


Is it as simple as which forecast comes closest to the actual observation? Probably not, as a number of factors necessitate a more sophisticated evaluation of forecast quality be undertaken. Such an evaluation would not be trivial and is beyond the time and space constraints of this note. Nonetheless, with apologies for simplification and omission, some of the factors include:

subsequent meteorologic conditions - the implicit assumption behind any forecast is that the meteorologic conditions during the remainder of the snow accumulation and melt season will be “normal.” While it may be difficult to adequately define what “normal” is, it is easier to discern conditions that are extreme or “not normal.” As such, a given forecast at a given time may have been the best forecast possible in light of known conditions, although ultimately turning out to be 20% too low; it just so happened that the ensuing meteorologic conditions were unusually wet. Just as a good forecast may be made to look bad by abnormal conditions in the future, the reverse situation is also possible.

natural variability of site’s streamflow - simply put, some rivers are much more difficult to forecast than others. Historically, such river flows may vary over a wide range and be quite sensitive to changing conditions, particularly in environs where the number of precipitation events are few. On the other hand, some river flows may be relatively constant with the effects of diverse conditions dampened. Oftentimes scale is a good indicator of the variability of flow at a given site. A 20% error on a small stream in Arizona may be more laudable than a 10% error on Lake Powell inflow.

character of the year - by definition, extreme events are rare and forecasting such events becomes more difficult. Because the number of past extreme events is small, less is known about the distribution and variability than in situations with “near-normal” populations. Even if it was possible to remove uncertainty about future meteorological conditions, there would still be more error associated with forecasting extreme events.

During the extreme conditions there is a demand that the forecaster make a more powerful (and potentially more valuable) statement: in effect, that “even normal conditions from here on out will not be enough to compensate for current abnormal snowpack and soil states.” It is during such events that consideration of information other than just the most probable forecast become especially important. Probability statements that convey the likelihood of exceeding a certain level (such as the reasonable maximum and minimum forecasts) help to underscore the uncertainty associated with the forecast.
So why do it? although it may not be a simple matter to grade a forecast, it is still useful for users and forecasters alike to review the previous year’s forecasts and adjusted observations (provisional as they may be with estimated diversions) so as to act on obvious problems and to gain perspective for the coming forecast season.