6  Best Practices in Ecological Forecasting

Ecological forecasting is an emerging field and a community of practice. A strong community of practice is associated with best practices that guide how to generate, analyze, and community forecasts that reflect state-of-the-art knowledge. The term “best practices” is used because “better” practices will always developed by an active research field so the “best” are the best at that time.

Multiple papers have explored best practices in ecological forecasting (e.g., Harris, Taylor, and White (2018), Lewis et al. (2022)), including best practices through an ethics lens (Hobday et al. (2019)).

The Lewis et al. (2022) best practices of forecasting, as written in the manuscript, are:

• Include uncertainty
• Report forecast accuracy
• Identify an end user
• Make iterative forecasts
• Automate forecasting workflows
• Make data available
• Archive forecasts
• Use null model comparisons
• Compare modeling approaches

The Hobday et al. (2019) principles of ethical forecasting, as written in the manuscript, are:

• Be open and transparent
• Do not deliver forecasts that would lead to unregulated impacts on the [environment]
• Undertake best practice skill assessment that tests the true skill of a model with out-of-sample testing.
• Do not ignore uncertainty
• Plan for and manage stakeholder expectations regarding continued delivery.
• Work to improve the literacy of all stakeholders around forecast use and interpretation, particularly on skill and uncertainty.
• Proactively explore the impact of the loss of a predictor variable in a forecast system, and be able to explain what the loss of performance is when one variable is removed
• Be vigilant for inequity in the use of forecasts between users, and the creation of winners and losers arising from the provision of information
• Scope the system context widely, seek deep domain and system knowledge, and consider scenario testing, as happens for [ecosystem] management regulations now.
• Consider the holistic outcome of the forecast system—if it is not achieving the overall goals, suspend delivery and work on improving the interaction of the forecast and the context in which it operates.

6.1 Reading

Hobday, A. J., Hartog, J. R., Manderson, J. P., Mills, K. E., Oliver, M. J., Pershing, A. J., & Siedlecki, S. (2019). Ethical considerations and unanticipated consequences associated with ecological forecasting for marine resources. ICES Journal of Marine Science. https://doi.org/10.1093/icesjms/fsy210

Lewis, A. S. L., Woelmer, W. M., Wander, H. L., Howard, D. W., Smith, J. W., McClure, R. P., et al. (2022). Increased adoption of best practices in ecological forecasting enables comparisons of forecastability. Ecological Applications, 32(2), e02500. https://doi.org/10.1002/eap.2500

6.2 Assignment

1. Read the two papers above
2. Find a documented example of an “operational” ecological forecast, either in the literature or website. An operational one means that it generated for applications beyond scientific literature, though ideally journal articles describe the forecasting approach and performance. However, many operational forecasts, like those produced by NOAA, are not clearly documented in the literature so website documentation is adequate for this assignment.
3. Describe the forecasted variable, forecasting approach, how it is iteratively updated through time, and how it addressed the best practices in the two papers above.

Harris, David J., Shawn D. Taylor, and Ethan P. White. 2018. “Forecasting Biodiversity in Breeding Birds Using Best Practices.” PeerJ 6. https://doi.org/10.7717/peerj.4278.

Hobday, Alistair J., Jason R. Hartog, John P. Manderson, Katherine E. Mills, Matthew J. Oliver, Andrew J. Pershing, and Samantha Siedlecki. 2019. “Ethical Considerations and Unanticipated Consequences Associated with Ecological Forecasting for Marine Resources.” ICES Journal of Marine Science. https://doi.org/10.1093/icesjms/fsy210.

Lewis, Abigail S. L., Whitney M. Woelmer, Heather L. Wander, Dexter W. Howard, John W. Smith, Ryan P. McClure, Mary E. Lofton, et al. 2022. “Increased Adoption of Best Practices in Ecological Forecasting Enables Comparisons of Forecastability.” Ecological Applications 32 (2): e02500. https://doi.org/10.1002/eap.2500.