Seasonal Streamflow Forecast in the Tocantins River Basin, Brazil: An Evaluation of ECMWF-SEAS5 with Multiple Conceptual Hydrological Models

Abstract

The assessment of seasonal streamflow forecasting is essential for appropriate water resource management. A suitable seasonal forecasting system requires the evaluation of both numerical weather prediction (NWP) and hydrological models to represent the atmospheric and hydrological processes and conditions in a specific region. In this paper, we evaluated the ECMWF-SEAS5 precipitation product with four hydrological models to represent seasonal streamflow forecasts performed at hydropower plants in the Legal Amazon region. The adopted models included GR4J, HYMOD, HBV, and SMAP, which were calibrated on a daily scale for the period from 2014 to 2019 and validated for the period from 2005 to 2013. The seasonal streamflow forecasts were obtained for the period from 2017 to 2019 by considering a daily scale streamflow simulation comprising an ensemble with 51 members of forecasts, starting on the first day of every month up to 7 months ahead. For each forecast, the corresponding monthly streamflow time series was estimated. A post-processing procedure based on the adjustment of an autoregressive model for the residuals was applied to correct the bias of seasonal streamflow forecasts. Hence, for the calibration and validation period, the results show that the HBV model provides better results to represent the hydrological conditions at each hydropower plant, presenting NSE and NSElog values greater than 0.8 and 0.9, respectively, during the calibration stage. However, the SMAP model achieves a better performance with NSE values of up to 0.5 for the raw forecasts. In addition, the bias correction displayed a significant improvement in the forecasts for all hydrological models, specifically for the representation of streamflow during dry periods, significantly reducing the variability of the residuals.

1. Introduction

Seasonal streamflow forecasting is very important for water resource management, including flood and drought control, water supply, hydropower generation, and irrigation [1,2,3]. According to Xu et al. (2014) [4], the use of forecast systems to predict future inflows can improve the performance of hydropower plants in terms of power generation. Hydropower production planning typically requires inflow forecasts for reservoirs at different lead times, being the seasonal forecasting (up to 6 months) necessary to establish yearly strategies. In that way, the predicted streamflow scenarios are used as input in optimization models to define policies for the optimal operation of the electric power system over a specific time horizon [5,6]. On the other hand, Del Sole (2004) [7] stated that a system is called unpredictable if the forecast distribution is identical, or even worse, to the climatological distribution. Hence, the definition of a seasonal meteorological forecasting system should bring more information than the climatology in order to support water managers in their decision-making process [8].

There are two main general approaches to defining a seasonal streamflow forecasting system: data-driven or statistical methods and dynamical methods. Data-driven methods aim to represent the spatial or/and temporal dependence between the given prediction, i.e., future seasonal streamflow, with some correlated variables or predictors, without any physical interpretation. Some common data-driven methods used for streamflow simulation and forecasting include the application of machine learning methods [9,10], such as support vector machines [11], regression-tree-based networks [12], and artificial neural networks (ANN) [13]. Alternatively, Zhao et al. (2016) [14] applied a Bayesian joint probability modeling approach to generate seasonal ensemble streamflow forecasts in Australia. Moreover, Hadi and Tombul (2018) [15] combined ANNs and wavelets to predict streamflows one month ahead in a river basin of Turkey.

On the other hand, dynamical methods rely on the use of numerical weather prediction (NWP) models or past observations to drive hydrological models. An early example of this approach for seasonal forecasting is the extended streamflow prediction (ESP) [16]. This method forces hydrological models with the current initial conditions of a watershed and past observed weather data to produce ensemble streamflow traces [17,18]. This approach is employed under the assumption that the historical meteorological data are representative of the region and have the same probability of occurrence in the future. Currently, this method is also used as a benchmark for seasonal streamflow forecasting systems that adopt more complex NWPs, such as general circulation models (GCMs), on a global scale, and regional climate models (RCM), providing meteorological information for future conditions in the atmosphere, which can be used as input in hydrological models for streamflow forecasting [19].

In recent years, the use of operational forecasting systems that coupled GCM or RCM with hydrological models, at a basin scale, has increased, thanks to the improvement in products, such as their accuracy and resolution [20,21]. World-scale seasonal forecasting products are mainly provided by global climate centers, such as the United States National Center for Environmental Prediction (NCEP), the UK Meteorological Office (UKMO), the Australian Bureau of Meteorology, and the European Centre for Medium-Range Weather Forecasts (ECMWF). The latter provides the Seasonal Forecasting System (SEAS), which constitutes the state-of-the-art global modeling of seasonal climates and forecasts. The SEAS is now in its fifth version and includes many improvements since its beginning in 1997, as described by Johnson et al. (2019) [22]. Ferreira et al. (2022) [23] evaluated the ECMWF-SEAS5 seasonal temperature and precipitation predictions over South America. The authors showed that the system presented a better performance in regions such as northern South America and northeastern Brazil, in which ECMWF-SEAS5 can reproduce the extreme precipitation anomalies observed in the last decades well. In addition, they indicated the need for improvement in the quality of the ECMWF-SEAS5 seasonal rainfall predictions for specific domains in South America, particularly the prognostic forecasts made as of 2017. Thus, there is still a need for regional studies in the literature that utilize ECMWF-SEAS5 to evaluate the representation of streamflow in tropical river basins through the use of hydrological models.

Hydrological models are typically classified into empirical, conceptual, and physical based, according to their parametrization and representation of the main hydrological processes. Empirical models represent the system by an input–output relationship between the available data. This relationship can be represented by linear or non-linear functions. Conceptual models use simplified mathematical conceptualizations of the watershed, representing the main hydrological process and fluxes by interconnected storage. Finally, physical-based models employ the main scientific principles of energy and mass conservation in water fluxes, taking into account the spatial heterogeneity conditions of the watershed into a grid resolution. The selection of each approach commonly relies on data availability, the heterogeneity of the watershed, and the application of the model itself [24]. Moreover, considering a unique hydrological model for all kinds of applications in water resources can be tricky. For that reason, many studies focus on comparing the performance of hydrological models with different levels of complexity [25]. In the case of conceptual hydrological models, Staudinger et al. (2011) [26] used the Framework for Understanding Structural Error (FUSE) to demonstrate that a single model structure could present a poor performance to simulate low flow and recession behavior over different seasons. Instead, they argued that combining different model structures could be a more robust approach. Ghimire et al. (2020) [27] evaluated four hydrological models, including GR4J, IHACRES-CWI, HEC-HMS, and SWAT, to simulate daily streamflow in the main river basin of Myanmar. The authors concluded that all hydrological models are useful for representing daily streamflow time series at the majority of selected stations for the period from 2001 to 2014. Jiang et al. (2007) [28] used six monthly water balance models to investigate the potential impacts of human-induced climate change on water availability in the Dongjian basin, South China. Jaiswal et al. (2020) [29] compared the use of conceptual and physical-based models to simulate the runoff in the Tandula basin of Chhattisgarh (India). Darbandsari and Coulibaly (2020) [24] made a comparison of seven lumped hydrological models in data-scarce watersheds using different precipitation-forcing datasets. The authors highlighted the performance of the ACSMA and GR4J hydrological models. Finally, Ávila et al. (2022) [30] compared four conceptual hydrological models (GR4J, HYMOD, HBV, and SMAP) and one semi-distributed hydrological model (MGB-IPH) in the calibration and validation stages to represent daily streamflow time series at six hydropower plants in the Tocantins River Basin. The authors argued that conceptual models with more simple structures can exhibit similar performance to the semi-distributed model.

The main objectives of the present study are the evaluation of the ECMWF-SEAS5 dataset in large-scale tropical river basins located in Brazil and the comparison of different conceptual hydrological modeling approaches to generate seasonal streamflow scenarios up to seven months ahead. The results will help the operation and decision-making of hydropower water reservoirs. The tested hydrological models are GR4J, HYMOD, SMAP, and HBV, applied to six different hydropower plants in the Tocantins River Basin, Brazil. A post-processing procedure proposed by Woldemeskel et al. (2018) [31] is applied to correct the seasonal streamflow forecast. Deterministic and probabilistic metrics are employed to evaluate the performance of each hydrological model. The remainder of this paper is organized as follows: Section 2 presents the methodology, including the description of the hydrological models, the input data, the performance metrics, and the post-processing approach to correct seasonal streamflow forecasts. Section 3 describes the case study and discusses the main results. Finally, Section 4 presents the main conclusions.

2. Methodology

The study compared four conceptual hydrological models (GR4J, HYMOD, HBV, and SMAP) to generate seasonal streamflow forecasts at different hydropower plants located in the Tocantins River Basin, using daily precipitation forecast data of the ECMWF-SEAS5 for the period 2017–2019. A bias correction procedure was applied for precipitation forecasts based on the delta method, considering the hindcasts of ECMWF-SEAS5. Furthermore, a post-processing procedure was performed to correct the seasonal streamflow forecast for all hydrological models. Figure 1 provides a general framework of the adopted methodology, including the model inputs and main processes.

2.1. Hydrological Models

This section briefly describes each adopted hydrological model. Figure 2 shows each model structure, and

Table 1. Characteristics of the model structure of the conceptual hydrologic models. P: Daily precipitation; PET: Potential evapotranspiration; LMT: Long-term monthly temperature; LMPET: Long-term monthly potential evapotranspiration.

Model Feature	GR4J	HYMOD	HBV	SMAP
Parameters	4	5	11	11
Input data	P; PET	P; PET	P; T; LMT; LMPET	P; PET
Conceptual storage	Production soil storage	Soil moisture layer	Soil moisture layer	Upper soil reservoir
	Routing soil storage	Quick flow reservoirs	Upper-zone storage	Second upper-soil reservoir
		Slow flow reservoir	Lower-zone storage	Lower soil reservoir
				Ground storage
Type of flows	Fast flow	Surface flow	Surface flow	Surface flow
	Slow flow	Ground water flow	Base flow	Base flow

summarizes the structural characteristics of each one, including the number of parameters, the conceptual storage and flow type, and required input data. The models were calibrated on a daily scale, using as input the observed total daily precipitation and potential evapotranspiration. The Shuffled Complex Evolution (SCE-UA) algorithm was adopted to estimate the parameters of each model using the Nash–Sutcliffe coefficient (NSE) as the objective function [32]. Each sub-catchment was calibrated in sequence from upstream to downstream, considering the hydropower plant time series at its outlet, and its parameters were obtained for each hydrological model. The calibration and validation stages were performed for the periods 2014–2019 and 2005–2013, respectively. The period from 2014–2019 was chosen for the calibration stage because it represents the longest period of continuous daily observations in the Tocantins River Basin. In addition, this selection allows for the adjustment of each parameter based on the most recent hydrological conditions observed in the sub-catchments. Given that all models have a lumped structure, we used a triangular weighting function for routing the hydrograph between sub-watersheds. More details about each hydrological model can be obtained in the provided references in each specific section.

2.1.1. HYMOD Model

The HYMOD (The HYdrological MODel) is a conceptual model with five parameters that simulate the daily runoff conditions by the use of a rainfall excess model according to the probability-distributed principle [33]. This probability-based equation considers the basin’s maximum water storage capacity ($C_{max}$) and the degree of spatial variability within the basin ($\beta$) to calculate the cumulative rate of storage capacity. Then, the effective precipitation is estimated according to the potential evapotranspiration (PET) and the soil moisture capacity. Finally, a portion of the excess precipitation is separated according to the $\alpha$ parameter and goes directly to the three identical quick-flow reservoirs to represent the surface flow, whereas the rest of the precipitation volume ($1-\alpha$) flows to the slow reservoir. Each kind of reservoir has its corresponding residence time ($K_q$ and $K_s$), composing the total runoff.

2.1.2. GR4J Model

GR4J (The Génie Rural à 4 paramètres Journalier) is a parsimonious model with four parameters developed by Perrin et al. (2003) [34] that employs two reservoirs (production and routing) and two unit hydrographs (HU1 and HU2) to represent the rainfall-runoff conditions in a watershed [35,36,37,38]. At first, the model estimates the effective precipitation (PE) based on PET, and part of the effective precipitation goes into the production storage where the percolation occurs. The runoff is generated both by leakage of percolated water and direct precipitation. Ninety percent of the runoff is routed by HU1 and the routing store. The other 10% of the runoff is routed by UH2, and the total runoff is obtained by adding these two runoff components.

2.1.3. SMAP Model

The SMAP (Soil Moisture Accounting Procedure) model proposed by Lopes et al. (1982) [39] considers a structure with three linear reservoirs to represent different layers of the soil, employing the Soil ConservationService (SCS) method for the separation of runoff. The Brazilian National Electrical System Operator (ONS) [40] proposed a new version of this model, called the SMAP-ONS model, by introducing one additional linear reservoir and other coefficients to adjust the temporal representation of the input data and the recession curves for the base and superficial flows. The SMAP-ONS has been applied in different Brazilian watersheds and combined with data-driven models to flood conditions [41,42] and daily inflow forecasts for hydropower generation [43].

2.1.4. HBV Model

The HBV (Hydrologiska Byrans Vattenbalansavdelning) model is a conceptual snow–rain water balance model proposed by Bergstrom (1992) [44]. The main input variables of the HBV model are daily precipitation, daily mean temperature, and mean interannual monthly evapotranspiration and temperature. The HBV model includes a soil routine, a response routine, and a snow routine. This last one can represent the precipitation as rain or snow, depending on the daily temperature. For regions where the mean temperature is greater than the melting point, and snow does not occur, the snow routine can be excluded, so the total precipitation goes directly to the soil routine. The total runoff is generated by two runoff reservoirs that use linear and non-linear functions and represent the quick flow (upper zone—UZ) and the base flow (lower zone—LZ). We adopted the HBV model version presented by Aghakouchak and Habib (2010) [45].

2.2. ECMWF-SEAS5 Data

The European Centre for Medium-Range Weather Forecasts—System 5 (ECMWF-SEAS5) consists of a 51-member ensemble starting on the first day of every month up to 7 months ahead (214 days) [22]. This system also includes retrospective seasonal forecasts from past decades (hindcast) with the aim of comparing and calibrating the forecasting system to historical records. The hindcasts have a 25-member ensemble starting on the first day of every month from 1993 to 2016. We considered the ECMWF-SEAS5 surface data of daily total precipitation, covering the hindcast data from January 2000 to December 2016 and forecast data from January 2017 to December 2020, with a horizontal grid resolution of 1${}^{\circ }$×1${}^{\circ }$.

Similarly to other forecasting systems, the ECWMF-SEAS5 presents a random and systematic error. To deal with the random error, the ECWMF-SEAS5 employs an ensemble method, such as all members are initialized on the same date but with slightly different initial state conditions. Those differences are generated in the atmospheric module, which presents an initial undisturbed condition for the first member of the ensemble, and then, for all other members, perturbations are applied to some fields to represent the uncertainty of the atmosphere’s initial state [22].

Moreover, the systematic errors are presented through the difference between the simulated atmospheric and oceanic states with the observed climatological conditions. In this study, we used the delta method and the hindcasts of the ECMWF-SEAS5 to correct the bias for the forecast period, assuming that the bias is constant over the forecast horizon. First, we calculated the daily climatology of the hindcast $\overline{P_{clim}\left(k\right)}$ for the period 2000–2016. Then, for a given position $\mathbf{k}=(lon,lat)$ on the grid, the corrected daily precipitation members for a time t can be estimated using the delta method [46,47], such as:

$$P_{corrected}(\mathbf{k},t)=P_{raw}(\mathbf{k},t)·{\displaystyle \frac{\overline{P_{clim}\left(\mathbf{k}\right)}}{\overline{P_{raw}\left(\mathbf{k}\right)}}}$$

(1)

2.3. Post-Processing Procedure

We adopted the post-processing method proposed by Woldemeskel et al. (2018) [31] to correct the streamflow forecasts. For this method, a statistical model is adjusted to the normalized streamflow forecast residual (${\eta }_t$) over a calibration period, such as:

$$\eta =Z\left(\stackrel{\sim }{Q_t}\right)-Z\left(Q_t^F\right)$$

(2)

where $\stackrel{\sim }{Q_t}$ is the observed streamflow, $Q_t^F$ is the median of the uncorrected streamflow forecast ensemble, and Z is a transformation function, defined in this study as the logarithmic (Log) transformation.

The residuals $\eta$ are standardized in order to account for the seasonal variations in the distribution, such as:

$$v_t=({\eta }_t-{\mu }_{\eta }^{m\left(t\right)})/{\sigma }_{\eta }^{m\left(t\right)}$$

(3)

where ${\mu }_{\eta }^{m\left(t\right)}$ and ${\sigma }_{\eta }^{m\left(t\right)}$ are the monthly mean and standard deviation of the residuals in the calibration period for the month $m\left(t\right)$.

Then, the standardized residuals $v_t$ can be described by a first-order autoregressive (AR(1)) model with Gaussian innovations such as:

$$v_{t+1}=\rho v_t+y_{t+1}$$

(4)

where $\rho$ is the AR(1) coefficient and $y_{t+1}\sim N(0,{\sigma }_y)$ is the innovation.

To perform the post-processed streamflow forecasts for a given period and ensemble member j, we calculated a sampled innovation $y_{t+1,j}\to N(0,{\sigma }_y)$. Then, we estimated the standardized residual $v_{t+1,j}$ (Equation (4)). Here $v_{t,j}$ is computed using Equation (3), and ${\eta }_{t,j}$ is computed using Equation (2), considering the observed and forecasted streamflow from time step t-1.

Next, we computed the normalized residuals ${\eta }_{t+1,j}$ as:

$${\eta }_{t+1,j}={\sigma }_{\eta }^{m\left(t\right)}{v}_{t+1,j}+{\mu }_{\eta }^{m\left(t\right)}$$

(5)

Finally, the post-processed streamflow forecast is calculated as:

$$Q_{t+1,j}^{PP}=Z^{-1}[Z\left(Q_{t+1}^F\right)+{\eta }_{t+1,j}]$$

(6)

2.4. Performance Metrics

The Nash–Sutcliffe coefficient (NSE) tests the predictive power of hydrological models and is used in this study as an objective function to calibrate the parameters of each hydrological model. The NSE presents a value equal to 1 when the model achieves perfect performance, whereas a value of 0 suggests that the model is as accurate as the mean of observed data [48]. In addition, the NSElog metric, which considers the logarithmic of the streamflows values, is useful for evaluating the representation of low flows. The NSE is calculated by Equation (7) as:

$$NSE=1-{\displaystyle \frac{{\sum }_{t=1}^T{(Q_{sim}\left(t\right)-Q_{obs}\left(t\right))}^2}{{\sum }_{t=1}^T{(Q_{obs}\left(t\right)-\overline{Q_{obs}})}^2}}$$

(7)

where $Q_{obs}\left(t\right)$ is the observed streamflow at the tth time step; $Q_{sim}\left(t\right)$ is the simulated streamflow at the tth time step; and $\overline{Q_{obs}}$ is the mean of the observed data.

The percent bias (PBIAS) measures the average tendency of the simulated values to be larger (positive PBIAS) or smaller (negative PBIAS) than their observed ones [49]. The PBIAS is estimated by Equation (8) as:

$$PBIAS={\displaystyle \frac{{\sum }_{t=1}^T(Q_{sim}\left(t\right)-Q_{obs}\left(t\right))}{{\sum }_{t=1}^T{Q}_{obs}\left(t\right)}}$$

(8)

The multi-criteria distance (DM) represents the Euclidean distance between the inverse of the NSE coefficient and the mean absolute percentage error (MAPE). This metric balances the power prediction of the hydrological model between long- and shot-term conditions. The DM metric is estimated by Equation (9) as:

$$DM=\sqrt{{(1-NSE)}^2+MAP{E}^2}$$

(9)

where the MAPE value is calculated as:

$$MAPE=\sum _{t=1}^T\left|{\displaystyle \frac{Q_{obs}\left(t\right)-Q_{sim}\left(t\right)}{Q_{obs}\left(t\right)}}\right|$$

(10)

The CRPS metric compares the difference between the cumulative distribution of the observed and forecast values and summarizes the reliability, sharpness, and bias attributes of the forecast [50].

$$CRPS={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\int }_{-\infty }^{\infty }{[F_i\left(y\right)-H_i\{y\le y_o\}]}^{2}dy$$

(11)

where $F_i$ is the cumulative distribution function of the forecast for year i; y is the forecast value; $y_o$ is the observed value; and $H_i\{y\le y_o\}$ is the piecewise Heaviside step function.

Given that an objective interpretation of the forecast error can not be made using only the CRPS, we also estimated the CRPSS, which compares the CRPS between the forecast system ($CRP{S}_{fest}$) and a benchmark ($CRPS{S}_{ref}$). In this case, we adopted the climatology of flows at each hydropower plant as the benchmark, and the CRPSS is estimated as:

$$CRPSS=1-{\displaystyle \frac{CRP{S}_{fest}}{CRPS{S}_{ref}}}$$

(12)

Here, higher values of CRPSS indicate better performance and a value of 0 represents the same performance as climatology.

The ROC curve (receiver operating characteristic curve) is a graphic that assesses the reliability of a forecast system by classifying the observed and forecasted values as discrete outcomes (true or false) [51]. Given a classifier, there are four possible outcomes: true positive, false positive, false negative, and true negative. A contingency table (

Table 2. Two-by-two contingency table.

Forecasted Outcome	Observed Outcome
	True	False
True	True positive (A)	False positive (B)
False	False negative (C)	True negative (D)

Note: A, B, C, and D are the numbers of true positives, false negatives, false positives, and true negatives, respectively.

) is constructed to represent the possible set of instances.

Based on Table 2, we can estimate the probability of detection (POD) and the probability of false detection (POFD) as:

$$POD={\displaystyle \frac{A}{A+C}}$$

(13)

$$POFD={\displaystyle \frac{B}{B+D}}$$

(14)

While the POD examines the fraction of observed events that was predicted correctly, the POFD evaluates the fraction of non-events that was incorrectly predicted. The value of the POD and POFD produces the ROC curve, the area under the ROC curve (AUC) can be used to summarize a ROC curve [52]. The value of the AUC ranges from 0 to 1, and a random guess procedure can have an AUC of 0.5, corresponding to the diagonal line in the ROC curve.

3. Case Study

3.1. Overview and Data

The Tocantins River Basin is located in the North, Central-West, and Northeast regions of Brazil and covers a drainage area of 306.200 km ${}^2$ (Figure 3). This basin is characterized by the presence of many water reservoirs used for many social and economic purposes and is one of the most important drainage areas in terms of hydropower production for the country [20,53,54]. In that way, short-term streamflow forecasts have been reported in previous studies to evaluate the hydropower generation and water resources conditions in this river basin [20,54], whereas seasonal streamflow forecasts have been less explored.

The Tocantins River has an annual mean temperature of 26 °C and an annual precipitation of 1.770 mm, approximately. The Köppen’s type climate classification indicates tropical with dry winter conditions in this region [55], and the most predominant biomes are the Brazilian Savanna and Cerrado, covering 97.8% of this drainage area [56]. The Tocantins River Basin presents a well-defined seasonal condition. The wet period occurs from October to April, and the dry period commonly occurs from May to September (Figure 3).

Observed daily rainfall data from 83 rain gauges were obtained from the Brazilian National Agency of Waters (ANA) for the period 2010–2019. Climatological records of 10 climatic stations from the Brazilian National Institute of Meteorology (INMET) provided data for temperature, humidity, sunshine, and wind speed, which were later used to calculate the potential daily evapotranspiration by the Pennam–Monteith equation [57]. Finally, naturalized streamflow data for the six hydropower plants of Tocantins’ River were provided by the Brazilian National Electrical System Operator (ONS).

3.2. Results Analysis

Before analyzing the results of the streamflow simulations by all hydrological models, we first evaluated the bias correction of the precipitation forecasts. For this study, we adopted the delta method to correct the systematic error, considering the hindcast daily climatology instead of the observed daily precipitation. This selection was made given the scarcity of rain gauges with continuous daily precipitation data distributed in this river basin. In order to validate the assumption, Figure 4 compares the hindcast and observed daily climatology at one rain gauge in the Tocantins River Basin. The graphic shows that the hindcast can well represent the periodic behavior of daily precipitation observed in this region. In addition, the figure compares the cumulative precipitation between the observed data, the hindcast daily climatology, the raw ensemble mean, and the corrected ensemble mean for the period between 1 March 2017 and 30 September 2017. The results show that for this period, the bias-corrected data exhibit a total precipitation volume close to the observed data and the hindcast climatology. On the other hand, the raw precipitation shows a drier condition for the same period.

Figure 5 presents the deterministic metrics obtained during the calibration and validation stages at each hydropower plant for all hydrological models. Based on the NSE and the NSElog metrics, the results show that for the calibration period, all hydrological models achieve satisfactory performance with values up to 0.5. In particular, the HBV model displayed the best performance, with DM values close to 0.2 and a mean bias percentage equal to 4%. For the validation period, the SMAP model exhibits the lower NSE values, presented at the Estreito UHE, and also shows the poorest performance to represent low flow values compared to the other hydrological models. For the validation period, all models present negative PBIAS values, indicating that the observed values are overall greater than the simulations. In addition, we remark on the performance of the GR4J and HYMOD models, as they achieved a good performance in representing the seasonal regimes in the Tocantins River Basin for both calibration and validation stages, despite their simplicities when compared to other models.

After the calibration of all hydrological models, we executed the seasonal forecasts of the ECMWF-SEAS5, considering a lead time of 214 days (7 months). In this study, we adopted the monthly average streamflow for each forecast instead of working with a daily scale. The main reason for this choice is that the Brazilian National Interconnected System (SIN) considers monthly streamflow time series for the medium- and long-term planning of its electrical power system [58,59]. Figure 6 compares the hydrographs of the observed streamflow, as well as the corrected and uncorrected forecasts obtained by each hydrological model at the Estreito UHE. In addition, we present the corresponding 90% and 50% CI interval for each case. It is worth mentioning that to avoid overfitting for the corrected forecasts, we followed the moving-window cross-validation framework proposed by [31]. The results show that the forecasts can represent the seasonal regimes observed in this region well. Furthermore, after the post-processing procedure, the forecasts show a significant improvement in the representation of both the recession curve and peak flows, specifically for the SMAP and HBV models. The simulated streamflow members also reflect less uncertainty during the dry periods, given the strongly seasonal conditions in this region, which is well represented by ECMWF-SEAS5. However, it is necessary to evaluate if the recession curve during those periods is under- or overestimated in comparison with the observed data.

Figure 7, Figure 8, Figure 9 and Figure 10 display the error bars of the PBIAS, NSE, NSElog, and DM metrics, respectively, for the entire period (2017–2019) as a function of the lead time (7 months), considering the mean of the streamflow members. The figures compare the performance of the uncorrected and corrected forecasts for all hydrological models. The results show that PBIAS (Figure 7) exposed an enhancement in the systematic error for all hydrological models, reducing the variability of the error bars by up to 10%. For the NSE metric, the results show that the SMAP model presents a better performance for the uncorrected forecasts, which decrease over the horizon time. Alternatively, the other models exhibit negative NSE values. After the post-processing procedure, all hydrological models show a significant improvement in the NSE metric with values greater than 0.7. The improvement in the value of the deterministic metrics is also presented for the NSElog and DM metrics after executing the post-processing procedure. In this case, the NSElog values for the uncorrected forecasts are greater than 0.6, whereas the corrected forecasts present NSElog values greater than 0.9, and the DM presents values close to 0.25 after the bias correction.

Figure 11 shows the relationships between the forecasted streamflow and the corresponding relative error for all hydrological models at three hydropower plants to examine the homoscedasticity condition of the residuals for the entire period. The relative error is calculated as the difference between the simulated and observed streamflow divided by the mean of the observed streamflow time series. In this case, the selected UHEs are located in the upper- (Serra da Mesa), middle- (Cana Brava), and downstream (Estreito) regions in order to present the error propagation through the river basin. Even though the residuals still present heteroscedasticity, being greater for wet periods and peak flows, the results show a significant reduction in the forecasted residuals after the bias correction, specifically for the Serra da Mesa UHE. In addition, the SMAP model presents fewer differences in the obtained residuals between the corrected and uncorrected forecasts.

Figure 12 shows the error bars of the CRPSS for all hydrological models along the 7-month horizon time. The figure describes the performance of the CRPSS for the periods that initiate with wet and dry conditions, as well as for the entire forecast period (all months). For the uncorrected forecasts, the results show that the dry months present greater bias with the observed data in comparison with the wet months. After the post-processing procedure, the bias-corrected forecast data present a significant improvement in the skill for the entire forecasted period after the first month. For the raws forecasts (uncorrected), the CRPSS shows that the SMAP model represents the seasonal variability of all hydropower plants compared to the climatology with more reliability, whereas GR4J exposed lower performance. This improvement is also displayed when evaluating the forecasts that initiate during the dry months, indicating less uncertainty in the prediction of streamflow data. On the other hand, the CRPSS values obtained for the wet months do not present greater differences between the corrected and uncorrected forecasts, with the exception of the third prediction month. The differences exposed in the CRPSS between those two periods are related to the strongly seasonal conditions in this region, making the dry periods exhibit less variability compared to wet periods. Therefore, the recession curves of the hydrograph are similar for different years.

Figure 13 presents the ROC curves obtained from the seasonal forecasts at the Lajeado UHE. In this case, we measure the accuracy of each model to represent the occurrence of different thresholds defined by the quantiles Q90, Q75, Q50, Q25, and Q10 of the monthly permanence curve. The results indicate that all hydrological models present better accuracies to represent the occurrence of streamflow equal to Q50 and Q75, respectively. The HBV exposed the lower accuracy to represent lower flows (Q90), whereas the GR4J and HYMOD models presented similar performances to represent high flows (Q10). Finally, based on this metric, the SMAP model indicates a more reliable performance to represent seasonal forecasts at this hydropower plant.

A broad analysis is presented in Figure 14, considering the AUC, which summarizes the ROC diagram into one numerical value and allows for an easier comparison of forecast systems. In this case, values close to 1 indicate better discrimination between events and non-events [60]. We compared the AUC for the corrected and uncorrected forecasts considering the thresholds Q90, Q75, Q25, and Q10. For this case, we excluded the median of the distribution represented by Q50. The results show that a greater improvement is presented after the bias correction procedure in the representation of low flows (Q90 and Q75). For high flows (Q10), the results exhibit better accuracy, especially for the hydropower located in the lower river basin.

4. Conclusions

This study evaluated the performance of the ECMWF-SEAS5 global precipitation product to predict seasonal streamflow regimes in a tropical river basin located in Brazil. The rationale for developing the obtained results was to address an improvement in the seasonal streamflow forecast system in Brazil, as well as in the operation of hydropower reservoirs and water resource management in the country. Therefore, it is expected that this evaluation will enhance the use of products of the ECMWF global forecast system in river basins located in South America. In general, the results showed that ECMWF-SEAS5 can represent the seasonal rainfall conditions in the studied area well, including the occurrence of extreme events. In addition, we compared four conceptual hydrological models with different structures in order to account for the uncertainties associated with the hydrological processes presented in a large-scale river basin. The results exposed that the hydrological models presented a satisfactory performance in representing the inflows at all six hydropower plants. However, none of the selected models presented a superior performance in representing the streamflow regimes for all the evaluation stages, including calibration, validation, and forecasting. For instance, for the calibration and validation stages, the HBV presented better adaptation to the observed data compared to the other hydrological models. For the validation stage, the SMAP model presented the lower NSE and NSElog metrics in some sub-catchments, with mean values close to 0.5 and 0.65, respectively. In addition, all hydrological models exposed a negative bias for this stage, indicating that observed streamflows are systematically greater than the simulations. Those differences are specifically observed during the recession curves of the hydrographs.

For the uncorrected forecasts, the results showed a poor performance based on the deterministic metrics for all sub-catchments and hydrological models. For instance, negative NSE values were exposed for the GR4J, HBV, and HBV models, whereas the SMAP model achieved values of up to 0.5. The applied post-processing scheme substantially improved the reliability of streamflow forecasts, both in terms of deterministic and probabilistic metrics. This improvement was exposed mainly in the representation of flows during dry periods. The main advantage observed for the adopted post-processing procedure is the moving window and cross-validation framework, which allows for representing the time-varying streamflow conditions in the watershed. Based on PBIAS, the forecasted streamflow presented values close to 10% for all hydrological models, with the exception of the first month. This behavior can be attributed to the differences in the initial conditions of the hydrological models and the watersheds at the beginning of each forecast. In this case, additional correction procedures can be performed to adjust the initial state conditions of each hydrological model.

According to CRPSS, we concluded that forcing different hydrological models with the ECMWF-SEAS5 precipitation product represents a seasonal streamflow forecast system more reliable than the climatology in tropical river basins located in South America. This result can be verified by comparing both the raw and corrected forecasts for the different lead times. After the bias correction, CRPSS presented a significant improvement in the forecasts executed during dry months. Alternatively, the bias correction showed an improvement for the forecasts executed during wet months only during the first three months. Finally, the ROC curve exposed some differences in the performance of the adopted hydrological models. For instance, the HBV model presented a greater probability of false detection to measure the occurrence of low flows (Q90), while GR4J and HYMOD have the same conditions for high flows (Q10).

This study highlights the importance of considering more than one hydrological modeling approach to represent the streamflow conditions in a large-scale river basin, given the uncertainties associated with the different hydrological processes. The comparison of different hydrological modeling approaches showed that the performance of each model can differ significantly according to the evaluated period. This variation can be related to the different uncertainty sources associated with the calibration procedure, the model structure, the parameters, and the input data. In addition, the perception of each hydrological model commonly follows the main hydrological process presented in the regions they are developed in. For instance, the SMAP model, adapted to represent the hydrological conditions of watersheds located in Brazil, presented a lower performance for the validation stage in comparison with GR4. However, this model presented better adjustment with the observed data for the seasonal forecasts. On the other hand, the HBV model, which can integrate a snow component and has the same number of parameters as the SMAP model, did not show a good performance for the forecasts, despite this model showing the best adjustment during the calibration stage.

For this study, we only considered conceptual approaches to representing streamflow conditions. However, this study could be broadly extended by considering the use of empirical or physical modeling approaches. The suggested directions for future studies included the comparison of additional global forecast systems and the evaluation of the adopted hydrological modes for sub-seasonal streamflow forecast and short-term forecasting coupled with data assimilation techniques.