Analysis of Artificial Neural Networks for Forecasting Photovoltaic Energy Generation with Solar Irradiance

Abstract

The growth in the use of solar energy has encouraged the development of techniques for short-term prediction of solar photovoltaic energy generation (PSPEG). Machine learning with Artificial Neural Networks (ANNs) is the most widely used technique to solve this problem. However, comparative studies of these networks with distinct structural configurations, input parameters and prediction horizon, have not been observed in the literature. In this context, the aim of this study is to evaluate the prediction accuracy of the Global Horizontal Irradiance (GHI), which is often used in the PSPEG, generated by ANN models with different construction structures, sets of input meteorological variables and in three short-term prediction horizons, considering a unique database. The analyses were performed with controlled environment and experimental configuration. The results suggest that ANNs using the input GHI variable provide better accuracy (approximately 10%), and their absence increases error variability. No significant difference (p>0.05) was identified in the prediction error models trained with distinct meteorological input data sets. The prediction errors were similar for the same ANN model in the different prediction horizons, and ANNs with 30 and 60 neurons with one hidden layer demonstrated similar or higher accuracy than those with two hidden layers.

Keywords:
forecasting solar power generation; artificial neural network; global horizontal irradiance

HIGHLIGHTS

The input GHI variable increased the prediction accuracy of the ANN models in 10%.

The meteorological variables do not influence the prediction errors statistically.

ANNs with 10 neurons demonstrated lower accuracy than those with 30 and 60 neurons.

ANNs with one hidden layer showed similar or higher accuracy than those with two layers.

INTRODUCTION

The increase in electricity consumption can be supplied by the diversification of the energy matrix with alternative renewable sources. Investments in this strategy are occurring recently in Brazil, with solar photovoltaic and wind energy, contributing to energetic sustainability through the process of decarbonisation of electricity sector in Brazil [¹]. In 2019, renewable energies represented 85% of Brazilian energy matrix [²]. Of this number, 62.7% comes from hydroelectric power plants, 9.4% from wind power plants, and 1.8% from solar photovoltaic power plants, which currently presents the largest growth in the country with 3 GW (GigaWatts) in 2020 [³].

In this scenario, it is important to study techniques for power generation prediction because, like other renewable energy sources, the photovoltaic solar energy generation has non-controllable features related to weather conditions, movements, height and thickness of clouds, temperature, intensity and duration of solar irradiance, air humidity, and others [⁴]. These features may cause intermittencies in the electric power generation process and increase the instability and insecurity of the energy grid [⁵, ⁶]. Therefore, the prediction of these intermittencies in the photovoltaic systems allows to optimize the control and dispatch of energy resources, and consequently mitigate the variation of the power injected into the electrical system [⁷, ⁸].

Several studies about the prediction of solar photovoltaic energy generation (PSPEG) have been published in the last decade [⁹] in technologically developed countries. However, due to recent investments in the use of these technologies in Latin America, little literature research is observed [¹⁰]. In general, the main classes of methods applied in the PSPEG are (i) statistical-time series, (ii) physical methods and (iii) ensemble methods [¹¹]. Nowadays, approaches based on Machine Learning (ML) and Artificial Intelligence (AI) [¹²] are the most commonly studied due to their ability to solve complex problems with non-linear data structures. [¹³, ¹⁴]. In this context, the most applied method in the PSPEG are Artificial Neural Networks (ANN's) [¹⁰] and, more recently Deep Learning models [¹⁵].

Scientific studies in the literature generally apply ANN models in predictions to measure and compare the error accuracy with other studies. In these studies, several characteristics may be considered, such as the different types of input and output information, network structures, and prediction horizons [¹⁶]. However, such studies usually analyze the prediction errors considering these characteristics individually, or with different experimental configuration and different databases.

In this scenario, the aim of this study is to evaluate the prediction accuracy of the Global Horizontal Irradiance (GHI) in the PSPEG context. In order to reach this goal, a controlled experimental environment was used for the conjoint analysis of different features related to ANN models, two sets of the input variables, different structures, and short-term prediction horizon. The experimental evaluation was conducted in a controlled environment in which a unique dataset of training, validation and test was adopted, providing a homogeneous experimental configuration for all evaluated ANN models. In this way, the relationship between ANN features and models configurations was statistically compared.

Theoretical Background and Related Works

The importance of the PSPEG is evidenced by the increase in the number of publications in the last decade [¹⁷] and, there is no literature consensus on the classification of prediction models. Therefore, in this research context, it is important to know the main concepts about the PSPEG research area [¹⁸]:

• - Prediction Horizon: it refers to the amount of time, starting from a time t in which the predictions are performed. This information is relevant for the selection of input data and the techniques employed in the prediction models [¹⁶]. However, there is no scientific consensus about the definition and nomenclature of these prediction horizons. For this study, following classification was considered: short-term (seconds to 60 minutes), medium-term (one to 24 hours), and long-term (one day to weeks ahead) [¹⁸].

• - Endogenous and Exogenous Data: it refers to the source of the input data for the prediction models. Prediction models with endogenous come from present or historical time series (TS), while exogenous data are derived from local or remote measurements, which include total sky images, cloud and wind data, temperature, humidity, solar irradiance, and others [¹⁶].

• - Classes of the Prediction Models: it refers to the two prediction approaches commonly applied. Direct Prediction provides directly the solar power energy generated in kilowatts (kW); and the Indirect Prediction first predicts the solar irradiation in Watts/meter² (W/m²), and uses the results to calculate the potential solar power energy generation (kW) [¹⁶]. The most commonly used solar irradiance in the PSPEG is the Global Horizontal Irradiance [¹⁶], adopted in this study.

• - Prediction Methods: refers to the several techniques and algorithms applied in the PSPEG field. Two scientific reviews of these methods are detailed in [¹¹, ¹⁹], and the accuracy of these methods varies depending on the prediction horizon.

Historically, the most widely used approach in the PSPEG is the Machine Learning [¹³, ¹⁴, ²⁰] with ANN models [²¹-²⁵]. The most cited researches in the literature, considering the citations number’s in Google Scholar in February 8th, 2021, are the ANN's proposed in [²¹] and [²²] for solar irradiance prediction over the 24-hour horizon. These studies were conducted in different locations with distinct meteorological data and ANN structures. In [²¹] it was found that the correlation coefficient ($r$) ranged from 98-99% for sunny days and 94-96% for cloudy days, while in [²²] o $r$ the r coefficient varied between 96% and 99% for cloudy days, and from 48-81% for rainy days. Another proposed study with ANN's for hourly predictions and up to six days ahead reported errors above 15% on same-day predictions, decreasing accuracy with increasing horizon [²³].

The paper described in [²⁴] specifically analyzed different ANN Multilayer Perceptron (MLP) structures in horizons of four or more days, finding that the ANN's structures should be defined according to the data and the prediction horizon. Finally, a study proposing a re-prediction approach based on ANN optimization to increase the accuracy of three prediction models was reported in [²⁵], improving the accuracy for short-term predictions for 5, 10, and 15 minutes.

Based on literature reviews [⁷,¹⁰,¹⁸] and previously cited researches, it was found that all works use different data sets from distinct geographic locations, and generally aim at proposing and analyzing ANN structures, or their optimization, to improve the accuracy for different horizons of short-term, medium-term, or long-term predictions. However, no accurate comparisons of ANN's models considering conjoint analysis of different features as input information, short-term prediction horizons and, ANN structures in the same homogeneous and controlled experimental environment. In this context, this study seeks to develop, evaluate and statistically compare a set of ANN models using a single database [²⁶], considering the use of different input meteorological information and short-term prediction horizons.

MATERIALS AND METHODS

In order to understand the machine learning process with an ANN, it is necessary to understand how it works. An ANN works similarly to the neurons of the human brain, capable of solving approximation, prediction, classification, standardization, and optimization problems [¹²]. Figure 1 schematically presents (a) an artificial neuron, where x ₁, x ₂, ..., x _n correspond to the inputs. For each input x _i there is a weight w _i corresponding to the sum of the inputs x _i which is pondered by the weights w _i, and the linear output u, where u = ∑_i w _i x _i. The activation output y of the neuron is obtained by applying of a function f to the linear output u, denoted by y = f(u). The f function is known as the activation function and can assume several nonlinear forms [²⁷]. In this study, ANN's of type Multilayer Perceptron (MLP) with the activation function RELu and the Backpropagation training algorithm [¹²] were employed.

Figure 1
Schematic representation of (a) an artificial neuron and (b) an Artificial Neural Network. Adapted from [²⁷].

Figure 1 (b) represents a simple ANN with structure (5×4×2) that has five inputs (x ₁, x ₂, ..., x ₅), four neurons and a hidden layer, and two outputs (y ₁ e y ₂). Therefore, an ANN is formed by several processor elements that execute simple functions and together, have the ability to solve complex problems due to their intrinsic computational ability to learn and generalize [²⁷].

In the PSPEG context, ANN models receive, as input, real databases with meteorological information and provide as prediction result the solar irradiance (Indirect Prediction) or generated solar energy (Direct Prediction). ANNs models make static predictions, that is, the next predicted value is estimated by model based on the previous real input value, considering the selected horizon prediction. A variety of the meteorological variables can be applied to the models, such as wind and cloud speed and direction, temperature, humidity, solar irradiance, atmospheric pressure, and others. In this study were employed the most used input and output variables studied in the PSPEG literature [¹⁸]. An analysis of the correlation of some these variables with solar irradiance is reported in [⁷].

Data Description

This study was conducted from the database [²⁶] composed of endogenous and exogenous data relative to meteorological information, solar irradiance, and images. This real and standardized information was collected over the full three-year period (2014-2016), with quality controlled and samples stored in the 1-minute time horizon. Table 1 presents the input variables used in this study and originated in Folsom City, California (United States of America) [²⁶].

Table 1
Data and database details applied in this study.

Parameter	Variable Description	Type		Horizon Prediction	Number of Samples
air_temp	Air temperature	Input			Training Dataset (2014-2015)	Test Dataset (2016)	Total
relhum	Relative humidity	Input		1 minutes	1.028.982	261.360	1.290.342
press	Atmospheric pressure	Input
windsp, max_windsp, winddir	Wind speed, Maximum wind speed, Wind direction	Input		15 minutes	68.601	17.424	86.025
precipitation	Precipitation (rain)	Input		60 minutes	17.152	4.356	21.508
month, hour, min	Month, hour, minute	Input
solar_irradiation	Global Horizontal Irradiance	Input/Output

The solar irradiance and meteorological information data from [²⁶] includes 1,290,342 available samples. These data were pre-processed for the short-term prediction horizons of 1, 15, and 60 minutes, and the number of samples, test and training for each prediction horizon are described in Table 1.

Tools and Technologies

Several computational tools implement ANNs. In this work, we used the Python programming language [²⁸] and Keras [²⁹], an open-source library which provide programming packages for ANNs that facilitate the manipulation of training, testing, and analysis of the models.

The development and execution of ANN's models, including the training and testing steps were conducted in the collaborative open-source programming environment Google Colab [³⁰]. A virtual machine and with the Nvidia K80s Graphics Processing Unit with 12 GB memory, were used. The code and results were combined in documents called notebooks and are available in the supplementary materials.

Experimental Configuration and Execution

ANNs are supervised learning algorithms that require previous iterative training to obtain the optimal weights and biases according to the model and data used [¹²]. In this sense, Figure 2 displays the components, flow and experimental execution steps designed for this study. Initially, the input variables (1), the temporal resolution horizons of the predictions (2), and the models structure in layer and neuron (3) were defined. After that, the database [²⁶] is preprocessed and the models are implemented (4), and the models are trained with data from 2014 and 2015 (5) and, then, tested with data from the year 2016 (6). In this study, all ANN models used the RELu activation function and the Backpropagation training algorithm [¹²], with up to 1000 training epochs. Finally, the errors of synthetically generated models were statistically analyzed (7) and the results are discussed (8).

Figure 2
Steps, flow and components of the experimental execution environment.

The prediction accuracy of ANN models can be evaluated with different statistical measures in the PSPEG. This study provides and applied the following statistical measures to compare the developed ANN models: Coefficient of Determination (R²), Mean Absolute Percentage Error (MAPE), and Root Mean Square Error (RMSE), detailed in equations (1), (2), and (3), respectively.

$R^2=1-\frac{{{\displaystyle \sum }}_{i=1}^N{\left(p_i-o_i\right)}^2}{{{\displaystyle \sum }}_{i=1}^N\left(p_i-o_i\right)}$ (1)

$MAPE=\frac{1}{N}{\displaystyle \sum }_{i=1}^N\left|\frac{p_i-o_i}{o_i}\right|$ (2)

$RMSE={\left[\frac{1}{M}{\displaystyle \sum }_{i=1}^N\left(p_i-o_i\right)\right]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}$ (3)

In these measures, $N$ represents the number of data points in the distribution, $M$ refers to the mean of the observed distribution, $p_i$ to the $i_{th}$ predicted point and, $o_i$ represents the $o_{th}$ observed point [³¹]. Comparisons of accuracy of models with different sample sizes was performed by normalizing the RMSE metric with reference to the mean $\mu =\frac{1}{M}{\displaystyle \sum }_{i=1}^M{o}_i$ of the observed data, where $nRMSE=RMSE/\mu$ [¹⁸].

Table 2 summarize the experimental settings and information related to accuracy from all ANN models utilized in this study. The experiments were performed for scenarios A and B that differ only by the absence (scenario A) and presence (scenario B) of the GHI input variable. Each scenario has two ANN sets with input variables (Input Sets). The first set is broader and used all seven available meteorological variables (Complete Input Set), while the second set (Reduced Input Set) is less extensive and contains the three more common meteorological variables. In this study three short-term prediction horizons (1, 15 and 60 minutes) were defined and evaluated for the output variable Global Horizon Irradiance (GHI).

According to Table 2, a total of 24 ANN models with distinct structures of one and two hidden layers were defined, developed, and analyzed. As a structure example, the ANN (10×15×15×1) has 10 neurons in the input layer, two hidden layers with 15 neurons each, and one neuron in output layer. For all models, the same preprocessed dataset [²⁶] was applied in the training (years 2014 and 2015) and testing (year 2016) steps.

The experimental configuration, described in Table 2, was designed to allow different conjoint analysis between (i) the use of the input variable GHI (Scenarios A and B) (ii) input variable sets (Input Sets), (iii) short-term prediction horizons (1, 15 and 60 minutes) and, (iv) ANN structures (layers and neuron numbers). In this context, the next section presents the experiment results and respective analysis and discussions.

Table 2
Experimental configuration of the all ANN models, variables and predictions horizons.

	Scenario A (Without GHI)		Scenario B (With GHI)
	Complete Input Set	Reduced Input Set	Complete Input Set	Reduced Input Set
Input Variables	Air temperature, Relative humidity, Atmospheric pressure, Wind speed, Maximum wind speed, Wind direction, Precipitation, Month, hour, minute	Air temperature, Wind speed, Wind direction, Month, hour and minute	Air temperature, Relative humidity, Atmospheric pressure, Wind speed, Maximum wind speed, Wind direction, Precipitation, Month, hour, minute, GHI	Air temperature, Wind speed, Wind direction, Month, hour and minute GHI
Output Variable	GHI	GHI	GHI	GHI
Prediction Horizons (minutes)	1, 15 and 60	1, 15 and 60	1, 15 and 60	1, 15 and 60
Structure ANN Models (input×neurons/layers×output)	(10×10×1) (10×30×1) (10×60×1) (10×5×5×1) (10×15×15×1) (10×30×30×1)	(6×10×1) (6×30×1) (6×60×1) (6×5×5×1) (6×15×15×1) (6×30×30×1)	(11×10×1) (11×30×1) (11×60×1) (11×5×5×1) (11×15×15×1) (11×30×30×1)	(7×10×1) (7×30×1) (7×60×1) (7×5×5×1) (7×15×15×1) (7×30×30×1)
Training Data (years)	2014 2015	2014 2015	2014 2015	2014 2015
Test Data (year)	2016	2016	2016	2016

RESULTS AND DISCUSSION

The experimental analysis was conducted with 24 different ANN structures, three prediction horizons, and two sets of input variables in each scenario (A and B) (Table 1). The propose methodological design of the experimental configuration and database employed allowed defining a controlled and homogeneous environment and, to evaluate and compare the distributions of prediction errors (accuracy) of the synthetic models. In the Google Colab environment, the training average and testing time for each ANN model varied from two to five hours depending on the prediction horizon, structure, and ANN inputs. The experiments result in scenarios A and B are exhibited and discussed in the next section.

Statistical Measures of the Prediction Errors

The experiments were run from the steps in Figure 2 and settings in Table 2. The results regarding the prediction errors (accuracy) of all ANN models are shown in Table 3 for scenarios A and B. These scenarios refer to the absence and presence of the input variable GHI, respectively. In each scenario the values of the prediction errors distribution are presented by model and configuration of Input Sets, ANN Structure, and Prediction Horizon. Four statistical metrics are used ($RMSE$, $nRMSE$, $MAPE$ and $R^2$) to represent the errors and facilitate a more complete analysis and understanding of the characteristics of the error distributions, and allow results comparison with other future studies. The $R^2$ metric expresses the fit of the predicted model data to the original data, and the closer to 1, the smaller error and the higher accuracy of the predicted model. The $RMSE$ (or $nRMSE)$ is the most widely used metric because it is sensitive to the variability of large individual errors. Finally, the $MAPE$ metric quantifies the overall prediction error based on mean percentage error [³¹].

For each model and Input Sets configuration, ANN Structure and Prediction Horizon, the mean ($m$) and standard deviation ($sd$) values of all metrics are presented in the Table 3. The results indicate that scenario A models generally have lower accuracy predictions ($RMSE$, $nRMSE$ and $MAPE$) in comparison to scenario B. Furthermore, the mean coefficient of determination ($R^2$), which indicates how close the test and predicted data are to one [³¹], is generally 10% lower in scenario A. In addition, the variability in model errors in scenario A is higher than in scenario B, demonstrating that the presence of the GHI input led to more stable predictions in the evaluated models.

The results in Table 3 demonstrate that, in general, the models group in scenario A with Reduced Input Set produces predictions with highest mean error and lowest mean $R^2$ values. In Scenario B this result was not observed. Therefore, in scenario A, the use of more meteorological information produced more accurate predictions.

Table 3
Statistical metrics of the prediction errors distributions for all ANN models.

Model’s Accuracy in Scenario A (Without GHI)
ANN Structure			Prediction Horizon	RMSE	nRMSE	MAPE	R2	ANN Structure			Prediction Horizon	RMSE	nRMSE	MAPE	R2
Complete Input Set	One Hidden Layer	(10×10×1)	1	0.05	0.09	0.18	0.89	Reduced Input Set	One Hidden Layer	(6×10×1)	1	0.08	0.12	0.25	0.79
			15	0.08	0.12	0.30	0.81				15	0.09	0.13	0.30	0.79
			60	0.09	0.12	0.27	0.83				60	0.09	0.12	0.29	0.84
		(10×30×1)	1	0.05	0.07	0.14	0.94			(6×30×1)	1	0.05	0.08	0.16	0.92
			15	0.06	0.07	0.18	0.93				15	0.06	0.08	0.19	0.91
			60	0.07	0.10	0.17	0.89				60	0.07	0.09	0.16	0.90
		(10×60×1)	1	0.04*	0.06	0.11	0.96			(6×60×1)	1	0.05	0.07*	0.15	0.92
			15	0.05	0.06	0.14	0.95				15	0.06	0.08	0.15	0.91
			60	0.06	0.07	0.15	0.94*				60	0.07	0.09	0.18	0.91
	Two Hidden Layers	(10×5×5×1)	1	0.07	0.11	0.26	0.82		Two Hidden Layers	(6×5×5×1)	1	0.09	0.14	0.36	0.74
			15	0.09	0.12	0.28	0.81				15	0.10	0.15	0.43	0.71
			60	0.10	0.13	0.31	0.80				60	0.11	0.15	0.36	0.73
		(10×15×15×1)	1	0.04*	0.06	0.13	0.95			(6×15×15×1)	1	0.05	0.07*	0.15	0.93
			15	0.06	0.07	0.13	0.93				15	0.07	0.09	0.15	0.90
			60	0.07	0.10	0.14	0.89				60	0.08	0.10	0.17	0.87
		(10×30×30×1)	1	0.04*	0.05*	0.09*	0.97			(6×30×30×1)	1	0.05	0.07*	0.10*	0.94*
			15	0.05	0.06	0.09*	0.96				15	0.06*	0.07*	0.10*	0.93
			60	0.06	0.08	0.14	0.93				60	0.07	0.08	0.13	0.92
			Mean (m)	0.06	0.09	0.18	0.90				Mean (m)	0.07	0.10	0.21	0.86
			Std. Dev. (sd)	0.02	0.03	0.07	0.06				Std. Dev. (sd)	0.02	0.03	0.10	0.08
Model’s Accuracy in Scenario B (With GHI)
ANN Structure			Prediction Horizon	RMSE	nRMSE	MAPE	R2	ANN Structure			Prediction Horizon	RMSE	nRMSE	MAPE	R2
Complete Input Set	One Hidden Layer	(11×10×1)	1	0.01	0.01	0.03	0.99*	Reduced Input Set	One Hidden Layer	(7×10×1)	1	0.01	0.01*	0.05	0.99*
			15	0.01	0.01	0.07	0.99*				15	0.01	0.02	0.09	0.99*
			60	0.01	0.02	0.07	0.99*				60	0.02	0.03	0.09	0.99*
		(11×30×1)	1	0.01	0.01	0.02	0.99*			(7×30×1)	1	0.00*	0.01*	0.02	0.99*
			15	0.00*	0.01	0.01*	0.99*				15	0.01	0.01*	0.03	0.99*
			60	0.02	0.02	0.05	0.99*				60	0.01	0.01*	0.02*	0.99*
		(11×60×1)	1	0.00*	0.01	0.02	0.99*			(7×60×1)	1	0.00*	0.01*	0.02*	0.99*
			15	0.00*	0.01	0.02	0.99*				15	0.01	0.01*	0.02*	0.99*
			60	0.00*	0.00*	0.01*	0.99*				60	0.01	0.01*	0.05	0.99*
	Two Hidden Layers	(11×5×5×1)	1	0.03	0.05	0.17	0.96		Two Hidden Layers	(7×5×5×1)	1	0.04	0.06	0.16	0.94
			15	0.05	0.08	0.19	0.92				15	0.05	0.07	0.17	0.93
			60	0.04	0.07	0.18	0.94				60	0.05	0.08	0.18	0.93
		(11×15×15×1)	1	0.03	0.05	0.08	0.97			(7×15×15×1)	1	0.02	0.04	0.07	0.98
			15	0.03	0.04	0.07	0.98				15	0.03	0.05	0.06	0.97
			60	0.04	0.06	0.07	0.95				60	0.03	0.04	0.08	0.98
		(11×30×30×1)	1	0.03	0.05	0.06	0.97			(7×30×30×1)	1	0.03	0.04	0.08	0.97
			15	0.02	0.03	0.05	0.99*				15	0.02	0.03	0.03	0.99*
			60	0.02	0.03	0.07	0.99*				60	0.03	0.05	0.05	0.98
			Mean (m)	0.02	0.03	0.07	0.98				Mean (m)	0.02	0.03	0.07	0.98
			Std. Dev. (sd)	0.02	0.02	0.06	0.02				Std. Dev. (sd)	0.02	0.02	0.05	0.02

• ^*
  Asterisks values are considered the best accuracy results for each statistical metric.

Analysis and Discussion

This section presents the statistical analyses and comparisons of the prediction errors for all the models in Table 3. Although information from four error metrics is provided, the analyses were primarily based on two main metrics: the $RMSE$ which is the most widely employed in the literature as it is sensitive to large individual errors and captures the variability of the distributions [¹⁸,³¹], and the $R^2$ which estimates the quality and accuracy of the model fit relative to the original data, i.e., if a $R^2$ is closer to 1, better accuracy and quality of the predicted model [³¹]. The analysis and discussion results in the next sections are focused on the use of the input variable GHI, the input variable sets, the prediction horizons, and the ANN structures.

The use of GHI Input Variable (Scenarios A and B)

In [⁷] it was reported that solar PV has a 98% of correlation coefficient with solar irradiance. The fast variation of solar irradiance, called solar ramp, is an important characteristic in the solar photovoltaic generation. The intermittency at a point due to a passing cloud can exceed 60% of the peak of solar irradiance in seconds. Thus, the solar irradiance and energy generation may fluctuate with high amplitude at different time scale [¹⁸]. This study aims to evaluate the influence of the GHI irradiance in relation to prediction accuracy of the different ANN structures and input set configurations (Reduced Input Set and Complete Input Set). Figure 3 show the coefficient of determination ($R^2$) of the prediction errors considering the similar ANN models in scenarios A and B. It can be observed that the models in scenario B (with GHI) exhibit, in all comparisons, better accuracy rates than scenario A models (without GHI).

Figure 3
Accuracy of predictions between Scenario A (Without GHI) and Scenario B (With GHI) models.

The mean coefficient of determination for all models in scenario A is $R^2=0.88$, while in scenario B $R^2=0.98$ was obtained. This shows that the predictions using the input variable GHI (scenario B) were approximately 10% more accurate. Furthermore, the accuracy difference between Scenarios A and B is more notable in ANN's models with Complete Input Set. Moreover, the accuracy difference between Scenarios A and B is more notable in ANN's models with the Complete Input Set input variables. Thus, the presence of the GHI input variable increased the predictions accuracy when compared to the Reduced Input Set models (Figure 3).

The Input Variables Sets

Analysis of the input variable sets was performed to evaluate their relevance with ANNs in the context of the PSPEG. In scenario A, although the Complete Input Set models show slightly lower prediction errors (R²) when compared to the Reduced Input Set models, the statistical comparison of errors ($RMSE$) in Table 4 indicates that the observed statistically difference is not significant (95%) within the same scenario (A or B, with p>0.05). This result was obtained by applying the analysis of variance ANOVA statistical test [³²] to compare all models together and, Tukey-Kramer post-test to compare the statistical difference of the models pairs [³³]. Therefore, in this study context, a particular ANN model and configuration did not have its prediction capacity affected by using the Complete Input Set or the Reduced Input Set.

On the other hand, the presence of the input GHI variable promotes a statistically significant and increase the prediction accuracy of ANNs models (Figure 3 and Table 4). This demonstrates that ANN models capture better the variability of GHI irradiance when this information is present in training and performing of the models.

Table 4
RMSE errors comparisons between models with Reduced Input Set and Complete Input Set.

Comparisons	p-value	Meaning (statistical difference)
Scenario A/ Complete Input Set × Scenario A/ Reduced Input Set	p > 0.05	Not significant
Scenario A/ Complete Input Set × Scenario B/ Complete Input Set	p < 0.001	Significant
Scenario A/ Complete Input Set × Scenario B/ Reduced Input Set	p < 0.001	Significant
Scenario A/ Reduced Input Set × Scenario B/ Complete Input Set	p < 0.001	Significant
Scenario A/ Reduced Input Set × Scenario B/ Reduced Input Set	p < 0.001	Significant
Scenario B/ Complete Input Set × Scenario B/ Reduced Input Set	p > 0.05	Not significant

The Forecasting Horizons

As reported, no significant difference was identified in the ANN models using the Complete Input Set or Reduced Input Set. Thus, the analysis of the three short-term prediction horizons (1, 15 and 60 minutes) was performed individually for each scenario (A and B) and the errors of all models were considered for each horizon analyzed. Figure 4 shows the mean coefficient of determination ($R^2$) and the standard deviation for each prediction horizon.

Figure 4
Mean and standard deviation of prediction errors (R²) for the evaluated horizons.

The results revealed that scenario A models have, in general, a slight decrease in accuracy when there is an increase in the prediction horizon, as well as a greater predictions variability in all horizons. On the other hand, in scenario B the mean precision value remained constant in all three horizons; therefore, it can be inferred that the prediction horizon had little influence on the precision for all models evaluated. Furthermore, it is observed in Figure 4 that the models of scenario A present lower precision indexes in comparison to scenario B, demonstrating again the influence of the input GHI variable.

ANN Models and Structures

The structure of an ANN can be evaluated by the number of neurons and hidden layers. The experiments were performed with one- and two-layer networks that totalize models with 10, 30, and 60 neurons in the hidden layer(s). Figure 5(a) presents the mean and standard deviation of the prediction errors of the models grouped by the number of layers in each scenario. Models with a single hidden layer in both scenarios, show higher mean accuracy than models with two layers, considering ANN's with the same amount of neurons. Consequently, it can be inferred that networks with a single hidden layer are able to capture and predict data variability equal or better than ANN's with two hidden layers (approximately 3%).

Figure 5(b) shows the mean prediction error for all ANNs with the total amount of 10, 30, and 60 neurons in the hidden layer(s), independently of the amount. This analysis seeks to evaluate whether the total amount of neurons in the hidden layer(s) of each ANN affected the prediction accuracy. In scenario A (without GHI) the increase in the number of neurons increased, more markedly, the mean accuracy of the models, demonstrated by the positive linear correlation [³²] with $r=0.75$. This positive correlation is less noticeable in scenario B, where $r=0.84$ was obtained. In addition, the models with 10 neurons in the hidden layer(s) demonstrated lower accuracy indexes than the other ANN's, especially in scenario A (without GHI) in which the accuracy was 11% lower ($R^2=0.79$) than in the other models ($R^2\ge 0.91$).

Figure 5
Means of all the prediction errors (R²) for (a) hidden layers and (b) total amount of neurons in each ANN.

Based on the analysis of the results in Table 3 and Figure 5(a), the ANN models and configurations that showed higher prediction accuracy belong to scenario B, independent of the use of Reduced Input Set or Complete Input Set inputs (Table 4). Considering this, the GHI irradiance predictions of all ANN's in scenario B using Reduced Input Set are presented in Figure 6. The GHI values were normalized according to [³⁴] and are presented for the 1, 15, and 30 minute horizons.

Figure 6
Sample of predictions of the ANN models with one and two hidden layers.

In Figure 6 the top graphics represent the models prediction with a single hidden layer, and the bottom graphics are networks with the same neurons numbers and two hidden layers. The exhibited period from May, 2 to May, 6, in 2016, was selected to demonstrate samples of the predictions on days of clear skies (2 and 3), slightly cloudy (4 and 5), and rainy (6). The points represent the synthetically generated predictions, and these points number is related to the horizon prediction. For example, the 1-minute horizon has a higher points amount than others. In addition, Table 5 summarizes the ANN structures with the best prediction performance (R²) in scenarios A and B, for the sets of input variables and evaluated prediction horizons.

Table 5
The best ANN structures in accuracy for all scenarios and experimental configurations.

	Input Set	Prediction Horizon	ANN Structure with Best Accuracy	R2	Hidden Layer(s)	Total Amount of Neurons
Scenario A (Without GHI)	Reduced	1	(6×30×30×1)	0.93	2	60
		15	(6×30×30×1)	0.95	2	60
		60	(6×30×30×1)	0.92	2	60
	Complete	1	(10×30×30×1)	0.96	2	60
		15	(10×30×30×1)	0.93	2	60
		60	(10×60×1)	0.94	1	60
Scenario B (With GHI)	Reduced	1	(7×30×1)	0.99	1	30
		15	(7×60×1)	0.99	1	60
		60	(7×30×1)	0.99	1	60
	Complete	1	(11×60×1)	0.99	1	60
		15	(11×30×1)	0.99	1	30
		60	(11×60×1)	0,99	1	60

As observed, in scenario A (without GHI) the best results were obtained mainly through ANN's of two hidden layers and 60 neurons. In scenario B (with GHI), the models with one hidden layer displayed better results. Therefore, the use of the GHI input allowed improving prediction accuracy in models with a single hidden layer and 30 or 60 neurons.

CONCLUSIONS

This study, ANN models applied to short-term prediction of solar photovoltaic energy generation were evaluated. The tools, environment, and methodology were designed to enable a controlled and homogeneous experimental configuration, and hence, provide adequate evaluation and comparison of the models prediction errors (accuracy). Furthermore, for reproducibility of this study, the scripts produced in Python language [³⁵] and used data are available in <https://sites.google.com/site/joylan/supplementary-material>.

The present study evaluated ANN's models with the following characteristics: use of the input variable GHI (scenarios A and B), input variable sets (Complete Input Set and Reduced Input Set), prediction horizons (1, 15 and 60 minutes) and ANN's structures. Although ANN's models have been used in the PSPEG for at least a decade [¹⁰], the analysis of these four features, presented in this study, have not been identified in the literature. In this context, the main results in this study were based on the evaluated features and are described below:

1. The input GHI variable increased the prediction accuracy (approximately 10%) for all horizons, structures, and ANN input configurations (Table 3 and Figure 3). While this result was expected [⁷], this study provides additional information that quantifies the relevance and accuracy of ANN models with GHI;

2. Prediction errors measured between the Reduced Input Set and Complete Input Set groups do not indicate a significant difference between them (Table 4). Therefore, the use of more meteorological variables did not affect statistically the prediction accuracy in evaluated models. Possibly, the variables chosen for the Reduced Input Set are the most relevant for prediction the output GHI variable. This information is relevant in the prediction systems design, because there are additional costs in the acquisition of sensors to capture information and, with this study, it is possible to choose, or not, their use;

3. The ANN's models evaluation in three short-term prediction horizons showed that there is a small decrease in accuracy when the prediction horizon increases in scenario A, where the great variability of the predictions is also highlighted (Figure 4). In scenario B, the mean precision and variability remained practically constant over the horizons. Therefore, changing the prediction horizon had little influence on the accuracy of the ANNs evaluated. For others prediction horizons the ANN models should be trained with new data;

4. The ANN structures analysis with same neurons number, in both scenarios, showed that the models with a single layer have a slightly better accuracy rate (approximately 3%) than the models with two hidden layers. The single hidden layer networks were able to learn the variability of the data equal to or better than the two hidden layer models (Figure 5(a)). Regarding the amount of total neurons of the ANNs, models with 30 and 60 neurons showed better accuracy rates (approximately 11%) relative to models with 10 neurons, especially with the absence of the input GHI variable (Figure 5(b)). Furthermore, the most accurate ANN models feature two hidden layers for scenario A (without GHI) and one hidden layer in scenario B (with GHI) (Table 5).

This study did not aim to improve the ANN models accuracy, but, to evaluate them with different structures, input variable configurations, and prediction horizons, in order to contribute to future research in the context of the PSPEG. In addition, the standardized database was selected to perform experiments with controlled and quality data. Another example of the contribution of this study is that the fact that ANNs with one layer show performance equal or superior to networks with two hidden layers is important to contribute to future works in the PSPEG with ANNs, since a larger number of layers and neurons requires more data and, consequently, more training time for the models [³⁶].

Finally, the results revealed that the GHI variable is essential to increase the accuracy of ANN models at the same time it decreases the error variability. The meteorological variables of the Complete Input Set did not provide significant improvement in accuracy for the models evaluated, and ANNs with 30 and 60 neurons in the hidden layer(s) showed better performance than the models with 10 neurons, which, possibly, were not able to capture the variability of the data because they constitute more simple evaluated ANN structures. For future research, we plan to apply and analyze Deep Learning models [¹⁵, ³⁶] in short-term predictions.

Acknowledgments

The authors would like to thank the support of the Coordination for the Improvement of Higher Education Personnel - CAPES, the Federal University of Latin American Integration - UNILA, Araucária Foundation of Support to the Scientific and Technological Development of the State of Paraná - FAPPR, the Brazilian National Council for Scientific and Technological Development - CNPq.

REFERENCES

• ¹
  Ministério de Minas e Energia. Plano Decenal de Expansão de Energia 2026. [internet]. Brasília: Empresa de Pesquisa Energética; 2017 [cited 2021 mar 7]. 271p. Available from: https://www.epe.gov.br/sites-pt/publicacoes-dados-abertos/publicacoes/PublicacoesArquivos/publicacao-40/PDE2026.pdf
  » https://www.epe.gov.br/sites-pt/publicacoes-dados-abertos/publicacoes/PublicacoesArquivos/publicacao-40/PDE2026.pdf
• ²
  Francesco La Camera. Renewable Capacity Statistics 2020. [internet]. Abu Dhabi: International Renewable Energy Agency; 2020 [cited 2021 mar 7]. 66 p. Available from: https://irena.org/-/media/Files/IRENA/Agency/Publication/2020/Mar/IRENA_RE_Capacity_Statistics_2020.pdf
  » https://irena.org/-/media/Files/IRENA/Agency/Publication/2020/Mar/IRENA_RE_Capacity_Statistics_2020.pdf
• ³
  Agência Nacional de Energia Elétrica. Sistema de Informações de Geração da ANEEL - SIGA. Agência Nacional de Energia Elétrica; 2020 [cited 2021 mar 5]. Available from: https://bit.ly/2IGf4Q0
  » https://bit.ly/2IGf4Q0
• ⁴
  Lappalainen K, Valkealahti S. Output power variation of different PV array configurations during irradiance transitions caused by moving clouds. Appl. Energy. 2017 Mar;190:902-910.
• ⁵
  Marcos J, Storkël O, Marroyo L, Garcia M, Lorenzo E. Storage requirements for PV power ramp-rate control. Sol. Energy. 2014 Nov; 99, 28-35. doi.org/10.1016/j.solener.2013.10.037.
  » https://doi.org/10.1016/j.solener.2013.10.037
• ⁶
  Shivashankar S, Mekhilef S, Mokhlis H, Karimi M. Mitigating methods of power fluctuation of photovoltaic (PV) sources - A review. Renew. Sustain. Energy Rev. 2016 Jun; 59:1170-84. doi.org/10.1016/j.rser.2016.01.059.
  » https://doi.org/10.1016/j.rser.2016.01.059
• ⁷
  Das UK, Tey KS, Seyedmahmoudian M, Mekhilef S, Idris MYI, Van Deventer W, Stojcevski A Forecasting of photovoltaic power generation and model optimization: A review. Renew. Sustain. Energy Rev. 2018 Aug; 81(1):912-928. doi.org/10.1016/j.rser.2017.08.017.
  » https://doi.org/10.1016/j.rser.2017.08.017.
• ⁸
  Jamaly M, Kleissl J. Robust cloud motion estimation by spatio-temporal correlation analysis of irradiance data. Sol. Energy. 2018 Jan; 159(October), 306-317. doi.org/10.1016/j.solener.2017.10.075.
  » https://doi.org/10.1016/j.solener.2017.10.075.
• ⁹
  Yang D. Kriging for NSRDB PSM version 3 satellite-derived solar irradiance. Sol. Energy. 2018, 171:876-883. doi.org/10.1016/j.solener.2018.06.055.
  » https://doi.org/10.1016/j.solener.2018.06.055
• ¹⁰
  Maciel JN, Ledesma JJG, Ando Junior OH. Forecasting Solar Power Output Generation: A Systematic Review with the Proknow-C. IEEE Latin America Transactions. 2021; 19(4):612-24. doi.org/10.1109/TLA.2021.9448544.
  » https://doi.org/10.1109/TLA.2021.9448544.
• ¹¹
  Sobri S, Koohi-Kamali S, Rahim NA. Solar photovoltaic generation forecasting methods: A review. Energy Convers. Manag. 2018 Jan; 156:459-497. doi.org/10.1016/j.enconman.2017.11.019.
  » https://doi.org/10.1016/j.enconman.2017.11.019.
• ¹²
  Russell S, Norvig P. Artificial Intelligence: A Modern Approach. 3rd ed. India: Pearson Education; 2015. 1164p.
• ¹³
  Voyant C, Notton G, Kalogirou S, Nivet ML, Paoli C, Motte F, Fouilloy A. Machine learning methods for solar radiation forecasting: A review. Renew. Energy. 2017 May; 105:569-582. doi.org/10.1016/j.renene.2016.12.095.
  » https://doi.org/10.1016/j.renene.2016.12.095.
• ¹⁴
  Obando ED, Carvajal SX, Pineda J. Solar Radiation Prediction Using Machine Learning Techniques: A Review. IEEE Lat. Am. Trans. 2019 Nov; 17(4):684-697.
• ¹⁵
  Brahma B, Wadhvani R. Solar irradiance forecasting based on deep learning methodologies and multi-site data. Symmetry. 2020 Nov, 12(11):1-20. doi.org/10.3390/sym12111830.
  » https://doi.org/10.3390/sym12111830.
• ¹⁶
  Antonanzas J, Osorio N, Escobar R, Urraca R, Martinez-de-Pison, F. J., & Antonanzas-Torres, F.. Review of photovoltaic power forecasting. Sol. Energy. 2016 Oct; 136:78-111. doi.org/10.1016/j.solener.2016.06.069.
  » https://doi.org/10.1016/j.solener.2016.06.069.
• ¹⁷
  Maciel JN, Wentz VH, Junior OHA, Ledesma JJG. Banco de Dados Preliminar com informações meteorológicas e de geração de energia fotovoltaica [internet]. Foz do Iguaçu(PR): Universidade Federal da Integração Latino-Americana, Anais do IX Encontro Anual de Iniciação Científica; 2020 Dec [cited 2021 Mar 4]. Available from https://portal.unila.edu.br/prppg/pesquisa/encontros-anuais-1/2020/AnaisEICTI2020_V1.2.pdf
  » https://portal.unila.edu.br/prppg/pesquisa/encontros-anuais-1/2020/AnaisEICTI2020_V1.2.pdf
• ¹⁸
  Blaga R, Sabadus A, Stefu N, Dughir C, Paulescu M, Badescu V. A current perspective on the accuracy of incoming solar energy forecasting. Prog. Energy Combust. Sci. 2019 Jan; 70:19-144. doi.org/10.1016/j.pecs.2018.10.003.
  » https://doi.org/10.1016/j.pecs.2018.10.003.
• ¹⁹
  Khatib T, Mohamed A, Sopian K. A review of solar energy modeling techniques. Renewable and Sustainable Energy Rev. 2012 Jun; 16(5):2864-2869. doi.org/10.1016/j.rser.2012.01.064.
  » https://doi.org/10.1016/j.rser.2012.01.064.
• ²⁰
  Akhter MN, Mekhilef S, Mokhlis H, Shah NM. Review on forecasting of photovoltaic power generation based on machine learning and metaheuristic techniques. IET Renew.Power Gener. 2019 Mar; 13(7):1009-1023. doi.org/10.1049/iet-rpg.2018.5649.
  » https://doi.org/10.1049/iet-rpg.2018.5649.
• ²¹
  Mellit A, Pavan AM. A 24-h forecast of solar irradiance using artificial neural network: Application for performance prediction of a grid-connected PV plant at Trieste, Italy. Sol. Energy. 2010 May; 84(5):807-821. doi.org/10.1016/j.solener.2010.02.006.
  » https://doi.org/10.1016/j.solener.2010.02.006.
• ²²
  Chen C, Duan S, Cai T, Liu B. Online 24-h solar power forecasting based on weather type classification using artificial neural network. Solar Energy. 2011 Nov; 85(11):2856-70. doi.org/10.1016/j.solener.2011.08.027.
  » https://doi.org/10.1016/j.solener.2011.08.027.
• ²³
  Marque’z R, Coimbra CFM. Forecasting of global and direct solar irradiance using stochastic learning methods, ground experiments and the NWS database. Sol. Energy. 2011 May; 85(5):746-756. doi.org/10.1016/j.solener.2011.01.007.
  » https://doi.org/10.1016/j.solener.2011.01.007.
• ²⁴
  Dumitru CD, Gligor A, Enachescu C. Solar Photovoltaic Energy Production Forecast Using Neural Networks. Procedia Technol. 2016 Feb; 22:808-815. doi.org/10.1016/j.protcy.2016.01.053.
  » https://doi.org/10.1016/j.protcy.2016.01.053.
• ²⁵
  Chu Y, Urquhart B, Gohari SMI, Pedro HTC, Kleissl J, Coimbra CFM. Short-term reforecasting of power output from a 48 MWe solar PV plant. Sol. Energy. 2015 Feb; 112:68-77. doi.org/10.1016/j.solener.2014.11.017.
  » https://doi.org/10.1016/j.solener.2014.11.017.
• ²⁶
  Pedro HTC, Larson DP, Coimbra CFM. A comprehensive dataset for the accelerated development and benchmarking of solar forecasting methods. J. Renew. Sustain. Energy. 2019 Jun; 11(3):036102. doi.org/10.1063/1.5094494.
  » https://doi.org/10.1063/1.5094494.
• ²⁷
  Rezende SO. Sistemas Inteligentes. 1st ed. [Barueri, SP]: Manole; 2003. 564 p.
• ²⁸
  Mining E. Python Machine Learning: Understand Python Libraries (Keras, NumPy, Scikit-Lear, TensorFlow) for Implementing Machine Learning Models in Order to Build Intelligent Systems. [place unknow]: Amazon Digital Services LLC - KDP Print US: 2019. 245 p.
• ²⁹
  Dürr O, Sick B, Murina E. Probabilistic Deep Learning: with Python, Keras and Tensorflow Probability. Manning Publications. [place unknow]. Simon and Schuster: 2020. 296 p.
• ³⁰
  Bisong E. Google Colaboratory. In Building Machine Learning and Deep Learning Models on Google Cloud Platform. [Berkeley, CA]: Apress: 2019. p. 59-64.
• ³¹
  Gueymard CA. A review of validation methodologies and statistical performance indicators for modeled solar radiation data: Towards a better bankability of solar projects. Renew. Sustain. Energy Rev. 2014 Nov; 39:1024-1034. doi.org/10.1016/j.rser.2014.07.117.
  » https://doi.org/10.1016/j.rser.2014.07.117.
• ³²
  Morettin PA, Bussab WO. Estatística básica. 9th ed. [São Paulo-SP]: Saraivauni; 2017. 568 p.
• ³³
  Hsu J. Multiple Comparisons: Theory and Methods. 1st ed. [Ohio-USA]: Chapman and Hall/CRC; 1996. 296 p.
• ³⁴
  Alsina EF, Bortolini M, Gamberi M, Regattieri A. Artificial neural network optimisation for monthly average daily global solar radiation prediction. Energy Convers. Manag. 2016 Jul; 120:320-329 doi.org/10.1016/j.enconman.2016.04.101.
  » https://doi.org/10.1016/j.enconman.2016.04.101.
• ³⁵
  Python Software Foundation [internet]. Python 3.9.0 documentation. 2020 [cited 2021 Mar 6]. Available from: https://docs.python.org/3/.
  » https://docs.python.org/3
• ³⁶
  Ajoy KP, Dobrivoje P. Computational Intelligence in Time Series Forecasting - Theory and Engineering Applications. 1st ed. [place unknow]: Springer-Verlag London; 2014. 372p.

• Funding:

  This research was funded by Triple Agenda Institutional Program of the Federal University of Latin American Integration, the Araucária Foundation of Support to the Scientific and Technological Development of the State of Paraná - FAPPR, grant number the CP 21/2018 PTI and CP 20/2018 PPP, the Brazilian National Council for Scientific and Technological Development - CNPq, grant number the 307223/2017-5, 407531/2018-1 and 303293/2020-9.

Edited by

Editor-in-Chief:

Alexandre Rasi Aoki

Associate Editor:

Alexandre Rasi Aoki

Publication Dates

• Publication in this collection
  14 July 2021
• Date of issue
  2021

History

• Received
  08 Mar 2021
• Accepted
  08 Apr 2021