Hourly-averaged solar plus wind power generation for Germany 2016: Long-term prediction, short-term forecasting, data mining and outlier analysis

Nationwide, hourly-averaged solar plus wind power generation (MW) data compiled for Germany for year 2016 is evaluated with ten influencing variables. Those variables cover, on an hourly basis, weather and ground-surface conditions and electricity prices. The transparent open box (TOB) algorithm accurately predicts and forecasts power generation (MW) for this dataset (prediction RMSE = 1175 MW and R-2 = 0.9804; hour ahead forecast RMSE = 1632 MW and R-2 = 0.9609) and meaningfully data mines the prediction outliers. Some 1.5 % of the data records display significant prediction errors. These records are mined to reveal that many of them form trends on a few specific days displaying unusual and rapidly changing weather conditions. Derivatives of ground level solar radiation, wind velocity and air pressure can meaningfully distinguish such unusual conditions and can be used to filter the dataset to further improve prediction accuracy. Derivatives and ratios of variables are also exploited to focus and modify feature selection for TOB analysis on approximately 10 % of the dataset (900 data records) responsible for the least accurate predictions. This more focused feature selection improves prediction accuracy for these more difficult to predict data records (RMSE improves from 3544 to 2630 MW; R-2 from 0.8027 to 0.8938).