Htydjtk

I have written a variable length sequence to seqeunce autoencoder in keras using this tutorial as a guideline: https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html. The idea to apply it to the problem of time series forecasting came from the winning entry of this kaggle competition: https://www.kaggle.com/c/web-traffic-time-series-forecasting

The model is trained on 1831 different time series of products but they all come from the same product category. Each series has a minimum length of 156 data points and a maximum of 208 (So 3-4 years of historical data). The data points in each series are upscaled daily data aggregated to the Monday of each week. Some of the series are intermittent some are not

The forecast horizon for each time series is 26 points into the future. The data is log(x + 1) transformed and then normalised by subtracting the mean of the dataset and then multiplying by the standard deviation of the dataset.

The features fed into the model are:

1. The time series


2. Lags of the the time series using 13, 26, 13, & 52


3. A yearly autocorrelation scaler per series which is tiled to be the length of each time series


4. A quarterly autocorrelation scaler per series which is tiled to be the length of each time series


5. A product popularity scaler per series which is tiled to the length of each time series.

The accuracy of the model is evaluated with two separate measures MASE and a variant of SMAPE:

Both of these metrics should be as close to zero as possible. The forecasts need to be made per series and thus I determine overall success by taking the median of all the MASE scores and calculating one SMAPE3 score by taking all the forecast and actual values and computing them in one sum for all the series instead of taking an average per series.

My model has a single encoder layer with 64 neurons and a single decoder layer also with 64 neurons. It uses Adam optimiser using all keras defaults and the mean absolute error as the loss function, as I believe this most closely reflects the error metrics it is evaluated with eventually. The model doesn't perform as well as I want it to and I am looking for ways to imporve it. Currently as measured on my validation set I have the two following scores:

Vol SMAPE3: 0.46 and MASE median: 0.75

The model is trained over 100 epochs with a batch size of 256. Below is the loss and some example plots of series and predictions the model makes:

To me the model doesn't seem to capture the variation in the series very well and almost looks like a linear learning. It appears to only really capture the level of the different series well. To improve it I would like to try and add the ability for the model to capture more of the variations in the series and predictions rather than the smooth kind of predictions it gives at the moment. Are there any suggestions for how to achieve this?