Should I only use scalar labels with Keras LSTM?












1












$begingroup$


I have an array X_train = (1110,25,2) and a y_train = (1110,5,2). It means I use arrays with length of 25 for inputs and length of 5 for labels. But when I use:



model = Sequential()
model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
model.add(Dropout(0.2))
model.add(Dense(units = 2))
model.compile(optimizer = 'adam', loss = 'mean_squared_error')
model.fit(X_train, y_train, epochs = 100 , batch_size = 25)


It gives me this error in the last line of the code:




ValueError: Error when checking target: expected dense_1 to have 2
dimensions, but got array with shape (1110, 5, 2) [Finished in 5.1s
with exit code 1]




The code works if I change the length of y_train to the 1, but I like to test longer y labels to train. What is the problem and how can I fix it?



EDIT:
I create X_train and y_train arrays with this code:



for i in range((len(training_set)%30) + 30 , len(training_set) - days ):
X_train.append(training_set_scaled[i-30:i-5])
y_train.append(training_set_scaled[i-5:i])
X_train, y_train = np.array(X_train), np.array(y_train)


This is the result of model.summary()



model.summary()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
lstm_45 (LSTM) (None, 25, 25) 2800
_________________________________________________________________
dropout_45 (Dropout) (None, 25, 25) 0
_________________________________________________________________
lstm_46 (LSTM) (None, 25, 25) 5100
_________________________________________________________________
dropout_46 (Dropout) (None, 25, 25) 0
_________________________________________________________________
lstm_47 (LSTM) (None, 25, 25) 5100
_________________________________________________________________
dropout_47 (Dropout) (None, 25, 25) 0
_________________________________________________________________
lstm_48 (LSTM) (None, 25) 5100
_________________________________________________________________
dropout_48 (Dropout) (None, 25) 0
_________________________________________________________________
dense_12 (Dense) (None, 2) 52
=================================================================
Total params: 18,152
Trainable params: 18,152
Non-trainable params: 0
_________________________________________________________________


EDIT2:
I have tried to solve my problem with RepeatVector() function in encoder-decoder approach with the following code:



model = Sequential()
model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
model.add(Dropout(0.2))
model.add(LSTM(units = 25, return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units = 25)) #, return_sequences = True))
model.add(Dropout(0.2))
model.add(RepeatVector(5))
model.add(LSTM(units = 5 ,return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units = 5 ,return_sequences = True ))
model.add(Dropout(0.2))
model.add(Dense(units = 2))


But I get this stupid result:
enter image description here










share|improve this question











$endgroup$

















    1












    $begingroup$


    I have an array X_train = (1110,25,2) and a y_train = (1110,5,2). It means I use arrays with length of 25 for inputs and length of 5 for labels. But when I use:



    model = Sequential()
    model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
    model.add(Dropout(0.2))
    model.add(Dense(units = 2))
    model.compile(optimizer = 'adam', loss = 'mean_squared_error')
    model.fit(X_train, y_train, epochs = 100 , batch_size = 25)


    It gives me this error in the last line of the code:




    ValueError: Error when checking target: expected dense_1 to have 2
    dimensions, but got array with shape (1110, 5, 2) [Finished in 5.1s
    with exit code 1]




    The code works if I change the length of y_train to the 1, but I like to test longer y labels to train. What is the problem and how can I fix it?



    EDIT:
    I create X_train and y_train arrays with this code:



    for i in range((len(training_set)%30) + 30 , len(training_set) - days ):
    X_train.append(training_set_scaled[i-30:i-5])
    y_train.append(training_set_scaled[i-5:i])
    X_train, y_train = np.array(X_train), np.array(y_train)


    This is the result of model.summary()



    model.summary()
    _________________________________________________________________
    Layer (type) Output Shape Param #
    =================================================================
    lstm_45 (LSTM) (None, 25, 25) 2800
    _________________________________________________________________
    dropout_45 (Dropout) (None, 25, 25) 0
    _________________________________________________________________
    lstm_46 (LSTM) (None, 25, 25) 5100
    _________________________________________________________________
    dropout_46 (Dropout) (None, 25, 25) 0
    _________________________________________________________________
    lstm_47 (LSTM) (None, 25, 25) 5100
    _________________________________________________________________
    dropout_47 (Dropout) (None, 25, 25) 0
    _________________________________________________________________
    lstm_48 (LSTM) (None, 25) 5100
    _________________________________________________________________
    dropout_48 (Dropout) (None, 25) 0
    _________________________________________________________________
    dense_12 (Dense) (None, 2) 52
    =================================================================
    Total params: 18,152
    Trainable params: 18,152
    Non-trainable params: 0
    _________________________________________________________________


    EDIT2:
    I have tried to solve my problem with RepeatVector() function in encoder-decoder approach with the following code:



    model = Sequential()
    model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
    model.add(Dropout(0.2))
    model.add(LSTM(units = 25, return_sequences = True))
    model.add(Dropout(0.2))
    model.add(LSTM(units = 25)) #, return_sequences = True))
    model.add(Dropout(0.2))
    model.add(RepeatVector(5))
    model.add(LSTM(units = 5 ,return_sequences = True))
    model.add(Dropout(0.2))
    model.add(LSTM(units = 5 ,return_sequences = True ))
    model.add(Dropout(0.2))
    model.add(Dense(units = 2))


    But I get this stupid result:
    enter image description here










    share|improve this question











    $endgroup$















      1












      1








      1


      1



      $begingroup$


      I have an array X_train = (1110,25,2) and a y_train = (1110,5,2). It means I use arrays with length of 25 for inputs and length of 5 for labels. But when I use:



      model = Sequential()
      model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
      model.add(Dropout(0.2))
      model.add(Dense(units = 2))
      model.compile(optimizer = 'adam', loss = 'mean_squared_error')
      model.fit(X_train, y_train, epochs = 100 , batch_size = 25)


      It gives me this error in the last line of the code:




      ValueError: Error when checking target: expected dense_1 to have 2
      dimensions, but got array with shape (1110, 5, 2) [Finished in 5.1s
      with exit code 1]




      The code works if I change the length of y_train to the 1, but I like to test longer y labels to train. What is the problem and how can I fix it?



      EDIT:
      I create X_train and y_train arrays with this code:



      for i in range((len(training_set)%30) + 30 , len(training_set) - days ):
      X_train.append(training_set_scaled[i-30:i-5])
      y_train.append(training_set_scaled[i-5:i])
      X_train, y_train = np.array(X_train), np.array(y_train)


      This is the result of model.summary()



      model.summary()
      _________________________________________________________________
      Layer (type) Output Shape Param #
      =================================================================
      lstm_45 (LSTM) (None, 25, 25) 2800
      _________________________________________________________________
      dropout_45 (Dropout) (None, 25, 25) 0
      _________________________________________________________________
      lstm_46 (LSTM) (None, 25, 25) 5100
      _________________________________________________________________
      dropout_46 (Dropout) (None, 25, 25) 0
      _________________________________________________________________
      lstm_47 (LSTM) (None, 25, 25) 5100
      _________________________________________________________________
      dropout_47 (Dropout) (None, 25, 25) 0
      _________________________________________________________________
      lstm_48 (LSTM) (None, 25) 5100
      _________________________________________________________________
      dropout_48 (Dropout) (None, 25) 0
      _________________________________________________________________
      dense_12 (Dense) (None, 2) 52
      =================================================================
      Total params: 18,152
      Trainable params: 18,152
      Non-trainable params: 0
      _________________________________________________________________


      EDIT2:
      I have tried to solve my problem with RepeatVector() function in encoder-decoder approach with the following code:



      model = Sequential()
      model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
      model.add(Dropout(0.2))
      model.add(LSTM(units = 25, return_sequences = True))
      model.add(Dropout(0.2))
      model.add(LSTM(units = 25)) #, return_sequences = True))
      model.add(Dropout(0.2))
      model.add(RepeatVector(5))
      model.add(LSTM(units = 5 ,return_sequences = True))
      model.add(Dropout(0.2))
      model.add(LSTM(units = 5 ,return_sequences = True ))
      model.add(Dropout(0.2))
      model.add(Dense(units = 2))


      But I get this stupid result:
      enter image description here










      share|improve this question











      $endgroup$




      I have an array X_train = (1110,25,2) and a y_train = (1110,5,2). It means I use arrays with length of 25 for inputs and length of 5 for labels. But when I use:



      model = Sequential()
      model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
      model.add(Dropout(0.2))
      model.add(Dense(units = 2))
      model.compile(optimizer = 'adam', loss = 'mean_squared_error')
      model.fit(X_train, y_train, epochs = 100 , batch_size = 25)


      It gives me this error in the last line of the code:




      ValueError: Error when checking target: expected dense_1 to have 2
      dimensions, but got array with shape (1110, 5, 2) [Finished in 5.1s
      with exit code 1]




      The code works if I change the length of y_train to the 1, but I like to test longer y labels to train. What is the problem and how can I fix it?



      EDIT:
      I create X_train and y_train arrays with this code:



      for i in range((len(training_set)%30) + 30 , len(training_set) - days ):
      X_train.append(training_set_scaled[i-30:i-5])
      y_train.append(training_set_scaled[i-5:i])
      X_train, y_train = np.array(X_train), np.array(y_train)


      This is the result of model.summary()



      model.summary()
      _________________________________________________________________
      Layer (type) Output Shape Param #
      =================================================================
      lstm_45 (LSTM) (None, 25, 25) 2800
      _________________________________________________________________
      dropout_45 (Dropout) (None, 25, 25) 0
      _________________________________________________________________
      lstm_46 (LSTM) (None, 25, 25) 5100
      _________________________________________________________________
      dropout_46 (Dropout) (None, 25, 25) 0
      _________________________________________________________________
      lstm_47 (LSTM) (None, 25, 25) 5100
      _________________________________________________________________
      dropout_47 (Dropout) (None, 25, 25) 0
      _________________________________________________________________
      lstm_48 (LSTM) (None, 25) 5100
      _________________________________________________________________
      dropout_48 (Dropout) (None, 25) 0
      _________________________________________________________________
      dense_12 (Dense) (None, 2) 52
      =================================================================
      Total params: 18,152
      Trainable params: 18,152
      Non-trainable params: 0
      _________________________________________________________________


      EDIT2:
      I have tried to solve my problem with RepeatVector() function in encoder-decoder approach with the following code:



      model = Sequential()
      model.add(LSTM(units = 25, return_sequences = True, input_shape = (25, 2)))
      model.add(Dropout(0.2))
      model.add(LSTM(units = 25, return_sequences = True))
      model.add(Dropout(0.2))
      model.add(LSTM(units = 25)) #, return_sequences = True))
      model.add(Dropout(0.2))
      model.add(RepeatVector(5))
      model.add(LSTM(units = 5 ,return_sequences = True))
      model.add(Dropout(0.2))
      model.add(LSTM(units = 5 ,return_sequences = True ))
      model.add(Dropout(0.2))
      model.add(Dense(units = 2))


      But I get this stupid result:
      enter image description here







      keras lstm error-handling






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited 12 hours ago







      user3486308

















      asked yesterday









      user3486308user3486308

      1213




      1213






















          1 Answer
          1






          active

          oldest

          votes


















          2












          $begingroup$

          Can you post a model summary using:



          model.summary()


          Also, elaborate on how exactly the Y_train dataset works with the X_train? It's not clear how the 25 time steps from X_train data correspond to the Y_train 5 outputs.






          share|improve this answer









          $endgroup$













          • $begingroup$
            Hi, I updated my question!
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
            $endgroup$
            – kylec123
            yesterday










          • $begingroup$
            I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Yes, I read data for 25 days, then predict next 5 days.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
            $endgroup$
            – user3486308
            yesterday











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "557"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46207%2fshould-i-only-use-scalar-labels-with-keras-lstm%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          2












          $begingroup$

          Can you post a model summary using:



          model.summary()


          Also, elaborate on how exactly the Y_train dataset works with the X_train? It's not clear how the 25 time steps from X_train data correspond to the Y_train 5 outputs.






          share|improve this answer









          $endgroup$













          • $begingroup$
            Hi, I updated my question!
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
            $endgroup$
            – kylec123
            yesterday










          • $begingroup$
            I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Yes, I read data for 25 days, then predict next 5 days.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
            $endgroup$
            – user3486308
            yesterday
















          2












          $begingroup$

          Can you post a model summary using:



          model.summary()


          Also, elaborate on how exactly the Y_train dataset works with the X_train? It's not clear how the 25 time steps from X_train data correspond to the Y_train 5 outputs.






          share|improve this answer









          $endgroup$













          • $begingroup$
            Hi, I updated my question!
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
            $endgroup$
            – kylec123
            yesterday










          • $begingroup$
            I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Yes, I read data for 25 days, then predict next 5 days.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
            $endgroup$
            – user3486308
            yesterday














          2












          2








          2





          $begingroup$

          Can you post a model summary using:



          model.summary()


          Also, elaborate on how exactly the Y_train dataset works with the X_train? It's not clear how the 25 time steps from X_train data correspond to the Y_train 5 outputs.






          share|improve this answer









          $endgroup$



          Can you post a model summary using:



          model.summary()


          Also, elaborate on how exactly the Y_train dataset works with the X_train? It's not clear how the 25 time steps from X_train data correspond to the Y_train 5 outputs.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered yesterday









          kylec123kylec123

          718




          718












          • $begingroup$
            Hi, I updated my question!
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
            $endgroup$
            – kylec123
            yesterday










          • $begingroup$
            I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Yes, I read data for 25 days, then predict next 5 days.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
            $endgroup$
            – user3486308
            yesterday


















          • $begingroup$
            Hi, I updated my question!
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
            $endgroup$
            – kylec123
            yesterday










          • $begingroup$
            I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Yes, I read data for 25 days, then predict next 5 days.
            $endgroup$
            – user3486308
            yesterday










          • $begingroup$
            Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
            $endgroup$
            – user3486308
            yesterday
















          $begingroup$
          Hi, I updated my question!
          $endgroup$
          – user3486308
          yesterday




          $begingroup$
          Hi, I updated my question!
          $endgroup$
          – user3486308
          yesterday












          $begingroup$
          I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
          $endgroup$
          – kylec123
          yesterday




          $begingroup$
          I'm still not clear on what exactly you are trying to predict. You have a sequence of variable length and want to predict the next 5 timesteps? Can you elaborate on this? Also, I assume, given the model.summary output, that you have return_sequences=False on the last lstm layer, correct?
          $endgroup$
          – kylec123
          yesterday












          $begingroup$
          I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
          $endgroup$
          – user3486308
          yesterday




          $begingroup$
          I am trying to create X_train with capturing 25 elements, then shift it 1 position then capture 25 elements till the end of the whole data. And do the same process with length of 5 for y_train array.
          $endgroup$
          – user3486308
          yesterday












          $begingroup$
          Yes, I read data for 25 days, then predict next 5 days.
          $endgroup$
          – user3486308
          yesterday




          $begingroup$
          Yes, I read data for 25 days, then predict next 5 days.
          $endgroup$
          – user3486308
          yesterday












          $begingroup$
          Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
          $endgroup$
          – user3486308
          yesterday




          $begingroup$
          Also the last LSTM layer is return_sequence = False cause I have connected it to a Dense(units = 2) layer that is my last layer.
          $endgroup$
          – user3486308
          yesterday


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Data Science Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46207%2fshould-i-only-use-scalar-labels-with-keras-lstm%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          How to label and detect the document text images

          Vallis Paradisi

          Tabula Rosettana