Using LSTM's on Multivariate Input AND Multivariate Output
$begingroup$
I have a very small dataset, only about 40 rows, that has historical usage data for a few categories (roughly 20). I strongly suspect that these categories are dependent in a partial-zero-sum-game fashion: if the usage of one category goes up, I'm expecting that of another to go down. My goal is to predict the next row of all categories.
I've looked at this post, but it's not predicting multiple variables. I also looked at this post, but it's still univariate output (albeit multiple time steps) and multivariate input. So far, I've been basing my approach on the typical LSTM post here at machinelearningmastery, but it's also a single-output-variable example, and a number of the functions used, such as scaler.inverse_transform don't appear to broadcast very well. I'm even having difficulties trying to scale back my full example to match his!
Any tips for scaling LSTM's up to multivariate output? Can the keras LSTM do this natively? If so, how would the code change?
Thanks for your time!
python keras time-series predictive-modeling lstm
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
$endgroup$
add a comment |
$begingroup$
I have a very small dataset, only about 40 rows, that has historical usage data for a few categories (roughly 20). I strongly suspect that these categories are dependent in a partial-zero-sum-game fashion: if the usage of one category goes up, I'm expecting that of another to go down. My goal is to predict the next row of all categories.
I've looked at this post, but it's not predicting multiple variables. I also looked at this post, but it's still univariate output (albeit multiple time steps) and multivariate input. So far, I've been basing my approach on the typical LSTM post here at machinelearningmastery, but it's also a single-output-variable example, and a number of the functions used, such as scaler.inverse_transform don't appear to broadcast very well. I'm even having difficulties trying to scale back my full example to match his!
Any tips for scaling LSTM's up to multivariate output? Can the keras LSTM do this natively? If so, how would the code change?
Thanks for your time!
python keras time-series predictive-modeling lstm
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
$endgroup$
add a comment |
$begingroup$
I have a very small dataset, only about 40 rows, that has historical usage data for a few categories (roughly 20). I strongly suspect that these categories are dependent in a partial-zero-sum-game fashion: if the usage of one category goes up, I'm expecting that of another to go down. My goal is to predict the next row of all categories.
I've looked at this post, but it's not predicting multiple variables. I also looked at this post, but it's still univariate output (albeit multiple time steps) and multivariate input. So far, I've been basing my approach on the typical LSTM post here at machinelearningmastery, but it's also a single-output-variable example, and a number of the functions used, such as scaler.inverse_transform don't appear to broadcast very well. I'm even having difficulties trying to scale back my full example to match his!
Any tips for scaling LSTM's up to multivariate output? Can the keras LSTM do this natively? If so, how would the code change?
Thanks for your time!
python keras time-series predictive-modeling lstm
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
$endgroup$
I have a very small dataset, only about 40 rows, that has historical usage data for a few categories (roughly 20). I strongly suspect that these categories are dependent in a partial-zero-sum-game fashion: if the usage of one category goes up, I'm expecting that of another to go down. My goal is to predict the next row of all categories.
I've looked at this post, but it's not predicting multiple variables. I also looked at this post, but it's still univariate output (albeit multiple time steps) and multivariate input. So far, I've been basing my approach on the typical LSTM post here at machinelearningmastery, but it's also a single-output-variable example, and a number of the functions used, such as scaler.inverse_transform don't appear to broadcast very well. I'm even having difficulties trying to scale back my full example to match his!
Any tips for scaling LSTM's up to multivariate output? Can the keras LSTM do this natively? If so, how would the code change?
Thanks for your time!
python keras time-series predictive-modeling lstm
python keras time-series predictive-modeling lstm
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
asked 7 hours ago
Adrian KeisterAdrian Keister
3641315
3641315
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
You can simply change the number of units at the last layer if you want to predict multiple outputs (or just said differently, one output with multiple features). For example from the link you shared, you can change the units of the Dense layer:
model = Sequential()
model.add(LSTM(50, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(2))
model.compile(loss='mae', optimizer='adam')
The above model would now predict the next step of an output with 2 "features".
Note that you output should be of shape num_samplesx2 now.
answered 6 hours ago
SaTaSaTa
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
1
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
|
show 11 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f44249%2fusing-lstms-on-multivariate-input-and-multivariate-output%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
You can simply change the number of units at the last layer if you want to predict multiple outputs (or just said differently, one output with multiple features). For example from the link you shared, you can change the units of the Dense layer:
model = Sequential()
model.add(LSTM(50, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(2))
model.compile(loss='mae', optimizer='adam')
The above model would now predict the next step of an output with 2 "features".
Note that you output should be of shape num_samplesx2 now.
answered 6 hours ago
SaTaSaTa
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
1
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
|
show 11 more comments
$begingroup$
You can simply change the number of units at the last layer if you want to predict multiple outputs (or just said differently, one output with multiple features). For example from the link you shared, you can change the units of the Dense layer:
model = Sequential()
model.add(LSTM(50, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(2))
model.compile(loss='mae', optimizer='adam')
The above model would now predict the next step of an output with 2 "features".
Note that you output should be of shape num_samplesx2 now.
answered 6 hours ago
SaTaSaTa
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
1
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
|
show 11 more comments
$begingroup$
You can simply change the number of units at the last layer if you want to predict multiple outputs (or just said differently, one output with multiple features). For example from the link you shared, you can change the units of the Dense layer:
model = Sequential()
model.add(LSTM(50, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(2))
model.compile(loss='mae', optimizer='adam')
The above model would now predict the next step of an output with 2 "features".
Note that you output should be of shape num_samplesx2 now.
answered 6 hours ago
SaTaSaTa
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
You can simply change the number of units at the last layer if you want to predict multiple outputs (or just said differently, one output with multiple features). For example from the link you shared, you can change the units of the Dense layer:
model = Sequential()
model.add(LSTM(50, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(2))
model.compile(loss='mae', optimizer='adam')
The above model would now predict the next step of an output with 2 "features".
Note that you output should be of shape num_samplesx2 now.
answered 6 hours ago
SaTaSaTa
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 6 hours ago
answered 6 hours ago
SaTaSaTa
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 6 hours ago
SaTaSaTa
112
answered 6 hours ago
SaTaSaTa
112
112
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
SaTa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
1
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
|
show 11 more comments
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
1
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
You wrote, "The above model would not predict..." Did you mean "The above model would now predict..."?
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Yes, thanks. I edited the answer.
$endgroup$
– SaTa
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Ah, gotcha. Thanks! I wouldn't have bothered, but it did change the meaning of the sentence a fair bit. I'll try it out!
$endgroup$
– Adrian Keister
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
$begingroup$
Yes, it certainly does :).
$endgroup$
– SaTa
6 hours ago
1
1
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
$begingroup$
num_samples is like how many samples you have for training. num_observations is how long is each sample, or how many time steps you have in each sample.
$endgroup$
– SaTa
5 hours ago
|
show 11 more comments
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f44249%2fusing-lstms-on-multivariate-input-and-multivariate-output%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
