Ensemble models - neural network input both original data and predictions of other models?
$begingroup$
From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one of the inputs for a neural network where the other input is the original data and the targets remain the same could improve the model? Why add extra complexity? basically the problem is a multi-dimensional regression problem and although the random forest gets a smaller MSE the neural network is bette at preservering some of the properties of the target labels. Therefore, I was wondering if by putting these two models together I would get a lower MSE while preserving some of the properties. Is it worth a shot? or will it just drastically overfit?
neural-network random-forest ensemble-modeling
asked Jul 18 '18 at 7:25
TankTank
305
$endgroup$
add a comment |
$begingroup$
From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one of the inputs for a neural network where the other input is the original data and the targets remain the same could improve the model? Why add extra complexity? basically the problem is a multi-dimensional regression problem and although the random forest gets a smaller MSE the neural network is bette at preservering some of the properties of the target labels. Therefore, I was wondering if by putting these two models together I would get a lower MSE while preserving some of the properties. Is it worth a shot? or will it just drastically overfit?
neural-network random-forest ensemble-modeling
asked Jul 18 '18 at 7:25
TankTank
305
$endgroup$
add a comment |
$begingroup$
From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one of the inputs for a neural network where the other input is the original data and the targets remain the same could improve the model? Why add extra complexity? basically the problem is a multi-dimensional regression problem and although the random forest gets a smaller MSE the neural network is bette at preservering some of the properties of the target labels. Therefore, I was wondering if by putting these two models together I would get a lower MSE while preserving some of the properties. Is it worth a shot? or will it just drastically overfit?
neural-network random-forest ensemble-modeling
asked Jul 18 '18 at 7:25
TankTank
305
$endgroup$
From my understanding in order to improve accuracy with ensemble models you need a wide range of independent ensemble methods. I was wondering whether using the ouput of a random forest model as one of the inputs for a neural network where the other input is the original data and the targets remain the same could improve the model? Why add extra complexity? basically the problem is a multi-dimensional regression problem and although the random forest gets a smaller MSE the neural network is bette at preservering some of the properties of the target labels. Therefore, I was wondering if by putting these two models together I would get a lower MSE while preserving some of the properties. Is it worth a shot? or will it just drastically overfit?
neural-network random-forest ensemble-modeling
neural-network random-forest ensemble-modeling
asked Jul 18 '18 at 7:25
TankTank
305
asked Jul 18 '18 at 7:25
TankTank
305
asked Jul 18 '18 at 7:25
TankTank
305
asked Jul 18 '18 at 7:25
TankTank
305
asked Jul 18 '18 at 7:25
TankTank
305
305
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
I suggest you test it by cross-validation to prevent overfitting. However, I guess because of a strong relationship between this variable and output your work will not be better. Instead, try to use an ensemble.
I read something about deep learning which in that way they use several models as a sequence and use results of previous models as inputs of the next models. Maybe if you continue this work it will be in some way a deep learning.
answered Jul 18 '18 at 9:07
parvijparvij
485214
$endgroup$
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
add a comment |
$begingroup$
I'm working on prediction of time series and my stacked LSTMs alone does not capture well the trend and seasonality of my data. I decided to add features from a Holts Winter regression as an additional input and the result is way better. It slightly overfits but in my case that's not an issue.
In my case, I like the fact that my statistic (holts winter) model can capture well the global trend, and the network can find other non linear relations between datapoints. I guess it's a good idea to combine several models.
answered 14 hours ago
nymanonymano
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f35642%2fensemble-models-neural-network-input-both-original-data-and-predictions-of-oth%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I suggest you test it by cross-validation to prevent overfitting. However, I guess because of a strong relationship between this variable and output your work will not be better. Instead, try to use an ensemble.
I read something about deep learning which in that way they use several models as a sequence and use results of previous models as inputs of the next models. Maybe if you continue this work it will be in some way a deep learning.
answered Jul 18 '18 at 9:07
parvijparvij
485214
$endgroup$
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
add a comment |
$begingroup$
I suggest you test it by cross-validation to prevent overfitting. However, I guess because of a strong relationship between this variable and output your work will not be better. Instead, try to use an ensemble.
I read something about deep learning which in that way they use several models as a sequence and use results of previous models as inputs of the next models. Maybe if you continue this work it will be in some way a deep learning.
answered Jul 18 '18 at 9:07
parvijparvij
485214
$endgroup$
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
add a comment |
$begingroup$
I suggest you test it by cross-validation to prevent overfitting. However, I guess because of a strong relationship between this variable and output your work will not be better. Instead, try to use an ensemble.
I read something about deep learning which in that way they use several models as a sequence and use results of previous models as inputs of the next models. Maybe if you continue this work it will be in some way a deep learning.
answered Jul 18 '18 at 9:07
parvijparvij
485214
$endgroup$
I suggest you test it by cross-validation to prevent overfitting. However, I guess because of a strong relationship between this variable and output your work will not be better. Instead, try to use an ensemble.
I read something about deep learning which in that way they use several models as a sequence and use results of previous models as inputs of the next models. Maybe if you continue this work it will be in some way a deep learning.
answered Jul 18 '18 at 9:07
parvijparvij
485214
edited Jul 18 '18 at 9:59
answered Jul 18 '18 at 9:07
parvijparvij
485214
answered Jul 18 '18 at 9:07
parvijparvij
485214
answered Jul 18 '18 at 9:07
parvijparvij
485214
485214
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
add a comment |
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
what do you mean by deep mining?
$endgroup$
– Tank
Jul 18 '18 at 9:10
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
$begingroup$
Sorry, deep learning
$endgroup$
– parvij
Jul 18 '18 at 9:59
add a comment |
$begingroup$
I'm working on prediction of time series and my stacked LSTMs alone does not capture well the trend and seasonality of my data. I decided to add features from a Holts Winter regression as an additional input and the result is way better. It slightly overfits but in my case that's not an issue.
In my case, I like the fact that my statistic (holts winter) model can capture well the global trend, and the network can find other non linear relations between datapoints. I guess it's a good idea to combine several models.
answered 14 hours ago
nymanonymano
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I'm working on prediction of time series and my stacked LSTMs alone does not capture well the trend and seasonality of my data. I decided to add features from a Holts Winter regression as an additional input and the result is way better. It slightly overfits but in my case that's not an issue.
In my case, I like the fact that my statistic (holts winter) model can capture well the global trend, and the network can find other non linear relations between datapoints. I guess it's a good idea to combine several models.
answered 14 hours ago
nymanonymano
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I'm working on prediction of time series and my stacked LSTMs alone does not capture well the trend and seasonality of my data. I decided to add features from a Holts Winter regression as an additional input and the result is way better. It slightly overfits but in my case that's not an issue.
In my case, I like the fact that my statistic (holts winter) model can capture well the global trend, and the network can find other non linear relations between datapoints. I guess it's a good idea to combine several models.
answered 14 hours ago
nymanonymano
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I'm working on prediction of time series and my stacked LSTMs alone does not capture well the trend and seasonality of my data. I decided to add features from a Holts Winter regression as an additional input and the result is way better. It slightly overfits but in my case that's not an issue.
In my case, I like the fact that my statistic (holts winter) model can capture well the global trend, and the network can find other non linear relations between datapoints. I guess it's a good idea to combine several models.
answered 14 hours ago
nymanonymano
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 14 hours ago
nymanonymano
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 14 hours ago
nymanonymano
12
answered 14 hours ago
nymanonymano
12
12
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
nymano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f35642%2fensemble-models-neural-network-input-both-original-data-and-predictions-of-oth%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
