Forward and backward process in pyTorch
$begingroup$
When i write a network, do i have to write the whole forward porp in nn.Module.forward()? I mean if i do some operations outside the net, does grad can correctly flow?
For example, i have 2 networks, in which the output of one is the input of the other(net1 -> midresults -> net2), if i do some operations on midresults(net1 -> midresults -> operations -> net2), can (net1+net2) be trained end to end?
pytorch
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
$endgroup$
add a comment |
$begingroup$
When i write a network, do i have to write the whole forward porp in nn.Module.forward()? I mean if i do some operations outside the net, does grad can correctly flow?
For example, i have 2 networks, in which the output of one is the input of the other(net1 -> midresults -> net2), if i do some operations on midresults(net1 -> midresults -> operations -> net2), can (net1+net2) be trained end to end?
pytorch
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
$endgroup$
add a comment |
$begingroup$
When i write a network, do i have to write the whole forward porp in nn.Module.forward()? I mean if i do some operations outside the net, does grad can correctly flow?
For example, i have 2 networks, in which the output of one is the input of the other(net1 -> midresults -> net2), if i do some operations on midresults(net1 -> midresults -> operations -> net2), can (net1+net2) be trained end to end?
pytorch
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
$endgroup$
When i write a network, do i have to write the whole forward porp in nn.Module.forward()? I mean if i do some operations outside the net, does grad can correctly flow?
For example, i have 2 networks, in which the output of one is the input of the other(net1 -> midresults -> net2), if i do some operations on midresults(net1 -> midresults -> operations -> net2), can (net1+net2) be trained end to end?
pytorch
pytorch
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
asked Mar 19 '18 at 11:16
Yang JiaoYang Jiao
11
11
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.
A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.
answered 16 hours ago
EugValEugVal
1064
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29256%2fforward-and-backward-process-in-pytorch%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.
A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.
answered 16 hours ago
EugValEugVal
1064
$endgroup$
add a comment |
$begingroup$
As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.
A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.
answered 16 hours ago
EugValEugVal
1064
$endgroup$
add a comment |
$begingroup$
As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.
A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.
answered 16 hours ago
EugValEugVal
1064
$endgroup$
As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.
A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.
answered 16 hours ago
EugValEugVal
1064
answered 16 hours ago
EugValEugVal
1064
answered 16 hours ago
EugValEugVal
1064
answered 16 hours ago
EugValEugVal
1064
1064
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29256%2fforward-and-backward-process-in-pytorch%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
