Should I pad zero or mean value in a convolution neural network? What about reflective padding?
$begingroup$
In convolution layers, sometimes you need to pad some (usually 1 or 2 pixel) 0s at the edges of the original image before applying your convolution kernel (filter).
However, 0 has different meanings.
In the case of 0 - 255 gray scale images, 0 means black.
In the case of normalized images ( minus mean then divided by std), 0 is the mean value.
Should I always pad 0, or should I sometimes pad mean value ?
If in some cases I should pad mean value, what are those cases?
Edit:
And when should I use reflective padding?
Further more, is reflective padding always better than zero padding?
machine-learning neural-network cnn
asked Sep 25 '18 at 9:08
user10253771user10253771
266
$endgroup$
add a comment |
$begingroup$
In convolution layers, sometimes you need to pad some (usually 1 or 2 pixel) 0s at the edges of the original image before applying your convolution kernel (filter).
However, 0 has different meanings.
In the case of 0 - 255 gray scale images, 0 means black.
In the case of normalized images ( minus mean then divided by std), 0 is the mean value.
Should I always pad 0, or should I sometimes pad mean value ?
If in some cases I should pad mean value, what are those cases?
Edit:
And when should I use reflective padding?
Further more, is reflective padding always better than zero padding?
machine-learning neural-network cnn
asked Sep 25 '18 at 9:08
user10253771user10253771
266
$endgroup$
$begingroup$
Why do you pad the pixels? Why with zero?
$endgroup$
– user2974951
Sep 25 '18 at 9:14
$begingroup$
@user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding.
$endgroup$
– user10253771
Sep 25 '18 at 9:47
add a comment |
$begingroup$
In convolution layers, sometimes you need to pad some (usually 1 or 2 pixel) 0s at the edges of the original image before applying your convolution kernel (filter).
However, 0 has different meanings.
In the case of 0 - 255 gray scale images, 0 means black.
In the case of normalized images ( minus mean then divided by std), 0 is the mean value.
Should I always pad 0, or should I sometimes pad mean value ?
If in some cases I should pad mean value, what are those cases?
Edit:
And when should I use reflective padding?
Further more, is reflective padding always better than zero padding?
machine-learning neural-network cnn
asked Sep 25 '18 at 9:08
user10253771user10253771
266
$endgroup$
In convolution layers, sometimes you need to pad some (usually 1 or 2 pixel) 0s at the edges of the original image before applying your convolution kernel (filter).
However, 0 has different meanings.
In the case of 0 - 255 gray scale images, 0 means black.
In the case of normalized images ( minus mean then divided by std), 0 is the mean value.
Should I always pad 0, or should I sometimes pad mean value ?
If in some cases I should pad mean value, what are those cases?
Edit:
And when should I use reflective padding?
Further more, is reflective padding always better than zero padding?
machine-learning neural-network cnn
machine-learning neural-network cnn
asked Sep 25 '18 at 9:08
user10253771user10253771
266
asked Sep 25 '18 at 9:08
user10253771user10253771
266
edited Sep 25 '18 at 9:50
user10253771
asked Sep 25 '18 at 9:08
user10253771user10253771
266
asked Sep 25 '18 at 9:08
user10253771user10253771
266
asked Sep 25 '18 at 9:08
user10253771user10253771
266
266
$begingroup$
Why do you pad the pixels? Why with zero?
$endgroup$
– user2974951
Sep 25 '18 at 9:14
$begingroup$
@user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding.
$endgroup$
– user10253771
Sep 25 '18 at 9:47
add a comment |
$begingroup$
Why do you pad the pixels? Why with zero?
$endgroup$
– user2974951
Sep 25 '18 at 9:14
$begingroup$
@user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding.
$endgroup$
– user10253771
Sep 25 '18 at 9:47
$begingroup$
Why do you pad the pixels? Why with zero?
$endgroup$
– user2974951
Sep 25 '18 at 9:14
$begingroup$
Why do you pad the pixels? Why with zero?
$endgroup$
– user2974951
Sep 25 '18 at 9:14
$begingroup$
@user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding.
$endgroup$
– user10253771
Sep 25 '18 at 9:47
$begingroup$
@user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding.
$endgroup$
– user10253771
Sep 25 '18 at 9:47
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Mean, zero, duplication (i.e., constant extrapolation), reflection, and symmetric padding are all valid and widely used methods of padding for conv layers. To my knowledge, there is no systemic study that says one is better than the other in all cases. In other words, one is not always better than any other. (Ideally, one would vary the padding type as a hyper-parameter I suppose.) I think intuitively that reflection and symmetric padding alters the local image structure and global statistics the least. (Clearly, the zeros from zero padding on the boundary are not truly part of the image, and the network has to learn this). However, these have different problems with "realism": for instance, reflection padding will "duplicate" a chair leg on the boundary if the padding is large enough, which may present its own problems depending on the task.
Overall, I think it doesn't matter too much, but it depends on the task and setup. For instance, in generative modelling and image-to-image translation, reflection padding avoid some artifacts on the boundary, as noted in the CycleGAN paper, for example.
An interesting recent paper is Partial Convolution based Padding by Liu et al, where they sidestep this issue by essentially having the convolution completely ignore the boundary. This is a partial convolution in the sense that it ignores the part of the filter that reaches outside of the image. The overall improvements seem to be relatively marginal, except that it does seem to help significantly on the boundary, as one would expect. See also this earlier work doing a similar thing in the segmentation context and this work which applies partial convolutions to inpainting. This approach to avoiding padding is seemingly well-principled.
TL;DR: test different paddings and see which is best for your architecture/task (also check the literature). If that's too much work, I default to reflect padding. If you want to get fancy or worry about boundary effects, use partial convolutions.
answered 2 days ago
user3658307user3658307
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f38758%2fshould-i-pad-zero-or-mean-value-in-a-convolution-neural-network-what-about-refl%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Mean, zero, duplication (i.e., constant extrapolation), reflection, and symmetric padding are all valid and widely used methods of padding for conv layers. To my knowledge, there is no systemic study that says one is better than the other in all cases. In other words, one is not always better than any other. (Ideally, one would vary the padding type as a hyper-parameter I suppose.) I think intuitively that reflection and symmetric padding alters the local image structure and global statistics the least. (Clearly, the zeros from zero padding on the boundary are not truly part of the image, and the network has to learn this). However, these have different problems with "realism": for instance, reflection padding will "duplicate" a chair leg on the boundary if the padding is large enough, which may present its own problems depending on the task.
Overall, I think it doesn't matter too much, but it depends on the task and setup. For instance, in generative modelling and image-to-image translation, reflection padding avoid some artifacts on the boundary, as noted in the CycleGAN paper, for example.
An interesting recent paper is Partial Convolution based Padding by Liu et al, where they sidestep this issue by essentially having the convolution completely ignore the boundary. This is a partial convolution in the sense that it ignores the part of the filter that reaches outside of the image. The overall improvements seem to be relatively marginal, except that it does seem to help significantly on the boundary, as one would expect. See also this earlier work doing a similar thing in the segmentation context and this work which applies partial convolutions to inpainting. This approach to avoiding padding is seemingly well-principled.
TL;DR: test different paddings and see which is best for your architecture/task (also check the literature). If that's too much work, I default to reflect padding. If you want to get fancy or worry about boundary effects, use partial convolutions.
answered 2 days ago
user3658307user3658307
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
Mean, zero, duplication (i.e., constant extrapolation), reflection, and symmetric padding are all valid and widely used methods of padding for conv layers. To my knowledge, there is no systemic study that says one is better than the other in all cases. In other words, one is not always better than any other. (Ideally, one would vary the padding type as a hyper-parameter I suppose.) I think intuitively that reflection and symmetric padding alters the local image structure and global statistics the least. (Clearly, the zeros from zero padding on the boundary are not truly part of the image, and the network has to learn this). However, these have different problems with "realism": for instance, reflection padding will "duplicate" a chair leg on the boundary if the padding is large enough, which may present its own problems depending on the task.
Overall, I think it doesn't matter too much, but it depends on the task and setup. For instance, in generative modelling and image-to-image translation, reflection padding avoid some artifacts on the boundary, as noted in the CycleGAN paper, for example.
An interesting recent paper is Partial Convolution based Padding by Liu et al, where they sidestep this issue by essentially having the convolution completely ignore the boundary. This is a partial convolution in the sense that it ignores the part of the filter that reaches outside of the image. The overall improvements seem to be relatively marginal, except that it does seem to help significantly on the boundary, as one would expect. See also this earlier work doing a similar thing in the segmentation context and this work which applies partial convolutions to inpainting. This approach to avoiding padding is seemingly well-principled.
TL;DR: test different paddings and see which is best for your architecture/task (also check the literature). If that's too much work, I default to reflect padding. If you want to get fancy or worry about boundary effects, use partial convolutions.
answered 2 days ago
user3658307user3658307
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
Mean, zero, duplication (i.e., constant extrapolation), reflection, and symmetric padding are all valid and widely used methods of padding for conv layers. To my knowledge, there is no systemic study that says one is better than the other in all cases. In other words, one is not always better than any other. (Ideally, one would vary the padding type as a hyper-parameter I suppose.) I think intuitively that reflection and symmetric padding alters the local image structure and global statistics the least. (Clearly, the zeros from zero padding on the boundary are not truly part of the image, and the network has to learn this). However, these have different problems with "realism": for instance, reflection padding will "duplicate" a chair leg on the boundary if the padding is large enough, which may present its own problems depending on the task.
Overall, I think it doesn't matter too much, but it depends on the task and setup. For instance, in generative modelling and image-to-image translation, reflection padding avoid some artifacts on the boundary, as noted in the CycleGAN paper, for example.
An interesting recent paper is Partial Convolution based Padding by Liu et al, where they sidestep this issue by essentially having the convolution completely ignore the boundary. This is a partial convolution in the sense that it ignores the part of the filter that reaches outside of the image. The overall improvements seem to be relatively marginal, except that it does seem to help significantly on the boundary, as one would expect. See also this earlier work doing a similar thing in the segmentation context and this work which applies partial convolutions to inpainting. This approach to avoiding padding is seemingly well-principled.
TL;DR: test different paddings and see which is best for your architecture/task (also check the literature). If that's too much work, I default to reflect padding. If you want to get fancy or worry about boundary effects, use partial convolutions.
answered 2 days ago
user3658307user3658307
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
Mean, zero, duplication (i.e., constant extrapolation), reflection, and symmetric padding are all valid and widely used methods of padding for conv layers. To my knowledge, there is no systemic study that says one is better than the other in all cases. In other words, one is not always better than any other. (Ideally, one would vary the padding type as a hyper-parameter I suppose.) I think intuitively that reflection and symmetric padding alters the local image structure and global statistics the least. (Clearly, the zeros from zero padding on the boundary are not truly part of the image, and the network has to learn this). However, these have different problems with "realism": for instance, reflection padding will "duplicate" a chair leg on the boundary if the padding is large enough, which may present its own problems depending on the task.
Overall, I think it doesn't matter too much, but it depends on the task and setup. For instance, in generative modelling and image-to-image translation, reflection padding avoid some artifacts on the boundary, as noted in the CycleGAN paper, for example.
An interesting recent paper is Partial Convolution based Padding by Liu et al, where they sidestep this issue by essentially having the convolution completely ignore the boundary. This is a partial convolution in the sense that it ignores the part of the filter that reaches outside of the image. The overall improvements seem to be relatively marginal, except that it does seem to help significantly on the boundary, as one would expect. See also this earlier work doing a similar thing in the segmentation context and this work which applies partial convolutions to inpainting. This approach to avoiding padding is seemingly well-principled.
TL;DR: test different paddings and see which is best for your architecture/task (also check the literature). If that's too much work, I default to reflect padding. If you want to get fancy or worry about boundary effects, use partial convolutions.
answered 2 days ago
user3658307user3658307
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 2 days ago
user3658307user3658307
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 2 days ago
user3658307user3658307
1855
answered 2 days ago
user3658307user3658307
1855
1855
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
user3658307 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f38758%2fshould-i-pad-zero-or-mean-value-in-a-convolution-neural-network-what-about-refl%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Why do you pad the pixels? Why with zero?
$endgroup$
– user2974951
Sep 25 '18 at 9:14
$begingroup$
@user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding.
$endgroup$
– user10253771
Sep 25 '18 at 9:47