Unable to figure out the linear embedding layer in the convolutional neural network?
$begingroup$
I have the network architecture from the paper "learning fine-grained image similarity with deep ranking" and I am unable to figure out how the output from the three parallel network is merged using the linear embedding layer.
The only information given on this layer, in the paper is
Finally, we normalize the embeddings from the three parts, and combine them with a linear embedding layer. The dimension of the embedding is 4096.
Can anyone help me in figuring out what exactly does the author mean when he is talking about this layer?
neural-network deep-network
edited Nov 23 '18 at 12:25
Snympi
132114
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
$endgroup$
bumped to the homepage by Community♦ 10 hours ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I have the network architecture from the paper "learning fine-grained image similarity with deep ranking" and I am unable to figure out how the output from the three parallel network is merged using the linear embedding layer.
The only information given on this layer, in the paper is
Finally, we normalize the embeddings from the three parts, and combine them with a linear embedding layer. The dimension of the embedding is 4096.
Can anyone help me in figuring out what exactly does the author mean when he is talking about this layer?
neural-network deep-network
edited Nov 23 '18 at 12:25
Snympi
132114
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
$endgroup$
bumped to the homepage by Community♦ 10 hours ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
$begingroup$
It's unfortunate for me that there is no answer for this question. Because I'm stuck with the exactly same issue. Did you figure it out?
$endgroup$
– LKM
Oct 10 '17 at 16:51
$begingroup$
I did not figure out the answer but i just concatenated the input from the three parts and passed it through a dense layer containing 4096 nodes.
$endgroup$
– A. Sam
Oct 12 '17 at 8:53
add a comment |
$begingroup$
I have the network architecture from the paper "learning fine-grained image similarity with deep ranking" and I am unable to figure out how the output from the three parallel network is merged using the linear embedding layer.
The only information given on this layer, in the paper is
Finally, we normalize the embeddings from the three parts, and combine them with a linear embedding layer. The dimension of the embedding is 4096.
Can anyone help me in figuring out what exactly does the author mean when he is talking about this layer?
neural-network deep-network
edited Nov 23 '18 at 12:25
Snympi
132114
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
$endgroup$
I have the network architecture from the paper "learning fine-grained image similarity with deep ranking" and I am unable to figure out how the output from the three parallel network is merged using the linear embedding layer.
The only information given on this layer, in the paper is
Finally, we normalize the embeddings from the three parts, and combine them with a linear embedding layer. The dimension of the embedding is 4096.
Can anyone help me in figuring out what exactly does the author mean when he is talking about this layer?
neural-network deep-network
neural-network deep-network
edited Nov 23 '18 at 12:25
Snympi
132114
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
edited Nov 23 '18 at 12:25
Snympi
132114
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
edited Nov 23 '18 at 12:25
Snympi
132114
edited Nov 23 '18 at 12:25
Snympi
132114
edited Nov 23 '18 at 12:25
Snympi
132114
132114
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
asked Oct 7 '16 at 6:16
A. SamA. Sam
1685
1685
bumped to the homepage by Community♦ 10 hours ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 10 hours ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
$begingroup$
It's unfortunate for me that there is no answer for this question. Because I'm stuck with the exactly same issue. Did you figure it out?
$endgroup$
– LKM
Oct 10 '17 at 16:51
$begingroup$
I did not figure out the answer but i just concatenated the input from the three parts and passed it through a dense layer containing 4096 nodes.
$endgroup$
– A. Sam
Oct 12 '17 at 8:53
add a comment |
$begingroup$
It's unfortunate for me that there is no answer for this question. Because I'm stuck with the exactly same issue. Did you figure it out?
$endgroup$
– LKM
Oct 10 '17 at 16:51
$begingroup$
I did not figure out the answer but i just concatenated the input from the three parts and passed it through a dense layer containing 4096 nodes.
$endgroup$
– A. Sam
Oct 12 '17 at 8:53
$begingroup$
It's unfortunate for me that there is no answer for this question. Because I'm stuck with the exactly same issue. Did you figure it out?
$endgroup$
– LKM
Oct 10 '17 at 16:51
$begingroup$
It's unfortunate for me that there is no answer for this question. Because I'm stuck with the exactly same issue. Did you figure it out?
$endgroup$
– LKM
Oct 10 '17 at 16:51
$begingroup$
I did not figure out the answer but i just concatenated the input from the three parts and passed it through a dense layer containing 4096 nodes.
$endgroup$
– A. Sam
Oct 12 '17 at 8:53
$begingroup$
I did not figure out the answer but i just concatenated the input from the three parts and passed it through a dense layer containing 4096 nodes.
$endgroup$
– A. Sam
Oct 12 '17 at 8:53
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
It's mentioned in the paper:
A local normalization layer normalizes the feature map
around a local neighborhood to have unit norm and zero
mean. It leads to feature maps that are robust to the differences
in illumination and contrast.
They take each part of the model and normalize it separately.
As for combining them, as you commented, to capture the most salient features, with under-complete representation no needs for the non-linearity.
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
$endgroup$
add a comment |
$begingroup$
Linear embedding layer must be just a fancy name for a dense layer with no activation.
'Linear' means there is no activation (activation is identity). And the embedding is rather a concept for a vector representation of the input data (e.g. word embeddings). I believe the elements from the second vector are simply added to the first one element-wise.
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f14412%2funable-to-figure-out-the-linear-embedding-layer-in-the-convolutional-neural-netw%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
It's mentioned in the paper:
A local normalization layer normalizes the feature map
around a local neighborhood to have unit norm and zero
mean. It leads to feature maps that are robust to the differences
in illumination and contrast.
They take each part of the model and normalize it separately.
As for combining them, as you commented, to capture the most salient features, with under-complete representation no needs for the non-linearity.
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
$endgroup$
add a comment |
$begingroup$
It's mentioned in the paper:
A local normalization layer normalizes the feature map
around a local neighborhood to have unit norm and zero
mean. It leads to feature maps that are robust to the differences
in illumination and contrast.
They take each part of the model and normalize it separately.
As for combining them, as you commented, to capture the most salient features, with under-complete representation no needs for the non-linearity.
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
$endgroup$
add a comment |
$begingroup$
It's mentioned in the paper:
A local normalization layer normalizes the feature map
around a local neighborhood to have unit norm and zero
mean. It leads to feature maps that are robust to the differences
in illumination and contrast.
They take each part of the model and normalize it separately.
As for combining them, as you commented, to capture the most salient features, with under-complete representation no needs for the non-linearity.
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
$endgroup$
It's mentioned in the paper:
A local normalization layer normalizes the feature map
around a local neighborhood to have unit norm and zero
mean. It leads to feature maps that are robust to the differences
in illumination and contrast.
They take each part of the model and normalize it separately.
As for combining them, as you commented, to capture the most salient features, with under-complete representation no needs for the non-linearity.
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
answered May 7 '18 at 5:17
Fadi BakouraFadi Bakoura
653111
653111
add a comment |
add a comment |
$begingroup$
Linear embedding layer must be just a fancy name for a dense layer with no activation.
'Linear' means there is no activation (activation is identity). And the embedding is rather a concept for a vector representation of the input data (e.g. word embeddings). I believe the elements from the second vector are simply added to the first one element-wise.
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
$endgroup$
add a comment |
$begingroup$
Linear embedding layer must be just a fancy name for a dense layer with no activation.
'Linear' means there is no activation (activation is identity). And the embedding is rather a concept for a vector representation of the input data (e.g. word embeddings). I believe the elements from the second vector are simply added to the first one element-wise.
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
$endgroup$
add a comment |
$begingroup$
Linear embedding layer must be just a fancy name for a dense layer with no activation.
'Linear' means there is no activation (activation is identity). And the embedding is rather a concept for a vector representation of the input data (e.g. word embeddings). I believe the elements from the second vector are simply added to the first one element-wise.
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
$endgroup$
Linear embedding layer must be just a fancy name for a dense layer with no activation.
'Linear' means there is no activation (activation is identity). And the embedding is rather a concept for a vector representation of the input data (e.g. word embeddings). I believe the elements from the second vector are simply added to the first one element-wise.
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
answered Jan 11 at 14:45
Dmytro PrylipkoDmytro Prylipko
4387
4387
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f14412%2funable-to-figure-out-the-linear-embedding-layer-in-the-convolutional-neural-netw%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
It's unfortunate for me that there is no answer for this question. Because I'm stuck with the exactly same issue. Did you figure it out?
$endgroup$
– LKM
Oct 10 '17 at 16:51
$begingroup$
I did not figure out the answer but i just concatenated the input from the three parts and passed it through a dense layer containing 4096 nodes.
$endgroup$
– A. Sam
Oct 12 '17 at 8:53