Vectorizing equation in MATLAB
$begingroup$
I am working on collaborative filtering using matrix factorization in MATLAB. I am using Gradient Descent for parameter learning. The cost function to optimize is :
$ J = {left| I odot (R - U V') right|}_{F}^{2} + lambda_1 {left| U right|}_{F}^{2} + lambda_2 {left| V right|}_{F}^{2} $
I have the following equations for updates :
$ nabla J_{u(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') v_j $
$ nabla J_{v(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') u_i $
The dimensions of the matrices are :
$I -> n * n$
$R -> n*n$
$U -> n*k$
$V -> n*k$
I am not able to come up with vectorized form of the gradient, so I have to loop over i's and j's explicitly, which is slowing down the code a lot.
recommender-system gradient-descent matlab
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
$endgroup$
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I am working on collaborative filtering using matrix factorization in MATLAB. I am using Gradient Descent for parameter learning. The cost function to optimize is :
$ J = {left| I odot (R - U V') right|}_{F}^{2} + lambda_1 {left| U right|}_{F}^{2} + lambda_2 {left| V right|}_{F}^{2} $
I have the following equations for updates :
$ nabla J_{u(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') v_j $
$ nabla J_{v(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') u_i $
The dimensions of the matrices are :
$I -> n * n$
$R -> n*n$
$U -> n*k$
$V -> n*k$
I am not able to come up with vectorized form of the gradient, so I have to loop over i's and j's explicitly, which is slowing down the code a lot.
recommender-system gradient-descent matlab
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
$endgroup$
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
1
$begingroup$
1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript?
$endgroup$
– AN6U5
Jul 24 '16 at 15:50
$begingroup$
Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down.
$endgroup$
– AN6U5
Jul 24 '16 at 15:57
$begingroup$
How did you derive the gradients in the first place? Derive them in matrix form and voila.
$endgroup$
– Jim
Sep 22 '16 at 19:54
add a comment |
$begingroup$
I am working on collaborative filtering using matrix factorization in MATLAB. I am using Gradient Descent for parameter learning. The cost function to optimize is :
$ J = {left| I odot (R - U V') right|}_{F}^{2} + lambda_1 {left| U right|}_{F}^{2} + lambda_2 {left| V right|}_{F}^{2} $
I have the following equations for updates :
$ nabla J_{u(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') v_j $
$ nabla J_{v(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') u_i $
The dimensions of the matrices are :
$I -> n * n$
$R -> n*n$
$U -> n*k$
$V -> n*k$
I am not able to come up with vectorized form of the gradient, so I have to loop over i's and j's explicitly, which is slowing down the code a lot.
recommender-system gradient-descent matlab
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
$endgroup$
I am working on collaborative filtering using matrix factorization in MATLAB. I am using Gradient Descent for parameter learning. The cost function to optimize is :
$ J = {left| I odot (R - U V') right|}_{F}^{2} + lambda_1 {left| U right|}_{F}^{2} + lambda_2 {left| V right|}_{F}^{2} $
I have the following equations for updates :
$ nabla J_{u(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') v_j $
$ nabla J_{v(i)} = -sum_j I_{ij} (r_{ij} - u_i v_j') u_i $
The dimensions of the matrices are :
$I -> n * n$
$R -> n*n$
$U -> n*k$
$V -> n*k$
I am not able to come up with vectorized form of the gradient, so I have to loop over i's and j's explicitly, which is slowing down the code a lot.
recommender-system gradient-descent matlab
recommender-system gradient-descent matlab
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
edited Sep 13 '17 at 11:21
Dawny33♦
5,53683289
5,53683289
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
asked Jun 24 '16 at 14:43
SHASHANK GUPTASHASHANK GUPTA
89511221
89511221
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ yesterday
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
1
$begingroup$
1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript?
$endgroup$
– AN6U5
Jul 24 '16 at 15:50
$begingroup$
Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down.
$endgroup$
– AN6U5
Jul 24 '16 at 15:57
$begingroup$
How did you derive the gradients in the first place? Derive them in matrix form and voila.
$endgroup$
– Jim
Sep 22 '16 at 19:54
add a comment |
1
$begingroup$
1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript?
$endgroup$
– AN6U5
Jul 24 '16 at 15:50
$begingroup$
Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down.
$endgroup$
– AN6U5
Jul 24 '16 at 15:57
$begingroup$
How did you derive the gradients in the first place? Derive them in matrix form and voila.
$endgroup$
– Jim
Sep 22 '16 at 19:54
1
1
$begingroup$
1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript?
$endgroup$
– AN6U5
Jul 24 '16 at 15:50
$begingroup$
1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript?
$endgroup$
– AN6U5
Jul 24 '16 at 15:50
$begingroup$
Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down.
$endgroup$
– AN6U5
Jul 24 '16 at 15:57
$begingroup$
Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down.
$endgroup$
– AN6U5
Jul 24 '16 at 15:57
$begingroup$
How did you derive the gradients in the first place? Derive them in matrix form and voila.
$endgroup$
– Jim
Sep 22 '16 at 19:54
$begingroup$
How did you derive the gradients in the first place? Derive them in matrix form and voila.
$endgroup$
– Jim
Sep 22 '16 at 19:54
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Is your question how to vectorize the equations for updating? You have a vector of i and j, and for each update you are summing over j. Calculate the vectors of each of the components and then sum the vector over j.
Providing actual code might help your question. As is, it seems vague.
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
$endgroup$
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f12396%2fvectorizing-equation-in-matlab%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Is your question how to vectorize the equations for updating? You have a vector of i and j, and for each update you are summing over j. Calculate the vectors of each of the components and then sum the vector over j.
Providing actual code might help your question. As is, it seems vague.
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
$endgroup$
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
add a comment |
$begingroup$
Is your question how to vectorize the equations for updating? You have a vector of i and j, and for each update you are summing over j. Calculate the vectors of each of the components and then sum the vector over j.
Providing actual code might help your question. As is, it seems vague.
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
$endgroup$
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
add a comment |
$begingroup$
Is your question how to vectorize the equations for updating? You have a vector of i and j, and for each update you are summing over j. Calculate the vectors of each of the components and then sum the vector over j.
Providing actual code might help your question. As is, it seems vague.
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
$endgroup$
Is your question how to vectorize the equations for updating? You have a vector of i and j, and for each update you are summing over j. Calculate the vectors of each of the components and then sum the vector over j.
Providing actual code might help your question. As is, it seems vague.
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
answered Jun 24 '16 at 15:00
HobbesHobbes
1,199313
1,199313
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
add a comment |
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
$begingroup$
Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui.
$endgroup$
– SHASHANK GUPTA
Jun 24 '16 at 15:19
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f12396%2fvectorizing-equation-in-matlab%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript?
$endgroup$
– AN6U5
Jul 24 '16 at 15:50
$begingroup$
Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down.
$endgroup$
– AN6U5
Jul 24 '16 at 15:57
$begingroup$
How did you derive the gradients in the first place? Derive them in matrix form and voila.
$endgroup$
– Jim
Sep 22 '16 at 19:54