My naive (ha!) Gaussian Naive Bayes classifier is too slow
$begingroup$
I am trying to build a film review classifier where I determine if a given review is positive or negative (w/ Python). I'm trying to avoid any other ML libraries so that I can better understand the processes. Here is my approach and the problems that I am facing:
- I mine thousands of film reviews as training sets and classify them as positive or negative.
- I parse through my training set and for each class, I build an array of unique words.
- For each document, I build a vector of TF-IDF values where the vector size is my number of unique words.
- I use a Gaussian classifier to determine: $$P(C_i|w)=P(C_i)P(w|C)=P(C_i)*dfrac{1}{sqrt{2pi}sigma_i}e^{-(1/2)(w-mu_i)^Tsigma_i^{-1}(w-mu_i)}$$ where $w$ is the my document in a vector, $C_i$ is a particular class, $mu_i$ is the mean vector and $sigma_i$ is my covariance matrix.
This approach seems to makes sense. My problem is that my algorithm is much too slow. As an example, I have sampled over 1,500 documents and I have determined over 40,000 unique words. This mean that each of my document vectors has 40,000 entries and if I were to build a covariance matrix, it would have dimensions 40,000 by 40,000. Even I were able to generate the entirety of $sigma_i$, but then I would have to compute the matrix product in the exponent, which will take an extraordinarily long time just to classify one document.
I have experimented with a multinomial approach, which is working well. I very curious on how to make this work more efficiently. I realise the matrix multiplication runtime can't be improved, and I was hoping for insight on how others are able to do this.
Some things I have tried:
- Filtered any stop words (but this still leaves me with tens of thousands of words)
- Estimated $sigma_i$ by summing over a couple of documents.
machine-learning python naive-bayes-classifier tfidf
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I am trying to build a film review classifier where I determine if a given review is positive or negative (w/ Python). I'm trying to avoid any other ML libraries so that I can better understand the processes. Here is my approach and the problems that I am facing:
- I mine thousands of film reviews as training sets and classify them as positive or negative.
- I parse through my training set and for each class, I build an array of unique words.
- For each document, I build a vector of TF-IDF values where the vector size is my number of unique words.
- I use a Gaussian classifier to determine: $$P(C_i|w)=P(C_i)P(w|C)=P(C_i)*dfrac{1}{sqrt{2pi}sigma_i}e^{-(1/2)(w-mu_i)^Tsigma_i^{-1}(w-mu_i)}$$ where $w$ is the my document in a vector, $C_i$ is a particular class, $mu_i$ is the mean vector and $sigma_i$ is my covariance matrix.
This approach seems to makes sense. My problem is that my algorithm is much too slow. As an example, I have sampled over 1,500 documents and I have determined over 40,000 unique words. This mean that each of my document vectors has 40,000 entries and if I were to build a covariance matrix, it would have dimensions 40,000 by 40,000. Even I were able to generate the entirety of $sigma_i$, but then I would have to compute the matrix product in the exponent, which will take an extraordinarily long time just to classify one document.
I have experimented with a multinomial approach, which is working well. I very curious on how to make this work more efficiently. I realise the matrix multiplication runtime can't be improved, and I was hoping for insight on how others are able to do this.
Some things I have tried:
- Filtered any stop words (but this still leaves me with tens of thousands of words)
- Estimated $sigma_i$ by summing over a couple of documents.
machine-learning python naive-bayes-classifier tfidf
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I am trying to build a film review classifier where I determine if a given review is positive or negative (w/ Python). I'm trying to avoid any other ML libraries so that I can better understand the processes. Here is my approach and the problems that I am facing:
- I mine thousands of film reviews as training sets and classify them as positive or negative.
- I parse through my training set and for each class, I build an array of unique words.
- For each document, I build a vector of TF-IDF values where the vector size is my number of unique words.
- I use a Gaussian classifier to determine: $$P(C_i|w)=P(C_i)P(w|C)=P(C_i)*dfrac{1}{sqrt{2pi}sigma_i}e^{-(1/2)(w-mu_i)^Tsigma_i^{-1}(w-mu_i)}$$ where $w$ is the my document in a vector, $C_i$ is a particular class, $mu_i$ is the mean vector and $sigma_i$ is my covariance matrix.
This approach seems to makes sense. My problem is that my algorithm is much too slow. As an example, I have sampled over 1,500 documents and I have determined over 40,000 unique words. This mean that each of my document vectors has 40,000 entries and if I were to build a covariance matrix, it would have dimensions 40,000 by 40,000. Even I were able to generate the entirety of $sigma_i$, but then I would have to compute the matrix product in the exponent, which will take an extraordinarily long time just to classify one document.
I have experimented with a multinomial approach, which is working well. I very curious on how to make this work more efficiently. I realise the matrix multiplication runtime can't be improved, and I was hoping for insight on how others are able to do this.
Some things I have tried:
- Filtered any stop words (but this still leaves me with tens of thousands of words)
- Estimated $sigma_i$ by summing over a couple of documents.
machine-learning python naive-bayes-classifier tfidf
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I am trying to build a film review classifier where I determine if a given review is positive or negative (w/ Python). I'm trying to avoid any other ML libraries so that I can better understand the processes. Here is my approach and the problems that I am facing:
- I mine thousands of film reviews as training sets and classify them as positive or negative.
- I parse through my training set and for each class, I build an array of unique words.
- For each document, I build a vector of TF-IDF values where the vector size is my number of unique words.
- I use a Gaussian classifier to determine: $$P(C_i|w)=P(C_i)P(w|C)=P(C_i)*dfrac{1}{sqrt{2pi}sigma_i}e^{-(1/2)(w-mu_i)^Tsigma_i^{-1}(w-mu_i)}$$ where $w$ is the my document in a vector, $C_i$ is a particular class, $mu_i$ is the mean vector and $sigma_i$ is my covariance matrix.
This approach seems to makes sense. My problem is that my algorithm is much too slow. As an example, I have sampled over 1,500 documents and I have determined over 40,000 unique words. This mean that each of my document vectors has 40,000 entries and if I were to build a covariance matrix, it would have dimensions 40,000 by 40,000. Even I were able to generate the entirety of $sigma_i$, but then I would have to compute the matrix product in the exponent, which will take an extraordinarily long time just to classify one document.
I have experimented with a multinomial approach, which is working well. I very curious on how to make this work more efficiently. I realise the matrix multiplication runtime can't be improved, and I was hoping for insight on how others are able to do this.
Some things I have tried:
- Filtered any stop words (but this still leaves me with tens of thousands of words)
- Estimated $sigma_i$ by summing over a couple of documents.
machine-learning python naive-bayes-classifier tfidf
machine-learning python naive-bayes-classifier tfidf
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
asked 13 mins ago
Ayumu KasuganoAyumu Kasugano
101
101
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Ayumu Kasugano is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Ayumu Kasugano is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f45067%2fmy-naive-ha-gaussian-naive-bayes-classifier-is-too-slow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Ayumu Kasugano is a new contributor. Be nice, and check out our Code of Conduct.
Ayumu Kasugano is a new contributor. Be nice, and check out our Code of Conduct.
Ayumu Kasugano is a new contributor. Be nice, and check out our Code of Conduct.
Ayumu Kasugano is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f45067%2fmy-naive-ha-gaussian-naive-bayes-classifier-is-too-slow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
