General question on EDA, correlations, classification, ML
$begingroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
asked 2 days ago
user69194user69194
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
asked 2 days ago
user69194user69194
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
asked 2 days ago
user69194user69194
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
machine-learning feature-extraction data-science-model
asked 2 days ago
user69194user69194
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
user69194user69194
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
user69194user69194
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
user69194user69194
11
asked 2 days ago
user69194user69194
11
11
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
user69194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered 2 days ago
SunilSunil
1045
$endgroup$
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
user69194 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46989%2fgeneral-question-on-eda-correlations-classification-ml%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered 2 days ago
SunilSunil
1045
$endgroup$
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
add a comment |
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered 2 days ago
SunilSunil
1045
$endgroup$
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
add a comment |
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered 2 days ago
SunilSunil
1045
$endgroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered 2 days ago
SunilSunil
1045
answered 2 days ago
SunilSunil
1045
answered 2 days ago
SunilSunil
1045
answered 2 days ago
SunilSunil
1045
1045
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
add a comment |
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
yesterday
add a comment |
user69194 is a new contributor. Be nice, and check out our Code of Conduct.
user69194 is a new contributor. Be nice, and check out our Code of Conduct.
user69194 is a new contributor. Be nice, and check out our Code of Conduct.
user69194 is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46989%2fgeneral-question-on-eda-correlations-classification-ml%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
