outliers detection with non normal distribution
$begingroup$
What are some techniques I can use for anomaly detection given a non-Normal distribution? I have less than twenty available observations.
anomaly-detection
edited Mar 22 '18 at 10:44
JahKnows
5,102625
asked Mar 22 '18 at 10:12
youngamyoungam
62
$endgroup$
add a comment |
$begingroup$
What are some techniques I can use for anomaly detection given a non-Normal distribution? I have less than twenty available observations.
anomaly-detection
edited Mar 22 '18 at 10:44
JahKnows
5,102625
asked Mar 22 '18 at 10:12
youngamyoungam
62
$endgroup$
$begingroup$
Checkout QQ plots also with what JahKnows said..
$endgroup$
– Aditya
Mar 22 '18 at 13:07
$begingroup$
@JahKnows - if the offer still stands, I would like to ask for an easy introduction.
$endgroup$
– user7677771
2 days ago
$begingroup$
@user7677771, probably best to ask a separate question to avoid reviving old posts. But, sure!
$endgroup$
– JahKnows
yesterday
add a comment |
$begingroup$
What are some techniques I can use for anomaly detection given a non-Normal distribution? I have less than twenty available observations.
anomaly-detection
edited Mar 22 '18 at 10:44
JahKnows
5,102625
asked Mar 22 '18 at 10:12
youngamyoungam
62
$endgroup$
What are some techniques I can use for anomaly detection given a non-Normal distribution? I have less than twenty available observations.
anomaly-detection
anomaly-detection
edited Mar 22 '18 at 10:44
JahKnows
5,102625
asked Mar 22 '18 at 10:12
youngamyoungam
62
edited Mar 22 '18 at 10:44
JahKnows
5,102625
asked Mar 22 '18 at 10:12
youngamyoungam
62
edited Mar 22 '18 at 10:44
JahKnows
5,102625
edited Mar 22 '18 at 10:44
JahKnows
5,102625
edited Mar 22 '18 at 10:44
JahKnows
5,102625
5,102625
asked Mar 22 '18 at 10:12
youngamyoungam
62
asked Mar 22 '18 at 10:12
youngamyoungam
62
asked Mar 22 '18 at 10:12
youngamyoungam
62
62
$begingroup$
Checkout QQ plots also with what JahKnows said..
$endgroup$
– Aditya
Mar 22 '18 at 13:07
$begingroup$
@JahKnows - if the offer still stands, I would like to ask for an easy introduction.
$endgroup$
– user7677771
2 days ago
$begingroup$
@user7677771, probably best to ask a separate question to avoid reviving old posts. But, sure!
$endgroup$
– JahKnows
yesterday
add a comment |
$begingroup$
Checkout QQ plots also with what JahKnows said..
$endgroup$
– Aditya
Mar 22 '18 at 13:07
$begingroup$
@JahKnows - if the offer still stands, I would like to ask for an easy introduction.
$endgroup$
– user7677771
2 days ago
$begingroup$
@user7677771, probably best to ask a separate question to avoid reviving old posts. But, sure!
$endgroup$
– JahKnows
yesterday
$begingroup$
Checkout QQ plots also with what JahKnows said..
$endgroup$
– Aditya
Mar 22 '18 at 13:07
$begingroup$
Checkout QQ plots also with what JahKnows said..
$endgroup$
– Aditya
Mar 22 '18 at 13:07
$begingroup$
@JahKnows - if the offer still stands, I would like to ask for an easy introduction.
$endgroup$
– user7677771
2 days ago
$begingroup$
@JahKnows - if the offer still stands, I would like to ask for an easy introduction.
$endgroup$
– user7677771
2 days ago
$begingroup$
@user7677771, probably best to ask a separate question to avoid reviving old posts. But, sure!
$endgroup$
– JahKnows
yesterday
$begingroup$
@user7677771, probably best to ask a separate question to avoid reviving old posts. But, sure!
$endgroup$
– JahKnows
yesterday
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
I would suggest a nearest neighbors approach. This technique is non-parametric, such that it does not assume your features follow any given distribution. The degree from which a novel instance can be classified as anomalous can set through some p-value estimation. These techniques are computationally expensive however due to your small dataset this may be well suited.
Check out:
Learning Minimum Volume Sets
http://www.stat.rice.edu/~cscott/pubs/minvol06jmlr.pdf
Anomaly Detection with Score functions based on Nearest Neighbor Graphs
https://arxiv.org/abs/0910.5461
New statistic in P-value estimation for anomaly detection
http://ieeexplore.ieee.org/document/6319713/
You can also use more rudimentary anomaly detection techniques such as a generalized likelihood ratio test. But, this is kind of old-school.
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
$endgroup$
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29411%2foutliers-detection-with-non-normal-distribution%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I would suggest a nearest neighbors approach. This technique is non-parametric, such that it does not assume your features follow any given distribution. The degree from which a novel instance can be classified as anomalous can set through some p-value estimation. These techniques are computationally expensive however due to your small dataset this may be well suited.
Check out:
Learning Minimum Volume Sets
http://www.stat.rice.edu/~cscott/pubs/minvol06jmlr.pdf
Anomaly Detection with Score functions based on Nearest Neighbor Graphs
https://arxiv.org/abs/0910.5461
New statistic in P-value estimation for anomaly detection
http://ieeexplore.ieee.org/document/6319713/
You can also use more rudimentary anomaly detection techniques such as a generalized likelihood ratio test. But, this is kind of old-school.
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
$endgroup$
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
add a comment |
$begingroup$
I would suggest a nearest neighbors approach. This technique is non-parametric, such that it does not assume your features follow any given distribution. The degree from which a novel instance can be classified as anomalous can set through some p-value estimation. These techniques are computationally expensive however due to your small dataset this may be well suited.
Check out:
Learning Minimum Volume Sets
http://www.stat.rice.edu/~cscott/pubs/minvol06jmlr.pdf
Anomaly Detection with Score functions based on Nearest Neighbor Graphs
https://arxiv.org/abs/0910.5461
New statistic in P-value estimation for anomaly detection
http://ieeexplore.ieee.org/document/6319713/
You can also use more rudimentary anomaly detection techniques such as a generalized likelihood ratio test. But, this is kind of old-school.
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
$endgroup$
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
add a comment |
$begingroup$
I would suggest a nearest neighbors approach. This technique is non-parametric, such that it does not assume your features follow any given distribution. The degree from which a novel instance can be classified as anomalous can set through some p-value estimation. These techniques are computationally expensive however due to your small dataset this may be well suited.
Check out:
Learning Minimum Volume Sets
http://www.stat.rice.edu/~cscott/pubs/minvol06jmlr.pdf
Anomaly Detection with Score functions based on Nearest Neighbor Graphs
https://arxiv.org/abs/0910.5461
New statistic in P-value estimation for anomaly detection
http://ieeexplore.ieee.org/document/6319713/
You can also use more rudimentary anomaly detection techniques such as a generalized likelihood ratio test. But, this is kind of old-school.
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
$endgroup$
I would suggest a nearest neighbors approach. This technique is non-parametric, such that it does not assume your features follow any given distribution. The degree from which a novel instance can be classified as anomalous can set through some p-value estimation. These techniques are computationally expensive however due to your small dataset this may be well suited.
Check out:
Learning Minimum Volume Sets
http://www.stat.rice.edu/~cscott/pubs/minvol06jmlr.pdf
Anomaly Detection with Score functions based on Nearest Neighbor Graphs
https://arxiv.org/abs/0910.5461
New statistic in P-value estimation for anomaly detection
http://ieeexplore.ieee.org/document/6319713/
You can also use more rudimentary anomaly detection techniques such as a generalized likelihood ratio test. But, this is kind of old-school.
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
answered Mar 22 '18 at 10:51
JahKnowsJahKnows
5,102625
5,102625
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
add a comment |
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
$begingroup$
I can elaborate on how these techniques work if you have difficulty with the paper. They're relatively easy concepts clouded in a lot of theory in the papers.
$endgroup$
– JahKnows
Mar 22 '18 at 10:52
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f29411%2foutliers-detection-with-non-normal-distribution%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Checkout QQ plots also with what JahKnows said..
$endgroup$
– Aditya
Mar 22 '18 at 13:07
$begingroup$
@JahKnows - if the offer still stands, I would like to ask for an easy introduction.
$endgroup$
– user7677771
2 days ago
$begingroup$
@user7677771, probably best to ask a separate question to avoid reviving old posts. But, sure!
$endgroup$
– JahKnows
yesterday