Anomaly Detection from available sensor data set? [on hold]
$begingroup$
I am working on a live sensor data set and looking for abnormal patterns (leading to a machine fault condition) from the available data set.
I am learning and new to the world of data science, but comfortable with Python. I have few questions that I am looking to get suggestions.
- What kind of algorithms would be best suited for this case?
- What are the basic steps for doing a predictive analysis in python?
Please correct me if my questions are not correctly framed.
machine-learning scikit-learn predictive-modeling anomaly-detection data-science-model
edited 11 hours ago
Stephen Rauch
1,52551330
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
$endgroup$
put on hold as too broad by Esmailian, Stephen Rauch, Sean Owen♦ 3 hours ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
|
show 2 more comments
$begingroup$
I am working on a live sensor data set and looking for abnormal patterns (leading to a machine fault condition) from the available data set.
I am learning and new to the world of data science, but comfortable with Python. I have few questions that I am looking to get suggestions.
- What kind of algorithms would be best suited for this case?
- What are the basic steps for doing a predictive analysis in python?
Please correct me if my questions are not correctly framed.
machine-learning scikit-learn predictive-modeling anomaly-detection data-science-model
edited 11 hours ago
Stephen Rauch
1,52551330
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
$endgroup$
put on hold as too broad by Esmailian, Stephen Rauch, Sean Owen♦ 3 hours ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
$begingroup$
Depends on your dataset's Complexity.. Welcome to the Site..
$endgroup$
– Aditya
Mar 22 '18 at 18:40
$begingroup$
I am dealing with time series data, symptoms of faulty conditions depends on multiple parameters.
$endgroup$
– Vikas Gaikwad
Mar 22 '18 at 18:43
2
$begingroup$
What will and what won't work, their is no assurance to that point in ML....It's just we say that it should work in this case... Doing an EDA is the first step, checking which components get replaced frequently,are there parts which stops working on same day, time of the week etc.. doing feature engineering (adding new features) and them applying a DL model which suits my understanding is what I would do..
$endgroup$
– Aditya
Mar 22 '18 at 19:02
$begingroup$
This is a really broad question - you are asking for information about a methodology that incorporates most of statistics...
$endgroup$
– Spacedman
May 27 '18 at 7:10
$begingroup$
what kind of sensor do you have?
$endgroup$
– Francesco Pegoraro
Sep 24 '18 at 14:23
|
show 2 more comments
$begingroup$
I am working on a live sensor data set and looking for abnormal patterns (leading to a machine fault condition) from the available data set.
I am learning and new to the world of data science, but comfortable with Python. I have few questions that I am looking to get suggestions.
- What kind of algorithms would be best suited for this case?
- What are the basic steps for doing a predictive analysis in python?
Please correct me if my questions are not correctly framed.
machine-learning scikit-learn predictive-modeling anomaly-detection data-science-model
edited 11 hours ago
Stephen Rauch
1,52551330
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
$endgroup$
I am working on a live sensor data set and looking for abnormal patterns (leading to a machine fault condition) from the available data set.
I am learning and new to the world of data science, but comfortable with Python. I have few questions that I am looking to get suggestions.
- What kind of algorithms would be best suited for this case?
- What are the basic steps for doing a predictive analysis in python?
Please correct me if my questions are not correctly framed.
machine-learning scikit-learn predictive-modeling anomaly-detection data-science-model
machine-learning scikit-learn predictive-modeling anomaly-detection data-science-model
edited 11 hours ago
Stephen Rauch
1,52551330
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
edited 11 hours ago
Stephen Rauch
1,52551330
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
edited 11 hours ago
Stephen Rauch
1,52551330
edited 11 hours ago
Stephen Rauch
1,52551330
edited 11 hours ago
Stephen Rauch
1,52551330
1,52551330
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
asked Mar 22 '18 at 18:38
Vikas GaikwadVikas Gaikwad
112
112
put on hold as too broad by Esmailian, Stephen Rauch, Sean Owen♦ 3 hours ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
put on hold as too broad by Esmailian, Stephen Rauch, Sean Owen♦ 3 hours ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
$begingroup$
Depends on your dataset's Complexity.. Welcome to the Site..
$endgroup$
– Aditya
Mar 22 '18 at 18:40
$begingroup$
I am dealing with time series data, symptoms of faulty conditions depends on multiple parameters.
$endgroup$
– Vikas Gaikwad
Mar 22 '18 at 18:43
2
$begingroup$
What will and what won't work, their is no assurance to that point in ML....It's just we say that it should work in this case... Doing an EDA is the first step, checking which components get replaced frequently,are there parts which stops working on same day, time of the week etc.. doing feature engineering (adding new features) and them applying a DL model which suits my understanding is what I would do..
$endgroup$
– Aditya
Mar 22 '18 at 19:02
$begingroup$
This is a really broad question - you are asking for information about a methodology that incorporates most of statistics...
$endgroup$
– Spacedman
May 27 '18 at 7:10
$begingroup$
what kind of sensor do you have?
$endgroup$
– Francesco Pegoraro
Sep 24 '18 at 14:23
|
show 2 more comments
$begingroup$
Depends on your dataset's Complexity.. Welcome to the Site..
$endgroup$
– Aditya
Mar 22 '18 at 18:40
$begingroup$
I am dealing with time series data, symptoms of faulty conditions depends on multiple parameters.
$endgroup$
– Vikas Gaikwad
Mar 22 '18 at 18:43
2
$begingroup$
What will and what won't work, their is no assurance to that point in ML....It's just we say that it should work in this case... Doing an EDA is the first step, checking which components get replaced frequently,are there parts which stops working on same day, time of the week etc.. doing feature engineering (adding new features) and them applying a DL model which suits my understanding is what I would do..
$endgroup$
– Aditya
Mar 22 '18 at 19:02
$begingroup$
This is a really broad question - you are asking for information about a methodology that incorporates most of statistics...
$endgroup$
– Spacedman
May 27 '18 at 7:10
$begingroup$
what kind of sensor do you have?
$endgroup$
– Francesco Pegoraro
Sep 24 '18 at 14:23
$begingroup$
Depends on your dataset's Complexity.. Welcome to the Site..
$endgroup$
– Aditya
Mar 22 '18 at 18:40
$begingroup$
Depends on your dataset's Complexity.. Welcome to the Site..
$endgroup$
– Aditya
Mar 22 '18 at 18:40
$begingroup$
I am dealing with time series data, symptoms of faulty conditions depends on multiple parameters.
$endgroup$
– Vikas Gaikwad
Mar 22 '18 at 18:43
$begingroup$
I am dealing with time series data, symptoms of faulty conditions depends on multiple parameters.
$endgroup$
– Vikas Gaikwad
Mar 22 '18 at 18:43
2
2
$begingroup$
What will and what won't work, their is no assurance to that point in ML....It's just we say that it should work in this case... Doing an EDA is the first step, checking which components get replaced frequently,are there parts which stops working on same day, time of the week etc.. doing feature engineering (adding new features) and them applying a DL model which suits my understanding is what I would do..
$endgroup$
– Aditya
Mar 22 '18 at 19:02
$begingroup$
What will and what won't work, their is no assurance to that point in ML....It's just we say that it should work in this case... Doing an EDA is the first step, checking which components get replaced frequently,are there parts which stops working on same day, time of the week etc.. doing feature engineering (adding new features) and them applying a DL model which suits my understanding is what I would do..
$endgroup$
– Aditya
Mar 22 '18 at 19:02
$begingroup$
This is a really broad question - you are asking for information about a methodology that incorporates most of statistics...
$endgroup$
– Spacedman
May 27 '18 at 7:10
$begingroup$
This is a really broad question - you are asking for information about a methodology that incorporates most of statistics...
$endgroup$
– Spacedman
May 27 '18 at 7:10
$begingroup$
what kind of sensor do you have?
$endgroup$
– Francesco Pegoraro
Sep 24 '18 at 14:23
$begingroup$
what kind of sensor do you have?
$endgroup$
– Francesco Pegoraro
Sep 24 '18 at 14:23
|
show 2 more comments
2 Answers
2
active
oldest
votes
$begingroup$
You are talking about anomaly detection, and there are many approaches. If you can create a training set, one-class SVM is a place to start, but even simple control charts can be useful, particularly with live streaming data.
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
$endgroup$
add a comment |
$begingroup$
If you have a dataset with labeled anomalies, then you can use a binary classification approach. If the anomalies are not labeled, then you have to look into outlier/novelty detection. The scikit-learn documenation has a good overview.
answered 17 hours ago
jonnorjonnor
2275
$endgroup$
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
You are talking about anomaly detection, and there are many approaches. If you can create a training set, one-class SVM is a place to start, but even simple control charts can be useful, particularly with live streaming data.
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
$endgroup$
add a comment |
$begingroup$
You are talking about anomaly detection, and there are many approaches. If you can create a training set, one-class SVM is a place to start, but even simple control charts can be useful, particularly with live streaming data.
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
$endgroup$
add a comment |
$begingroup$
You are talking about anomaly detection, and there are many approaches. If you can create a training set, one-class SVM is a place to start, but even simple control charts can be useful, particularly with live streaming data.
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
$endgroup$
You are talking about anomaly detection, and there are many approaches. If you can create a training set, one-class SVM is a place to start, but even simple control charts can be useful, particularly with live streaming data.
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
answered Mar 27 '18 at 22:02
HEITZHEITZ
63126
63126
add a comment |
add a comment |
$begingroup$
If you have a dataset with labeled anomalies, then you can use a binary classification approach. If the anomalies are not labeled, then you have to look into outlier/novelty detection. The scikit-learn documenation has a good overview.
answered 17 hours ago
jonnorjonnor
2275
$endgroup$
add a comment |
$begingroup$
If you have a dataset with labeled anomalies, then you can use a binary classification approach. If the anomalies are not labeled, then you have to look into outlier/novelty detection. The scikit-learn documenation has a good overview.
answered 17 hours ago
jonnorjonnor
2275
$endgroup$
add a comment |
$begingroup$
If you have a dataset with labeled anomalies, then you can use a binary classification approach. If the anomalies are not labeled, then you have to look into outlier/novelty detection. The scikit-learn documenation has a good overview.
answered 17 hours ago
jonnorjonnor
2275
$endgroup$
If you have a dataset with labeled anomalies, then you can use a binary classification approach. If the anomalies are not labeled, then you have to look into outlier/novelty detection. The scikit-learn documenation has a good overview.
answered 17 hours ago
jonnorjonnor
2275
answered 17 hours ago
jonnorjonnor
2275
answered 17 hours ago
jonnorjonnor
2275
answered 17 hours ago
jonnorjonnor
2275
2275
add a comment |
add a comment |
$begingroup$
Depends on your dataset's Complexity.. Welcome to the Site..
$endgroup$
– Aditya
Mar 22 '18 at 18:40
$begingroup$
I am dealing with time series data, symptoms of faulty conditions depends on multiple parameters.
$endgroup$
– Vikas Gaikwad
Mar 22 '18 at 18:43
2
$begingroup$
What will and what won't work, their is no assurance to that point in ML....It's just we say that it should work in this case... Doing an EDA is the first step, checking which components get replaced frequently,are there parts which stops working on same day, time of the week etc.. doing feature engineering (adding new features) and them applying a DL model which suits my understanding is what I would do..
$endgroup$
– Aditya
Mar 22 '18 at 19:02
$begingroup$
This is a really broad question - you are asking for information about a methodology that incorporates most of statistics...
$endgroup$
– Spacedman
May 27 '18 at 7:10
$begingroup$
what kind of sensor do you have?
$endgroup$
– Francesco Pegoraro
Sep 24 '18 at 14:23