My small script on value alteration in columns of a data not working
0
$begingroup$
I have a data set which has "Speed" as one of the columns (features). The column contains both zero and non-zero values. I want to randomly set 10% of the non-zero values to zeros. This will change the corresponding "class" label to zeros. I mean any value set to zero, its corresponding class value will be zero as well. I have done this but it is give me errors. Though due to error, I cannot tell it will give me the update/result I want.
file_path = 'Processed_data/data1.csv'
df = pd.read_csv(file_path)
per_change = 0.1
attr = 'Speed'
target = 'Class'
df_spd = df[df['Speed'] > 0.]
num_rows_to_change = int(df.shape[0] * per_change)
num_with_zero_initial = df[df[attr] == 0].shape[0]
assert df_spd.shape[0] > num_rows_to_change,
'Number of rows with non-zero speed is less than 10% of the original dataset.'
df_update = df_spd.sample(num_rows_to_change)
df_update[attr] = 0.
df_update[target] = 0.
df.update(df_update)
update_list = df_update.index.tolist()
num_with_zero_final = df[df['Speed'] == 0].shape[0]
assert num_with_zero_final == num_with_zero_initial + num_rows_to_change,
'Number of rows needed to change not equal to number of rows changed.'
df.to_csv('changed.csv')
pandas
edited yesterday
Kiritee Gak
1,2991420
asked 2 days ago
elvin ugonnaelvin ugonna
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
0
$begingroup$
I have a data set which has "Speed" as one of the columns (features). The column contains both zero and non-zero values. I want to randomly set 10% of the non-zero values to zeros. This will change the corresponding "class" label to zeros. I mean any value set to zero, its corresponding class value will be zero as well. I have done this but it is give me errors. Though due to error, I cannot tell it will give me the update/result I want.
file_path = 'Processed_data/data1.csv'
df = pd.read_csv(file_path)
per_change = 0.1
attr = 'Speed'
target = 'Class'
df_spd = df[df['Speed'] > 0.]
num_rows_to_change = int(df.shape[0] * per_change)
num_with_zero_initial = df[df[attr] == 0].shape[0]
assert df_spd.shape[0] > num_rows_to_change,
'Number of rows with non-zero speed is less than 10% of the original dataset.'
df_update = df_spd.sample(num_rows_to_change)
df_update[attr] = 0.
df_update[target] = 0.
df.update(df_update)
update_list = df_update.index.tolist()
num_with_zero_final = df[df['Speed'] == 0].shape[0]
assert num_with_zero_final == num_with_zero_initial + num_rows_to_change,
'Number of rows needed to change not equal to number of rows changed.'
df.to_csv('changed.csv')
pandas
edited yesterday
Kiritee Gak
1,2991420
asked 2 days ago
elvin ugonnaelvin ugonna
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
1
$begingroup$
Please write the error of the presented code.
$endgroup$
– Alireza Zolanvari
2 days ago
add a comment |
0
0
0
$begingroup$
I have a data set which has "Speed" as one of the columns (features). The column contains both zero and non-zero values. I want to randomly set 10% of the non-zero values to zeros. This will change the corresponding "class" label to zeros. I mean any value set to zero, its corresponding class value will be zero as well. I have done this but it is give me errors. Though due to error, I cannot tell it will give me the update/result I want.
file_path = 'Processed_data/data1.csv'
df = pd.read_csv(file_path)
per_change = 0.1
attr = 'Speed'
target = 'Class'
df_spd = df[df['Speed'] > 0.]
num_rows_to_change = int(df.shape[0] * per_change)
num_with_zero_initial = df[df[attr] == 0].shape[0]
assert df_spd.shape[0] > num_rows_to_change,
'Number of rows with non-zero speed is less than 10% of the original dataset.'
df_update = df_spd.sample(num_rows_to_change)
df_update[attr] = 0.
df_update[target] = 0.
df.update(df_update)
update_list = df_update.index.tolist()
num_with_zero_final = df[df['Speed'] == 0].shape[0]
assert num_with_zero_final == num_with_zero_initial + num_rows_to_change,
'Number of rows needed to change not equal to number of rows changed.'
df.to_csv('changed.csv')
pandas
edited yesterday
Kiritee Gak
1,2991420
asked 2 days ago
elvin ugonnaelvin ugonna
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have a data set which has "Speed" as one of the columns (features). The column contains both zero and non-zero values. I want to randomly set 10% of the non-zero values to zeros. This will change the corresponding "class" label to zeros. I mean any value set to zero, its corresponding class value will be zero as well. I have done this but it is give me errors. Though due to error, I cannot tell it will give me the update/result I want.
file_path = 'Processed_data/data1.csv'
df = pd.read_csv(file_path)
per_change = 0.1
attr = 'Speed'
target = 'Class'
df_spd = df[df['Speed'] > 0.]
num_rows_to_change = int(df.shape[0] * per_change)
num_with_zero_initial = df[df[attr] == 0].shape[0]
assert df_spd.shape[0] > num_rows_to_change,
'Number of rows with non-zero speed is less than 10% of the original dataset.'
df_update = df_spd.sample(num_rows_to_change)
df_update[attr] = 0.
df_update[target] = 0.
df.update(df_update)
update_list = df_update.index.tolist()
num_with_zero_final = df[df['Speed'] == 0].shape[0]
assert num_with_zero_final == num_with_zero_initial + num_rows_to_change,
'Number of rows needed to change not equal to number of rows changed.'
df.to_csv('changed.csv')
pandas
pandas
edited yesterday
Kiritee Gak
1,2991420
asked 2 days ago
elvin ugonnaelvin ugonna
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited yesterday
Kiritee Gak
1,2991420
asked 2 days ago
elvin ugonnaelvin ugonna
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited yesterday
Kiritee Gak
1,2991420
edited yesterday
Kiritee Gak
1,2991420
edited yesterday
Kiritee Gak
1,2991420
1,2991420
asked 2 days ago
elvin ugonnaelvin ugonna
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
elvin ugonnaelvin ugonna
11
asked 2 days ago
elvin ugonnaelvin ugonna
11
11
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
elvin ugonna is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
$begingroup$
Please write the error of the presented code.
$endgroup$
– Alireza Zolanvari
2 days ago
add a comment |
1
$begingroup$
Please write the error of the presented code.
$endgroup$
– Alireza Zolanvari
2 days ago
1
1
$begingroup$
Please write the error of the presented code.
$endgroup$
– Alireza Zolanvari
2 days ago
$begingroup$
Please write the error of the presented code.
$endgroup$
– Alireza Zolanvari
2 days ago
add a comment |
1 Answer
1
active
oldest
votes
0
$begingroup$
FYI, I did not go through your code as it is fairly straight forward assuming I had understood it right.
>> import pandas as pd
>> import random
>> df = pd.DataFrame({'a': np.random.rand(10), b: np.random.rand(10)})
>> print(df)
a b
0 0.127409 0.508811
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.046671 0.592971
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
>> idx = random.sample(df.index, int(len(df)*0.2)) # random indices from dataframe selection, 20% of them are selected based on 0.2
>> print(idx)
[6, 0]
>> df[df.index.isin(idx)] = [0, 0]
>> df
a b
0 0.000000 0.000000
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.000000 0.000000
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
Hope it helps.
answered yesterday
Kiritee GakKiritee Gak
1,2991420
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
elvin ugonna is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47366%2fmy-small-script-on-value-alteration-in-columns-of-a-data-not-working%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
$begingroup$
FYI, I did not go through your code as it is fairly straight forward assuming I had understood it right.
>> import pandas as pd
>> import random
>> df = pd.DataFrame({'a': np.random.rand(10), b: np.random.rand(10)})
>> print(df)
a b
0 0.127409 0.508811
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.046671 0.592971
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
>> idx = random.sample(df.index, int(len(df)*0.2)) # random indices from dataframe selection, 20% of them are selected based on 0.2
>> print(idx)
[6, 0]
>> df[df.index.isin(idx)] = [0, 0]
>> df
a b
0 0.000000 0.000000
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.000000 0.000000
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
Hope it helps.
answered yesterday
Kiritee GakKiritee Gak
1,2991420
$endgroup$
add a comment |
0
$begingroup$
FYI, I did not go through your code as it is fairly straight forward assuming I had understood it right.
>> import pandas as pd
>> import random
>> df = pd.DataFrame({'a': np.random.rand(10), b: np.random.rand(10)})
>> print(df)
a b
0 0.127409 0.508811
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.046671 0.592971
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
>> idx = random.sample(df.index, int(len(df)*0.2)) # random indices from dataframe selection, 20% of them are selected based on 0.2
>> print(idx)
[6, 0]
>> df[df.index.isin(idx)] = [0, 0]
>> df
a b
0 0.000000 0.000000
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.000000 0.000000
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
Hope it helps.
answered yesterday
Kiritee GakKiritee Gak
1,2991420
$endgroup$
add a comment |
0
0
0
$begingroup$
FYI, I did not go through your code as it is fairly straight forward assuming I had understood it right.
>> import pandas as pd
>> import random
>> df = pd.DataFrame({'a': np.random.rand(10), b: np.random.rand(10)})
>> print(df)
a b
0 0.127409 0.508811
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.046671 0.592971
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
>> idx = random.sample(df.index, int(len(df)*0.2)) # random indices from dataframe selection, 20% of them are selected based on 0.2
>> print(idx)
[6, 0]
>> df[df.index.isin(idx)] = [0, 0]
>> df
a b
0 0.000000 0.000000
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.000000 0.000000
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
Hope it helps.
answered yesterday
Kiritee GakKiritee Gak
1,2991420
$endgroup$
FYI, I did not go through your code as it is fairly straight forward assuming I had understood it right.
>> import pandas as pd
>> import random
>> df = pd.DataFrame({'a': np.random.rand(10), b: np.random.rand(10)})
>> print(df)
a b
0 0.127409 0.508811
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.046671 0.592971
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
>> idx = random.sample(df.index, int(len(df)*0.2)) # random indices from dataframe selection, 20% of them are selected based on 0.2
>> print(idx)
[6, 0]
>> df[df.index.isin(idx)] = [0, 0]
>> df
a b
0 0.000000 0.000000
1 0.345239 0.674797
2 0.824521 0.381567
3 0.893538 0.062142
4 0.307070 0.769546
5 0.872883 0.175192
6 0.000000 0.000000
7 0.799977 0.632761
8 0.932829 0.456906
9 0.188867 0.470296
Hope it helps.
answered yesterday
Kiritee GakKiritee Gak
1,2991420
answered yesterday
Kiritee GakKiritee Gak
1,2991420
answered yesterday
Kiritee GakKiritee Gak
1,2991420
answered yesterday
Kiritee GakKiritee Gak
1,2991420
1,2991420
add a comment |
add a comment |
elvin ugonna is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
elvin ugonna is a new contributor. Be nice, and check out our Code of Conduct.
elvin ugonna is a new contributor. Be nice, and check out our Code of Conduct.
elvin ugonna is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47366%2fmy-small-script-on-value-alteration-in-columns-of-a-data-not-working%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
Please write the error of the presented code.
$endgroup$
– Alireza Zolanvari
2 days ago