How to use math.log10 function on whole pandas dataframe
I want to take the logarithm of every value in a pandas dataframe. I have tried this but it does not work:
#Reading data from excel and rounding values on 2 decimal places
import math
import pandas as pd
data = pd.read_excel("DataSet.xls").round(2)
log_data= math.log10(data)
It gives me this error:
TypeError: must be real number, not DataFrame
Do you have any idea what to do?
python pandas numpy
edited yesterday
yatu
12.6k31341
asked yesterday
AleksandarAleksandar
988
add a comment |
I want to take the logarithm of every value in a pandas dataframe. I have tried this but it does not work:
#Reading data from excel and rounding values on 2 decimal places
import math
import pandas as pd
data = pd.read_excel("DataSet.xls").round(2)
log_data= math.log10(data)
It gives me this error:
TypeError: must be real number, not DataFrame
Do you have any idea what to do?
python pandas numpy
edited yesterday
yatu
12.6k31341
asked yesterday
AleksandarAleksandar
988
add a comment |
I want to take the logarithm of every value in a pandas dataframe. I have tried this but it does not work:
#Reading data from excel and rounding values on 2 decimal places
import math
import pandas as pd
data = pd.read_excel("DataSet.xls").round(2)
log_data= math.log10(data)
It gives me this error:
TypeError: must be real number, not DataFrame
Do you have any idea what to do?
python pandas numpy
edited yesterday
yatu
12.6k31341
asked yesterday
AleksandarAleksandar
988
I want to take the logarithm of every value in a pandas dataframe. I have tried this but it does not work:
#Reading data from excel and rounding values on 2 decimal places
import math
import pandas as pd
data = pd.read_excel("DataSet.xls").round(2)
log_data= math.log10(data)
It gives me this error:
TypeError: must be real number, not DataFrame
Do you have any idea what to do?
python pandas numpy
python pandas numpy
edited yesterday
yatu
12.6k31341
asked yesterday
AleksandarAleksandar
988
edited yesterday
yatu
12.6k31341
asked yesterday
AleksandarAleksandar
988
edited yesterday
yatu
12.6k31341
edited yesterday
yatu
12.6k31341
edited yesterday
yatu
12.6k31341
12.6k31341
asked yesterday
AleksandarAleksandar
988
asked yesterday
AleksandarAleksandar
988
asked yesterday
AleksandarAleksandar
988
988
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
Use the numpy version, not math
import numpy as np
np.log10(df)
answered yesterday
ecortazarecortazar
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
1
(Note that if numpy has no equivalent of the function you want, you can also usenp.vectorize(function)to turn any scalar function into a vector function.)
– Christoph Burschka
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
1
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
add a comment |
From what it seems math.log10 cannot handle neither pandas dataframes nor ndarrays.
So one option would be to go with numpy, which also includes a function to compute the base 10 logarithm, np.log10, and reconstruct the dataframe as pointed out in other solutions.
Or if you want to go with math.log10, and the same would apply to other functions that cannot be directly vectorized, you can use DataFrame.applymap to apply math.log10 to the dataframe elementwise. Do note however that this solution will be slower than a vectorized approach using np.log10.
Use case
Here's an example of how this could be done using DataFrame.applymap:
df = pd.DataFrame(np.random.randint(1,5,(6,6)), columns=list('abcdef'))
print(df)
a b c d e f
0 3 4 1 1 2 1
1 4 4 4 3 4 1
2 4 3 3 1 4 1
3 3 4 1 3 1 1
4 1 2 3 4 2 1
5 1 3 3 1 4 3
df.applymap(math.log10)
a b c d e f
0 0.477121 0.602060 0.000000 0.000000 0.30103 0.000000
1 0.602060 0.602060 0.602060 0.477121 0.60206 0.000000
2 0.602060 0.477121 0.477121 0.000000 0.60206 0.000000
3 0.477121 0.602060 0.000000 0.477121 0.00000 0.000000
4 0.000000 0.301030 0.477121 0.602060 0.30103 0.000000
5 0.000000 0.477121 0.477121 0.000000 0.60206 0.477121
For the numpy solution, you could take the np.log10 of the dataframe, and reconstruct it as:
pd.DataFrame(np.log10(data), index=df.index, columns=df.columns)
answered yesterday
yatuyatu
12.6k31341
add a comment |
You may want to use the applymap method to apply math.log10 on the whole dataframe, here is the documentation.
You can test it:
df.applymap(math.log10)
edited yesterday
IanS
8,56232763
answered yesterday
Valentin MercierValentin Mercier
9710
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55024529%2fhow-to-use-math-log10-function-on-whole-pandas-dataframe%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Use the numpy version, not math
import numpy as np
np.log10(df)
answered yesterday
ecortazarecortazar
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
1
(Note that if numpy has no equivalent of the function you want, you can also usenp.vectorize(function)to turn any scalar function into a vector function.)
– Christoph Burschka
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
1
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
add a comment |
Use the numpy version, not math
import numpy as np
np.log10(df)
answered yesterday
ecortazarecortazar
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
1
(Note that if numpy has no equivalent of the function you want, you can also usenp.vectorize(function)to turn any scalar function into a vector function.)
– Christoph Burschka
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
1
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
add a comment |
Use the numpy version, not math
import numpy as np
np.log10(df)
answered yesterday
ecortazarecortazar
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Use the numpy version, not math
import numpy as np
np.log10(df)
answered yesterday
ecortazarecortazar
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered yesterday
ecortazarecortazar
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered yesterday
ecortazarecortazar
2715
answered yesterday
ecortazarecortazar
2715
2715
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
ecortazar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
1
(Note that if numpy has no equivalent of the function you want, you can also usenp.vectorize(function)to turn any scalar function into a vector function.)
– Christoph Burschka
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
1
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
add a comment |
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
1
(Note that if numpy has no equivalent of the function you want, you can also usenp.vectorize(function)to turn any scalar function into a vector function.)
– Christoph Burschka
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
1
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
Thanks my friend!!! Do you have any idea what to do if some values in pandas dataframe are 0? Then, I cant use log function
– Aleksandar
yesterday
1
1
(Note that if numpy has no equivalent of the function you want, you can also use
np.vectorize(function) to turn any scalar function into a vector function.)– Christoph Burschka
yesterday
(Note that if numpy has no equivalent of the function you want, you can also use
np.vectorize(function) to turn any scalar function into a vector function.)– Christoph Burschka
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
@Aleksandar At that point, you'll have to decide what you want to happen to the zeros. Numpy can handle whatever you choose without difficulty.
– Draconis
yesterday
1
1
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
@ChristophBurschka: But if you do that, it's going to be way slower than a "natively" vectorized function, as well as producing silently wrong results if you aren't careful about having consistent output dtypes.
– user2357112
yesterday
add a comment |
From what it seems math.log10 cannot handle neither pandas dataframes nor ndarrays.
So one option would be to go with numpy, which also includes a function to compute the base 10 logarithm, np.log10, and reconstruct the dataframe as pointed out in other solutions.
Or if you want to go with math.log10, and the same would apply to other functions that cannot be directly vectorized, you can use DataFrame.applymap to apply math.log10 to the dataframe elementwise. Do note however that this solution will be slower than a vectorized approach using np.log10.
Use case
Here's an example of how this could be done using DataFrame.applymap:
df = pd.DataFrame(np.random.randint(1,5,(6,6)), columns=list('abcdef'))
print(df)
a b c d e f
0 3 4 1 1 2 1
1 4 4 4 3 4 1
2 4 3 3 1 4 1
3 3 4 1 3 1 1
4 1 2 3 4 2 1
5 1 3 3 1 4 3
df.applymap(math.log10)
a b c d e f
0 0.477121 0.602060 0.000000 0.000000 0.30103 0.000000
1 0.602060 0.602060 0.602060 0.477121 0.60206 0.000000
2 0.602060 0.477121 0.477121 0.000000 0.60206 0.000000
3 0.477121 0.602060 0.000000 0.477121 0.00000 0.000000
4 0.000000 0.301030 0.477121 0.602060 0.30103 0.000000
5 0.000000 0.477121 0.477121 0.000000 0.60206 0.477121
For the numpy solution, you could take the np.log10 of the dataframe, and reconstruct it as:
pd.DataFrame(np.log10(data), index=df.index, columns=df.columns)
answered yesterday
yatuyatu
12.6k31341
add a comment |
From what it seems math.log10 cannot handle neither pandas dataframes nor ndarrays.
So one option would be to go with numpy, which also includes a function to compute the base 10 logarithm, np.log10, and reconstruct the dataframe as pointed out in other solutions.
Or if you want to go with math.log10, and the same would apply to other functions that cannot be directly vectorized, you can use DataFrame.applymap to apply math.log10 to the dataframe elementwise. Do note however that this solution will be slower than a vectorized approach using np.log10.
Use case
Here's an example of how this could be done using DataFrame.applymap:
df = pd.DataFrame(np.random.randint(1,5,(6,6)), columns=list('abcdef'))
print(df)
a b c d e f
0 3 4 1 1 2 1
1 4 4 4 3 4 1
2 4 3 3 1 4 1
3 3 4 1 3 1 1
4 1 2 3 4 2 1
5 1 3 3 1 4 3
df.applymap(math.log10)
a b c d e f
0 0.477121 0.602060 0.000000 0.000000 0.30103 0.000000
1 0.602060 0.602060 0.602060 0.477121 0.60206 0.000000
2 0.602060 0.477121 0.477121 0.000000 0.60206 0.000000
3 0.477121 0.602060 0.000000 0.477121 0.00000 0.000000
4 0.000000 0.301030 0.477121 0.602060 0.30103 0.000000
5 0.000000 0.477121 0.477121 0.000000 0.60206 0.477121
For the numpy solution, you could take the np.log10 of the dataframe, and reconstruct it as:
pd.DataFrame(np.log10(data), index=df.index, columns=df.columns)
answered yesterday
yatuyatu
12.6k31341
add a comment |
From what it seems math.log10 cannot handle neither pandas dataframes nor ndarrays.
So one option would be to go with numpy, which also includes a function to compute the base 10 logarithm, np.log10, and reconstruct the dataframe as pointed out in other solutions.
Or if you want to go with math.log10, and the same would apply to other functions that cannot be directly vectorized, you can use DataFrame.applymap to apply math.log10 to the dataframe elementwise. Do note however that this solution will be slower than a vectorized approach using np.log10.
Use case
Here's an example of how this could be done using DataFrame.applymap:
df = pd.DataFrame(np.random.randint(1,5,(6,6)), columns=list('abcdef'))
print(df)
a b c d e f
0 3 4 1 1 2 1
1 4 4 4 3 4 1
2 4 3 3 1 4 1
3 3 4 1 3 1 1
4 1 2 3 4 2 1
5 1 3 3 1 4 3
df.applymap(math.log10)
a b c d e f
0 0.477121 0.602060 0.000000 0.000000 0.30103 0.000000
1 0.602060 0.602060 0.602060 0.477121 0.60206 0.000000
2 0.602060 0.477121 0.477121 0.000000 0.60206 0.000000
3 0.477121 0.602060 0.000000 0.477121 0.00000 0.000000
4 0.000000 0.301030 0.477121 0.602060 0.30103 0.000000
5 0.000000 0.477121 0.477121 0.000000 0.60206 0.477121
For the numpy solution, you could take the np.log10 of the dataframe, and reconstruct it as:
pd.DataFrame(np.log10(data), index=df.index, columns=df.columns)
answered yesterday
yatuyatu
12.6k31341
From what it seems math.log10 cannot handle neither pandas dataframes nor ndarrays.
So one option would be to go with numpy, which also includes a function to compute the base 10 logarithm, np.log10, and reconstruct the dataframe as pointed out in other solutions.
Or if you want to go with math.log10, and the same would apply to other functions that cannot be directly vectorized, you can use DataFrame.applymap to apply math.log10 to the dataframe elementwise. Do note however that this solution will be slower than a vectorized approach using np.log10.
Use case
Here's an example of how this could be done using DataFrame.applymap:
df = pd.DataFrame(np.random.randint(1,5,(6,6)), columns=list('abcdef'))
print(df)
a b c d e f
0 3 4 1 1 2 1
1 4 4 4 3 4 1
2 4 3 3 1 4 1
3 3 4 1 3 1 1
4 1 2 3 4 2 1
5 1 3 3 1 4 3
df.applymap(math.log10)
a b c d e f
0 0.477121 0.602060 0.000000 0.000000 0.30103 0.000000
1 0.602060 0.602060 0.602060 0.477121 0.60206 0.000000
2 0.602060 0.477121 0.477121 0.000000 0.60206 0.000000
3 0.477121 0.602060 0.000000 0.477121 0.00000 0.000000
4 0.000000 0.301030 0.477121 0.602060 0.30103 0.000000
5 0.000000 0.477121 0.477121 0.000000 0.60206 0.477121
For the numpy solution, you could take the np.log10 of the dataframe, and reconstruct it as:
pd.DataFrame(np.log10(data), index=df.index, columns=df.columns)
answered yesterday
yatuyatu
12.6k31341
edited yesterday
answered yesterday
yatuyatu
12.6k31341
answered yesterday
yatuyatu
12.6k31341
answered yesterday
yatuyatu
12.6k31341
12.6k31341
add a comment |
add a comment |
You may want to use the applymap method to apply math.log10 on the whole dataframe, here is the documentation.
You can test it:
df.applymap(math.log10)
edited yesterday
IanS
8,56232763
answered yesterday
Valentin MercierValentin Mercier
9710
add a comment |
You may want to use the applymap method to apply math.log10 on the whole dataframe, here is the documentation.
You can test it:
df.applymap(math.log10)
edited yesterday
IanS
8,56232763
answered yesterday
Valentin MercierValentin Mercier
9710
add a comment |
You may want to use the applymap method to apply math.log10 on the whole dataframe, here is the documentation.
You can test it:
df.applymap(math.log10)
edited yesterday
IanS
8,56232763
answered yesterday
Valentin MercierValentin Mercier
9710
You may want to use the applymap method to apply math.log10 on the whole dataframe, here is the documentation.
You can test it:
df.applymap(math.log10)
edited yesterday
IanS
8,56232763
answered yesterday
Valentin MercierValentin Mercier
9710
edited yesterday
IanS
8,56232763
edited yesterday
IanS
8,56232763
edited yesterday
IanS
8,56232763
8,56232763
answered yesterday
Valentin MercierValentin Mercier
9710
answered yesterday
Valentin MercierValentin Mercier
9710
answered yesterday
Valentin MercierValentin Mercier
9710
9710
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55024529%2fhow-to-use-math-log10-function-on-whole-pandas-dataframe%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
