2365 views|0 replies

3836

Posts

19

Resources
The OP
 

How to set numpy floating point precision? [Copy link]

Just calculated the same number both ways, but in numpy, it gives an error
[[ 0.910221324013388510820732335560023784637451171875]]
[[-0.9102213240133882887761274105287156999111175537109375]]
This number is the same as e^(-15), but then it's different. How do I handle this error?

Is there a way to limit floating point precision?

Since I use these numbers to calculate exponents, even small differences can lead to frustrating errors...

Do you care about the actual precision of the result, or about getting exactly the same number from both calculations?
If you just want the same number, you can use np.around() to round the result to some appropriate number of decimal places. However, by doing this you will only reduce the precision of the result.

If you really want to compute the result more precisely, you could try using np.longdouble type as your input array, which, depending on your architecture and compiler, might give you 80-bit or 128-bit floating point representation instead of the standard 64-bit np.double * .

You can compare the number of decimal places of precision using np.finfo:

print np.finfo(np.double).precision
# 15

print np.finfo(np.longdouble).precision
# 18
Note that not all numpy functions support long double - some will downcast to double.

*However, some compilers (such as Microsoft Visual C++) always treat long double as a synonym for double, in which case there is no difference in precision between np.longdouble and np.double.

This post is from DSP and ARM Processors
 

Just looking around
Find a datasheet?

EEWorld Datasheet Technical Support

EEWorld
subscription
account

EEWorld
service
account

Automotive
development
circle

Copyright © 2005-2024 EEWORLD.com.cn, Inc. All rights reserved 京B2-20211791 京ICP备10001474号-1 电信业务审批[2006]字第258号函 京公网安备 11010802033920号
快速回复 返回顶部 Return list