float is wrong
#1

Hi, when I set a float to 0.005 and print it, it says "0.004999". Is there any way to fix that? I need it to be exactly 0.005.
Reply
#2

try 0.005001
Reply
#3

If I set it to 0.005001 and print it then it's correct (0.005001). 0.5 is correct as well (0.500000). But 0.005 is 0.004999. What's up with it?
Reply
#4

floats are not exact that is why you always use <= or => when making comparisons.
Reply
#5

I've already see that somewhere. It's about compiling.
I don't know why but it's compiling like 0.49999999999999999999...
It's the same thing with 1.0 : 0.99999...
Reply
#6

Quote:
Originally Posted by Pottus
Посмотреть сообщение
floats are not exact that is why you always use <= or => when making comparisons.
Well 0.5 is exact for me. Is it possible to round 0.004999 to 0.005 as a workaround?
Reply
#7

It's the same thing as trying to display 1/3 in base 10: 0.33333... You know that 1/3 + 1/3 + 1/3 equals 1 but a computer will say its 0.33 + 0.33 + 0.33 = 0.99999... That's just how floating point arithmetic works. As far as I know it's not possible to round the fractional part.

http://www.youtube.com/watch?v=PZRI1IfStY0
Reply
#8

Quote:
Originally Posted by Vince
Посмотреть сообщение
It's the same thing as trying to display 1/3 in base 10: 0.33333... You know that 1/3 + 1/3 + 1/3 equals 1 but a computer will say its 0.33 + 0.33 + 0.33 = 0.99999... That's just how floating point arithmetic works. As far as I know it's not possible to round the fractional part.

http://www.youtube.com/watch?v=PZRI1IfStY0
Alright, thanks. I think I'll just add like 0.000001 when I need to show it somewhere.
Reply


Forum Jump:


Users browsing this thread: 4 Guest(s)