float is wrong
#7

It's the same thing as trying to display 1/3 in base 10: 0.33333... You know that 1/3 + 1/3 + 1/3 equals 1 but a computer will say its 0.33 + 0.33 + 0.33 = 0.99999... That's just how floating point arithmetic works. As far as I know it's not possible to round the fractional part.

http://www.youtube.com/watch?v=PZRI1IfStY0
Reply


Messages In This Thread
float is wrong - by GoldenLion - 12.02.2017, 18:35
Re: float is wrong - by Jefff - 12.02.2017, 18:39
Re: float is wrong - by GoldenLion - 12.02.2017, 18:42
Re: float is wrong - by Pottus - 12.02.2017, 19:00
Re: float is wrong - by Dayrion - 12.02.2017, 19:22
Re: float is wrong - by GoldenLion - 12.02.2017, 19:35
Re: float is wrong - by Vince - 12.02.2017, 19:55
Re: float is wrong - by GoldenLion - 12.02.2017, 20:13

Forum Jump:


Users browsing this thread: 2 Guest(s)