PDA

View Full Version : calculator bug?



sandyrepope
22-12-2007, 02:07
I've been playing around with the two calculator examples and I think I have found a bug in the code. Do this:
1 press the 6 button
2 press the 1/x button
answer: .166666666666667
3 press the 1/x button
answer: 5.99999999999999

I'm not too good at math but this seems to give the wrong answer. Could someone let me know why it gives the above result?
I've tried this in the Microsoft calculator and it gives a different answer. Pressing the 1/x button the second time gives an answer of 6.

Thanks
Sandy

ErosOlmi
22-12-2007, 09:52
Sandy,

you are right. The problem are the number of decimal places used when converting from number to string and the other way round. thinBasic is limited with the limit of the compiler used. Max number of decimals in converting number to strings are 18 (in calculator example 15 are used). Internal calculations are done with the precision of EXTENDED numbers (so very high) but continuous conversions /string/number/string introduce a lot of roundings.

Microsoft Calculator uses much more decimal places to perform conversions.

I will see what I can do because I do not like that too.

Ciao
Eros