Nearly had a heart attack just now. I'm working on a fixed-point arithmetic include script for BigInt and was writing an example script with some, erm, examples.
After the simple, boring, stuff I had the script set the number of digits to 1,000. Square root of 2, cube root of 2. And 1/10! seemed like a good idea. The result was 0.5 and 999 other digits. That was not the near-heart attack.
Rewrote the test as 1 divided by 2, then the result by 3, then the result by 4, etc. The result for 0.1666666666 divided by 4 was 4.041666666666. Yes, there's a "4" in the beginning there. That was the near heart-attack.
Luckily it turned out it was just the convert-internal-fixed-point-value-to-string function that was incorrect, not the BigInt module. What I had was this.
What it should have been was this.If Len(a)<=fxp_dgt Then a=RIGHT$(Repeat$(fxp_dgt,"0"+a),fxp_dgt+1)
(The closing parenthesis of the Repeat$ should be after the "0", not after the a.)If Len(a)<=fxp_dgt Then a=RIGHT$(Repeat$(fxp_dgt,"0")+a,fxp_dgt+1)
Man, I should find a hobby that's a little bit easier on my nerves. Maybe there's an opening at the bomb squad.
Bookmarks