samcoinc
11-05-2009, 07:32 AM
Could someone explain this? (problems in code - this is a sample from the debugger)
?int(39.564/13.188)
2
?int((13.188*3)/13.188)
3
?int(35.256/11.752)
2
I am beating my head against the wall.
(sorry - adding - this is access 97)
Thanks
sam
chergh
11-05-2009, 07:50 AM
Its a known issue with the IEEE specification for double datatype.
There is more info at: http://www.cpearson.com/excel/rounding.htm
It is as relevant to access as it is to excel.
You can work round it, for example if you use:
?int(cdec(39.564)/cdec(13.188))
you should see the correct answer.
samcoinc
11-05-2009, 08:03 AM
Wow - How did I miss this info. :)
initial testing seems to work correctly. You are a life saver!
Thank you very much.
sam
Its a known issue with the IEEE specification for double datatype.
There is more info at: http://www.cpearson.com/excel/rounding.htm
It is as relevant to access as it is to excel.
You can work round it, for example if you use:
?int(cdec(39.564)/cdec(13.188))
you should see the correct answer.