The INT function (1 Viewer)

tmyers

Well-known member
Local time
Today, 12:41
Joined
Sep 8, 2020
Messages
1,090
I have a question that involves the INT function within Access.
I have been using it to do my rounding to the nearest tenth but just have a single anomaly that I cant seem to explain.

My expression is INT(-10*[Price]/(1-[GP]))/-10.
When I ran this against $19.00 with a 5% GP value, it returned $20.10. When I put this into our Excel sheet and take $19.00 and divide it by .95, I get $20.00 even as well as when I simply do it in a calculator.

Can anyone explain why a .10 appeared? At the end of the day it rounded up so it would be in our favor, but I worry that something like that could happen elsewhere and next time it might not just be a dime worth.
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 12:41
Joined
Feb 19, 2002
Messages
42,971
What data type are you using for price and gp? If you are using single or double, you will be getting floating point errors.
 

tmyers

Well-known member
Local time
Today, 12:41
Joined
Sep 8, 2020
Messages
1,090
What data type are you using for price and gp? If you are using single or double, you will be getting floating point errors.
Price is Currency and GP is number (single)
 

theDBguy

I’m here to help
Staff member
Local time
Today, 09:41
Joined
Oct 29, 2018
Messages
21,358
Can anyone explain why a .10 appeared?
Hi. I think it may have to do with "negative" numbers. I'll have to find a reference to verify that thought.
 

plog

Banishment Pending
Local time
Today, 11:41
Joined
May 11, 2011
Messages
11,611
My expression is INT(-10*[Price]/(1-[GP]))/-10.
When I ran this against $19.00 with a 5% GP value, it returned $20.10.

I think you screwed up your inputs or your formula in your database. I created a table with those fields, input 19 for the price and .05 for the GP and created a query with your function and it spit out 20.
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 11:41
Joined
Feb 28, 2001
Messages
26,999
TheDBguy's original thought is correct. There is a marked difference between INT and FIX when dealing with negative numbers, and I think MAYBE you might have used the wrong one.


The INT of a negative number rounds DOWN (away from zero) because INT returns the greatest integer LESS THAN the input number. The FIX of a negative number rounds to the number for which ABS(FIX(number)) is smaller than ABS(number) - i.e. towards zero.
 

Users who are viewing this thread

Top Bottom