VBA Function to convert mm to inches accuracy

NewbieJohnny

New member
Local time
Today, 12:29
Joined
May 23, 2012
Messages
4
I have a public function in a module that converts a number given in millimeters to inches. The problem is that it does not convert accuratly and I'm stumped as to why? The code is:

Public Function CvrtInch(Dia As Single)

CvrtInch = Dia / 25.4

End Function

When I run the function in the immediate window for a Dia variable of 25.4, I get an answer of 0.999999984981507. Any ideas where the inaccuracy is coming from?
 
What "inaccuracy" ? For floating point operations with variables of type Single this is as accurate as you can ever expect. You want smaller "inaccuracy" then use Double
 
I want 25.4 / 25.4 to equal 1... Is this not possible. Sorry, I'm new to programming.
 
Floting point numbers (Single, Double) are not represented accurately at the last decimals - that is impossible.

If you round the result to the number of decimals of your input, you will get your 1.0. Any decimals beyond the number of those input make no sense anyway.

You could, as a "cheat", use Currency for anything with less than 5 values beyond the decimal point.
 

Users who are viewing this thread

Back
Top Bottom