inmediate window calculation

[cpampas]

I wonder why in the inmediate window the calculation :

?68/18032
3,77107364685004E-03

when in fact the correct result is : 0,003771

 

[theDBguy]

Hi.

3,7710...E-03 is the same as 0,003771

The E-03 simply means you can move the decimal point 3 places to the left.

PS. I think it's called "exponential notation"

 

[Isaac]

My calculator gives me:

0.00377107364685

 

[Cronk]

0.003771 is the same as 3.7711E-3 with rounding

3.77107364685004E-03 is showing the same answer but with a higher precision

 

[cpampas]

Humm,
I still dont get it. how can it be the same number, with diferent format or rounding? after all :

?3,77107364685004E-03 > 3 returns True
?0.003771073 > 3 returns False

Hence diferent number ?

 

[CJ_London]

it is just format to show higher precision. your 'correct' result is truncated

if you only want to go to 6dp, use the round function

?round(68/18032,6)
0.003771

To answer your last post, you have used a comma

?3,77107364685004E-03 > 3

and here a dot
?0.003771073 > 3

change the dot to a comma and you get

?0,003771073 > 3
0 True

in the first example change the comma to a dot and you get

?3.77107364685004E-03 > 3
False

perhaps you are in a country that uses comma's rather that dots for decimal points?

 

[arnelgp]

when in fact the correct result is : 0,003771

actually more than that:

?cdec(68)/cdec(18032)

0.0037710736468500443655723159

 

[cpampas]

yes, you are right CJ_London in my example if I change the dot to a comma , I get the same result
i realize now that 0.003771 is the same as 3.7711E-3

Thanks for helping

 

[arnelgp]

change the dot to a comma and you get

?0,003771073 > 3

that does not prove anything.
there are 2 statement on that expression:
1. 0
2. 003771073 > 3

it is similar to:
debug.print 0, 003771073 > 3

so, 3771073 > 3 is always True.