Comparing floating-point numbers
#1
I'm down crazy (angry) with lua tonight:

function func1()
LOG(var1);
LOG(var2);
LOG("type var1 is ".. type(var1));
LOG("type var2 is ".. type(var2));

LOG("var1 - var2 = ".. (var1 - var2));

if var1 == var2 then
  LOG("EQ");
else
  LOG("NEQ");
end

Output:
Code:
Log: [02:05:47] 0.8
Log: [02:05:47] 0.8
Log: [02:05:47] type var1 is number
Log: [02:05:47] type var2 is number
Log: [02:05:47] var1 - var2 = -1.1102230246252e-16
Log: [02:05:47] NEQ
Reply
Thanks given by:
#2
Thats just IEEE floating point. Which is why for the server we use "var1 - var2 < x" where x is something like 0.0001 for equality.

http://floating-point-gui.de/errors/comparison/
Reply
Thanks given by:
#3
I'm fix the error
var1 = tostring(var1);
var2 = tostring(var2);

LOG(var1);
LOG(var2);

...
Reply
Thanks given by:
#4
That's a bit inefficient, worktycho's way with epsilon is better, imo.
Reply
Thanks given by:
#5
(12-20-2015, 10:13 PM)jan64 Wrote: That's a bit inefficient, worktycho's way with epsilon is better, imo.

I'm do not understand how does it work in worktycho's way.
Reply
Thanks given by:
#6
something like this:

Code:
if math.abs( var1 - var2 ) < 1e-3 then
   -- equal
else
    -- not equal
end
Reply
Thanks given by:




Users browsing this thread: 2 Guest(s)