Nearly equal with relative and absolute tolerance

[dvd101x]

Hi,

This initial commit is intended for exploration of #2838. It overwrites the current nearlyEqual completely to be as similar to python's Math.isclose as possible (according to the description in the documentation)

It currently fails 6 tests:

• 2 tests in rotationMatrix
• 1 test in nearlyEqual (for bigNumber)
• 3 tests in number / nearlyEqual

No effort was made to either adapt the tests or fix the function. Everything is left as is to review the failing tests and aid in the decision making.

Pleas note that the functions are changed abruptly only for exploration and can be returned to their original form upon review.

Also no calls for nearlyEqual were modified, thus only relative tolerance is being used. If the default values are changed to relTol = Number.EPSILON, absTol = 1e-12 then a few more tests fail.

[josdejong]

Thanks, this is a great start!

Some thoughts:

1. Only 8 failing unit tests is not that much :). I don't see failing tests related to rotationMatrix though, I see 4 tests for eigs failing?
2. I expect more tests to fail as soon as we update all places where nearlyEqual(...) is invoked to pass both relTol and absTol.
3. I think we may need to add some additional unit tests to test relTol and absTol individually.
4. We should decide upon good default values relTol and absTol, and how we map values for both when a user configures epsilon (for backward compatibility). Would it make sense to default to relTol = epsilon ?? 1e-12 and absTol = epsilon ?? 1e-12?
5. In general I like to use full words rather than abbreviations, like relativeTolerance and absoluteTolerance. However in this case the words are quite long. Still thinking about that.
6. I see the tests where run on node.js 16, I don't understand why, it should only run node 18 and 20 (see config)? Is something in this branch not up to date?

[dvd101x]

1. That's odd, I don't see eigs failing, maybe this is not up to date.
2. I agree
3. I agree
4. I think the current behavior is similar to having an absTol about 1000 times smaller than relTol. I see on the julia docs for isaprox some ways to calculate those default values.
5. I agree these names don't match mathjs very well just took the python names and got a warning related to the camel case. I'm sure there are more appropriate names. Regarding abbreviations it could be argued that config is an abbreviation.
6. At some point I saw that line trying to update in my PR but I returned it to what it was. Probably it was my mistake.

[dvd101x]

A quick comparison for names and values.

language	function	relative tolerance	absolute tolerance
python	isclose	rel_tol = 1e-09	abs_tol = 0.0
numpy	isclose	rtol = 1e-05	atol = 1e-08
julia	isapprox	rtol = atol>0 ? 0 : √eps	atol = 0

[josdejong]

(1) You can see the test results by clicking them in the list with checks in this PR, for example https://github.com/josdejong/mathjs/actions/runs/7853061106/job/21432241083?pr=3152. I guess indeed that you somehow have an old version or so (that may also explain why the PR is still testing on Node 16).

(5) ha ha, that's true. I'm ok with either absTol / relTol or absoluteTolerance / relativeTolerance, I don't have a very strong opinion in this regard (in general, I like the non abbreviated name, but in this case it is quite long).

[dvd101x]

(1) I was at v 11.9.1 (sorry about that). Should be updated to v12.4.0 by now. (5) Ok

Now that it's updated, I see the following errors:

• round as predicted at #3136
• eigs
• round

What would be the next steps?

[josdejong]

I think the next step is to work out this PR in detail: update all places where nearlyEqual(...) is invoked, and then fix all failing unit tests.

[dvd101x]

Ok, thanks! I will do that and report back.

[dvd101x]

Hi, just reporting back.

Progress has been slower than what I originally expected but I'm still on it.

[josdejong]

Thanks for the update David, good to know.

[dvd101x]

Hi Jos

I have some good news!

I found a good fit with a relative tolerance the size of the current config.epsilon 1e-12 and an absolute tolerance of 1e-15. I did some empirical tests and works similarly in most cases.

In the latest submit I made all calls for nearlyEqual to use relTol = config.epsilon and absTol = config.epsilon * 1e-3 and this eliminates all the failing tests except for the nearlyEqual tests.

I would like your advise with the following:

1. Shall I remove config.epsilon in favor of config.relTol and config.absTol? (naming to be defined)
2. Shall I change the failing nearlyEqual tests to accommodate for the proposed nearlyEqual or make a compromise between the proposed nearlyEqual and the current tests?
3. Shall I make a mathjs function nearlyEqual available so that the user can choose absTol and relTol without changing config or shall it be a different topic outside this PR?

Appendix

Here is the mathjs code I used to test for the tolerances: (this works on the current version 12.4.1

config({epsilon:1e-12});

# Define function nearly equal
nearlyEqual(a, b, relTol, absTol) = abs(a-b) <= max(relTol * max(abs(a), abs(b)), absTol);

# Define rtol and atol 
rtol = config().epsilon;
atol = rtol * 1e-3;

# Tested various values of x
X = [0, 0.1, 0.5, 1, 10, 1e3, 1e6, 1e12];
x = X[1];

# Ys
powers = -2:-1:-16;
Y = x + map(powers, _(d) = pow(10, d));

# Proposed
nearlyEqualResults = map(Y, _(y) = nearlyEqual(x, y, rtol, atol));

# Current
equalResults = x == Y;

# These are the powers where the current nearly equals differ from the proposed one
differences = nearlyEqualResults != equalResults;

result = {
  powersFailing:powers[differences],
  x:x,
  current:equalResults[differences],
  proposed:nearlyEqualResults[differences],
  rtol:rtol,
  atol:atol
};

print("Testing with:\n
rtol: $rtol
atol: $atol
x: $x\n
yields:
powers faling: $powersFailing
proposal: $proposed
current: $current
", result)

I read the following document to realize that nearlyEqual might need to be one of the mathjs functions.

• MDN: Number.EPSILON

[josdejong]

Yay, that sounds promising!

1. Yes, though we do need backward compatibility in the function config(). When people are still using .epsilon, we can translate it internally into the new options relTol and abstol and output a console warning about the deprecation.
2. I think we have to adjust the failing nearlyEqual tests. This will be a breaking change, and I suppose the failing tests are due to the behavior now being different, so the tests should be adapted accordingly. (please correct me if I'm wrong)
3. I'm not sure if we should expose the nearlyEqual function, it may make sense 🤔. I prefer to discuss that in a separate topic to keep this PR focused, OK?

Looking at the code example, you could definitely use #2953 🤣

[dvd101x]

Thanks!

1. OK: sounds good, I was worried it was too much change.
2. OK: most are due to the new formula. There is one that might be debatable that says nearlyEqual: should do exact comparison when epsilon is null or undefined. In my opinion it should resort to default values of relTol and absTol when they are null or undefined
3. OK: Will make a separate discussion.

Absolutely! I was trying to change my point of view about the shorthand and I did.

PD: Also could have used #2701 to cycle through the values of X

[josdejong]

👍

I see you're pushing new commits. Please let me know when the PR is ready for review (or if you hit new challenges).

[dvd101x]

Hi Jos,

Thanks for the follow up.

I'm still tweaking some stuff and will let you know when it's ready for review.

[dvd101x]

Hi Jos, this is ready for review

[josdejong]

Thanks, that is good news! I'll look into it asap.

[dvd101x]

Hi Jos, thanks for the thorough review!

I'm also glad there where not so many errors. The reason might be that the values for the new tolerances were picked to be close to the existing behavior. Maybe at some point in the future the values might change and there will be more discrepancies but at least there is the option to adjust as needed case by case.

I will work on the inline comments and report back probably during the weekend.

[dvd101x]

Hi Jos, it's ready for review again. I included some inline comments with a few questions.

[dvd101x]

Hi Jos, the two left comments were addressed.

Please review.

[josdejong]

Thanks David, I'm really happy with how this PR worked out, it's much cleaner than I had expected.

Merging this PR now in the v13 branch. I expect to publish v13 somewhere the coming weeks with the breaking changes implemented so far.

[dvd101x]

Thanks Jos!

Looking forward to v13!

[josdejong]

Published now in v13.0.0 🎉