nadavrot
Any plan for trigonometric functions?
Hey @fxpineau. Yes, I think that I'll work on implementing them. I already implemented e and pi, and started reading on log and exp. The paper that you attached looks excellent. Btw, if you are interested in implementing something then that would be great. You can start with something small. I would love to review the pull request.
I landed a quick implementation of sine(). It's incomplete, and we are still missing support for Inf,NaN, etc. And the Taylor series is unbounded, but the implementation can already compute a bunch of normal values in double precision and get bit accurate results.
For a lib I am working on, I am computing rotation matrices obtained from Euler angles and compare them to (high precision) matrices provided by the literature.
I am doing one shot computations (hard coding the result in the lib) and I finally chose to write bc scripts.
If cosine() is also implemented, I will compare bc and arpfloat results.
I may also extend the lib with high precision computations and arpfloat seems a very good candidate to me.
I implemented Sin, Cos, Tan, Exp, Log, Pow, and a few constants (pi, e, ln2). Can you think of other functions or constants?
Doing a quick cprofile of nanoGPT model training on M1 CPU for the perf7 proposal:
| ncalls | tottime | percall | cumtime | percall | filename:lineno(function) |
|---|---|---|---|---|---|
| 440 | 6.682 | 0.015 | 6.682 | 0.015 | {method 'run_backward' of 'torch._C._EngineBase' objects} |
| 1920 | 1.183 | 0.001 | 1.183 | 0.001 | {built-in method torch.tanh} |
It looks like torch.tanh() is really expensive and could use some love.
BSD tanh() seems really branchy. GNU tahh()
