chainrules for chebpoly constructors

[stevengj]

It would be nice, in principle, to define chainrules for the chebpoly constructors, so that we support computation of sensitivities of e.g. polynomial evaluations to the data used to construct the polynomial.

In principle Zygote can do this for us, but it seems to be incredibly slow. (Before we implemented the rrule in #9, it was taking over an hour to backpropagate through a ChebPoly evaluation.)

There are a couple of pieces to implement here. First, chain rules for chebinterp and chebregression, which should be very straightforward since they are linear operations. Second, we need to fix the @notimplemented in the rrule for ChebPoly evaluation (#9) to optionally compute the tangent for sensitivity to polynomial coefficients and bounds — this will need a "thunk" tangent of some sort in order to defer the computation until/unless it is actually needed, because the coefficient tangent is both large and expensive to compute.

[isentropic]

@stevengj is this issue similar to what I asked here https://discourse.julialang.org/t/differentiating-through-chebyshev-coeff-basis/110234 So far I tried ClassicalOrthogonalPolynomials.jl, ApproxFun.jl, FastChebInterp.jl and Polynomials.jl none of which let me do this unfortunately

[stevengj]

I don't know what this sentence means:

The problem is that I need the derivative in terms of the coefficients of the approximation

You want the derivative of what with respect to what?

(Or you want something completely different, like the Chebyshev coefficients of the function derivative?)