nsiccha
nsiccha
Yeah, after having a really short look, I also arrived at those two files. Haven't had the opportunity to port i anything yet though.
Also, any benchmark should probably include multiple chains run in parallel and combined to estimate mixing/ESS.
You are of course free to do whatever you want, but I don't think it's reasonable to publish/keep online, link to and quote benchmarks which are known to have major...
> I think the fairest comparisons would be achieved by using exactly the same algorithm for all sampling algorithms. Yeah, that's what should be done. > What we need more...
> IMO that would not make the benchmark fairer. I disagree. In my experience, people fitting ODE models generally care about performance, and this includes changing some easily accessible compiler...
> So it would be good for example if we could have a version with special compiler flags and the default Yeah, I think it would also be a good...
I'm guessing it can also happen that some "optimizations" can even have an adverse effect in some cases, which would also be interesting to see.
(Estimated) Effective sample size per second.
@bob-carpenter for the niche but important set of models with odes, the number of gradient evaluations is (potentially) much less informative than the runtime.
I'd find it great if there were some way to get more metrics about (e.g.) the ODE solution process. But, I think for now just closing the gap between reported...