Herbie Bradley

> @herbiebradley I see that your checklist is marked fully complete. Is this ready for review, or is more work needed? More work needed, will update tomorrow

Working on a basic library for Anthropic's evals myself, will hopefully put something out this week. lm-eval2 (the refactor of `lm-evaluation-harness`) will support OpenAI style model-graded evals too

I did some tests and it looks like numpy does not short-circuit:  This issue seems to confirm it: https://github.com/numpy/numpy/issues/3446 However, there may be performance improvements by switching to `np.where`...

Thank you for the feedback @CWen001 - we will add some of the indexes you mention to the list of metrics included in the package in the next release. In...

I think this compatibility issue has been solved simply by both involved packages upgrading significantly and improving their GDAL compatibility. Happy to unpin `rasterio` now, see https://github.com/ai4er-cdt/geograph/pull/81#issuecomment-1186013757

@reshinthadithyan you might want to add your scripts to this branch before we merge?

If you can get the commit message and hash then it should be simple to adapt the code at #31 to fit Bitbucket.

Yes, these seem like sensible criteria, I think that should be everything we need.

I meant the length of the combined data, but after checking with Louis we decided this doesn't need to be filtered because the constraint is too highly variable and model-dependent....

Updated to remove Python specific stuff, to allow for scraping all languages.