David González Martínez

Add a ParseDatePipe to the common library

19

### Is there an existing issue that is already proposing this? - [X] I have searched the existing issues ### Is your feature request related to a problem? Please describe...

add support for retain_graph in backward

2

Prerequisite for https://github.com/tinygrad/tinygrad/issues/5858. Simply allows not deleting the graph information when running backward (defaults to False). Most common usage is gradient accumulation or second order derivatives. Defaults to False as...

nvidia fp8 support

10

Supports fp8 arithmetic and m16n8k32 tensor cores, with both e4m3 and e5m2 variants. Arithmetic is supported as a graph rewrite rule that casts fp8 arithmetic to float, and stores the...

Proposal: supporting second derivatives

4

Second derivatives are useful on some cases (see https://github.com/tinygrad/tinygrad/pull/5701), but not possible by the way autodiff works in Tinygrad. Here is my take on how they could be supported with...

```python import torch x = torch.tensor(2.0, requires_grad=True) y = torch.tensor(3.0) intermediate = x + y res = 2 + intermediate res.backward() print(f"Gradient of loss with respect to x: {x.grad}") print(f"Gradient...

cuda support for fp8 arithmetic

2

Adds support for CUDA fp8 arithmetic for e4m3 and e5m2. This is done with a simple pattern matcher that casts to float and then back to the corresponding dtype. Breakdown...

fix fine_tuning_tutorial.ipynb

1

Fixes trying to access a variable that was not defined (incorrect casing).

 In Lab 02, the `grad_norm_bound` value in the last section is computed incorrectly. It should be calculated as `(25 * np.linalg.norm(np.dot(A.T, A), 2) + np.linalg.norm(np.dot(A.T, b))) / A.shape[0]` in...