Protonu comments

Results 3 comments of


                                            Protonu

Cannot correctly compute cross entropy loss (backward) when the reduction used is "sum" and torch executor is used.

A similar issue that shows up is this code: ``` from collections.abc import Sequence import itertools from functools import partial from typing import Any # NOTE: Dependency on fdm and...

Cannot correctly compute cross entropy loss (backward) when the reduction used is "sum" and torch executor is used.

The interesting thing is it for the nvfuser backend it fails, but if I turn on the custom decomposition for cross entropy loss (for nvFuser) it works correctly.

Add custom decompositions for cross entropy loss for the nvfuser executor

>just wondering if you have some numbers of cross entropy loss to compare this with the existing ones e.g. https://github.com/Lightning-AI/lightning-thunder/blob/c6928015914fdbdd708fd8e87fbd9d9c1b4a40ef/thunder/executors/triton_crossentropy.py? @crcrpar I did compare performance against torchcompile (which uses Triton...