microsoft
Make gmm lower triangle params row-major
The comment at https://github.com/awf/ADBench/blob/37f7f15137e5c8759785840917e4d773819bfddf/usr/awf/Julia/gmm3.jl#L14 says that the elements in the lower triangle are inserted row-major.
This is the sensible order for most of the languages we support, but it appears that Manual inserts them column-major, and when we check for consistency, this will bite.
I believe it's worthwhile to make the (nontrivial) effort to make Manual row-major.
I just checked the document, https://github.com/awf/ADBench/blob/0a4f34e371deec4c45d3654bca4260f996730622/Documents/ms.tex#L201-L204
Which is also column-major, so this is a significant change.
To check my understanding, we have
- Julia: row-major
- Manual: column-major
- Document: column-major
Which one(s) are we going to change?
My proposal is to change all to row-major. But that actually disfavours Julia, which is col-major by default... More thought needed.
Yep, it just bit me. Apparently, the reason julia/zygote code was producing incorrect results was just that - it treated packed lower triangle as row-major. And for d = 2 it didn't matter, hence the correct results for that case. I do propose, however, to make everything column-major (already fixed julia/zygote pack/unpack code to do just that in my branch), because everywhere else (c+, python, doc) it is column-major and changing that would be a pain. On the other hand, if some AD tool could benefit from it being row-major, our architecture allows it to do the rearranging during non-time-measured prepare and output stages.
