phusion
Proper SHA3 implementation?
Hey,
I'm trying to find a SHA3 library for Ruby, but it seems like every library (this one included) is, if I'm not mistaken, using an old version of Keccak for validation, not the actual standardized SHA3.
The most recent SHA3 test vectors say that the hash of the blank string should be:
A7FFC6F8BF1ED76651C14756A061D662F580FF4DE43B49FA82D80A4B80F8434A
however, in this library, it works out to:
C5D2460186F7233C927E7DB2DCC703C0E500B653CA82273B7BFAD8045D85A470
After some research, I found some pages that calculate both the original keccak and the sha3, and it confirms that the first output is correct. Here's one such page:
https://www.npmjs.com/package/js-sha3
And here's a blank test vector from the original docs:
http://csrc.nist.gov/groups/ST/toolkit/documents/Examples/SHA3-256_Msg0.pdf
Which agrees with everything else, and shows that the output is wrong.
Is there any chance of getting this fixed? Or of adding an option to use the new variation?
It appears to be a padding issue.. NIST changed the padding to be:
SHA3-256(M) = K ECCAK [512] (M || 01, 256);
But this implementation doesn't account for the 01 bits in the padding. After padding the empty string, the first byte should be '06', not '01', to account for the extra 01 at the start (6 = 00000110, reversing endian = 01100000, the current padding is 10000......, but the proper padding is that one, 01 then 10000.......
I am not maintaining this gem anymore. I welcome anybody who can take over.
I had also written a nodejs version of this library, and that library is now maintained by another version. I believe the kekkac-sha3 issue has been fixed there so maybe someone can port over the changes.
Sent from my Android phone.
It appears to be a padding issue.. NIST changed the padding to be:
SHA3-256(M) = K ECCAK 512;
But this implementation doesn't account for the 01 in the padding.
— Reply to this email directly or view it on GitHub https://github.com/phusion/digest-sha3-ruby/issues/5#issuecomment-149087749 .
Great stuff! :-)
On Sun, Mar 20, 2016 at 11:37 PM, Barry Allard [email protected] wrote:
@iagox86 https://github.com/iagox86 Fixed in PR #6 https://github.com/phusion/digest-sha3-ruby/pull/6
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/phusion/digest-sha3-ruby/issues/5#issuecomment-199146008
For posterity
This gem is no longer maintained, so anyone wanting an actual SHA3 hash should just:
gem install specific_install && gem specific_install https://github.com/steakknife/digest-sha3-ruby
Or
gem 'digest-sha3-ruby', github: 'steakknife/digest-sha3-ruby'
cc: @iagox86
A lot of people still installs this gem from rubygems.org, so I believe merging @steakknife 's PR would really be helpful and avoid problems to users.
Thanks! There's alternatives listed in the PR's readme. However, Phusion folks are probably busy doing client work and seem to abandon stuff like I do. Maybe Github needs a "community vote to approve merge" feature should owners go incommunicado? On Sat, Aug 12, 2017 at 4:20 AM konsolebox [email protected] wrote:
A lot of people still installs this gem from rubygems.org, so I believe merging @steakknife https://github.com/steakknife 's PR would really be helpful and avoid problems to users.
— You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/phusion/digest-sha3-ruby/issues/5#issuecomment-321975067, or mute the thread https://github.com/notifications/unsubscribe-auth/AANsMHXu2eRWvpjFSR83urrinMa5ZdTTks5sXYqPgaJpZM4GRDsm .
(I hope the Phusion team won't mind the shameless plug here)
For anyone interested in a FIPS202 (SHA3, not Keccak), my gem sha3 is actively maintained (although latest stable version is over 2-years old now -- not much has changed with implementation, and it fully passes vector tests).
Also worth mentioning that it will uses x86 optimized C implementation whenever possible, and is order of magnitude faster than reference code.
gem install sha3
or
gem 'sha3'
(https://github.com/johanns/sha3).
I think that's the one I ended up using :-)
On Dec 20, 2017 21:35, "Johanns Gregorian" [email protected] wrote:
(I hope the Phusion team won't mind the shameless plug here)
For anyone interested in a FIPS202 (SHA3, not Keccak), my gem sha3 is actively maintained (although latest stable version is over 2-years old now -- not much has changed with implementation, and all it fully passes vector tests).
Also worth mentioning that it will uses x86 optimized C implementation whenever possible, and is order of magnitude faster than reference code.
gem install sha3
or
gem 'sha3'
(https://github.com/johanns/sha3).
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/phusion/digest-sha3-ruby/issues/5#issuecomment-353262494, or mute the thread https://github.com/notifications/unsubscribe-auth/AAgITDAWqHOm3sDBEzFIDbWd0GKQ_qiAks5tCe4UgaJpZM4GRDsm .
For context, a lot of the reason that libraries commonly use the old version of Keccak is that Ethereum started using it before the SHA3 implementation was finalized. It stuck because there was no change in security level, and it would be a consensus breaking change. Now, Ethereum related libraries still use Keccak, but commonly mistake it for SHA3.
The most recent SHA3 test vectors say that the hash of the blank string should be:
A7FFC6F8BF1ED76651C14756A061D662F580FF4DE43B49FA82D80A4B80F8434A
however, in this library, it works out to:
C5D2460186F7233C927E7DB2DCC703C0E500B653CA82273B7BFAD8045D85A470
There is historic significance to this change. See also this thread: Ethereum: Difference between keccak256 and sha3
Note, the author on Stack Exchange posts the same hashes. This gem does not implement it incorrectly, it just does implement an earlier version of the standard, now used by Ethereum.
If you need a proper SHA3 implementation, you can use the sha3 gem: https://rubygems.org/gems/sha3
However, if you need the keccak used by Ethereum, I would recommend renaming this repository accordingly: https://rubygems.org/gems/keccak
