App adds an additional character at the end of the hash

[Calos]

Hi,

Thanks again for the app. Been using it for some time!

I had an issue crop up today that I haven't seen before. I just created an account on my PC, and went to log in on my phone, but couldn't because the passwords didn't match. I compared the output, and the app is giving me a hash 1 character longer than the browser extension I use and the pwdhash website. The app added a trailing character (not NULL or space), the hash is otherwise identical.

Unfortunately, some attempts are re-creating the issue have failed. I probed similar length URL + user password combos, and similar patters of characters/special characters, but the only case the issue shows up is with the URL and password I need to use.

Obviously I want to avoid giving up the URL and my user password, and I'm also pretty sure that this is not enough information for you to do any serious troubleshooting. En lieu of other ideas, as a start I was hoping that you could show me how you resolved the previous issue I submitted, regarding trailing NULL characters being appended. Somehow you were able to determine that the "official" pwdhash also had the trailing NULL characters, though it isn't evident from the browser. Just hoping that seeing what the raw output of the official algorithm is may give some clue.

Thanks again!

[phw]

Thanks for reporting this. As you said, this might be hard to track down. It would probably help to know what character get attached.

The way I tracked down the NULL byte issue was by entering a affected URL / password combination on https://www.pwdhash.com and looking at the resulting value with JavaScript (using Firebug or a similar browser extension):

document.getElementsByName("hashedPassword")[0].value

To get the ASCII code of the last character you can use:

document.getElementsByName("hashedPassword")[0].value.substr(-1).charCodeAt(0)

An example URL / password combination is at https://github.com/phw/Android-PwdHash/blob/2f74d68bbc5b25b32e382a487fd10ffe9b9e8185/tests/src/com/uploadedlobster/PwdHash/test/HashedPasswordTest.java#L52 . I am not sure if this issue was visible in all browsers, but at least in Firefox it is.

In the case of the NULL bytes I solved it by simply stripping them from the result in the App, as this is what the browsers do when copying the value.

Maybe if you find out what character gets appended the algorithm will reveal the issue.

[Calos]

Thanks for the feedback.

Unfortunately I have no real Javascript experience, especially in the browser, so I wasn't sure how to do what you suggested. Instead I parsed through the code quick and Firebug let me put a break where the result is completed, and lets me inspect it.

The result of the pwdhash.js Generate() function contains 3 more elements than the apparent final hash. The extra elements appear in Firebug as this symbol; one of them is the last element: �

Hopefully that is helpful. Not sure that that symbol is meaningful or just Firebugs filler for something it doesn't understand. I'll need some guidance to get the ASCII code, as you mentioned.

Thanks again!

[phw]

Sorry for the really long delay. That � is really just a filler, but if you open firebug there is a tab labeled "Console" where you can enter and execute arbitrary JavaScript. If you copy and paste the above code it will output the ASCII code of the last character in the hashed password input field. Instead of the -1 in the substr call you can pass -2 and -3 to get the codes for the second last or third last character.

I am lacking time right now, but I will definitely run some own tests.

[Calos]

No problem, thanks for the follow-up.

charCodeAt returns 0 for each instance of that symbol... so apparently strings are treated like arrays in JavaScript, and 0, being an invalid char code, indicates null?

At any rate, just a guess, but if it helps... I was looking through your source, and found in applyConstraints where you added a line to fix the previous issue:

String result = hash.substring(0, startingSize);
<snip>
result = result.replace("\0", "");
return rotate(result, nextExtra());

So to me it appears result is a string, and you're doing a standard string replacement. From some quick Googling, "\0" is how you match null elements - great. However, from some reading, it sounds like the standard replace function only replaces the first instance of the match it finds. So I'm guessing that because it shows up three times in the string (twice before the end), it isn't removing the last one, and may essentially be the same issue as before. Modifying that replacement to the following seems to work from Firebug's console:

document.getElementsByName("hashedPassword")[0].value.replace(/\0/g,"")

As for why null values are showing up in a new place - I have no idea :-)

[phw]

That is quite confusing. A charcode of 0 is indeed the ASCII code for a null byte. But that's exactly the case that is supposed to be fixed. And String.replace is supposed to replace all occurrences of the search string, not only the first one. There is even a test for it in https://github.com/phw/Android-PwdHash/blob/master/tests/src/com/uploadedlobster/PwdHash/test/HashedPasswordTest.java#L5.

Or maybe the issue is that the app removes the null bytes? Maybe it's some kind of incompatibility between different pwdhash implementations. A few questsions:

1. Do you use a pwdhash browser extension? If yes, which one?
2. Login with the browser extension for the problematic password works, right?
3. Does the login also work if you generate the password on pwdhash.com and copy and paste it?
4. Is 3 true for different browsers?

In addition I will make you a build of my app which does not replace the null bytes, I'll let you know.

[Calos]

Unfortunately I think you are correct about incompatibility between implementations, which could make this rather ugly. What's unsettling is that I get different results from the webpage on different browsers. I have the following two browser extensions:

Chrome:
https://chrome.google.com/webstore/detail/my-hash-password-lite/golphaljalbhbjgaommnjeekjagdihni
Firefox (v1.7)
http://crypto.stanford.edu/PwdHash/

A. The browser extension for Chrome produces a hash consistent with the webpage on Firefox (though the webpage on Firefox contains the null bytes, which get removed with copy/paste) (I can log in with both). These match what your app produces. B. The webpage in Chrome produces a different result, which is what I submitted this about to begin with - it is identical to (A.) except that it is missing the final character. It seems Chrome's built-in dev tools also include a JavaScript console, running the below command shows no null bytes and and also does not include the last character.

document.getElementsByName("hashedPassword")[0].value

C. The browser extension for Firefox does not log in successfully, if I try to count characters in the password field, I suspect that there are 3 additional characters corresponding to the three null bytes; also worth noting is that, ignoring the extra three characters, the length matches case (A.) and not case (B.) D. I just tested rekonq on the webpage as well since it is available on this system, result is the same as Chrome, probably not surprising as I think they both use Chromium as a base

My impression has been that the original algorithm has a maximum length for the hash - I think 22 characters? This is just based off of playing with the webpage in the past. Interestingly, case (A.) above produces a 23 character hash. Playing with it again, for long passwords of repeating 1234567890 on both Chrome and Firefox, it looks like eventually it starts making a 22 character hash with "AAAA" appended (26 total characters).

So, it seems like this is not a bug of your app, it actually matches the majority of my test cases. I suppose I should try to contact the creators of the algorithm to define what the expected output should be and implement it better to get consistent results. Thanks again for all of the help. Sorry, I should have tested across multiple browsers/browser extensions first, but I never would have dreamed that the webpage would give different results in different browsers.

A couple asides (trying not to deviate too far from the subject at hand): -- string.replace - does this have different behavior in the context of Android? If I Google it (just in the context of Javascript), all of the feedback I get is that by default it only replaces the first match, and that you need to set the /g switch to match globally. If I run it in Firebug it does only replace the first null byte, whereas /\0/g replaces them all. -- What do you use to build Android apps? Curious because it's something I'm interested in getting into; installed the ADT-SDK today and was considering trying to build this app as a start.