fixes whitelist issue 46

[codetothepoint]

Fixes the logic to correctly honor whitelists, and updates respective test coverage.

see also: https://github.com/greengerong/prerender-java/issues/46

[codetothepoint]

Would like to request a review from @greengerong.

[greengerong]

Thanks. For normal case, even though the url is in white list, but we still need to check user agent. For your case is a special case, I think you can give a switch to enable check user-agent or not.

Do you think is it ok?

[codetothepoint]

That makes perfect sense. I like the idea of adding a switch to check User-Agent. I will update my pull request. Thank you!

• Scott

On Wed, May 9, 2018, 7:52 AM green [email protected] wrote:

Thanks. For normal case, even though the url is in white list, but we still need to check user agent. For your case is a special case, I think you can give a switch to enable check user-agent or not.

Do you think is it ok?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/greengerong/prerender-java/pull/47#issuecomment-387765862, or mute the thread https://github.com/notifications/unsubscribe-auth/ADD7SMsuZMmi3r4yc4saBWZSdg1OTmQDks5twwKSgaJpZM4T3cW7 .

[codetothepoint]

@greengerong: I appreciate your feedback, and agree with your suggestion. I have updated the pull request to support a new config parameter called ignoreUserAgent. Let me know what you think! Thank you.

• Scott

[thoop]

I just commented on your #46 issue too, but wanted to add my same point here for visibility:

We don't recommend serving Prerendered pages to users when using our prerendering service since we remove javascript tags to prevent crawlers from trying to execute the javascript and clearing out the static html that has been prerendered.

So if that's the reason for ignoring the user agent check, there might be issues that prevent this from working as you expect for your users.

[greengerong]

@codetothepoint The code have conflicting.

[thoop]

Does this code fix any whitelist issue anymore? Looking into the changes, it just adds an option to ignore the user agent check, which would cause prerendered pages to be returned to all users. We don't recommend serving Prerender.io pages to users since the <script> tags are removed from all prerendered pages and your javascript would no longer work for users.