jetpack icon indicating copy to clipboard operation
jetpack copied to clipboard
verified publisher icon Automattic

Improve WAF logging

Open miguelxpn opened this issue 2 years ago • 4 comments

Proposed changes:

  • Adds more information to the WAF logs
  • Adds a new toggle that allow users to opt-in to share more detailed debug data such as POST params and request headers for blocked requests.

Other information:

  • [ ] Have you written new tests for your changes, if applicable?
  • [ ] Have you checked the E2E test CI results, and verified that your changes do not break them?
  • [ ] Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Yes. This PR increases the data that we collect for the WAF logs for blocked requests (GET params, user-agent, referer, content-type, request uri) and adds a opt-in toggle to share with us some more additional data (post params and request headers).

Testing instructions:

  • Create a test site with a Scan subscription.
  • Install Protect and toggle the share data and share debug data on.
  • Make a few requests that are to be blocked.
  • Verify that the information was properly logged in the log file.
  • Turn off the debug data and verify that the post params and headers aren't being logged.

miguelxpn avatar Feb 01 '24 03:02 miguelxpn

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

  • To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the improve-waf-logs branch.

  • To test on Simple, run the following command on your sandbox:

    bin/jetpack-downloader test jetpack improve-waf-logs

Interested in more tips and information?

  • In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
  • Read more about our development workflow here: PCYsg-eg0-p2
  • Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

github-actions[bot] avatar Feb 01 '24 03:02 github-actions[bot]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

  • :white_check_mark: Include a description of your PR changes.
  • :white_check_mark: Add a "[Status]" label (In Progress, Needs Team Review, ...).
  • :white_check_mark: Add testing instructions.
  • :white_check_mark: Specify whether this PR includes any changes to data or privacy.
  • :white_check_mark: Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation :robot:

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

Once your PR is ready for review, check one last time that all required checks appearing at the bottom of this PR are passing or skipped. Then, add the "[Status] Needs Team Review" label and ask someone from your team review the code. Once reviewed, it can then be merged. If you need an extra review from someone familiar with the codebase, you can update the labels from "[Status] Needs Team Review" to "[Status] Needs Review", and in that case Jetpack Approvers will do a final review of your PR.

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

  • WordPress.com Simple releases happen daily.
  • WoA releases happen weekly.
  • Releases to self-hosted sites happen monthly. The next release is scheduled for April 2, 2024 (scheduled code freeze on April 1, 2024).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

Protect plugin:

  • Next scheduled release: April 2, 2024.
  • Scheduled code freeze: March 25, 2024.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

github-actions[bot] avatar Feb 01 '24 03:02 github-actions[bot]

I can see that when the share debug toggle is enabled the additional data is added to the jetpack-waf/waf-blocklog which is then eventually consumed by logstash. However, the entry in the wp_jetpack_waf_blocklog table remains identical to an entry with only share data enabled. Is that intentional or are we wanting more information there also?

Since we're not really using the table for anything at the moment I think it's fine to leave it as is.

miguelxpn avatar Feb 09 '24 04:02 miguelxpn

The new data WAF will collect is enough for starting a better threat intel on the data. The only extra data I'd add would be the source IP address, just to correlate attacks.

I think the IP would be considered PII, so we can only collect it if the user opts in.

miguelxpn avatar Mar 06 '24 03:03 miguelxpn