zap-extensions icon indicating copy to clipboard operation
zap-extensions copied to clipboard
verified publisher icon zaproxy

[WIP] WebSocket active scan infrastructure

Open eakirtas opened this issue 7 years ago • 7 comments

This pull request its part of WebSocket Active Scan project which described in issue #4276.

[WS-Active-Scan] WebSocket active scan infrastructure

(on early stage yet)

This commit introduces a new active scanner feature on WebSocket add-on. The WebSocket scanner infrastructure based on WebSocket Treemap and Structure analyzer which are introduced in zaproxy/zap-extensions #1742.

Short Intro

WebSocket Protocol comes with many differences in reference to Http protocol. The most common one is the format of the messages/payloads. In contrast with HTTP, WebSocket protocol does not define the specific format of the messages will be transmitted over the TCP connection. That means at the same connection may be transferred messages with a different format. In addition, WebSocket protocol keeps the TCP connection opened in order to offer us a full duplex client/server communication in contrast with the classic request/response technique. Those are the most challenging parts which are tried to overcome by this PR. In any other case, this implementation attempts to emulate the existing HTTP active scanner.

The concept

There as some utility classes which are managed the whole scanning process.

  • WebSocketActiveScanManager: The class which manage the scans and links the active scanner with the rest functionality of WebSocket Add-on. This class can be used in order to add a new WebSocketActivePlugin (Explained above) and start a new Active Scan. Note: The scan will not run if there is another scan was already running.

  • WebSocketPluginFactory: Used for storing and loading plugins.

  • WebSocketTarget: Stores the basic information about the active scan targets.

  • WebSocketScanner: Implements the runnable class. This class is responsible to separate the targets onto different host process (if there are more than two). That could be happened when the user selects to scan all nodes under the root. In addition, if there are available threads WebSocketScanner can create different tasks for those different hosts in order to run scans in parallel. Finally, If the user selects a leaf node (ex. MessageNode) the WebSocketScanner will create a new HostProcess in order to handle the communication with the server.

  • WebSocketHostProcess: The host process represents a scan on a specific host. It has to distribute the targets over the appropriate plugins. In addition, WebSocketHostProcess offers to the plugins some basic utilities. Those utilities help plugins to establish a new WebSocket Connection, send messages, receive the response messages from the appropriate channel etc. I should note that every plugin has to open a new channel, so every time a new message received the WebSocketHostProcess be able to inform the appropriate plugin. The WebSocketScanner has many host process (which could be running on a different thread).

All those classes aim to support the scanning plugins. Every plugin have to test the application (either on server site or client site) in order to find a set of vulnerabilities. To simplify the whole process I create some abstracts class (as the HTTP active scanner does). In order to write a new plugin, we should extend the following classes. An important note: The plugins are categorized in such a way in order to test the different "parts" of the WebSocket Connection. This means that a plugin must scan some specific node's type. That depends on the class you extend. Finally, the target is defined by "user" (either from UI or API). Target could be either recursive or not. The target (which reached to the plugin) could be any available (except folder) under the HostFolder. The WebSocketActivePlugin is the class which is responsible to iterate over the nodes.

  • WebSocketActivePlugin: This is the superclass of all plugin. You typically will not extend this class directly but provides key methods that you will need to use and abstract methods that will need to be implemented. (exactly as AbstractPlugin). WebSocketActivePlugin iterates over the targets, "ask" the scanner if it is interested to scan this target (by applyScan method) and then gives them the target to scan it. Implement boolean applyScan(StructuralWebSocketNode structuralWebSocketNode) in order to choose which nodes you to apply scan and void canNode(StructuralWebSocketNode structuralWebSocketNode) to scan it. It is able to apply scans on nodes: {HANDSHAKE, MESSAGE_OBJECT, MESSAGE_TEXT}

  • WebSocketActiveHandshakeScan: Extend this class in order to perform some tests on the handshake process. Implement boolean applyScanToHandshake(HttpMessage handshake) in order to choose which handshakes you want to scan. And of course, implement scanHandshake(HttpMessage handshake) in order to scan the handshake. WebSocketActivePlugin is able to apply scans on nodes with type: {HANDSHAKE}

  • WebSocketMessageNodeScan : Extend this class in order to perform scans on nodes. WebSocketMessageNodes stores the WebSocketMessages, PayloadAnalyzers, PayloadStructure etc. Implement boolean applyScan(WebSocketMessageNode messageNode) in order to choose in which nodes you want to apply your scan and void scanMessageNode(WebSocketMessageNode messageNode) to scan it. It is able to apply scans on nodes with type: {MESSAGE_OBJECT, MESSAGE_TEXT}

  • WebSocketMessageDTOScan: Extend this class in order to perform a scan on Messages. Implement boolean applyScan(WebSocketMessageDTO message) in order to choose in which Messages you want to apply your scan and scanMessage(WebSocketMessageDTO message) to scan it. It is able to apply scans on nodes with type: {MESSAGE_OBJECT, MESSAGE_TEXT}

  • WebSocketPayloadScan: Extend this class in order to perform scans on Payloads. Implement boolean applyScan(Object message) in order to choose in which and scanPayload(Object message) to scan it. It is able to apply scans on nodes: {MESSAGE_OBJECT, MESSAGE_TEXT}

  • WebSocketStructureScan: Extend this class in order to perform scans on Paylaod Structure. This scan based on Payload Structure which introduced on zaproxy/ zap-extensions #1742. Payload structure exported from PayloadAnalyzers. Those interfaces aim to analyze the WebSocket message in a structural format (expressed as a data structure). You can implement those in order to adjust it in your test case. The basic idea behind that is to create a formal structure in order to iterate over the different parameter (keys and values). If your messages can be mapped in a PayloadStructure you can use it to scan different parameters and create your own attacks vectors with analyzers. (In an abstract way PayloadAnalyzers has the role to deserialize/serialize the payloads). Implement boolean applyScan(PayloadAnalyzer analyzer, PayloadStructure structure) in order to choose in which and scanMessageStructure(PayloadAnalyzer analyzer, PayloadStructure structure) to scan it. PayloadAnalyzer and PayloadStructure should not be null. It is able to apply scans on nodes with type: {MESSAGE_OBJECT, MESSAGE_TEXT}

  • WebSocketMessageParamScan: Extend this class in order to perform scans on parameters. This plugin scanning a target parameter by parameter. PayloadAnalyzer and PayloadStructure should not be null. It is able to apply scans on nodes with type: {MESSAGE_OBJECT, MESSAGE_TEXT}.

All Plugins have to implement and the following methods:

  • void messageReceived(WebSocketMessageDTO message) : Notify the plugin that a new message received over the channel
  • void connectionStateChanged(WebSocketProxy.State state, WebSocketProxy proxy):State of connection changed
  • String getName() Return the name of the plugin
  • int getCode() returns the unique ID of the plugin

Plugins

  • ScriptWebSocketMessageActivePlugin: This class extends WebSocketMessageNodeScan and uses the ExtensionScript. Calls the script is written by the user. Implement in your script the WebSocket ActiveMessageScript interface in order to create an active scan.

  • WebSocketActiveMessageScript: Should be implemented by active scan scripts

    • [Optional] boolean applyScan(WebSocketMessageNode webSocketMessageNode) return true by default
    • scan(WebSocketMessageNode webSocketMessageNode, ScriptWebSocketMessageActivePlugin parent);
    • void messageReceived(WebSocketMessageDTO message)
    • [Optional] void connectionStateChanged(WebSocketProxy.State state)

  • WebsocketActiveScan Template.js is a template script

User Interface

Gives the ability to user to select the starting node.

Issues

  • [ ] Address the concurrency issues
  • [ ] ....

TODO

  • [ ] Clean up the code
  • [ ] Add documentation
  • [ ] ...

(Not ready for code Review yet)

eakirtas avatar Aug 10 '18 19:08 eakirtas

This pull request introduces 9 alerts when merging 9da329a6057d94debfffee336f445daa7aabbfc3 into b33ac46d6bb7db10ed20d53a51ff48dd8d3010de - view on LGTM.com

new alerts:

  • 2 for Equals on incomparable types
  • 2 for Implicit conversion from array to string
  • 1 for Container contents are never accessed
  • 1 for Inconsistent equals and hashCode
  • 1 for Type mismatch on container modification
  • 1 for Reference equality test of boxed types
  • 1 for Dereferenced variable may be null

Comment posted by LGTM.com

psiinon avatar Aug 10 '18 19:08 psiinon

This pull request introduces 9 alerts when merging 9da329a6057d94debfffee336f445daa7aabbfc3 into de4537e305d018e8a185c0b7feb5f87576f021c9 - view on LGTM.com

new alerts:

  • 2 for Equals on incomparable types
  • 2 for Implicit conversion from array to string
  • 1 for Container contents are never accessed
  • 1 for Inconsistent equals and hashCode
  • 1 for Type mismatch on container modification
  • 1 for Reference equality test of boxed types
  • 1 for Dereferenced variable may be null

Comment posted by LGTM.com

psiinon avatar Aug 14 '18 13:08 psiinon

This pull request introduces 9 alerts when merging 4ec16be53b694cac55a7019c6b6ddf2be0eb74be into 0ceff13117188accffd473b5c36075b6d79ef55a - view on LGTM.com

new alerts:

  • 2 for Equals on incomparable types
  • 2 for Implicit conversion from array to string
  • 1 for Container contents are never accessed
  • 1 for Inconsistent equals and hashCode
  • 1 for Type mismatch on container modification
  • 1 for Reference equality test of boxed types
  • 1 for Dereferenced variable may be null

Comment posted by LGTM.com

psiinon avatar Mar 27 '19 09:03 psiinon

Conflicts Resolved, and branch rebased on latest change of #1742

eakirtas avatar Mar 27 '19 11:03 eakirtas

This pull request introduces 4 alerts when merging 99a97ca3d1d1e68e0d517dda9e99a906ba1c2042 into 0ceff13117188accffd473b5c36075b6d79ef55a - view on LGTM.com

new alerts:

  • 1 for Container contents are never accessed
  • 1 for Equals on incomparable types
  • 1 for Type mismatch on container modification
  • 1 for Expression always evaluates to the same value

Comment posted by LGTM.com

psiinon avatar Mar 27 '19 12:03 psiinon

The pull request is now updated (after build changes, zaproxy/zaproxy#5302).

thc202 avatar Apr 15 '19 23:04 thc202

This pull request introduces 3 alerts when merging 7e43047bb703cfc57bbb6cf1e6d5a96d08591aa5 into a2b44c5ce80cfd0e8b2031606dead1ba836c943e - view on LGTM.com

new alerts:

  • 1 for Equals on incomparable types
  • 1 for Type mismatch on container modification
  • 1 for Expression always evaluates to the same value

Comment posted by LGTM.com

lgtm-com[bot] avatar May 03 '19 14:05 lgtm-com[bot]