crewAI icon indicating copy to clipboard operation
crewAI copied to clipboard

feat: Add network activity monitoring to tool execution verifier

Open qizwiz opened this issue 4 months ago • 1 comments

This PR adds network activity monitoring to the tool execution verification system to detect when tools fabricate results without making actual network calls.

Summary

  • Add NetworkEvent dataclass to capture network request evidence
  • Add NetworkMonitor class to hook HTTP libraries and capture activity
  • Enhance ToolExecutionWrapper to include network monitoring during execution
  • Update ExecutionRecord to store network activity evidence
  • Enhance complete_execution to accept and store network events
  • Add AgentExecutionInterface and enhance_tool_for_verification utilities

Verification

This addresses the need to verify that tools making network requests actually execute those requests rather than fabricating responses. The system can now distinguish between:

  • LIKELY_REAL: Network activity detected during tool execution
  • LIKELY_FAKE: No network activity detected when network calls were expected

Backward Compatibility

All existing functionality is preserved and all original tests continue to pass.


[!NOTE] Add HTTP network activity monitoring to tool execution verification and store captured events with execution records, with docs, demos, and tests.

  • Verification utilities:
    • Add NetworkEvent and NetworkMonitor to capture HTTP activity by hooking requests/urllib.
    • Enhance ToolExecutionWrapper to monitor executions and persist events via ExecutionRegistry.complete_execution; extend ExecutionRecord with network_activity.
    • Introduce AgentExecutionInterface and helper wrappers in src/crewai/utilities/tool_execution_verifier.py.
  • Documentation:
    • New guides: NETWORK_MONITORING_FEATURE.md, src/crewai/utilities/NETWORK_MONITORING_README.md (design, usage, verification logic).
  • Examples & tests:
    • Add src/crewai/utilities/demo_network_monitoring_feature.py and src/crewai/utilities/test_network_monitoring_concept.py.
    • Adjust existing tests for clearer exception assertions and minor security annotations.

Written by Cursor Bugbot for commit 2c267e3b0780422fde741bb70160749aef27686f. This will update automatically on new commits. Configure here.

qizwiz avatar Sep 29 '25 22:09 qizwiz

This PR is stale because it has been open for 45 days with no activity.

github-actions[bot] avatar Nov 14 '25 12:11 github-actions[bot]