datadog-static-analyzer icon indicating copy to clipboard operation
datadog-static-analyzer copied to clipboard

[STAL-1960] Add tree-sitter query wrapper

Open jasonforal opened this issue 1 year ago • 0 comments

What problem are you trying to solve?

As we refactor the JS runtime, we want to be able to efficiently send data between v8 and Rust.

What is your solution?

This PR is part of a larger chain of PRs. However:

  • This PR is mainly about ergonomics, writing code now to make future code less verbose, and thus easier to review. We do this by wrapping tree_sitter::Query to provide a better API for our use case:
    • We want to be able to tag a tree_sitter::Node with a u32 id. Because this all starts with a tree-sitter query, which "captures" nodes, we implement a generic struct that will allow us to cleanly implement this (e.g. TSCaptureContent<tree_sitter::Node> will be transformed into TSCaptureContent<NodeId>)
  • However, there are also significant memory efficiency improvements here, here, and here.
  • A single commit converts our existing use to this new wrapper.

Alternatives considered

What the reviewer should know

  • Don't pay too much attention to https://github.com/DataDog/datadog-static-analyzer/commit/2bd26d50c44f8d71b431b097fd0952e4e820ca2c, other than verifying correctness. Basically all of the existing implementation of tree-sitter query captures will need to be deleted/refactored, so this commit is disposable.
  • The memory efficiency improvements will enable a future PR which will reuse a single tree_sitter::QueryCursor for every query, instead of allocating it and dropping it for every file/rule combo.

jasonforal avatar May 16 '24 14:05 jasonforal