eyalleshem
eyalleshem
Hi , Currently the lib support different databases (postgresql-sql, mysql , etc .. ) dialect only in the tokenizer. My question is - if there was any discussions/plans about supporting...
This PR implements zero-copy tokenization by using borrowed strings (`&str`) instead of owned strings (`String`) for identifiers, string literals, and comments. This eliminates unnecessary string allocations during the tokenization process....
his PR adds a lifetime parameter to the Token enum to enable zero-copy tokenization in the future. Backward Compatibility: To maintain backward compatibility, we renamed Token to BorrowedToken. This allows...