Parallelization

[eapache]

Perhaps via https://github.com/grosser/parallel or some other gem.

Parsing files (in particular large Ruby files) is slow, so parallelizing it would be a big speed-up on multi-core systems.

[bry-guy]

I gave this a try over the past few days, utilizing Parallel.each for parsing the AST, such as:

    def add_files(files)
      Parallel.each do |file|
        @output.extra("Adding `#{file}`")
        parse_file(file)
      end
    end

Unfortunately, since parsing the AST depends upon db's @table and @meta instance variables, putting the parsing into threads via parallel meant that those instance variables were not shared across threading contexts. Unwinding this is probably possible, but wasn't dead-simple. I think using https://github.com/ruby-concurrency/concurrent-ruby may require less surgery within starscope.

[eapache]

Yeah, that's possible.

In practice I've found, for very large ruby projects, https://github.com/eapache/starscope/issues/95 and https://github.com/eapache/starscope/issues/154 dominate as performance issues anyway. If I had the time to do performance surgery, that's where I'd start.