EFForg
Possible extension refresh?
Type: other There's still a bunch of open and old issues which is not still resolved/finished.
The release process: #2697 Improving UI/UX: #1291, #17868 Performance: #12232: Extension excessive memory usage (applies to Chrome too) #16551: Use more performant IndexedDB instead of serialized WebStorage
Solutions:
- Refactor everything in a new branch and migrate rulesets into other repo.
- Rewrite code and make modules for most browsers and environments (see #17268).
~~Sorry for the long response, my laptop charger just died.~~
In details:
- #981: There was a discussion about improving the release process, does it was resolved in release #14730?
- #2697: Why there's still no separate repository for only the rulesets? As I see, the migration process will not be so smooth for the ruleset maintainers.
- #1291, #1292: @J0WI has already mentioned that some features are missing from Chrome/WebExtension version.
- #17868: I don't see any progress in the discussion of UI/UX feedback.
- #12232: While it says that this extension using 40-80 MB of memory, mine is using over 120 MB.
I suppose the issues above can be resolved by making "beta_2.0" (or something like that) branch and apply backward-incompatible changes there.
Also, maybe it's better to make some things from scratch than rewriting that.
Generally, I like this idea. I personally feel like the extension (not rulesets) development is slowing down to crawl. We should discuss it with actual EFF employees though. I can try developing (locally, I don't want to develop my own "competitor" to HTTPS Everywhere) my own attempt on how the extension could work if it were possible to rebuild it from scratch using today's best practices. I also want to take advantage of the fact all currently targeted browsers (including Tor Browser) support ECMAScript modules natively now.
#12232: While it says that this extension using 40-80 MB of memory, mine is using over 120 MB.
I might have missed something but I don't see any claim on the memory usage in #12232 matched your description. I am guessing that you are referring to #16092. This measurement calculate the theoretical memory consumption by the rulesets, not the extension itself.
I am open to the idea of refreshing the extension since the current code have some assumption which limit the performance. I believe this will also imply a breaking change to the ruleset format. If https://github.com/EFForg/https-everywhere/issues/17143#issuecomment-447156379 can be resolved, we should rewrite the whole extension in WASM, such that the improvement can be very obvious. However, I am concerned that #17268 will limit our effort to fully rewrite the extension given the uncertainty on the Google's side.
@cschanaj I think we should use #17268 as an opportunity to grow. Brave devs have confirmed they will not introduce anti-adblock code and will neutralize such code if it will be added by upstream. Currently there is no decision for Ungoogled Chromium.
I've closed https://github.com/EFForg/https-everywhere/issues/981 since the release process has improved significantly since this issue was opened.
I will leave #17868 until @zoracon is back in a few months.
https://github.com/EFForg/https-everywhere/issues/12232 will be greatly improved by the addition of the WASM module: https://github.com/EFForg/https-everywhere/pull/18093
https://github.com/EFForg/https-everywhere/issues/1292 is not currently prioritized.
@Saiv46 Not much, sadly. We have a WebAssembly-based rewriter engine now, so the extension should be a little bit faster and consume less memory, but the extension was not significantly rewritten yet.
@Saiv46 So an extension refresh was done on the UI side without touching functionality under the hood in 2018.
After the initial UI overhaul, feedback was taken for when we have the bandwidth to do the next round of UI updates.
Last year was a big step towards memory optimization with WASM by @Hainish.
Once I am finally done with some backlog concerning rulesets clean up. I want to refocus on optimizing the code.
I just ask for patience since capacity is limited usually to focus on all aspects of the project at once. A complete refresh of UI, memory, and ruleset optimization at the same time is not a possibility at the moment, especially with maintenance. But we do our best to tackle it in rounds throughout the year.
@Saiv46 @zoracon What do you think about splitting the extension into two branches, a "greenfield" branch which re-development work happens on, and a "brownfield" branch which will have the current extension code and which will still get bugfixes, but work on new features will happen in the "greenfield" branch?
This seems to be much more reasonable than trying to pile more and more features and changes on a code base which already has too much accumulated cruft left from the past.
The releases will be made from the "brownfield" branch until the "greenfield" branch will become stable and have feature parity with the "brownfield" branch.
@pipboy96 This is a decent solution if we had an official "dev" and "staging" environment for the extension. Without that in place we won't get the full benefits of such a set up. I will think on this on this, and try to come up with a reasonable way of tackling this.
@zoracon Can we begin developing a "greenfield" version this fall, so that we may have a minimum viable product finished by the end of this year? If people outside EFF start working on a rewrite, can they transfer ownership of it to EFF once MVP is finished?
@pipboy96 I can create that working branch for fall, for now I would like to work on the backend since there is no planned major UX work until next year. If we could have a working branch that prioritizes javascript refactoring, legacy code, and possibly python3 update (instead of pinning to python3.6 only) I think that would be a good start for this branch.
Edit:
To add to this, MV3 (https://github.com/EFForg/https-everywhere/issues/17268) is not currently prioritized. The plan is to port to MV3 once they set an actual EoL on MV2. But in order to be prepared, we have to breakdown what we currently have into more modular and compartmentalized code.