block Syntax improvement project

So in the past few months I had begun to notice that there were many issue reports on GitHub about substantial problems with list version syntaxes, so I decided to begin volunteering to try to fix it, which is currently going pretty poorly even after I got pull/merge privileges on GitHub. So as far as I am personally concerned, it's time to make a solid checklist so I won't have to check in on 20 different issue reports to remember what I wanted to fix, and to easier show what I need help from Ador and badmojr with.

Stage 1 (Most important)

[ ] Become able to see the [Travis / Azure Pipeline / whatever else] logs for the list generator script(s).
[ ] Make the list generator recognise changes made to build.sh and filter.sh scripts on GitHub.

Stage 2

[x] Remove entries with double-dots. (#547, #531)
[ ] Remove entries with backslashes in them. (#561, #531, #326, plausibly #219)
[ ] Remove entries that end with dots (#547, #531, #529)
[ ] Remove entries containing "<", ">", " ", and other HTML elements (#547, #531)
[ ] Add an [Adblock Plus n.n] header to the adblocker list versions. (#358, #172)
[ ] A way to edit the GitHub Readme without having it be reverted 6 hours later.

Stage 3

[ ] An easier and more direct way to submit whitelistings. (#537, #558, #533 #527, etc.)
[ ] Implement a max-length of 240~247 characters per entry for RPZ list versions. (#526)
[ ] Implement a max-length of 63 characters per entry for Unbound list versions. (#477)
[ ] Combine the IPv4 and IPv6 versions of dnsmasq into one version. (#246)

Website-related bonus stuff

[ ] Possibly create separate Telegram servers for issue reports and Rose-bot button clicks.
[ ] https://app.energized.pro/ is down. (#559)
[ ] All watchdog-(...).energized.pro image files are down, causing https://energized.pro/check/ to display false positives for all lists.

Sep 05 '20 01:09 DandelionSprout

Maybe adding to stage 2:

evaluate source reliability / quality , e.g. Airelle's as pointed out here https://github.com/EnergizedProtection/block/issues/552#issuecomment-686731548 | https://github.com/EnergizedProtection/block/issues/538#issuecomment-687201630 | https://github.com/EnergizedProtection/block/issues/460#issuecomment-687843050 | https://github.com/EnergizedProtection/block/issues/556#issuecomment-686843325 | https://github.com/EnergizedProtection/block/issues/352#issuecomment-687882999 | https://github.com/EnergizedProtection/block/issues/563#issuecomment-690943987

Sep 05 '20 06:09 ghost

Okay, so it turns out that the extent to which the GitHub files are ignored by Energized, is far bigger than I initially believed (My initial beliefs was that I had somehow broke something when writing #560).

In particular, #530 still hasn't gone live after more than 3 weeks, which not only means that Energized is still blocking Falun Gong domains due to the political beliefs of neoHosts Full; but also that the past 48 hours of my life that I spent on this repo was a complete waste.

Either Ador needs to set a lot of things straight here within the next few days and explain how (and/or make it easier) to contribute to Energized, or I'm leaving this project.

Sep 05 '20 08:09 DandelionSprout

I think this is probably the best place to mention this, but the adblock lists need to be simplified. For example, in the blu list, there is the rule ||pushwoosh.com^, yet there are 66 other entries in that list that all end with pushwoosh.com, even though the adblock syntax would block all of it due to the ||pushwoosh.com^ rule. An even worse example is ||siteintercept.qualtrics.com^, where subdomains of that site account for 771 rules, and that's not accounting for the 4327 hits for instances of qualtrics.com domains, which at that point you might as well just add a rule for ||qualtrics.com^.

Edit: Also there's a whole bunch of instances where there's ||badsite.com and then ||www.badsite.com. Once again, adblock syntax takes care of this. There are 11,517 instances of ||www. in the blu list, and they're all completely redundant rules.

Sep 06 '20 20:09 kah0922

Okay, so with Ador having been AFK for a month and a half now without doing even cursory checks of reported issues (Many of them very critical), my patience is done for. There is no way I can salvage a project whose salvation methods can only be accessed by one man who is AFK for weeks or months at a time.

My involvement as an Energized Protection team member is hereby over.

Oct 11 '20 20:10 DandelionSprout

Too bad, really, but cannot be helped then. It seems that even just reporting false positives a/o syntax issues is a waste of time under the circumstance.

Oct 11 '20 21:10 ghost

Okay, so with Ador having been AFK for a month and a half now without doing even cursory checks of reported issues (Many of them very critical), my patience is done for. There is no way I can salvage a project whose salvation methods can only be accessed by one man who is AFK for weeks or months at a time.

My involvement as an Energized Protection team member is hereby over.

We'll be sad to see you go away like that. How about you and I work together while Ador's AFK? Feel free to ping me on Telegram: @AvinashReddy3108 is my username there.

Oct 15 '20 09:10 AvinashReddy3108