Addon icon indicating copy to clipboard operation
Addon copied to clipboard

Encountering issues with Chinese URL encoding on some websites

Open Keyezi opened this issue 1 year ago • 3 comments

The following content was expressed using machine translation. If there are any errors or issues, please let me know for supplementation. Thank you.

When I search for '宠物'(pets) on this website and enable cleanURLs, it can cause garbled URLs, indirectly leading to the inability to search properly.

https://s.1688.com/selloffer/offer_search.htm?keywords=%B3%E8%CE%EF

This is the result that should appear under normal circumstances when searching: image

When I enabled cleanURLs, it seemed that they were not properly 'url encoded' and were accompanied by some classic Chinese encoding issues. Please refer to the article I provided at the bottom for more details. image——————————————————————————————————————

锟斤拷 garbled code problem

https://zh.wikipedia.org/wiki/%E4%B8%AD%E6%96%87%E4%BA%82%E7%A2%BC#%E9%94%9F%E6%96%A4%E6%8B%B7%E4%B9%B1%E7%A0%81%E9%97%AE%E9%A2%98

When converting between Unicode encoding and Simplified Chinese encoding systems (such as GB 2312, GBK, GB 18030, CP936), some Simplified Chinese encoded text does not exist in Unicode encoding, and Unicode records it as an "unrecognized character (U+FFFD)" as an internal code, while it is represented as EF BF BD in UTF-8. When multiple EF BF BDs appear consecutively and are interpreted in Simplified Chinese encoding, they will be parsed as multiple "锟斤拷". The encoding of the three characters is 锟(0xEFBF), 斤(0xBDEF), and 拷 (0xBFBD). ——————————————————————————————————————————————————

URL Encode

https://en.wikipedia.org/wiki/Percent-encoding

Keyezi avatar Dec 27 '24 04:12 Keyezi

Related to #308

ccuser44 avatar Jan 18 '25 00:01 ccuser44

We’re aware of the issue, and an update has already been submitted to Mozilla for review. It should be approved within the next few days, resolving the problem.

Should be fixed by https://github.com/ClearURLs/Addon/pull/415.

KevinRoebert avatar Jan 29 '25 20:01 KevinRoebert

The issue still persists in version 1.27.3.

dongfengweixiao avatar Feb 17 '25 12:02 dongfengweixiao