Anki-Android
Anki-Android copied to clipboard
feat: Add Han Unification detection in note editor (Stage 1)
Purpose / Description
Implements Stage 1 of Han Unification issue detection to help users who are learning Japanese (or other CJK languages) identify rendering problems in their flashcards.
Fixes
- #19431
Approach
Stage 1: Detection & Logging Only
This is the first stage of a multi-stage implementation. Stage 1 focuses on detection and logging without any user-facing UI changes.
- Detection at Note Edit/Add Time (Not Review Time)
- Plain Text Detection (Not HTML Parsing)
-
Curated Character List (Not All CJK)
- Checks for 23 known problematic CJK characters with significant visual differences
- Reduces false positives compared to checking all 20,000+ CJK Unified Ideographs
How Has This Been Tested?
Learning (optional, can help others)
Han Unification Background:
- Wikipedia: Han Unification - Overview of the Unicode concept
- Unicode CJK Unified Ideographs - Technical details
- W3C: Language Tags in HTML - How lang attributes solve this
Checklist
Please, go through these checks before submitting the PR.
- [x] You have a descriptive commit message with a short title (first line, max 50 chars).
- [x] You have commented your code, particularly in hard-to-understand areas
- [x] You have performed a self-review of your own code
- [ ] UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)
- [ ] UI Changes: You have tested your change using the Google Accessibility Scanner
Windows test DeckPickerTest > [1] > ContextMenu creates deck shortcut when selecting CREATE_SHORTCUT[1] has become flaky
unrelated to this PR - noting for posterity
DeckPickerTest > [1] > ContextMenu creates deck shortcut when selecting CREATE_SHORTCUT[1] FAILED
java.lang.IllegalStateException: throwOnShowError: android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views. Expected: SDK 35 Main Thread @kotlinx.coroutines.test runner#3416 Calling: DefaultDispatcher-worker-4 @coroutine#3438
at com.ichi2.anki.CrashReportData$Companion.throwIfDialogUnusable(CoroutineHelpers.kt:719)
at com.ichi2.anki.CoroutineHelpersKt.showError(CoroutineHelpers.kt:259)
at com.ichi2.anki.CoroutineHelpersKt.showError(CoroutineHelpers.kt:607)
at com.ichi2.anki.CoroutineHelpersKt.showError$default(CoroutineHelpers.kt:604)
at com.ichi2.anki.CoroutineHelpersKt.runCatching(CoroutineHelpers.kt:203)
at com.ichi2.anki.CoroutineHelpersKt$runCatching$1.invokeSuspend(CoroutineHelpers.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:34)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:98)
at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:65)
at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:243)
at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:147)
at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:470)
at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$kotlinx_coroutines_core(CancellableContinuationImpl.kt:504)
at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$kotlinx_coroutines_core$default(CancellableContinuationImpl.kt:493)
at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:359)
at kotlinx.coroutines.ResumeOnCompletion.invoke(JobSupport.kt:1541)
at kotlinx.coroutines.JobSupport.notifyCompletion(JobSupport.kt:1625)
at kotlinx.coroutines.JobSupport.completeStateFinalization(JobSupport.kt:316)
at kotlinx.coroutines.JobSupport.finalizeFinishingState(JobSupport.kt:233)
at kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath(JobSupport.kt:946)
at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:894)
at kotlinx.coroutines.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:859)
at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:99)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:47)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:124)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:89)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:820)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:717)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:704)
Caused by:
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views. Expected: SDK 35 Main Thread @kotlinx.coroutines.test runner#3416 Calling: DefaultDispatcher-worker-4 @coroutine#3438
at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:11023)
at android.view.ViewRootImpl.requestLayout(ViewRootImpl.java:2473)
at android.view.View.requestLayout(View.java:28020)
at org.robolectric.shadows.ShadowView$ViewReflector$$Reflector51.requestLayout(Unknown Source)
at org.robolectric.shadows.ShadowView.requestLayout(ShadowView.java:268)
at android.view.View.requestLayout(View.java)
at android.view.View.setLayoutParams(View.java:20530)
at android.view.WindowManagerGlobal.updateViewLayout(WindowManagerGlobal.java:464)
at android.view.WindowManagerImpl.updateViewLayout(WindowManagerImpl.java:165)
at android.app.Activity.onWindowAttributesChanged(Activity.java:4364)
at androidx.appcompat.view.WindowCallbackWrapper.onWindowAttributesChanged(WindowCallbackWrapper.java:114)
at android.view.Window.dispatchWindowAttributesChanged(Window.java:1323)
at com.android.internal.policy.PhoneWindow.dispatchWindowAttributesChanged(PhoneWindow.java:3197)
at android.view.Window.setFlags(Window.java:1309)
at org.robolectric.shadows.ShadowWindow$WindowReflector$$Reflector62.setFlags(Unknown Source)
at org.robolectric.shadows.ShadowWindow.setFlags(ShadowWindow.java:54)
at android.view.Window.setFlags(Window.java)
at android.view.Window.clearFlags(Window.java:1283)
at com.ichi2.anki.CoroutineHelpersKt$withProgressDialog$2.invokeSuspend(CoroutineHelpers.kt:473)
at app//kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:34)
Good first step, no functional changes to the code, but lets us improve on it later