keyman icon indicating copy to clipboard operation
keyman copied to clipboard
verified publisher icon keymanapp

fix(ios): selection range sync for long contexts

Open jahorton opened this issue 1 year ago • 5 comments

Fixes corrections and predictions after recently-typed paragraph breaks.

So... I forgot to properly remap the selection-range coordinates when updating the internal web-engine's context across paragraph boundaries in #10728. Fortunately, the mapping calculations are pretty simple - we only avoid resetting context if there's a very specific relation between the old and new left-hand contexts, after all.

User Testing

To easily navigate to this page from a mobile device:

image

Sample text:

Almost half of all people in the world today speak an Indo-European language.

Over the last couple of hundred years, linguists have figured out a lot about the first Indo-European language including many

TEST_NEW_PARAGRAPH_PREDICTIONS: Using the sample text specified above as existing context, verify that predictive text operates as expected.

  1. Using Keyman as a system keyboard on an iOS device, paste the "sample text" above into an iOS app.

    • Personal recommendation: Notes app
    • If using Simulator to test: Messages app

  2. After pasting, the text insertion point should be at the end of the pasted text. If not, make it so.

  3. Hit ENTER twice.

  4. If no predictions are shown at this specific moment, this is fine - it's a separate issue.

  5. Type p, then hit backspace to delete it.

  6. Verify that the usual default suggestions are displayed: "the", "to", "and", etc.

  7. Type ot and verify that suggestions mostly starting with o and t are displayed.

  8. Select others and verify that the expected results occur.

  9. Once applied, directly apply suggestions that pop up at least ten times. Verify that each is applied as expected.

TEST_PREDICTION_AFTER_DELETIONS: Using the same sample text as specified above, verify that predictive text operates as expected after deleting lots of text.

  1. Using Keyman as a system keyboard on an iOS device, paste the "sample text" above into an iOS app.
    • Personal recommendation: Notes
    • If using Simulator to test: Messages
  2. After pasting, the text insertion point should be at the end of the pasted text. If not, make it so.
  3. Hit ENTER twice.
  4. Type others.
  5. Press and hold backspace until you reach the middle of the second paragraph - try to stop in the middle of a word.
  6. If not in the middle of a word, tap backspace a few times until the text insertion point is in the middle of a word.
  7. Verify that the suggestions shown are all suggestions that make sense for the half-word.
    • Example: if you've deleted part of about and are now at abo, the following make sense as the first suggestions: "about", "above", "abortion". Other suggestions should include at least two of the three letters as well.
  8. Apply one of the displayed suggestions and verify that it acts as expected.

TEST_PREDICTION_AFTER_MOVE: Using the same sample text as specified above, verify that predictive text operates as expected after deleting lots of text.

  1. Using Keyman as a system keyboard on an iOS device, paste the "sample text" above into an iOS app.
    • Personal recommendation: Notes
    • If using Simulator to test: Messages
  2. After pasting, the text insertion point should be at the end of the pasted text. If not, make it so.
  3. Move the text insertion point into the middle of the word peo ple, which is in the middle of the first paragraph.
  4. Verify that the suggestions shown are all suggestions that make sense for peo: you might expect to see "people", "people's", "peoples", "person", etc.
  5. Apply one of the displayed suggestions and verify that it acts as expected.

jahorton avatar Mar 06 '24 08:03 jahorton

User Test Results

Test specification and instructions

  • ✅ TEST_NEW_PARAGRAPH_PREDICTIONS (PASSED): Tested with the attached PR build on iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Hit Enter key twice. 3. Typed p, then hit the backspace key to delete it. 4. Verified that the usual default suggestions are displayed. 5. Typed 'ot' and verified that the suggestions mostly started with o and t are displayed. 6. Selected 'others' from the suggestion bar and verified that it displayed on the text input screen. 7. Directly applied suggestions from that pop up at least ten time. Verified that the applied suggestions are displayed on the input screen. (notes)
  • ✅ TEST_PREDICTION_AFTER_DELETIONS (PASSED): Tested with the attached PR build on iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Hit Enter key twice. 3. Typed 'others'. 4. Pressed and held down the backspace key a few times until the text insertion point is in the middle of the word. 5. Verified that the suggestions shown are all suggestions that make sense for the half-word. 6. Applied one of the displayed suggestions and verified that it showed the expected result on the screen. (notes)
  • ✅ TEST_PREDICTION_AFTER_MOVE (PASSED): Tested with the attached PR build (17.0.283-beta-test-10956) on an iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Moved the txt insertion point into the middle of the word (eg., in between peo and ple)which is in the middle of the first paragraph. 3. Verified that the suggestions shown are all suggestions that are related to 'peo'. 4. I selected 'person' from the banner and verified that it replaced 'people' on the text input screen with 'Person'. (notes)

Test Artifacts

keymanapp-test-bot[bot] avatar Mar 06 '24 08:03 keymanapp-test-bot[bot]

Test Results

  • TEST_NEW_PARAGRAPH_PREDICTIONS (PASSED): Tested with the attached PR build on iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Hit Enter key twice. 3. Typed p, then hit the backspace key to delete it. 4. Verified that the usual default suggestions are displayed. 5. Typed 'ot' and verified that the suggestions mostly started with o and t are displayed. 6. Selected 'others' from the suggestion bar and verified that it displayed on the text input screen. 7. Directly applied suggestions from that pop up at least ten time. Verified that the applied suggestions are displayed on the input screen.

bharanidharanj avatar Mar 08 '24 11:03 bharanidharanj

Test Results

  • TEST_PREDICTION_AFTER_DELETIONS (PASSED): Tested with the attached PR build on iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Hit Enter key twice. 3. Typed 'others'. 4. Pressed and held down the backspace key a few times until the text insertion point is in the middle of the word. 5. Verified that the suggestions shown are all suggestions that make sense for the half-word. 6. Applied one of the displayed suggestions and verified that it showed the expected result on the screen.

bharanidharanj avatar Mar 08 '24 12:03 bharanidharanj

Test Results

  • TEST_PREDICTION_AFTER_MOVE (FAILED): Tested with the attached PR build on iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Moved the txt insertion point into the middle of the word (eg., in between peo and ple)which is in the middle of the first paragraph. 3. Noticed that the unpredicted words appeared on the suggestion bar. Seems to be an issue.

bharanidharanj avatar Mar 08 '24 12:03 bharanidharanj

@keymanapp-test-bot retest TEST_PREDICTION_AFTER_MOVE

Glad I spec'd that user test. I believe I've identified and fixed the issues causing the problem now; "hard context resets" were also using the codepaths meant for non-reset cases by mistake.

jahorton avatar Mar 12 '24 16:03 jahorton

The build has failed a few times now with this error set during the "Upload to TestFlight" step:

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]: 2024-03-15 23:44:33.983 DEBUG: Adding upload task 90131 for part 1.

19:44:34 
  

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]:     "LocalUploadTask <F57B37EC-277D-4824-B286-C3393A2D27DE>.<90130>"

19:44:34 
  

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]: ), NSLocalizedDescription=A server with the specified hostname could not be found., NSErrorFailingURLStringKey=https://northamerica-1.object-storage.apple.com/itmspod11-assets-massilia-200001/PurpleSource211%2Fv4%2Fcd%2F7a%2F94%2Fcd7a94f9-e61d-72b8-7d63-a7d98b7e09d0%2FzdNd26rtHz3H6IPNT3PgTTlx-mNkSbv9_5UvwiUtan8_U003d-1710506561722?uploadId=837bda20-e2c9-11ee-a5ef-783fd2d59ff3&Signature=SvvrmRTL%2BQnksXg31t5hLfwQxMo%3D&AWSAccessKeyId=MKIAP7F9QNTEY48OTE7F&partNumber=1&Expires=1711111361, NSErrorFailingURLKey=https://northamerica-1.object-storage.apple.com/itmspod11-assets-massilia-200001/PurpleSource211%2Fv4%2Fcd%2F7a%2F94%2Fcd7a94f9-e61d-72b8-7d63-a7d98b7e09d0%2FzdNd26rtHz3H6IPNT3PgTTlx-mNkSbv9_5UvwiUtan8_U003d-1710506561722?uploadId=837bda20-e2c9-11ee-a5ef-783fd2d59ff3&Signature=SvvrmRTL%2BQnksXg31t5hLfwQxMo%3D&AWSAccessKeyId=MKIAP7F9QNTEY48OTE7F&partNumber=1&Expires=1711111361, _kCFStreamErrorDomainKey=12}

19:44:34 
  

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]: 2024-03-15 23:44:33.984 DEBUG: Created new upload task (0x1537303b0) for part 1.

19:44:34 
  

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]: 2024-03-15 23:44:33.984 DEBUG: Saving uploader state (CDUploaderStateUploadAssetDescription) for identifier 'com.apple.cds_4BCE69C2-E5DF-42FA-9C0E-B783323F9738'.

19:44:34 
  

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]: 2024-03-15 23:44:33.985 DEBUG: There is one part remaining to upload.

19:44:34 
  

19:44:34 
  DEBUG [2024-03-15 23:44:33.98]: [altool]: 2024-03-15 23:44:33.987 DEBUG: LOST 0 bytes for part 1.

This is constantly repeated in the TC build logs.

jahorton avatar Mar 15 '24 13:03 jahorton

Hmm, altool was deprecated wasn't it? Or was it only specific tasks for altool. Can you break that into a separate issue?

mcdurdin avatar Mar 15 '24 15:03 mcdurdin

I've upgraded fastlane and retried another build, which worked, so retrying this ios build now.

mcdurdin avatar Mar 16 '24 02:03 mcdurdin

Test Results

  • TEST_PREDICTION_AFTER_MOVE (PASSED): Tested with the attached PR build (17.0.283-beta-test-10956) on an iPhone 13 Mobile (iOS 17.4) device and here is my observation: 1. Copied the given text then pasted it in Notes app. 2. Moved the txt insertion point into the middle of the word (eg., in between peo and ple)which is in the middle of the first paragraph. 3. Verified that the suggestions shown are all suggestions that are related to 'peo'. 4. I selected 'person' from the banner and verified that it replaced 'people' on the text input screen with 'Person'.

..before selection

..after selection

bharanidharanj avatar Mar 18 '24 13:03 bharanidharanj

Changes in this pull request will be available for download in Keyman version 17.0.290-beta

keyman-server avatar Mar 19 '24 18:03 keyman-server