FCP video noise reduction produces incorrect frame cadence on retimed rate conformed clip

Open joema4 opened this issue 2 years ago • 0 comments

Apple Feedback Assistant ID: FB10578973

If the FCP video noise reduction (NR) effect is applied to a retimed clip which has been rate conformed, it can produce incorrect frame cadence. E.g, a 29.97 clip retimed using “automatic speed” (80% in this case) in a 23.98 project should have 1:1 cadence. IOW frames ABCDEFG should play in order ABCDEFG. However if NR is applied, the frame order becomes ACDDEGHH, with some frames erroneously dropped and other frames duplicated.

The NR effect produces numerous different cadence errors which vary depending on the clip frame rate, project frame rate and retiming amount. This problem has existed at least since 10.5.2 and continues through 10.6.6 on both Intel and Apple Silicon. It does not seem related to codec but I tested mostly on ProRes 422. The timeline should be rendered to cache with and without NR to produce consistent behavior. If the timeline is not rendered, in the problem cases the frame cadence may unpredictably change during playback.

It does not happen from only rate conforming or only retiming. The clip must be rate conformed and retimed. It only happens with some combinations of clip frame rates and project frame rates if the clips are retimed and NR is applied. It does not seem sensitive to amount of retiming. For a given scenario, behavior is consistent and 100% reproducible, not intermittent (assuming a cached timeline).

Each combination of clip/project frame rates and retiming is expected to produce a specific cadence with varying perceptual smoothness. In some problem cases when that proper cadence is incorrectly altered by NR, it may be difficult to visually detect. In other cases it is very apparent. I list below some of the test scenarios which show the problem. For brevity “24” means 23.98, “30” means 29.97, and “60” means 59.94. I used alphabetic frame notation, e.g, frames 1,2,3,4 are denoted as ABCD. This notation allows clearer representation.

This was reported on 7-6-22 as Apple Support Case 101743180767. This was filed using the MacOS Feedback Assistant app 7-7-22 as bug FB10578973.

30 in 24, rate conformed but not retimed: OK with NR applied.
30 in 24, rate conformed, retimed to 80%, no NR: plays smoothly with 1:1 cadence; frames ABCDE play as ABCDE.
30 in 24, rate conformed, retimed to 80%, NR: plays roughly; ABCDEFGH becomes ACDDEGHH.
30 in 24, rate conformed, retimed to 90%, no NR: ABCDEFGHIJK becomes ABCDEFGHJ (expected cadence).
30 in 24, rate conformed, retimed to 90%, NR: ABCDEFGHI becomes ABDDEFIIJ (erroneous cadence).
24 in 30, auto speed no NR: expected 1:1 cadence, ABCDE remains ABCDE.
24 in 30, auto speed, NR: ABCDEFGHIJK becomes ABCDDFHIJJL (Verified also on Intel 10.5.2 and Catalina 10.15.7).
24 in 30, retimed to 180%, no NR: ABCDEFGHIJKL becomes ABDEGHIKLN (expected cadence).
24 in 30, retimed to 180%, NR: ABCDEFGHIJKL becomes ACEFFIJLN (erroneous cadence).
60 in 24: All tested retiming and NR combinations worked OK for 60 in 24.
30 in 60, automatic speed, no NR: expected 1:1 cadence, ABCDE remains ABCDE.
30 in 60, automatic speed, NR: cadence becomes 2:2, ABCDE becomes AABBCCDD (Verified also on Intel 10.5.2 and Catalina 10.15.7).
30 in 60, retimed to 150%, no NR: ABCDEFGHIJK becomes ABCCDEFFGHII (expected cadence).
30 in 60, retimed to 150%, NR: ABCDEFGHIJKLMN becomes AACCDDFFGGI (erroneous cadence).
60 in 30: All tested retiming and NR combinations worked OK for 60 in 30.

TO REPRODUCE:

(1) Configure FCP Preferences>Playback>Background Render to OFF. (2) Import to FCP the attached clip 1080p_2398_PR422.mov (does not matter whether “in place” or copied to library). (3) Create a custom 1080p project with 29.97 frame rate. (4) Add clip to project. (5) Retime clip to 80% using Modify>Retime>Automatic Speed. (6) Add built-in timecode effect to clip. (7) Modify timecode effect to display frames, not HMSF. This makes it easier to see cadence behavior. (8) Render clip to cache with CTRL+R. (9) Apply build-in FCP video noise reduction effect to clip. (10) Render to cache with CTRL+R. (11) Uncheck NR effect and play clip to observe smoothness and frame cadence. (12) Enable NR effect and play clip. The playback will be rough due to cadence errors. (13) To examine other problematic cases, create additional projects and use attached clips with parameters from the above scenarios.

Files to reproduce: https://www.dropbox.com/sh/6vpxc1z0a4r36s0/AAC06hZqWyky-sTuEGBOVdMna?dl=0

May 31 '23 15:05 joema4