pyocd flash big image is too slow for RT1060

[Hoohaha]

hi @flit,

We are using pyocd as the major flash tool for our sdk application develop and testing. Thanks working on this excellent tool! We used gdbserver and gdb to load applications , and added timeout handler for pyocd gdbserver process to prevent pyocd is hang there. You can see below logs that the flash programming is failed due to pyocd exceed max timeout 300s. Sometimes it can success withinn 2mintues, sometimes it cannot. Do you known how to improve the flash programming performance?

[2021-09-07 15:23:09,718 INFO] gdbserver: "c:\\program files\\python39\\python.exe" -m pyocd gdbserver --no-wait --port 6100 -u 02290000016dc16200000000000000000000000097969905 -O semihost_console_type=console -O keep_unwritten=false
[2021-09-07 15:23:11,234 INFO] C:/PROGRA~2/GNUARM~1/102020~1/bin/AR470E~1.EXE --exec C:/Dpc/binary/ota_demo_wifi_nxp.elf --silent
[2021-09-07 15:23:11,272 INFO] (gdb) target remote localhost:6100
[2021-09-07 15:23:13,482 INFO] (gdb) monitor reset --halt
[2021-09-07 15:23:13,682 INFO] (gdb) monitor halt
[2021-09-07 15:23:14,448 INFO] (gdb) load
[2021-09-07 15:28:11,280 WARNING] pid: 12720 exceeded timeout[Timeout=300(s)], force killed
[2021-09-07 15:28:11,342 WARNING] pid: 12416 exceeded timeout[Timeout=300(s)], force killed
[2021-09-07 15:28:11,357 ERROR] Loading section boot_hdr, size 0xa0 lma 0x60040000

Loading section .interrupts, size 0x400 lma 0x60040400

Loading section .text, size 0x10ae60 lma 0x60040800

Loading section .ARM, size 0x8 lma 0x6014b660

Loading section .init_array, size 0x8 lma 0x6014b668

Loading section .fini_array, size 0x4 lma 0x6014b670

Loading section .data, size 0x2a0c lma 0x6014b674

Loading section .ncache.init, size 0xd0 lma 0x6014e080

Remote communication error.  Target disconnected.: Not a directory.


[2021-09-07 15:28:11,379 INFO] Debug session is closed!
[2021-09-07 15:28:11,379 ERROR] gdbserver console: 
0000854:INFO:board:Target type is mimxrt1060
0000886:INFO:dap:DP IDR = 0x0bd11477 (v1 MINDP rev0)
0000902:INFO:ap:AHB-AP#0 IDR = 0x04770041 (AHB-AP var4 rev0)
0000949:INFO:rom_table:AHB-AP#0 Class 0x1 ROM table #0 @ 0xe00fd000 (designer=00e part=88c)
0000949:INFO:rom_table:[0]<e00fe000:ROM class=1 designer=43b part=4c8>
0000949:INFO:rom_table:  AHB-AP#0 Class 0x1 ROM table #1 @ 0xe00fe000 (designer=43b part=4c8)
0000986:INFO:rom_table:  [0]<e00ff000:ROM class=1 designer=43b part=4c7>
0000986:INFO:rom_table:    AHB-AP#0 Class 0x1 ROM table #2 @ 0xe00ff000 (designer=43b part=4c7)
0001002:INFO:rom_table:    [0]<e000e000:SCS v7-M class=14 designer=43b part=00c>
0001002:INFO:rom_table:    [1]<e0001000:DWT v7-M class=14 designer=43b part=002>
0001002:INFO:rom_table:    [2]<e0002000:FPB v7-M class=14 designer=43b part=00e>
0001018:INFO:rom_table:    [3]<e0000000:ITM v7-M class=14 designer=43b part=001>
0001065:INFO:rom_table:  [1]<e0041000:ETM M7 class=9 designer=43b part=975 devtype=13 archid=4a13 devid=0:0:0>
0001071:INFO:rom_table:  [2]<e0042000:CTI class=9 designer=43b part=906 devtype=14 archid=0000 devid=40800:0:0>
0001071:INFO:rom_table:[1]<e0040000:TPIU M7 class=9 designer=43b part=9a9 devtype=11 archid=0000 devid=ca1:0:0>
0001087:INFO:rom_table:[2]<e0043000:TSGEN class=15 designer=43b part=101>
0001087:INFO:target_imxrt:IMXRT Boot Mode: Internal Boot
0001087:INFO:cortex_m:CPU core #0 is Cortex-M7 r1p1
0001149:INFO:cortex_m:FPU present: FPv5-D16-M
0001149:INFO:dwt:4 hardware watchpoints
0001149:INFO:fpb:8 hardware breakpoints, 1 literal comparators
0001171:INFO:gdbserver:Semihosting will be output to console
0001566:INFO:gdbserver:GDB server started on port 6100 (core 0)
0003544:INFO:gdbserver:Client connected to port 6100!
0003729:INFO:target_imxrt:IMXRT Boot Mode: Internal Boot
0003729:INFO:target_imxrt:IMXRT Boot Device: 0
[---|---|---|---|---|---|---|---|---|----]
[0005018:INFO:target_imxrt:IMXRT Boot Mode: Internal Boot
0005033:INFO:target_imxrt:IMXRT Boot Device: 0
===============

[flit]

Hi @Hoohaha ! Always good to hear from you.

I don't know immediately what the cause is. Usually, flash performance is dominated by the flash itself. But it's very strange that the programming time can vary so much. We'll need to dig much deeper to understand the root cause.

One consideration is that by default, pyocd will scan existing flash contents in an attempt to skip erasing/programming pages that don't change. For active development when you are changing code a lot, this can be a big speed-up. But if you are programming entirely new content, it actually slows down the programming process. Usually it's not noticeable, but can be if you are programming a very large image. The smart_flash session option controls this feature. (I see you have keep_unwritten disabled, so I won't describe that.)

Questions:

• Does pyocd completely hang, or just progress very slowly? Meaning, if you had a much longer timeout, would it ever complete?
• Do you have an example file I can use to reproduce the problem? You can send it by email if you don't want to attach it. I don't have an RT106x, only an RT1052 and RT102x.
• Does it happen on all RT Series devices?
• Do you know if the pattern is the same for different external memory types? (QSPI vs HyperFlash, etc)
• Does it happen if you directly use pyocd flash instead of going through gdb? That shouldn't change anything, but it's worth testing.
• Does it happen if you disable smart flashing, e.g. -Osmart_flash=0?

[fbeaulier]

Hi @flit, I have a similar issue with an MIMXRT1061DVL6A board.

Flashing is fast for a binary image < ~350ko If image is larger it slows down above the first 350ko, the bargraph stops for few seconds and then it resumes but slowly. I never got a timeout issue (image not big enough probably), it's just slow.

The used Flash is a 8Mo QSPI NOR, an ISSI, the same one they put on the eval board.

I flash using the pyocd flash command, not through gdb.

With -Osmart_flash=0 no much change, may be a bit slower.

Note : it starts to slow down at the time the MCU resets all GPIO to high impedance (my LED goes OFF)

Hope this helps to find the issue.

[flit]

Hi @fbeaulier, thanks for the additional info.

Note : it starts to slow down at the time the MCU resets all GPIO to high impedance (my LED goes OFF)

Do you mean that your board has a GPIO-driven LED, which stays on (thus driven low?) for the first, faster, part of the flashing, and then turns off (thus the GPIO is either Hi-Z or driven opposite polarity) at the same time the flashing pauses and slows down?

This could be an important clue. Right now it's not clear what it means, since the MCU should be reset before flashing even starts. And presumably that would reset the GPIOs?

[fbeaulier]

Hi @flit,

Do you mean that your board has a GPIO-driven LED, which stays on (thus driven low?) for the first, faster, part of the flashing, and then turns off (thus the GPIO is either Hi-Z or driven opposite polarity) at the same time the flashing pauses and slows down?

Yes it's exactly that. The LED is driven using a transistor so GPIO is high when it's ON. So it looks like the GPIO is going Hi-Z or low when the flash pauses. (And yes an MCU reset puts all the GPIOs to Hi-Z)

[fbeaulier]

Hi @flit,

Any update on that issue ?

Thanks

[flit]

I'm sorry, I just haven't had a chance to look at this yet. (I also don't have an RT1060 board, so hopefully it's reproducible on the RT1050.)

[Hoohaha]

Sorry I missed your reply

Does pyocd completely hang, or just progress very slowly? Meaning, if you had a much longer timeout, would it ever complete?

Tested with latest version(v0.32.x), the progress updating is very slow, but sometimes it seems is stopped, Maybe it would be effected by old existing flash content.

Inorder to avoid such issue, we convert ELF to plain binary then invoke pyocd to flash. The stability get much better.

Does it happen on all RT Series devices?

It's strange that It is much better on RT1050-EVk than on RT1060.

Does it happen if you directly use pyocd flash instead of going through gdb? That shouldn't change anything, but it's worth testing.

Same results.

Does it happen if you disable smart flashing, e.g. -Osmart_flash=0?

Helpless, tried with various combinations for options keep_unwritten, smart_flash

[flit]

Thanks for the info @Hoohaha. That is really strange that RT1050 is better.

[flit]

Happy to say I have an MIMXRT1060-EVK now. 😄 I'll try reproducing this issue as soon as I get a chance.

[rikorsev]

Hi all, Just curious, is there some progress with the issue?

[flit]

Sorry, nothing new.