Daxbot
[BUG] SDO block transfer upload lasts 3-4 times more time than reading large files via segmented sdo transfer using the version 1.4.8
This issue can be related to another issue, which does not let to ˙setBlockSize() larger than 30. However, If I still use my original test with segmented transfer, the duration time of the transfer get increased even more. If I set the block size of the SDO upload bigger than 30, the SDO upload simply just throw an SDO timeout error. There is no problem with SDO download though. ˙setBlockSize() to the maximum 127 works with it as well.
Interesting, a 65kB transfer using segmented takes about 163ms in the test suite but the same transfer using block takes almost 30 seconds.
After some investigation I've determined that this isn't a bug so much as a bad interaction with Node's timers. Node cannot schedule a timer faster than 1 ms which means that when using timers each CAN message in a block transfer has a minimum of 1 ms delay between them. I believe this is the bulk of the time discrepancy you're experiencing.
A workaround is to instead schedule the send with "setImmediate" so that it happens on the next loop iteration, rather than on the next 1ms time slice.
We don't see this with the segmented transfer because each 8-byte segment is acknowledged, so the node library can fire a new segment after each ACK rather than on a timer with a fixed frequency.
The fix in 6.2.0 is I have added setBlockInterval to the server implementation. If the value is set to zero, then setImmediate is used for each segment of a block. I'm keeping the default at 1 ms per send for now. Take a look at the test for implementation:
https://github.com/Daxbot/node-canopen/blob/main/test/protocol/test_sdo.js#L343-L381
@wilkinsw Thank you for your investigation and the quick fix. Unfortunately, however, it still does not work fully for me. When I set the setBlockInterval to 0 and set setBlockSize larger than 15, I get SDO Timeout error when I start a sdoUpload of a large file which was sdoDownloaded previously to the same SDO object.
This is how I create a new SDO device:
const deviceObject = this._getCANOpenDeviceObject(node, loopback)
deviceObject.eds = edsPath ? Eds.fromFile(edsPath) : null
this._devices[node] = new Device(deviceObject)
const sdoCobIdTx = 0x600 | node // client to server
const sdoCobIdRx = 0x580 | node // server to client
const sdoServerCobIdTx = 0x580 | node // server to client
const sdoServerCobIdRx = 0x600 | node // client to server
this._devices[node].eds.addSdoClientParameter(node, sdoCobIdTx, sdoCobIdRx)
this._devices[node].eds.addSdoServerParameter(node, sdoServerCobIdTx, sdoServerCobIdRx)
this._devices[node].start()
this._devices[node].sdo.setBlockSize(15)
this._devices[node].sdoServer.setBlockSize(127)
this._devices[node].sdoServer.setBlockInterval(0)
and handle sdoUpload using blockTransfer:
const entry = {
deviceId: node,
index: index,
subIndex: subIndex ? subIndex : null,
dataType: this._dataTypeToCanOpen(dataType),
timeout: timeout,
cobIdRx: 0x580 | node,
blockTransfer: blockTransfer,
blockInterval: 0
}
const device = this._getDevice(node)
const sdoValue = await device.sdo.upload(entry)
From the logs it seems when I use larger value than 15 for the this._devices[node].sdo.setBlockSize(15) the sdo server sends out the first block of the messages with the preset size but than it wont send the next one just wait until the timeout expires.
I think I've fixed it. There was a race condition in several methods where the event that sends the CAN message happens before the state of the transfer gets updated. Since the event emitter handles callbacks synchronously, in certain scenarios you'll end up with message responses being handled by the Server/Client object before the original sending method even finishes.
Most of my tests decouple the send and receive methods with setImmediate() to avoid overflowing the call stack, so I never really noticed.
Fix in version 6.2.1
@wilkinsw Unfortunately, I still have the same issue with the latest version. If I use larger BlockSize value than 15 I just get an SDO Timeout error.
The block transfer works by sending a large "block" of messages without any explicit flow control. It sounds like the receiver may be running out of buffer space, or otherwise missing a message during the dump which is why lower block size works.
What does your setup look like? Two separate devices or just one over a virtual bus? Are you using node-canopen on both ends, or talking to a microcontroller?
@wilkinsw
What does your setup look like? Two separate devices or just one over a virtual bus? Are you using node-canopen on both ends, or talking to a microcontroller?
I am using node-canopen on both sides. Basically, I set up sdo and sdoServer devices at the same time with the same Id and use them like in your example, so I download the data from the client to the server and the upload it from the server to the client using loopback true.
I think you're creating a conflict. The loopback option is just to support running tests. All messages generated by a Device with loopback set will be emitted into its own receive() method. Your sending Device is probably handling the entire SDO transaction ping-ponging back and forth with itself before it tries to handle the other Device's response (and immediately drops it because the transfer is done).
It is undefined behavior if multiple Devices on a CAN bus have the same ID. Each Device must be a uniquely addressable entity. If you have two Device objects (as in new Device(...)) then try disabling loopback and make sure each device has a unique CAN id.
Note that if both objects need to initiate transfers you'll end up defining two SDO address pairs. For example if your two devices are 0xA and 0xB you will want to define both 0x58A/0x60A for transfers initiated from 0xA and 0x58B/0x60B for transfers initiated from 0xB.
@wilkinsw I am sorry for the late reply. I have been busy for the last two weeks. I really much appreciate your support on this issue. I have updated my code as you suggested. So my setup is to have a sdoClient and an sdoServer (with different deviceID-s this time). Also, I set the cobIdRx and the cobIdTx according to the deviceID-s and not using the loopback anymore. I'd like to transfer a 4KB sized "large" file from the client to the server and read it back via block transfer. But unfortunately, I still have the same issue while I do the sdo upload (reading back the file). There's no problem with downloading the large file before it, though. On the CAN logs I can see some sdo block transfer messages but only around the half of the block size, then the server waits until the timeout and sends the sdo transfer abort message. Before I run this test with the 4KB file, I read smaller data also with block transfer which works without a problem but maybe because it is only 20 bytes.
I wrote a simple file transfer example, both read and write, and 40kB transfers work fine. Please see https://github.com/Daxbot/node-canopen/tree/main/examples/sdo/file_transfer and see if you can modify it to show your problem.
Once you have that give me the git diff and I'll see if I can reproduce.
@wilkinsw Thanks for your reply. Unfortunately, your suggested solution won't work in my use case because I always need to run the sdoClient and the sdoServer in the same application. I guess this is the crucial point here. Also, I run this application in a docker container, so there can be an issue with limited resource access. Do you think could it help if I increase the allocated memory or the available cpu load allocated to that docker container?
I realize I should have started with this. Does npm test pass in your docker container if you clone the repository directly? The tests have a 64kB transfer in a single process, and that sample is a 40kB transfer in two processes. Both work fine for me on a vcan (socketcan) interface. Since I'm unable to reproduce your issue I can't really help with it.