FIRST-Tech-Challenge
LynxI2cReadMultipleBytesCommand always reads extra byte
We are communicating with serial I2C device which can send more than 100 bytes in a single response. Every response from the device contains header and cargo, first two bytes in header contain response length (including both cargo and header length). Per our observation, LynxI2cReadMultipleBytesCommand always reads one extra byte
Ie, per host request, device needs to send 6 cargo bytes 01 02 03 04 05 06. In this case devise will send back respond with 10 bytes 0A 00 FF FF 01 02 03 04 05 06
0A 00 FF FF - header - 4 bytes
01 02 03 04 05 06 - cargo - 6 bytes
0A - first byte=total packet length
In every read request, host is required to read at least four bytes (header)
Example expected exchange:
Scenario 1 – Host does not know expected response length
Host -> Read 4 bytes [header]
Device -> 0A 00 FF FF (I have 10 0A bytes to send)
Host -> Read 10 bytes
Device -> 0A 00 FF FF 01 02 03 04 05 06 (here are 10 bytes = 4 bytes in header + 6 bytes in cargo)
Scenario 2 – Host does know expected response length
Host -> Read 10 bytes [header + cargo]
Device -> 0A 00 FF FF 01 02 03 04 05 06 (ending 10 bytes including header and cargo)
Scenario 3 – Host does know expected response length, but has limited incoming buffer (7 bytes)
Host -> Read 7 bytes [header + cargo]
Device -> 0A 00 FF FF 01 02 03 (sending 7 bytes including header and first 3 bytes from cargo, host will need to come back and read 3 more cargo bytes = 0A – 07, or 7 total bytes (cargo + header))
Host -> Read 7 bytes [header + cargo]
Device -> 07 00 FF FF 04 05 06 (sending 7 bytes, 4 in header and 3 remaining bytes from cargo)
Actual exchange
Scenario 1 – Host does not know expected response length - data loss
Host -> Read 4 bytes [header]
Device -> 0A 00 FF FF (I have 10 0A bytes to send) [in this transaction Hub actually reads 5 bytes from the devices instead of 4]
Host -> Read 10 bytes
Device -> 09 00 FF FF 02 03 04 05 06 00 (here are 9 bytes = 4 bytes in header + 5 bytes in cargo + dummy byte) – this response indicates that instead of requesting and reading 4 bytes, host actually read 5 (4 bytes in header + 1 cargo), and device excluded original first cargo byte from second response. Byte 01 is lost
Scenario 2 – Host does know expected response length (response will fit into buffer)- no data loss
Host -> Read 10 bytes [header + cargo]
Device -> 0A 00 FF FF 01 02 03 04 05 06 (I am sending 10 bytes including header and cargo)
In other words, based on what we currently see, Expansion Hub always reads extra byte from the device and we are missing information when expected cargo length exceeds 100 bytes since we need to read it in chunks
Scenario 3 – Host does know expected response length, but has limited incoming buffer (7 bytes, [actual limit is 100, but behavior is the same]) - data loss
Host -> Read 7 bytes [header + cargo]
Device -> 0A 00 FF FF 01 02 03 (sending 7 bytes including header and cargo, total length 10)
Host -> Read 7 bytes [header + cargo]
Device -> 06 00 FF FF 05 06 00 (sending 7 bytes, 4 in header, 2 remaining bytes from cargo and one dummy byte - byte 04 is lost)
In other words, LynxI2cReadMultipleBytesCommand always reads one extra byte from I2C device beyond of what is requested. It does not create any issues if requested length is greater than number of bytes that device needs to send, but causes data loses in case if device needs to send more than requested in single read request
I'm guessing you're trying to use LynxI2cReadMultipleBytesCommands directly (instead of using the SDK's built-in methods) to enable reads without specifying a register since the SDK doesn't support that? If so, are you also manually handling sending the read status queries until the response is available?
How have you verified that the Lynx module is actually reading an extra byte? Are you saying that the cbRead in the query response is actually greater than the number of bytes you requested to be read? Or, if not, have you hooked up a scope or data analyzer to the bus lines and actually confirmed that the I2C bus master is reading an extra byte?
I'm guessing you're trying to use
LynxI2cReadMultipleBytesCommands directly (instead of using the SDK's built-in methods) to enable reads without specifying a register since the SDK doesn't support that? If so, are you also manually handling sending the read status queries until the response is available?
Correct, I implemented my read process in the same way as it is done in LynxI2cDeviceSynchV1.readTimeStamped with only difference that I do not send register number to the device (having dummy reg number in read request sent to device does not affect device behavior and device responds with same data regardless of Register Address byte being present in the playload)
How have you verified that the Lynx module is actually reading an extra byte? Are you saying that the
cbReadin the query response is actually greater than the number of bytes you requested to be read? Or, if not, have you hooked up a scope or data analyzer to the bus lines and actually confirmed that the I2C bus master is reading an extra byte?
cbRead in the query response actually equals to the number of bytes I request. I got my suspicions based on the response I get from the device (Ie, when I send two LynxI2cReadMultipleBytesCommand one after another to get two chunks of data from the device, second response from device indicates how many bytes were read from the device in the first transaction and it is always one more than I request) and validated behavior by connecting target device to another I2C host
I got my suspicions based on the response I get from the device (Ie, when I send two LynxI2cReadMultipleBytesCommand one after another to get two chunks of data from the device, second response from device indicates how many bytes were read from the device in the first transaction and it is always one more than I request) and validated behavior by connecting target device to another I2C host
Interesting. I am going to put my scope on the Expansion Hub I2C lines and see what actually goes over the wire when firing a LynxI2cReadMultipleBytesCommand.
Wow. Excellent sleuthing. I was incredulous, but scoping the I2C lines reveals that you are exactly correct.
First, I tested your theory using the built-in IMU in the Expansion Hub. I first send a write command to set the register address to 0x00 (which is the CHIP_ID register), then sent a LynxI2cReadMultipleBytesCommand asking for 2 bytes. But the scope reveals exactly what you've described - an extra byte being read for a total of 3 bytes read instead of 2. The 3rd byte read comes back as 0x32 which is the value the datasheet indicates will be returned from the MAG_ID register (addr 0x02).
I also tested communicating with a Maxbotix MB1242 sensor, which is a device that has no concept of registers. I send the ping command, waited for 50ms, and requested the 2-byte integer ranging result. Once again, a 3rd byte was read beyond the intended 2-byte payload.
Excellent :) Now when it is confirmed that is not just me, how long do you think it will take REV to fix it?
Not sure. But given that this is a firmware bug, and firmware releases don't happen nearly as often as SDK releases (last firmware release was September 2018) I wouldn't get my hopes up too much.
We got our new PixyCam2 (an I2C device that uses a request/response style communication on the bus instead of registers), coded up a basic I2C wrapper for the device following the advice here and using the very convenient new write(byte[]) and read(int) methods on I2cDeviceSynchSimple and ran into what we presume is this bug. Any use of read(n) seems to gobble up n+1 bytes so that a subsequent call to read(m) will be missing that n+1th byte at the start of the returned data. Given that the PixyCam2's communication model (like the examples above) involves a dynamically sized payload in the response, this is a problem.
Any chance this issue (open for >2 years) might be addressed? If not in the hub firmware, perhaps there is a workaround in the SDK where the extra byte might be read off the wire, cached, and returned on a subsequent call to read() until the firmware issue can be addressed?
Any chance this issue (open for >2 years) might be addressed? If not in the hub firmware, perhaps there is a workaround in the SDK where the extra byte might be read off the wire, cached, and returned on a subsequent call to read() until the firmware issue can be addressed?
My recollection is that the extra byte is lost inside the Hub firmware and never makes it over the wire to the SDK.
the extra byte is lost inside the Hub firmware and never makes it over the wire to the SDK.
Ahhh. :( We'll ping Rev.
The only command that I see where the length of response is actually variable is getMainFeatures(). For the rest (unless I missed some), the specification tells you how long the response to a particular command will be. For getMainFeatures(), your best bet is probably to just issue a 100-byte read (the max you can do in a single transaction) for the time being.
The only command that I see where the length of response is actually variable is getMainFeatures()
getBlocks() in the Color Connected Components portion of the interface has a dynamic aspect and is the one of interest for us.
your best bet is probably to just issue a 100-byte read (the max you can do in a single transaction) for the time being.
You raise an excellent point for a workaround however. There is probably little downside to attempting to read 100 bytes when something far less is there. It appears that the SDK just quietly pads out the missing bytes with 0xff. We saw that behavior when we tried a subsequent read and discovered earlier bytes were missing. We are also considering using the presence of a 2 byte checksum in these responses to continue with a 2 read approach, sacrificing one of the bytes in the checksum (and thus, not doing much of a checksum) in order to preserve the data integrity in the payload. We will test and respond here.
your best bet is probably to just issue a 100-byte read (the max you can do in a single transaction) for the time being.
So this worked for us. There is a measurable performance penalty for "reading" those bytes that are not there but it is also faster to read 40 bytes in a single read operation than to read 40 bytes in two 20 byte operations. Depending on what sort of data is coming back and how important it is, it's probably a good practice to aim for a total number of bytes that approximates what you are looking for instead of maxing it out at 100. For example, with the PixyCam2, we are interested in a list of detected "blocks". There is an option in the request to specify the maximum # of blocks to be returned. By capping this at a low number, we can predict the maximum # of bytes of relevant data that can be in a response and use that for our call to read(). Thanks for the help with the workaround for now.
Absolutely! All of your findings are as I'd expect, and specifying the max # of blocks sounds like an ideal way to handle this.