i2cdevlib icon indicating copy to clipboard operation
i2cdevlib copied to clipboard

FIFO overflow!

Open ajuancer opened this issue 7 years ago • 30 comments

Hi, after some problems i've been posting here, now i'm reading FIFO overflow in the serial monitor. As i have read the example code, i know that the code it very inefficient, so I decide to add mpu.resetFIFO(); after the line of code that i write, but the problem continues. I don't know what to do, someone know? (here is the code i aded) if (ypr[1]* 180/M_PI >= 5.4) { Serial.println("NICE"); printStringWithShift(mensaje1, 40); mpu.resetFIFO(); }

ajuancer avatar Oct 14 '18 16:10 ajuancer

same to me: (IDE 1.8.5, Adafruit Feather M4)

ypr 74.65 -12.55 9.63 ypr 74.60 -12.41 9.58 ypr 74.57 -12.36 9.58 ypr 74.52 -12.38 9.64 ypr 74.43 -12.46 9.71 ypr 74.19 -12.59 9.80 ypr 73.76 -12.80 10.05 ypr 73.24 -12.94 10.35 ypr 72.68 -12.93 10.59 ypr 72.08 -12.77 10.83 ypr 71.46 -12.47 11.08 ypr 70.82 -12.15 11.29 ypr 70.18 -11.85 11.48 ypr 69.54 -11.56 11.69 ypr 68.89 -11.29 11.93 FIFO overflow! ypr 49.25 1.73 15.39 ypr 48.20 2.31 15.72 ypr 47.27 2.92 15.99 ypr 46.50 3.35 16.27 ypr 45.88 3.68 16.48 ypr 45.30 3.98 16.57 ypr 44.57 4.38 16.81 ypr 43.99 4.66 16.82 ypr 43.22 4.84 16.93 ypr 42.32 5.02 16.97

^~~~~~~~~~~~~~~^ after this last line the program freezes completely, no Serial output any more at all. Pressing reset button does not the restart the program correctly then.

dsyleixa avatar Oct 15 '18 18:10 dsyleixa

this is my current code: (IDE 1.8.5, Adafruit Feather M4)

https://github.com/jrowberg/i2cdevlib/blob/master/Arduino/MPU6050/examples/MPU6050_DMP6/MPU6050_DMP6.ino

dsyleixa avatar Oct 15 '18 18:10 dsyleixa

I think the only one who can truly help us Mr. Rowberg (@jrowberg )

ajuancer avatar Oct 16 '18 19:10 ajuancer

@ developers / @jrowberg : will there soon be a FIFO overflow fix?

dsyleixa avatar Oct 18 '18 11:10 dsyleixa

PS: is the FIFO overflow caused by the i2cdevlib source code or by the MPU6050 hardware?

dsyleixa avatar Oct 20 '18 10:10 dsyleixa

I think it's caused by the source coee

ajuancer avatar Oct 20 '18 11:10 ajuancer

@jrowberg - what do you think?

dsyleixa avatar Oct 20 '18 12:10 dsyleixa

The FIFO overflow condition occurs when the IMU reports (via interrupt register read) that the FIFO has overflowed--note the use of MPU6050_INTERRUPT_FIFO_OFLOW_BIT in one of the if conditions. FIFO queuing is handled entirely by the IMU; it's not an application-level buffer.

When the FIFO overflows, fundamentally this means that the application cannot read data as fast as the IMU is generating it. Often this is caused by the CPU having too many other things to do between reads. It may be that the I2C data rate is too slow (100 kHz never helps, if the I2C clock is still set there). Monitoring the I2C data lines with a logic analyzer is the easiest way to see how often the reads are occurring and if they are using a lot of the CPU's processing time. Serial output is also often very slow, though this is less of a problem when using USB CDC for host output instead of hardware serial at slow baud rates. It is hard to say where other slow execution points may occur without detailed profiling tools.

You can reduce the DMP output frequency by increasing the rate divisor value inside the ..._MotionApps20.h file:

https://github.com/jrowberg/i2cdevlib/blob/900b8f959e9fa5c3126e0301f8a61d45a4ea99cc/Arduino/MPU6050/MPU6050_6Axis_MotionApps20.h#L272-L274

The data will be generated more slowly, but it will allow the CPU to keep up.

jrowberg avatar Oct 20 '18 14:10 jrowberg

I actually doubt that the CPU is too slow, because

  • I am just running the DMP6 example, nothing else,
  • I am running the i2c bus at 400 kHz as it's the original code
  • I am using a ARM Cortex M4 board at 120MHz cpu clock which is ~ 8x faster than AVR clock.

So if it does not get stuck on AVRs by FIFO overflow: why should it do that on an M4 ? Perhaps the FIFO thing is not the actual reason why the program hangs up after some seconds on my M4 as already assumed by me?

dsyleixa avatar Oct 20 '18 14:10 dsyleixa

Can you check the I2C lines with a logic analyzer?

jrowberg avatar Oct 20 '18 14:10 jrowberg

no, I don't have such a thing

dsyleixa avatar Oct 20 '18 14:10 dsyleixa

Can you check the voltage levels with a multimeter? If either is stuck at GND after the hang occurs, then the MPU itself may be misbehaving (which is an oft-reported issue, see #252).

jrowberg avatar Oct 20 '18 14:10 jrowberg

I'll check the voltage soon!

1st intermediate result: with this lib https://github.com/TKJElectronics/KalmanFilter it never hangs up, always runs fine (unfortunately no yaw output provided here)

dsyleixa avatar Oct 20 '18 15:10 dsyleixa

This library takes the raw accel/gyro data as inputs; to use it then, you must have the MPU-6050 operating in a very different mode (DMP inactive), so it is less likely that it will misbehave. In fact, I don’t recall ever seeing any reports of the IMU freezing in raw data mode.

Sent with GitHawk

jrowberg avatar Oct 20 '18 15:10 jrowberg

2nd intermediate result:

I had to migrate to another PC (a Intel i5, the former one was an older and slower core2duo E6550). On this new PC it currently runs fine during about 5 min without any FIFO issues on Serial monitor, strangely.

I'll see if I can reproduce the FIFO thing on the old PC when I'll be back there and then will report about voltage levels in case it hangs up again.

dsyleixa avatar Oct 20 '18 15:10 dsyleixa

Update: this "freezing" error showing me intermediate FIFO overflow on Serial console is always reproducable just at my old core2duo (Win7-64 pro, Serial.begin(155200), Arduino portable 1.8.5 ), attached to either PC USB port. When freezing there, both SDA and SCL have 3.28V voltage level

FTM, not reproducable though on my new PC (also Win7-64 pro, Serial.begin(155200), identical Arduino portable 1.8.5 ).

So as it concerns just to me, I assume it was perhaps a hardware issue (M4, UART, USB), not a i2cdevlib or a MPU_DMP6 source issue

  • of course no affecting the OP's issue report at the topic opening post at all or in either way.

dsyleixa avatar Oct 21 '18 08:10 dsyleixa

@jrowberg What do I have to add on line 272?

ajuancer avatar Oct 26 '18 21:10 ajuancer

@ajuancer You don’t have to add anything; just increase the divisor value from 0x01 to something bigger, like 0x02 or 0x03. The larger this value is, the slower the DMP will generate data.

jrowberg avatar Oct 26 '18 23:10 jrowberg

This is an interesting thread. I did experience overflow before and might check out line 272...wherever that is.

Sent from my iPhone

On Oct 27, 2018, at 10:52 AM, Jeff Rowberg [email protected] wrote:

@ajuancer You don’t have to add anything; just increase the divisor value from 0x01 to something bigger, like 0x02 or 0x03. The larger this value is, the slower the DMP will generate data.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

vicoz2009 avatar Oct 27 '18 01:10 vicoz2009

@jrowberg Nothing... I still have the same error, don't know what to do

ajuancer avatar Oct 28 '18 16:10 ajuancer

All the code I have write is here: https://github.com/ajuancer/smart_bicycle/blob/master/move-matrix.ino

ajuancer avatar Nov 04 '18 17:11 ajuancer

The FIFO overflow problem finally went away with setting #define MPU6050_DMP_FIFO_RATE_DIVISOR 0x04 but there still is an occasionally lock up for several seconds before data resumes. This is a Arduino Nano connected to a Windows machine with the Arduino IDE serial port.

john-bessire avatar Jan 01 '19 00:01 john-bessire

Interestingly, this happens on my breadboard setup (nano + MPU6050) too ...when the pitch or the roll is beyond a small initial value.

And more interestingly, if i touch (or bring my finger close enough) the insulation of the wire that connects INT on the MPU to the arduino interrupt pin - this problem does not happen !!

Need to explore this !

Update : Since the INT is pulled LOW, added a pull up resistor to the INT pin of the nano and the problem disappeared !!

madusudananr avatar Nov 04 '19 18:11 madusudananr

Try putting a very small capacitor (5 to 10 pf) between the INT signal line and ground. This will shunt any hi frequency EMI to ground.

From: Madusudanan Rajaraman [email protected] Sent: Monday, November 4, 2019 1:15 PM To: jrowberg/i2cdevlib [email protected] Cc: Subscribed [email protected] Subject: Re: [jrowberg/i2cdevlib] FIFO overflow! (#408)

Interestingly, this happens on my breadboard setup (nano + MPU6050) too ...when the pitch or the roll is beyond a small initial value.

And more interestingly, if i touch wire insulation that connects INT on the MPU to the arduino interrupt pin - this problem does not happen !!

Need to explore this !

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_jrowberg_i2cdevlib_issues_408-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAAQSKABD354INXKSX4SYI5LQSBRDVA5CNFSM4F3MZ3H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDAGPMI-23issuecomment-2D549480369&d=DwMCaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=K7d3gmAO5S_4tNJtGZTjqk_fX6BDwFy4YoYtJzD9Ds8&m=TXTEWNTqASsk6WmEHNTUrUPi2KooiuzuipJWbuuiN4Y&s=5-agP6G56do9iBX6AdaCJK50Dg1mxQ9xaj0xDwXuuNs&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAQSKAHU72LWM2LRW76JW5LQSBRDVANCNFSM4F3MZ3HQ&d=DwMCaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=K7d3gmAO5S_4tNJtGZTjqk_fX6BDwFy4YoYtJzD9Ds8&m=TXTEWNTqASsk6WmEHNTUrUPi2KooiuzuipJWbuuiN4Y&s=xmgEgDAq-ZT4YBK9MhT_obeyfmjX6wksE1IQe511Sks&e=.

jcrosbyME avatar Nov 04 '19 19:11 jcrosbyME

That was the first instinct - but I put a larger capacitor - which obviouslly wouldn't have worked - dumb of me.

Will try the small cap.
Update : Tried a 22pf cap. That didn't work - in fact the nano didn't receive a single interrupt.

Also the pullup resistor worked, although not sure if it is the right thing to do.

madusudananr avatar Nov 05 '19 04:11 madusudananr

I have been through hell over the last week with intermittent hangs and bad data in a deployed circuit (although everything worked perfectly on a small breadboard). I now firmly believe that nearly all the unreliability issues experienced by many people are due to electrical problems, not the I2CDev library or the MPU6050 library. The problems began for me when the connection between the Arduino and the MPU6050 was longer than 8-10cm (not very long really !).

The solution is so simple. Add a SERIES resistor in the SDL and SDA connections (near the Arduino). Resistance should be approx 30 - 200 Ohm (100 Ohm worked for me). The resistor damps 'ringing' on the lines and it is a known practice in electronic engineering forums. This is NOT THE PULLUP RESISTOR - it serves a different function. There can of course be other electrical problems but this is a great starting point.

I appreciate that this is a software discussion, but I think many people will save themselves some pain by trying this. If I2C communication is reliable then the timeout problem in i2cDev becomes secondary, since it is never needed.

elpidiovaldez avatar Dec 15 '19 05:12 elpidiovaldez

To All,

I agree! From what I have been reading on this forum the problem is electrical, not software. Especially when a bread board works with no motors involved and starts to behave badly when motors get involved.

I also use teensy processors which eliminates some of the 5 vs 3.3 problems. I realize that solution is not for everyone but having an Arm processor clocked at 96Mhz does help 😊

Again, my purpose is to make others think, not step on anyone’s toes. The power of these forums is getting many people to think and contribute not to have the “best answer”.

Series resistors on the SDL/SDA lines helps greatly but the Interrupt problem requires another approach but I am not positive what is the best method. I am seriously considering putting my processor and 6050 in a shielded can to help shield from rfi/efi, and adding shielding and capacitors to the power leads on the motors.

FWIW many project managers these days want to “fix it in software, we can’t change the hardware”. Unless it is a combined approach the bugs remain. So do we have a true electrical engineer out there that has experience in RFI shielding and prevention???

msaine

From: elpidiovaldez [email protected] Sent: Saturday, December 14, 2019 11:02 PM To: jrowberg/i2cdevlib [email protected] Cc: Subscribed [email protected] Subject: Re: [jrowberg/i2cdevlib] FIFO overflow! (#408)

I have been through hell over the last week with intermittent hangs and bad data in a deployed circuit (although everything worked perfectly on a small breadboard). I now firmly believe that nearly all the unreliability issues experienced by many people are due to electrical problems, not the I2CDev library or the MPU6050 library. The problems began for me when the connection between the Arduino and the MPU6050 was longer than 8-10cm (not very long really !).

The solution is so simple. Add a SERIES resistor in the SDL and SDA connections (near the Arduino). Resistance should be approx 30 - 200 Ohm (100 Ohm worked for me). The resistor damps 'ringing' on the lines and it is a known practice in electronic engineering forums. This is NOT THE PULLUP RESISTOR - it serves a different function. There can of course be other electrical problems but this is a great starting point.

I appreciate that this is a software discussion, but I think many people will save themselves some pain by trying this. If I2C communication is reliable then the timeout problem in i2cDev becomes secondary, since it is never needed.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/jrowberg/i2cdevlib/issues/408?email_source=notifications&email_token=AAWY7E6FPBEG5JQ4KHR7DBTQYW23DA5CNFSM4F3MZ3H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG4RQGY#issuecomment-565778459, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAWY7E747UCQITNOHS3NL6TQYW23DANCNFSM4F3MZ3HQ.

msaine avatar Dec 15 '19 16:12 msaine

Hi,

You might consider a 10pF to 20 pF capacitor from the interrupt line to ground to form a low pass filter. This will effectively shunt the typically higher frequency noise to ground. From what I have seen, this is a fairly common remedy to noise on a digital line. This value of the capacitor can be increased, but at some point it will act as a delay to the interrupt or distort the leading and trailing edges of the signal to a slope. If this works it explains why the breadboard worked when the final circuit didn’t as there is parasitic capacitance in the breadboard.

            -Jay Crosby

From: msaine [email protected] Sent: Sunday, December 15, 2019 11:01 AM To: jrowberg/i2cdevlib [email protected] Cc: Crosby, Jay [email protected]; Comment [email protected] Subject: Re: [jrowberg/i2cdevlib] FIFO overflow! (#408)

To All,

I agree! From what I have been reading on this forum the problem is electrical, not software. Especially when a bread board works with no motors involved and starts to behave badly when motors get involved.

I also use teensy processors which eliminates some of the 5 vs 3.3 problems. I realize that solution is not for everyone but having an Arm processor clocked at 96Mhz does help 😊

Again, my purpose is to make others think, not step on anyone’s toes. The power of these forums is getting many people to think and contribute not to have the “best answer”.

Series resistors on the SDL/SDA lines helps greatly but the Interrupt problem requires another approach but I am not positive what is the best method. I am seriously considering putting my processor and 6050 in a shielded can to help shield from rfi/efi, and adding shielding and capacitors to the power leads on the motors.

FWIW many project managers these days want to “fix it in software, we can’t change the hardware”. Unless it is a combined approach the bugs remain. So do we have a true electrical engineer out there that has experience in RFI shielding and prevention???

msaine

From: elpidiovaldez <[email protected]mailto:[email protected]> Sent: Saturday, December 14, 2019 11:02 PM To: jrowberg/i2cdevlib <[email protected]mailto:[email protected]> Cc: Subscribed <[email protected]mailto:[email protected]> Subject: Re: [jrowberg/i2cdevlib] FIFO overflow! (#408)

I have been through hell over the last week with intermittent hangs and bad data in a deployed circuit (although everything worked perfectly on a small breadboard). I now firmly believe that nearly all the unreliability issues experienced by many people are due to electrical problems, not the I2CDev library or the MPU6050 library. The problems began for me when the connection between the Arduino and the MPU6050 was longer than 8-10cm (not very long really !).

The solution is so simple. Add a SERIES resistor in the SDL and SDA connections (near the Arduino). Resistance should be approx 30 - 200 Ohm (100 Ohm worked for me). The resistor damps 'ringing' on the lines and it is a known practice in electronic engineering forums. This is NOT THE PULLUP RESISTOR - it serves a different function. There can of course be other electrical problems but this is a great starting point.

I appreciate that this is a software discussion, but I think many people will save themselves some pain by trying this. If I2C communication is reliable then the timeout problem in i2cDev becomes secondary, since it is never needed.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/jrowberg/i2cdevlib/issues/408?email_source=notifications&email_token=AAWY7E6FPBEG5JQ4KHR7DBTQYW23DA5CNFSM4F3MZ3H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG4RQGY#issuecomment-565778459, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAWY7E747UCQITNOHS3NL6TQYW23DANCNFSM4F3MZ3HQ.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_jrowberg_i2cdevlib_issues_408-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAAQSKAAOXYQQ7SOTDWBCICLQYZIFRA5CNFSM4F3MZ3H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG44GXI-23issuecomment-2D565822301&d=DwMFaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=K7d3gmAO5S_4tNJtGZTjqk_fX6BDwFy4YoYtJzD9Ds8&m=Xh36C1yAc9JliCsp2hETNIdZBSKoNNyqUk-Ai3-nNrk&s=Y9l79udOoWkkuF0qg7LizOPGJkquqsDY6qaqg5L_gsc&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAQSKAEHYANWP4TFZKJGZSTQYZIFRANCNFSM4F3MZ3HQ&d=DwMFaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=K7d3gmAO5S_4tNJtGZTjqk_fX6BDwFy4YoYtJzD9Ds8&m=Xh36C1yAc9JliCsp2hETNIdZBSKoNNyqUk-Ai3-nNrk&s=AH-6xm9gNB-MpTk1bPVY-ML2MsLg3tfjGJ8kY4UG8Kg&e=.

jcrosbyME avatar Dec 16 '19 15:12 jcrosbyME

Hi all,

It may be that the series resistors mentioned by Msaine are not actually working as series termination resistors but instead are blocking noise from being conducted down the I2C wires. This can be tested by replacing the series resistors with ferrite beads with a resistance of several hundred ohms at low frequencies in the MHz range eg Bourns Inc. FB20022-4B-RC.

While not offering anything over a simple series resistor on I2C interconnects, ferrite beads may also help on interrupt (in addition to the small capacitors suggested by Jay Crosby) and power wires where the power loss from a resistor cannot be tolerated.

Ferrite beads of this type are not really a production solution, but they may help in determining where noise is coming from and whether the problem is one of radiated or conducted noise.

Cheers, Ralph

ralphs99 avatar Dec 16 '19 16:12 ralphs99

Series resistors on the SCL and SDA lines made a huge difference to the reliability for me on a bread board implementatation, but eventually the system hung again. I tried putting a series resistor into the INT line too on the grounds that it might also suffer from ringing. That made the breadboard implementation perfectly reliable. Unfortunately when I put the series resistors on the circuit board, in the robot, the system was still unreliable (to avoid doubt there were no motors or PWM in use during the testing). It beats me - the wires were shorter in this implementation than on the breadboard. Does anybody know if there should be a difference due to the resistors being at the Arduino side vs. the sensor side of the connections ?

I cannot believe how difficult it is to use this sensor. I have spent a full week just building and rebuilding the circuit.

Edit: I finally have everything working reliably. Series resistors helped very well at times, but ultimately I think it comes down to the length of the connection between MPU6050 and microcontroller. Keep it REALLY short (< 3cm) and everything is fine, even without series resistors.

elpidiovaldez avatar Dec 17 '19 14:12 elpidiovaldez