File write on ESP8266 using SPIFFS sometimes puts incorrect characters in file

[danny-levinson]

Some specific combinations of writes & seeks put DEL characters (0xff) into files, usually in place of CRLF (0x0d+0x0a). Using LittleFS instead of SPIFFS eliminates the problem.

Here is an example program demonstrating the problem behaviour:

#include "Arduino.h"

#include <FS.h>

#define USE_SPIFFS X
#ifdef USE_SPIFFS
#define FileSys SPIFFS
#else
#include <LittleFS.h>
#define FileSys LittleFS
#endif

void setup()
{
	/*
	 * write out 80 x's, then go back 6 characters and write 4 y's and a CR-LF
	 */
	FileSys.begin();
	File f = FileSys.open("/writetest.txt", "w");
	f.print("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx");
	f.seek(-6, SeekCur);
	f.print("yyyy\r\n");
	f.close();

	/*
	 * dump the last 10 characters as bytes in hex
	 */
	unsigned char buff[120] = "";
	char cbuff[120] = "";
	f = FileSys.open("/writetest.txt", "r");
	f.read(buff, 80);
	for (int i=70; i<80; i++) {
		sprintf(cbuff+(i-70)*3, "%02x ", buff[i]);
	}
	f.close();
	/*
	 * print the dump out to the serial monitor
	 */
	Serial.begin(115200);
	Serial.printf("output: |%s|\n", cbuff);
}

void loop()
{
	delay(100);
}

Incorrect output using SPIFFS (generated using Sloeber 4.4 IDE):

SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635
output: |78 78 78 78 79 79 79 79 ff ff |

Expected output produced using LittleFS

SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635
output: |78 78 78 78 79 79 79 79 0d 0a |

The problem behaviour manifests on both the Sloeber 4.4 IDE and on the Arduino (1.8.8) IDE. Incorrect data is output on the Arduino IDE with the esp8266 board package versions 3.0.0, 3.0.1 and 3.0.2, but not with 2.5.0 and 2.7.4. Also tested with two esp8266 boards in case one was defective.

I've switched to LittleFS now and I realise SPIFFS is deprecated so I am not expecting any action on this. I am just posting the issue as a warning. I would however appreciate knowing if someone else is able to replicate the problem.

[robertlipe]

I have no inside info, so this is just a random internet stranger shouting comments.

On DOS-based systems, if you didn't want it to corrupt your writes, you needed to open the reads in writes in binary as opposed to the default of text mode. That would be FILE* f = fopen("cowgoesmoo", "wb") instead of just ...fopen(name, "w").

Arduino-esp8266 doesn't mention "b" at https://arduino-esp8266.readthedocs.io/en/2.5.0/filesystem.html

but it's mentioned at https://forum.arduino.cc/t/storing-byte-array-in-spiffs/935343/5 and https://www.esp32.com/viewtopic.php?t=15781. Maybe that means this was addressed in a later ESP-IDF or Arduino-esp32 or maybe the doc is out of date. Either way, trying a a "b" in there to force binary mode sounds like an easy thing to try. That would fix what you're describing on DOS and, I think, Windows systems. I think UNIX-y systems assume by default that you don't want your data corrupted; they're stubborn like that.

But if this has been open for two years, I wonder if Spiffs is actively maintained anyway.

Good luck!