OTA problem with async

[vatsake]

1. Choose a firmware for ESP
2. Start uploading the firmware
3. Let it upload for 1 second and then cancel the upload.
4. When you NOW upload a new firmware, the ESP wont boot at all.

[gibo77]

OTA will have issues if your 3.3Vdc power supply is not solid. Example flaky connections such as bread board and not enough amperage. The Wifi needs more power when doing OTA.

I found this the hard way.

[vatsake]

OTA will have issues if your 3.3Vdc power supply is not solid. Example flaky connections such as bread board and not enough amperage. The Wifi needs more power when doing OTA.

I found this the hard way.

I don't think it's a power issue, because it finishes the upload. After ESP.restart() it doesn't boot. I would understand if it crashes during the update.

[vatsake]

Is there a way to get upload status?

Maybe if I could call Update.end() when file aborted, the esp will function as normal.

else if(_authenticated && upload.status == UPLOAD_FILE_ABORTED){
        Update.end();
        if (_serial_output) Serial.println("Update was aborted");
}

[vatsake]

Can anyone else confirm this? I tried 5 different ESP8266s.

[vatsake]

11:28:56.038 -> Update Start: sketch_jun15b.ino.bin
//Here I refreshed webpage and started upload again
11:29:03.069 -> Update Start: sketch_jun15b.ino.bin
11:29:03.069 -> ERROR[0]: No Error
11:29:09.577 -> Update Success: 361600B
11:29:09.577 -> Rebooting...
11:29:09.809 -> 
11:29:09.809 ->  ets Jan  8 2013,rst cause:2, boot mode:(3,7)
11:29:09.809 -> 
11:29:09.809 -> load 0x4010f000, len 3460, room 16 
11:29:09.856 -> tail 4
11:29:09.856 -> chksum 0xcc
11:29:09.856 -> load 0x3fff20b8, len 40, room 4 
11:29:09.856 -> tail 4
11:29:09.856 -> chksum 0xc9
11:29:09.856 -> csum 0xc9
11:29:09.856 -> v00058480
11:29:09.856 -> @cp:B0
11:29:17.965 -> ld
11:29:17.965 -> e:2
11:29:17.965 ->  ets Jan  8 2013,rst cause:3, boot mode:(3,7)
11:29:17.965 -> 
11:29:17.965 -> ets_main.c 

And now ESP wont come up at all. Pleease help.

[zekageri]

In my understanding the OTA should compare the hash of the two firmware. If the new firmware hash is broken it will load the older one and boot up. Can we see your update code?

[vatsake]

I used the same as in the example. - Raw copy paste.

[zekageri]

I can see the problem. My sketch can not handle the refresh of the firmware update page too. I'm interested in a solution too. We should implement some kind of a timeout to the packets and close the request and end the update too...

[vatsake]

I can see the problem. My sketch can not handle the refresh of the firmware update page too. I'm interested in a solution too. We should implement some kind of a timeout to the packets and close the request and end the update too...

Thank you! Atleast now I know the problem isn't with my schematics.

[zekageri]

My sketch looks like this:

server.on("/newFirmware", HTTP_POST, [](AsyncWebServerRequest *request) {},
        [](AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len, bool final) {
            byte errCode = 1;
            if (!index) {
                hsh_Server.firmwareUpdating = true;
                int fileSize = UPDATE_SIZE_UNKNOWN;

                if (request->hasArg("fileSize")) {
                    fileSize = request->arg("fileSize").toInt();
                }

                if (!Update.begin(fileSize, U_FLASH)) {
                    #if SS_DEBUG_MODE
                        Serial.println("Update begin failed!");
                    #endif
                    errCode = 2;
                    final = true;
                }
            }

            if (Update.write(data, len) != len) {
                #if SS_DEBUG_MODE
                    Serial.println("Update write failed!");
                #endif
                errCode = 3;
                final = true;
            }

            if (final) {
                if (errCode == 2) {
                    request->send(500, "text/plain", "Update begin failed! Size mismatch or invalid file!");
                } else if (errCode == 3) {
                    request->send(500, "text/plain", "Update write failed!");
                    hsh_Server.sendFirmwareProgress(100, true, false);
                    hsh_Server.firmwareUpdating = false;
                } else if (!Update.end(true)) {
                    AsyncWebServerResponse *response = request->beginResponse(500, "text/plain", "Failed");
                    hsh_Server.sendFirmwareProgress(100, true, false);
                    request->send(response);
                    hsh_Server.firmwareUpdating = false;
                } else {
                    request->onDisconnect(firmwareRespHandler);
                    AsyncWebServerResponse *response = request->beginResponse(200, "text/plain", "Succeded");
                    hsh_Server.sendFirmwareProgress(100, true, true);
                    request->send(response);
                }
            };
        });

What i'm thinking on:

#define FIRMWARE_UPDATE_TIMEOUT 1000
long lastFirmwareUpdt_Packet_MS = 0;

void checkFirmwareUpdate_Timeout(){
    if( hsh_Server.firmwareUpdating && millis() - lastFirmwareUpdt_Packet_MS >= FIRMWARE_UPDATE_TIMEOUT ){
        Update.end();
        hsh_Server.firmwareUpdating = false;
    }
}

server.on("/newFirmware", HTTP_POST, [](AsyncWebServerRequest *request) {},
        [](AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len, bool final) {
            lastFirmwareUpdt_Packet_MS = millis();
            byte errCode = 1;
            if (!index) {
                hsh_Server.firmwareUpdating = true;
                int fileSize = UPDATE_SIZE_UNKNOWN;

                if (request->hasArg("fileSize")) {
                    fileSize = request->arg("fileSize").toInt();
                }

                if (!Update.begin(fileSize, U_FLASH)) {
                    #if SS_DEBUG_MODE
                        Serial.println("Update begin failed!");
                    #endif
                    errCode = 2;
                    final = true;
                }
            }

            if (Update.write(data, len) != len) {
                #if SS_DEBUG_MODE
                    Serial.println("Update write failed!");
                #endif
                errCode = 3;
                final = true;
            }

            if (final) {
                if (errCode == 2) {
                    request->send(500, "text/plain", "Update begin failed! Size mismatch or invalid file!");
                } else if (errCode == 3) {
                    request->send(500, "text/plain", "Update write failed!");
                    hsh_Server.sendFirmwareProgress(100, true, false);
                    hsh_Server.firmwareUpdating = false;
                } else if (!Update.end(true)) {
                    AsyncWebServerResponse *response = request->beginResponse(500, "text/plain", "Failed");
                    hsh_Server.sendFirmwareProgress(100, true, false);
                    request->send(response);
                    hsh_Server.firmwareUpdating = false;
                } else {
                    request->onDisconnect(firmwareRespHandler);
                    AsyncWebServerResponse *response = request->beginResponse(200, "text/plain", "Succeded");
                    hsh_Server.sendFirmwareProgress(100, true, true);
                    request->send(response);
                }
            };
        });

void loop(){
    checkFirmwareUpdate_Timeout();
}

[zekageri]

Here is a simplified example. I'm testing it now.

#define FIRMWARE_UPDATE_TIMEOUT 1000
long lastFirmwareUpdt_Packet_MS = 0;
boolean firmwareUpdating = true;

void checkFirmwareUpdate_Timeout(){
    if( firmwareUpdating && millis() - lastFirmwareUpdt_Packet_MS >= FIRMWARE_UPDATE_TIMEOUT ){
        Update.end();
        firmwareUpdating = false;
    }
}

server.on("/newFirmware", HTTP_POST, [](AsyncWebServerRequest *request) {},
        [](AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len, bool final) {
            lastFirmwareUpdt_Packet_MS = millis();
            if (!index) {
                firmwareUpdating = true;
                int fileSize = UPDATE_SIZE_UNKNOWN;

                if (request->hasArg("fileSize")) {
                    fileSize = request->arg("fileSize").toInt();
                }

                if ( !Update.begin(fileSize, U_FLASH) ) {
                    Serial.println("Update begin failed!");
                    request->send(500, "text/plain", "Update failed!");
                    Update.end();
                    firmwareUpdating = false;
                }
            }

            if ( Update.write(data, len) != len ) {
                request->send(500, "text/plain", "Update failed!");
                Update.end();
                firmwareUpdating = false;
            }

            if (final) {
                if( !Update.end(true) ){
                    request->send(500, "text/plain", "Update write failed!");
                    Update.end();
                    firmwareUpdating = false;
                }else{
                    request->send(500, "text/plain", "Update write failed!");
                    Update.end();
                    firmwareUpdating = false;
                }
            }
        });

void loop(){
    checkFirmwareUpdate_Timeout();
}

[vatsake]

But what if you call Update.end() in

if (!index) {

}

?

[zekageri]

It will end the update before it can begin. This line is executed on the start of the update.

[zekageri]

Thinking about this:

void firmwareUpdateEnd( boolean isSuccess, AsyncWebServerRequest *request = NULL );

#define FIRMWARE_UPDATE_TIMEOUT 2000

boolean firmwareUpdating  = false;
long lastFirmwareUpdt_Packet_MS  = 0;

void checkFirmwareUpdate_Timeout(){
    if( firmwareUpdating && millis() - lastFirmwareUpdt_Packet_MS >= FIRMWARE_UPDATE_TIMEOUT ){
        firmwareUpdateEnd(false);
    }
}

void firmwareUpdateEnd( boolean isSuccess, AsyncWebServerRequest *request ){
    firmwareUpdating = false;
    sendFirmwareProgress(100, true, isSuccess); // <-- used to indicate the progress for the client via websockets.
    Update.end();
    if( isSuccess && request != NULL ){
        request->send(200, "text/plain", "Success");
    }else if( request != NULL ) {
        request->send(500, "text/plain", "Failed!");
    }
}

void firmwareHandler() {
    server.on("/newFirmware", HTTP_POST, [](AsyncWebServerRequest *request) {},
        [](AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len, bool final) {
            hsh_Server.lastFirmwareUpdt_Packet_MS = millis();

            if (!index) {
                hsh_Server.firmwareUpdating = true;
                int fileSize = UPDATE_SIZE_UNKNOWN;

                if (request->hasArg("fileSize")) {
                    fileSize = request->arg("fileSize").toInt();
                }

                if ( !Update.begin( fileSize, U_FLASH ) ) {
                    firmwareUpdateEnd(false);
                    return;
                }
            }

            if (Update.write(data, len) != len) {
                firmwareUpdateEnd(false);
                return;
            }

            if (final) {
                if (!Update.end(true)) {
                    firmwareUpdateEnd(false, request);
                }else{
                    firmwareUpdateEnd(true, request);
                }
            }
        });
}

void loop(){
    checkFirmwareUpdate_Timeout();
}

[vatsake]

But what if you call Update.end() in

if (!index) {

}

?

i Just tried it like this, it worked also. -- if you start uploading second time, it will cancel the previous ota.

[zekageri]

So you did not specify anything other then this:

if (!index) {
    Update.end();
}

?

[vatsake]

server.on("/update", HTTP_POST, [](AsyncWebServerRequest *request){
    shouldReboot = !Update.hasError();
    AsyncWebServerResponse *response = request->beginResponse(200, "text/plain", shouldReboot?"OK":"FAIL");
    response->addHeader("Connection", "close");
    request->send(response);
  },[](AsyncWebServerRequest *request, String filename, size_t index, uint8_t *data, size_t len, bool final){
    if(!index){
      Update.end();
      Serial.printf("Update Start: %s\n", filename.c_str());
      Update.runAsync(true);
      if(!Update.begin((ESP.getFreeSketchSpace() - 0x1000) & 0xFFFFF000)){
        Update.printError(Serial);
      }
    }
    if(!Update.hasError()){
      if(Update.write(data, len) != len){
        Update.printError(Serial);
      }
    }
    if(final){
      if(Update.end(true)){
        Serial.printf("Update Success: %uB\n", index+len);
      } else {
        Update.printError(Serial);
      }
    }
  });

[zekageri]

I see. But this approach can not cover other scenarios. What if you set a flag to indicate that the firmware is updating to the rest of your program. It will stay true if you not refresh the webpage and start a new update but anything else.

[zekageri]

I did it like this, just in case. ( I'm on esp32 btw )

void firmwareRespHandler() {
    hsh_Performance.beginRestart();
}

void sSystem::checkFirmwareUpdate_Timeout(){
    if( firmwareUpdating && millis() - lastFirmwareUpdt_Packet_MS >= FIRMWARE_UPDATE_TIMEOUT ){
        firmwareUpdateEnd(false);
    }
}

void sSystem::firmwareUpdateEnd( boolean isSuccess, AsyncWebServerRequest *request ){
    firmwareUpdating = false;
    sendFirmwareProgress(100, true, isSuccess);
    Update.end();
    if( isSuccess && request != NULL ){
        request->onDisconnect(firmwareRespHandler);
        request->send(200, "text/plain", "Success");
    }else if( request != NULL ) {
        request->send(500, "text/plain", "Failed!");
        hshDisplay.addNoty("Failed to update firmware!");
    }
}

void sSystem::firmwareHandler() {
    server.on("/newFirmware", HTTP_POST, [](AsyncWebServerRequest *request) {},
        [](AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len, bool final) {
            hsh_Server.lastFirmwareUpdt_Packet_MS = millis();

            if (!index) {
                Update.end();
                hsh_Server.firmwareUpdating = true;
                int fileSize = UPDATE_SIZE_UNKNOWN;

                if (request->hasArg("fileSize")) {
                    fileSize = request->arg("fileSize").toInt();
                }

                if ( !Update.begin( fileSize, U_FLASH ) ) {
                    hsh_Server.firmwareUpdateEnd(false);
                    return;
                }
            }

            if (Update.write(data, len) != len) {
                hsh_Server.firmwareUpdateEnd(false);
                return;
            }

            if (final) {
                if (!Update.end(true)) {
                    hsh_Server.firmwareUpdateEnd(false, request);
                }else{
                    hsh_Server.firmwareUpdateEnd(true, request);
                }
            }
        });
}

void loop(){
    checkFirmwareUpdate_Timeout();
}

[vatsake]

I'm no 100% sure, but the traditional ota returns error if user cancels update (ERR::USER_CANCEL or something), but asyncwebserver doesn't. Probably has some timeout feature. Asyncwebserver should implement this as well :)

[zekageri]

Nah, i think it is out of scope for the webserver. It is two different library. You can use the Update class with any other thing. I have a NodeJS server where i store firmwares. The ESP can automatically check for new firmware by itself on the server and pulls it for itself via a regular HTTPS request. And it uses Update too. Not related to Async webserver.

[vatsake]

But asyncwebserver should atleast have some kind of event that notifies that the upload (any file upload) is cancelled. Ordinary webserver has

else if(_authenticated && upload.status == UPLOAD_FILE_ABORTED){
        Update.end();
        if (_serial_output) Serial.println("Update was aborted");
      }

[stale[bot]]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[vatsake]

But asyncwebserver should atleast have some kind of event that notifies that the upload (any file upload) is cancelled. Ordinary webserver has

else if(_authenticated && upload.status == UPLOAD_FILE_ABORTED){
        Update.end();
        if (_serial_output) Serial.println("Update was aborted");
      }

no stale!, plz add this.

[zekageri]

+1

[stale[bot]]

[STALE_CLR] This issue has been removed from the stale queue. Please ensure activity to keep it openin the future.

[stale[bot]]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.