hassio-nextcloud-backup icon indicating copy to clipboard operation
hassio-nextcloud-backup copied to clipboard
verified publisher icon Sebclem

Unable to upload backup above 1G in size :(

Open vincegre opened this issue 3 years ago • 8 comments

A clear and concise description of what the bug is:

extension has worked fine till backup size has passed 1Go in size (same problem on 2 HA instances and different Nextcloud servers). If I upload myself the backup on the NC it should go I have no problem either in web or using Nextcloud sync client. Problem is independant of NC version (problem happens with NC 21/22/23

Steps to reproduce the behavior:

happens all the time as soon as backup file goes above 1Go

A clear and concise description of what you expected to happen.

backup to continue to work as it was in the past !

System information

  • Home Assistant Version : 2022.4.0
  • Home Assistant Supervisor Version : 2022.04.0
  • Nextcloud Version : 23
  • Addon version : 0.16.8
  • If any, reverse proxy in fort on Nextcloud : no reverse proxy on NC instances involved

Logs in HA during a backup:

[12:57:56] INFO: Starting Node...
> [email protected] start
> node ./bin/www
[2022-04-08 12:57:58] [info]: 	Log level: debug
[2022-04-08 12:57:58] [info]: 	Backup timeout: 5400000
[2022-04-08 12:57:58] [info]: 	Satus : Go !
[2022-04-08 12:57:58] [info]: 	Initializing and checking webdav client...
[2022-04-08 12:57:58] [info]: 	Starting auto clean cron...
[2022-04-08 12:57:58] [info]: 	Starting Cron...
[2022-04-08 12:57:58] [info]: 	Hassio API : Go !
[2022-04-08 12:57:59] [debug]: 	Nextcloud connection:  Success !
[2022-04-08 12:57:59] [debug]: 	Path /Config_softs/ already exist.
[2022-04-08 12:57:59] [debug]: 	Path /Config_softs/HA/ already exist.
[2022-04-08 12:57:59] [info]: 	Nextcloud connection : Go !
[2022-04-08 12:58:27] [info]: 	Stopping addons...
[2022-04-08 12:58:27] [info]: 	... Ok
[2022-04-08 12:58:27] [info]: 	Creating new snapshot...
[2022-04-08 12:58:27] [debug]: 	Addon to backup:
[2022-04-08 12:58:27] [debug]: 	a0d7b954_adguard,core_check_config,5c53de3b_esphome,core_configurator,c4d25c74_edge,a0d7b954_glances,a0d7b954_grocy,core_mariadb,core_mosquitto,229cc4d7_nextcloud_backup,a0d7b954_phpmyadmin,core_samba,core_ssh,a0d7b954_unifi
[2022-04-08 12:58:27] [debug]: 	Folders to backup:
[2022-04-08 12:58:27] [debug]: 	homeassistant,ssl,share,media,addons/local
[2022-04-08 13:16:07] [info]: 	Snapshot created with id c795316f
[2022-04-08 13:16:07] [info]: 	Downloading snapshot c795316f...
[2022-04-08 13:16:07] [debug]: 	Snapshot size: 2081.25
[2022-04-08 13:16:49] [info]: 	Download success !
[2022-04-08 13:16:49] [debug]: 	Snapshot dl size : 2081.25
[2022-04-08 13:16:52] [info]: 	Uploading snap...
[2022-04-08 13:16:52] [debug]: 	...URI: https://host.hiden/remote.php/webdav/Config_softs/HA/Manual/Manual-2022.4.1-2022-04-08_1258.tar
[2022-04-08 13:16:52] [debug]: 	...rejectUnauthorized: true
[2022-04-08 13:17:58] [error]: 	Fail to upload snapshot to nextcloud (HTTPError: Response code 500 (Internal Server Error)) !
[2022-04-08 13:17:58] [error]: 	HTTPError: Response code 500 (Internal Server Error)
    at Request._onResponseBase (/opt/nextcloud_backup/node_modules/got/dist/source/core/index.js:899:31)
    at Request._onResponse (/opt/nextcloud_backup/node_modules/got/dist/source/core/index.js:934:24)
    at ClientRequest.<anonymous> (/opt/nextcloud_backup/node_modules/got/dist/source/core/index.js:948:23)
    at Object.onceWrapper (events.js:520:26)
    at ClientRequest.emit (events.js:412:35)
    at ClientRequest.origin.emit (/opt/nextcloud_backup/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)
    at HTTPParser.parserOnIncomingClient (_http_client.js:647:27)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:127:17)
    at TLSSocket.socketOnData (_http_client.js:515:22)
    at TLSSocket.emit (events.js:400:28)
(node:352) UnhandledPromiseRejectionWarning: Fail to upload snapshot to nextcloud (HTTPError: Response code 500 (Internal Server Error)) !
(Use `node --trace-warnings ...` to show where the warning was created)
(node:352) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:352) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Side note: why logs show a non-sense URL (https://host.hiden) for upload that is not the one setup in extension ??

Thanks

Vincèn

vincegre avatar Apr 08 '22 11:04 vincegre

Hi,

I think it's related to your Nextcloud setup, did you find anything in your Nextcloud logs ?

Ps: the hidden.host is just something I have added for privacy when you share your logs here

Sebclem avatar Apr 08 '22 11:04 Sebclem

I think it's related to your Nextcloud setup, did you find anything in your Nextcloud logs ?

Well as stated in my bug report I have no problem to upload files at these NC instances either with the synchronisation client or through the web interface that are quite larger than HA backups so I don't think NC is the problem !

All I could find in NC but not sure it's related :/ Do you use the lib axios for NC access ??

{"reqId":"7WwiOracS5sqgHQjARAd","level":0,"time":"2022-04-08T11:17:12+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"7WwiOracS5sqgHQjARAd","level":0,"time":"2022-04-08T11:17:12+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"7WwiOracS5sqgHQjARAd","level":0,"time":"2022-04-08T11:17:12+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"j3rQvQlRjouqFpP3EWZc","level":0,"time":"2022-04-08T11:17:13+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"j3rQvQlRjouqFpP3EWZc","level":0,"time":"2022-04-08T11:17:13+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"j3rQvQlRjouqFpP3EWZc","level":0,"time":"2022-04-08T11:17:13+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"XgwrCNqweCHiBIGvNlyq","level":0,"time":"2022-04-08T11:17:17+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"XgwrCNqweCHiBIGvNlyq","level":0,"time":"2022-04-08T11:17:17+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"XgwrCNqweCHiBIGvNlyq","level":0,"time":"2022-04-08T11:17:17+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"pAc5Tn5Es5Ur66KF4d8X","level":0,"time":"2022-04-08T11:17:18+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"pAc5Tn5Es5Ur66KF4d8X","level":0,"time":"2022-04-08T11:17:18+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"pAc5Tn5Es5Ur66KF4d8X","level":0,"time":"2022-04-08T11:17:18+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"9HkhbjnDSe1OkT0iIfa2","level":0,"time":"2022-04-08T11:17:22+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"9HkhbjnDSe1OkT0iIfa2","level":0,"time":"2022-04-08T11:17:22+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"9HkhbjnDSe1OkT0iIfa2","level":0,"time":"2022-04-08T11:17:22+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"ACong6b5pZBi5QadlHwt","level":0,"time":"2022-04-08T11:17:23+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"ACong6b5pZBi5QadlHwt","level":0,"time":"2022-04-08T11:17:23+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"ACong6b5pZBi5QadlHwt","level":0,"time":"2022-04-08T11:17:23+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"CXCpnaVrFFa97tjN27XG","level":0,"time":"2022-04-08T11:17:27+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"CXCpnaVrFFa97tjN27XG","level":0,"time":"2022-04-08T11:17:27+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"CXCpnaVrFFa97tjN27XG","level":0,"time":"2022-04-08T11:17:27+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"Zo1GY4KIU2Syc0doBAA4","level":0,"time":"2022-04-08T11:17:28+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"Zo1GY4KIU2Syc0doBAA4","level":0,"time":"2022-04-08T11:17:28+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"Zo1GY4KIU2Syc0doBAA4","level":0,"time":"2022-04-08T11:17:28+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"8NyS2Hab0MmriC64c4iJ","level":0,"time":"2022-04-08T11:17:32+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"8NyS2Hab0MmriC64c4iJ","level":0,"time":"2022-04-08T11:17:32+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"8NyS2Hab0MmriC64c4iJ","level":0,"time":"2022-04-08T11:17:32+00:00","remoteAddr":"51.154.57.249","user":"--","app":"maps","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Manual/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"4pOnuylU8TxSQXvH3KJd","level":0,"time":"2022-04-08T11:17:33+00:00","remoteAddr":"51.154.57.249","user":"--","app":"camerarawpreviews","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"axios/0.24.0","version":"23.0.3.2"}
{"reqId":"4pOnuylU8TxSQXvH3KJd","level":0,"time":"2022-04-08T11:17:33+00:00","remoteAddr":"51.154.57.249","user":"--","app":"serverDI","method":"PROPFIND","url":"/remote.php/webdav/Config_softs/HA/Auto/","message":"The requested alias \"PreviewManager\" is deprecated. Please request \"OCP\\IPreview\" directly. This alias will be removed in a future Nextcloud version.","userAgent":"axios/0.24.0","version":"23.0.3.2"}

Ps: the hidden.host is just something I have added for privacy when you share your logs here

I better understand and may I suggest to replace that very bad taste of fake url by something easier like "yourhosthere» or "hidden for confidentiality" for example ?

vincegre avatar Apr 08 '22 12:04 vincegre

Hi, I experience the same issue. I can download the backups from Home Assistant and upload them via the Web UI, the same endpoint effectively the addon would use. But the addon return 413. I could not find a related entry in the nextcloud logs. Nextcloud is running behind the Nginx Proxy Manager, but it does not matter if I use the external accessible URL or the internal IP address. Also the nextcloud system information shows the correct limits (16 gb in my instance), but only the addon is not able to upload 1.6 or 1.8 gb files.

vdwals avatar Oct 26 '22 07:10 vdwals

Hi, I experience the same issue. I can download the backups from Home Assistant and upload them via the Web UI, the same endpoint effectively the addon would use. But the addon return 413. I could not find a related entry in the nextcloud logs. Nextcloud is running behind the Nginx Proxy Manager, but it does not matter if I use the external accessible URL or the internal IP address. Also the nextcloud system information shows the correct limits (16 gb in my instance), but only the addon is not able to upload 1.6 or 1.8 gb files.

For 413 related error, please see #9

Sebclem avatar Oct 26 '22 08:10 Sebclem

i had the same error with HTTP 413 Error. I used the official nextcloud docker image. My Nextcloud runs with apache webserver behind a traeffic proxy and i must add LimitRequestBody 0 in apache webserver configuration. i do this with a separate docker volume /docker/nextcloud/nextcloud.conf:/etc/apache2/conf-enabled/nextcloud.conf

now it works

sippeeey avatar Jun 07 '23 09:06 sippeeey