[Bug]: NC 26 overwrites `storage_id` with `mount_id` in `mounts` table causing various issues with shares and mounts down the line
⚠️ This issue respects the following points: ⚠️
- [X] This is a bug, not a question or a configuration/webserver/proxy issue.
- [X] This issue is not already reported on Github (I've searched it).
- [X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- [X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
- [X] I agree to follow Nextcloud's Code of Conduct.
Bug description
After the update to Nextcloud 26, all storage_id fields in the mounts table are overridden with the mount_id value, if it exists.
This causes a multitude of issues down the line:
- Shares in external storages don't work
- Some shares in non-external storages don't work if the
numeric_idof that storage happens to line up with amount_idof an external storage - Internal links don't work for some folders with the above characteristics.
- Deleting the mount can cause cache deletion of the whole main storage
- Causes #37414 (see the discussion over there for details)
I tried manually changing back the storage_id field back to the correct value, and it was changed back to the incorrect one immediately
Steps to reproduce
- Have some normal storage setup
- Create and mount an external storage as a user
- Check the Database
(Not tested)
Expected behavior
storage_id fields in the mounts table contain the numeric_id of the storages table that they belong to.
Installation method
Community Web installer on a VPS or web space
Nextcloud Server version
26
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.1
Web server
Apache (supported)
Database engine version
MySQL
Is this bug present after an update or on a fresh install?
Updated to a major version (ex. 22.2.3 to 23.0.1)
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- [X] Default user-backend (database)
- [ ] LDAP/ Active Directory
- [ ] SSO - SAML
- [ ] Other
Configuration report
No response
List of activated Apps
Enabled:
- activity: 2.18.0
- admin_audit: 1.16.0
- bruteforcesettings: 2.6.0
- calendar: 4.3.1
- camerarawpreviews: 0.8.1
- circles: 26.0.0
- cloud_federation_api: 1.9.0
- comments: 1.16.0
- contacts: 5.2.0
- contactsinteraction: 1.7.0
- cospend: 1.5.8
- dashboard: 7.6.0
- dav: 1.25.0
- federatedfilesharing: 1.16.0
- federation: 1.16.0
- files: 1.21.1
- files_external: 1.18.0
- files_pdfviewer: 2.7.0
- files_rightclick: 1.5.0
- files_sharing: 1.18.0
- files_trashbin: 1.16.0
- files_versions: 1.19.1
- firstrunwizard: 2.15.0
- logreader: 2.11.0
- lookup_server_connector: 1.14.0
- nextcloud_announcements: 1.15.0
- notes: 4.7.2
- notifications: 2.14.0
- oauth2: 1.14.0
- password_policy: 1.16.0
- photos: 2.2.0
- previewgenerator: 5.2.1
- privacy: 1.10.0
- provisioning_api: 1.16.0
- recommendations: 1.5.0
- related_resources: 1.1.0-alpha1
- serverinfo: 1.16.0
- settings: 1.8.0
- sharebymail: 1.16.0
- support: 1.9.0
- survey_client: 1.14.0
- suspicious_login: 4.4.0
- systemtags: 1.16.0
- tasks: 0.14.5
- text: 3.7.2
- theming: 2.1.1
- twofactor_backupcodes: 1.15.0
- updatenotification: 1.16.0
- user_status: 1.6.0
- viewer: 1.10.0
- weather_status: 1.6.0
- workflowengine: 2.8.0
Disabled:
- twofactor_totp: 8.0.0-alpha.0
- user_ldap: 1.16.0
Nextcloud Signing status
No response
Nextcloud Logs
No response
Additional info
No response
Cc @icewind1991
Is there any work/progress ongoing in this, which is not visible here?
26.0.1 seems to fix it (at least for us)
This is not fixed, but it is quite a breaking bug. A fix would be highly appreciated!
After upgrade to NC 26.0.1 one user "userA" with external storage enabled now sees the activity stream of another user "user B"... E.g. user B creates a new file within his root storage (which is not shared), user A sees this file creation within his activity stream as created under his external storage (but this file does not exist within the external storage and is not accessible for him).
I assume this must be caused by this issue.
After double checking I got further feedback from users of my nextcloud instance who now see the activity of other nextcloud users. For those cases I can confirm that numeric_id (oc_storages) and storage_id and mount_id (oc_mounts) are identical. To circumvent this issue I thought it might be a good idea to just remove the external mounts of these individual users and add them again globally (admin account). But by doing so, the numeric_id (oc_storages) of the affected storage was reassigned and all existing shares of this storage are no gone!
This issue seems to leak private data (activity stream) and can cause data lost (by means of lost shares).
@icewind1991 isn't it required to raise the importance of this issue?
@szaimen @icewind1991 what is the progress on this issue? If necessary, I can provide more details / do more investigation on my side
I cannot reproduce this issue locally.
I tried manually changing back the storage_id field back to the correct value, and it was changed back to the incorrect one immediately
When the storage_id is changed back to the incorrect value, does the id column change?
Can you try applying the following debug patch: https://github.com/nextcloud/server/commit/1ae6e04cf1f1f9ce450eb21334535825134647f6
And provide the generated log items from when the incorrect values are written.
The id column appears not to change.
Here's the logs:
[no app in context] Warnung: Exception: updating mount info at /Daniel/files/Photos/[Mountpoint]/ to {"storage_id":3,"root_id":402117,"user_id":"Daniel","mount_point":"\/Daniel\/files\/Photos\/[Mountpoint]\/","mount_id":3,"mount_provider_class":"OCA\\Files_External\\Config\\ConfigAdapter"} at <<closure>>
0. /var/www/[Nextcloud folder]/lib/private/Files/Config/UserMountCache.php line 144
OC\Files\Config\UserMountCache->updateCachedMount("*** sensitive parameters replaced ***")
1. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 474
OC\Files\Config\UserMountCache->registerMounts()
2. /var/www/[Nextcloud folder]/lib/private/Files/Mount/Manager.php line 95
OC\Files\SetupManager->setupForPath()
3. /var/www/[Nextcloud folder]/lib/private/Files/View.php line 1390
OC\Files\Mount\Manager->find()
4. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/Directory.php line 216
OC\Files\View->getFileInfo()
5. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Tree.php line 78
OCA\DAV\Connector\Sabre\Directory->getChild()
6. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 971
Sabre\DAV\Tree->getNodeForPath()
7. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1662
Sabre\DAV\Server->getPropertiesIteratorForPath()
8. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1647
Sabre\DAV\Server->writeMultiStatus()
9. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 346
Sabre\DAV\Server->generateMultiStatus()
10. /var/www/[Nextcloud folder]/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPropFind()
11. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 472
Sabre\DAV\Server->emit()
12. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 253
Sabre\DAV\Server->invokeMethod()
13. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 321
Sabre\DAV\Server->start()
14. /var/www/[Nextcloud folder]/apps/dav/lib/Server.php line 366
Sabre\DAV\Server->exec()
15. /var/www/[Nextcloud folder]/apps/dav/appinfo/v2/remote.php line 35
OCA\DAV\Server->exec()
16. /var/www/[Nextcloud folder]/remote.php line 171
require_once("/var/www/[Nextcloud folder]/ ... p")
PROPFIND /remote.php/dav/files/Daniel/Uni2
from [IPv6] by Daniel at 2023-05-02T14:31:33+00:00
Here's a snapshot of some lines of the oc_mounts table compared to the backup from a couple of months ago:
| id | storage_id | root_id | user_id | mount_point | mount_id | mount_provider_class | |
|---|---|---|---|---|---|---|---|
| Backup | 572 | 16 | 14195 | Daniel | /Daniel/files/Photos/[Mountpoint]/ | 3 | NULL |
| Current | 610 | 3 | 402117 | Daniel | /Daniel/files/Photos/[Mountpoint]/ | 3 | OCA\Files_External\Config\ConfigAdapter |
| Backup | 413 | 9 | 7192 | Daniel | /Daniel/files/[other ext]/ | 2 | NULL |
| Current | 413 | 2 | 7192 | Daniel | /Daniel/files/[other ext]/ | 2 | OCA\Files_External\Config\ConfigAdapter |
Edit: Just realized that I actually know where the change in the id and root_id came from:
I manually updated this specific external drive BEFORE the update to NC26 because the remote path changed;
but the other ones have the same issue where mount_id and storage_id are identical.
It just so happens that this in particular is the mount with mount_id 3 which lines up with my personal storage being storage_id 3 and thus creating all the issues in my specific case
Edit 2: Updated the table to include another example from the DB where I didn't mess around with the share settings in recent months
In case this helps:
I tried manually changing the storage_id for the other mount (mount_id 2) as well,
and when it was changed back, Nextcloud also tried to add even more other lines to the DB that actually already existed..?
Logs
Multiple variations of this log:
[no app in context] Warnung: Exception: adding mount info at /[other User]/ to {"storage_id":4,"root_id":21,"user_id":"[other User]","mount_point":"\/[other User]\/","mount_id":null,"mount_provider_class":"OC\\Files\\Mount\\LocalHomeMountProvider"} at <<closure>>
0. /var/www/[Nextcloud folder]/lib/private/Files/Config/UserMountCache.php line 134
OC\Files\Config\UserMountCache->addToCache()
1. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 304
OC\Files\Config\UserMountCache->registerMounts()
2. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 230
OC\Files\SetupManager->afterUserFullySetup()
3. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 510
OC\Files\SetupManager->setupForUser()
4. /var/www/[Nextcloud folder]/lib/private/Files/Mount/Manager.php line 220
OC\Files\SetupManager->setupForProvider()
5. /var/www/[Nextcloud folder]/lib/private/Files/Node/Root.php line 448
OC\Files\Mount\Manager->getMountsByMountProvider()
6. /var/www/[Nextcloud folder]/lib/private/Files/Node/LazyUserFolder.php line 67
OC\Files\Node\Root->getByIdInPath()
7. /var/www/[Nextcloud folder]/apps/files_sharing/lib/SharedStorage.php line 146
OC\Files\Node\LazyUserFolder->getById()
8. /var/www/[Nextcloud folder]/apps/files_sharing/lib/SharedStorage.php line 532
OCA\Files_Sharing\SharedStorage->init()
9. /var/www/[Nextcloud folder]/lib/private/Files/Storage/Wrapper/Wrapper.php line 536
OCA\Files_Sharing\SharedStorage->getWrapperStorage()
10. /var/www/[Nextcloud folder]/lib/private/Files/Storage/Wrapper/Wrapper.php line 536
OC\Files\Storage\Wrapper\Wrapper->getDirectDownload()
11. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/File.php line 577
OC\Files\Storage\Wrapper\Wrapper->getDirectDownload()
12. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/FilesPlugin.php line 395
OCA\DAV\Connector\Sabre\File->getDirectDownload()
13. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/PropFind.php line 95
OCA\DAV\Connector\Sabre\FilesPlugin->OCA\DAV\Connector\Sabre\{closure}("*** sensitive parameters replaced ***")
14. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/FilesPlugin.php line 405
Sabre\DAV\PropFind->handle()
15. /var/www/[Nextcloud folder]/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
OCA\DAV\Connector\Sabre\FilesPlugin->handleGetProperties()
16. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1052
Sabre\DAV\Server->emit()
17. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 984
Sabre\DAV\Server->getPropertiesByNode()
18. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1662
Sabre\DAV\Server->getPropertiesIteratorForPath()
19. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1647
Sabre\DAV\Server->writeMultiStatus()
20. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 346
Sabre\DAV\Server->generateMultiStatus()
21. /var/www/[Nextcloud folder]/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPropFind()
22. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 472
Sabre\DAV\Server->emit()
23. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 253
Sabre\DAV\Server->invokeMethod()
24. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 321
Sabre\DAV\Server->start()
25. /var/www/[Nextcloud folder]/apps/dav/lib/Server.php line 366
Sabre\DAV\Server->exec()
26. /var/www/[Nextcloud folder]/apps/dav/appinfo/v2/remote.php line 35
OCA\DAV\Server->exec()
27. /var/www/[Nextcloud folder]/remote.php line 171
require_once("/var/www/[Nextcloud folder]/ ... p")
PROPFIND /remote.php/dav/files/Daniel/
from [IPv6] by Daniel at 2023-05-02T14:46:51+00:00
Multiple variations of this log:
[no app in context] Warnung: Exception: adding mount info at /Daniel/ to {"storage_id":3,"root_id":11,"user_id":"Daniel","mount_point":"\/Daniel\/","mount_id":null,"mount_provider_class":"OC\\Files\\Mount\\LocalHomeMountProvider"} at <<closure>>
0. /var/www/[Nextcloud folder]/lib/private/Files/Config/UserMountCache.php line 134
OC\Files\Config\UserMountCache->addToCache()
1. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 304
OC\Files\Config\UserMountCache->registerMounts()
2. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 230
OC\Files\SetupManager->afterUserFullySetup()
3. /var/www/[Nextcloud folder]/lib/private/Files/Filesystem.php line 378
OC\Files\SetupManager->setupForUser()
4. /var/www/[Nextcloud folder]/lib/private/Cache/File.php line 56
OC\Files\Filesystem::initMountPoints()
5. /var/www/[Nextcloud folder]/lib/private/Cache/File.php line 177
OC\Cache\File->getStorage()
6. /var/www/[Nextcloud folder]/lib/base.php line 866
OC\Cache\File->gc()
7. <<closure>>
OC::{closure}("*** sensitive parameters replaced ***")
8. /var/www/[Nextcloud folder]/lib/private/Hooks/EmitterTrait.php line 105
call_user_func_array()
9. /var/www/[Nextcloud folder]/lib/private/Hooks/PublicEmitter.php line 40
OC\Hooks\BasicEmitter->emit()
10. /var/www/[Nextcloud folder]/lib/private/User/Session.php line 401
OC\Hooks\PublicEmitter->emit()
11. /var/www/[Nextcloud folder]/lib/private/User/Session.php line 621
OC\User\Session->completeLogin("*** sensitive parameters replaced ***")
12. /var/www/[Nextcloud folder]/lib/private/User/Session.php line 355
OC\User\Session->loginWithPassword("*** sensitive parameters replaced ***")
13. /var/www/[Nextcloud folder]/lib/private/User/Session.php line 452
OC\User\Session->login("*** sensitive parameters replaced ***")
14. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/Auth.php line 111
OC\User\Session->logClientIn("*** sensitive parameters replaced ***")
15. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Auth/Backend/AbstractBasic.php line 103
OCA\DAV\Connector\Sabre\Auth->validateUserPass("*** sensitive parameters replaced ***")
16. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/Auth.php line 226
Sabre\DAV\Auth\Backend\AbstractBasic->check()
17. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/Auth.php line 133
OCA\DAV\Connector\Sabre\Auth->auth()
18. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Auth/Plugin.php line 180
OCA\DAV\Connector\Sabre\Auth->check()
19. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Auth/Plugin.php line 135
Sabre\DAV\Auth\Plugin->check()
20. /var/www/[Nextcloud folder]/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\Auth\Plugin->beforeMethod()
21. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 456
Sabre\DAV\Server->emit()
22. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 253
Sabre\DAV\Server->invokeMethod()
23. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 321
Sabre\DAV\Server->start()
24. /var/www/[Nextcloud folder]/apps/dav/appinfo/v1/caldav.php line 121
Sabre\DAV\Server->exec()
25. /var/www/[Nextcloud folder]/remote.php line 171
require_once("/var/www/[Nextcloud folder]/ ... p")
REPORT /remote.php/caldav/calendars/Daniel/[other ext]/
from [IPv6] by Daniel at 2023-05-02T14:46:54+00:00
Finally the log that changes the line back:
[no app in context] Warnung: Exception: updating mount info at /Daniel/files/[other ext]/ to {"storage_id":2,"root_id":7192,"user_id":"Daniel","mount_point":"\/Daniel\/files\/[other ext]\/","mount_id":2,"mount_provider_class":"OCA\\Files_External\\Config\\ConfigAdapter"} at <<closure>>
0. /var/www/[Nextcloud folder]/lib/private/Files/Config/UserMountCache.php line 144
OC\Files\Config\UserMountCache->updateCachedMount("*** sensitive parameters replaced ***")
1. /var/www/[Nextcloud folder]/lib/private/Files/SetupManager.php line 474
OC\Files\Config\UserMountCache->registerMounts()
2. /var/www/[Nextcloud folder]/lib/private/Files/Mount/Manager.php line 95
OC\Files\SetupManager->setupForPath()
3. /var/www/[Nextcloud folder]/lib/private/Files/View.php line 1390
OC\Files\Mount\Manager->find()
4. /var/www/[Nextcloud folder]/apps/dav/lib/Connector/Sabre/Directory.php line 216
OC\Files\View->getFileInfo()
5. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Tree.php line 78
OCA\DAV\Connector\Sabre\Directory->getChild()
6. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 971
Sabre\DAV\Tree->getNodeForPath()
7. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1662
Sabre\DAV\Server->getPropertiesIteratorForPath()
8. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 1647
Sabre\DAV\Server->writeMultiStatus()
9. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 346
Sabre\DAV\Server->generateMultiStatus()
10. /var/www/[Nextcloud folder]/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPropFind()
11. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 472
Sabre\DAV\Server->emit()
12. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 253
Sabre\DAV\Server->invokeMethod()
13. /var/www/[Nextcloud folder]/3rdparty/sabre/dav/lib/DAV/Server.php line 321
Sabre\DAV\Server->start()
14. /var/www/[Nextcloud folder]/apps/dav/lib/Server.php line 366
Sabre\DAV\Server->exec()
15. /var/www/[Nextcloud folder]/apps/dav/appinfo/v2/remote.php line 35
OCA\DAV\Server->exec()
16. /var/www/[Nextcloud folder]/remote.php line 171
require_once("/var/www/[Nextcloud folder]/ ... p")
PROPFIND /remote.php/dav/files/Daniel/[some folder]
from [IPv6] by Daniel at 2023-05-02T15:11:33+00:00
No new lines were actually added to the DB; Only the one line in the DB was changed back to the wrong state
After a couple of hours of digging through the code, I found the culprit!
Commit e3bafcc7a813f6adb305c2e35057b925c278d4bd from 3 months ago introduces this bug, but not for the reason you think:
The commit adds a new variable $numericStorageId to the class MountPoint where it caches the result of $this->getStorage()->getStorageCache()->getNumericId()
HOWEVER: the class PersonalMount which inherits from MountPoint through ExternalMountPoint already had a variable with the same name since forever!
https://github.com/nextcloud/server/blob/fc371facd202d7ede86f1adb37fc78a8c478deff/apps/files_external/lib/Lib/PersonalMount.php#L35-L41
The variable over there actually contains the mount id (because the terms 'storage' and 'mount' seem to be used interchangeably throughout the whole nextcloud codebase )
The constructor parameter $storageId is mounted to store exactly the same value as $mountId over here:
$storageConfig->getId() in this context actually stores the mount id of the storage which is a mount to store your data mounted on some external storage mount.
https://github.com/nextcloud/server/blob/e785ae345e85a5a4ca12f19a389ce6ab7f69a048/apps/files_external/lib/Config/ConfigAdapter.php#L153-L164
This now causes the new "caching code" to suddenly return the mount_id for the storage_id but only for personal external storages, which then populates the mounts table (which actually is only a cache, so no permanent damage done), which then causes all the issues downstream.
To fix this issue on my instance, I just renamed the property $numericStorageId in apps/files_external/lib/Lib/PersonalMount.php to a different name and afterwards, the DB table updated itself back to the correct state and all shares worked again 🎉
I'm not sure if that is the correct way; it might actually be a good idea to completely drop the $storageId parameter of the class additionally to the renaming to make all this make more sense...
I had the same problem. Renaming the var solved it, thanks
Did anyone have a fix for it..? I've seen today this problem with NC28 Raspberry Pi 5 Ububtu Server 24 LTS.. I run Nextcloud via snap.. On the files in the external drive there's no sharing button, in the files uploaded on the SD is present and works regularly.. Thanks