server Sharing still broken with ldap groups

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Steps to reproduce

Login as administrator to your nextcloud
Activate LDAP user and group backend app
Configure your ldap settings in http://localhost/settings/admin/ldap
Create a group in your ldap server, called sharing_group
Create a folder in your nextcloud, called shared_folder
Share the folder shared_folder with the group from ldap sharing_group
Now (after sharing) create a user called sharing_user in your ldap server
Add the created ldap user to the ldap group sharing_group
Login to your nextcloud as the newly created user sharing_user

Expected behaviour

User sharing_user should be inside the ldap group sharing_group (can be seen by the user sharing_user in http://localhost/settings/user)
User sharing_user therefore should see the folder shared_folder since this is shared with the group sharing_group where the user is a member

Actual behaviour

✔️ User sharing_user IS inside the ldap group sharing_group
❌ User sharing_user CANNOT see the folder shared_folder

The bug: LDAP users created after the share creation are not able to view the share to their group (which they normally should be able to).

The problem / a workaround

The problem might be located in the sharing or ldap app. It could be that the share is not automatically accepted. Therefore I went into the database using docker exec -it <<<<containername-of-db>>>> mysql -u<<<<username-of-db>>>> -p <<<<name-of-database>>>> and displayed the table with all shares:

MariaDB [nextcloud]> SELECT * FROM oc_share;
+----+------------+--------------------------------------+----------+-----------+---------------+--------+-----------+-------------+-------------+-------------+----------------+-------------+------------+----------+------------+-------+-----------+------------+------------------+------+---------------+-------+
| id | share_type | share_with                           | password | uid_owner | uid_initiator | parent | item_type | item_source | item_target | file_source | file_target    | permissions | stime      | accepted | expiration | token | mail_send | share_name | password_by_talk | note | hide_download | label |
+----+------------+--------------------------------------+----------+-----------+---------------+--------+-----------+-------------+-------------+-------------+----------------+-------------+------------+----------+------------+-------+-----------+------------+------------------+------+---------------+-------+
|  1 |          1 | sharing_group                        | NULL     | root      | root          |   NULL | folder    | 89          | NULL        |          89 | /shared_folder |          31 | 1610308767 |        0 | NULL       | NULL  |         0 | NULL       |                0 | NULL |             0 | NULL  |
|  2 |          2 | 5d6f515c-e7c2-103a-9933-21e2415ffd4b | NULL     | root      | root          |      1 | folder    | 89          | NULL        |          89 | /shared_folder |          31 | 1610308767 |        1 | NULL       | NULL  |         0 | NULL       |                0 | NULL |             0 | NULL  |
|  3 |          2 | cc512224-e7c4-103a-9934-21e2415ffd4b | NULL     | root      | root          |      1 | folder    | 89          | NULL        |          89 | /shared_folder |          31 | 1610308767 |        1 | NULL       | NULL  |         0 | NULL       |                0 | NULL |             0 | NULL  |
+----+------------+--------------------------------------+----------+-----------+---------------+--------+-----------+-------------+-------------+-------------+-------------+-------------+------------+----------+------------+-------+-----------+------------+------------------+------+---------------+-------+

Now we notice that the group we actually shared the folder with, supposedly did not accept this share (see row with id = 1, see column accepted). Obviously a group cannot accept a share manually, however I would have assumed that this would happen automatically, with such system-level sharing targets.

Now I just tried to set accepted to 1 in the affected line. And indeed it worked. 🎉 Now all ldap users see the shared folder shared_folder since they are in the group sharing_group. This works especially for ldap users created after the share was created.

MariaDB [nextcloud]> UPDATE `oc_share` SET `accepted` = 1 WHERE `share_with` = 'sharing_group' AND `file_target` = '/shared_folder';
Query OK, 1 row affected (0.005 sec)
Rows matched: 1  Changed: 1  Warnings: 0

Therefore I would limit the bug to this area, that shares to ldap groups are not automatically accepted (in the database). Groups obviously cannot accept shares by itself since no one really is the group. Is this maybe the underlying bug?

Not working workarounds

There are many hints and workarounds posted here in many issues but none of them fixed the problem for me. Here is a list of the not working ones:

The user sharing_user can accept the share by itself on http://localhost/apps/files/?dir=/&view=pendingshares (the pending share is visible but clicking accept results in a 404 api request)
Starting cronjobs manually via docker exec -it --user www-data -it <<<<container-name>>>> php /var/www/html/cron.php (doesn't change anything)
The OCC Repair command docker exec -it --user www-data -it <<<<container-name>>>> php occ maintenance:repair (doesn't change anything)
Setting sharing.force_share_accept to true via docker exec --user www-data -it <<<<container-name>>>> php occ config:app:set sharing force_share_accept --value=true (doesn't change anything)

Possibly related issues

#20133
#20617
#20937
#19520
#24016

There are quite a few issues regarding this topic but unfortunately the same issue still exists for the current nextcloud version.

Server configuration

Operating system: Docker version 20.10.2, build 2291f61

Web server: Apache

Database: 10.5.8-MariaDB-1:10.5.8+maria~focal

PHP version: 7.4.14

Nextcloud version: 20.0.4

Updated from an older Nextcloud/ownCloud or fresh install: Fresh install, but also happends with updated instances (since Nextcloud 20)

Where did you install Nextcloud from: Docker Hub

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - accessibility: 1.6.0
  - activity: 2.13.4
  - bruteforcesettings: 2.0.1
  - cloud_federation_api: 1.3.0
  - comments: 1.10.0
  - contactsinteraction: 1.1.0
  - dashboard: 7.0.0
  - dav: 1.16.2
  - federatedfilesharing: 1.10.2
  - federation: 1.10.1
  - files: 1.15.0
  - files_pdfviewer: 2.0.1
  - files_rightclick: 0.17.0
  - files_sharing: 1.12.1
  - files_trashbin: 1.10.1
  - files_versions: 1.13.0
  - files_videoplayer: 1.9.0
  - firstrunwizard: 2.9.0
  - groupfolders: 8.2.0
  - logreader: 2.5.0
  - lookup_server_connector: 1.8.0
  - nextcloud_announcements: 1.9.0
  - notifications: 2.8.0
  - oauth2: 1.8.0
  - password_policy: 1.10.1
  - photos: 1.2.1
  - privacy: 1.4.0
  - provisioning_api: 1.10.0
  - recommendations: 0.8.0
  - serverinfo: 1.10.0
  - settings: 1.2.0
  - sharebymail: 1.10.0
  - support: 1.3.0
  - survey_client: 1.8.0
  - systemtags: 1.10.0
  - text: 3.1.0
  - theming: 1.11.0
  - twofactor_backupcodes: 1.9.0
  - updatenotification: 1.10.0
  - user_ldap: 1.10.2
  - user_status: 1.0.1
  - viewer: 1.4.0
  - weather_status: 1.0.0
  - workflowengine: 2.2.0
Disabled:
  - admin_audit
  - encryption
  - files_external

Nextcloud configuration:

Config report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "20.0.4.0",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "maintenance": false
    }
}

Are you using external storage, if yes which one: None

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP

LDAP configuration (delete this part if not used)

LDAP config

+-------------------------------+-----------------------------------------------------------------------+
| Configuration                 | s01                                                                   |
+-------------------------------+-----------------------------------------------------------------------+
| hasMemberOfFilterSupport      |                                                                       |
| homeFolderNamingRule          |                                                                       |
| lastJpegPhotoLookup           | 0                                                                     |
| ldapAgentName                 | cn=query,dc=ldap,dc=example,dc=com                           |
| ldapAgentPassword             | ***                                                                   |
| ldapAttributesForGroupSearch  |                                                                       |
| ldapAttributesForUserSearch   |                                                                       |
| ldapBackupHost                |                                                                       |
| ldapBackupPort                |                                                                       |
| ldapBase                      | dc=ldap,dc=example,dc=com                                          |
| ldapBaseGroups                | ou=groups,dc=ldap,dc=example,dc=com                                |
| ldapBaseUsers                 | ou=people,dc=ldap,dc=example,dc=com                                |
| ldapCacheTTL                  | 600                                                                   |
| ldapConfigurationActive       | 1                                                                     |
| ldapDefaultPPolicyDN          |                                                                       |
| ldapDynamicGroupMemberURL     |                                                                       |
| ldapEmailAttribute            | mail                                                                  |
| ldapExperiencedAdmin          | 0                                                                     |
| ldapExpertUUIDGroupAttr       |                                                                       |
| ldapExpertUUIDUserAttr        |                                                                       |
| ldapExpertUsernameAttr        |                                                                       |
| ldapExtStorageHomeAttribute   |                                                                       |
| ldapGidNumber                 | gidNumber                                                             |
| ldapGroupDisplayName          | cn                                                                    |
| ldapGroupFilter               | (&(|(objectclass=posixGroup)))                                        |
| ldapGroupFilterGroups         |                                                                       |
| ldapGroupFilterMode           | 0                                                                     |
| ldapGroupFilterObjectclass    | posixGroup                                                            |
| ldapGroupMemberAssocAttr      | memberUid                                                             |
| ldapHost                      | production-ldap                                                       |
| ldapIgnoreNamingRules         |                                                                       |
| ldapLoginFilter               | (&(|(objectclass=inetOrgPerson)(objectclass=posixAccount))(uid=%uid)) |
| ldapLoginFilterAttributes     |                                                                       |
| ldapLoginFilterEmail          | 0                                                                     |
| ldapLoginFilterMode           | 0                                                                     |
| ldapLoginFilterUsername       | 1                                                                     |
| ldapMatchingRuleInChainState  | unknown                                                               |
| ldapNestedGroups              | 0                                                                     |
| ldapOverrideMainServer        |                                                                       |
| ldapPagingSize                | 500                                                                   |
| ldapPort                      | 389                                                                   |
| ldapQuotaAttribute            |                                                                       |
| ldapQuotaDefault              |                                                                       |
| ldapTLS                       | 0                                                                     |
| ldapUserAvatarRule            | default                                                               |
| ldapUserDisplayName           | cn                                                                    |
| ldapUserDisplayName2          |                                                                       |
| ldapUserFilter                | (|(objectclass=inetOrgPerson)(objectclass=posixAccount))              |
| ldapUserFilterGroups          |                                                                       |
| ldapUserFilterMode            | 0                                                                     |
| ldapUserFilterObjectclass     | inetOrgPerson;posixAccount                                            |
| ldapUuidGroupAttribute        | auto                                                                  |
| ldapUuidUserAttribute         | auto                                                                  |
| turnOffCertCheck              | 0                                                                     |
| turnOnPasswordChange          | 0                                                                     |
| useMemberOfToDetectMembership | 1                                                                     |
+-------------------------------+-----------------------------------------------------------------------+

Client configuration

Browser: Google Chrome 87.0.4280.141

Operating system: Windows 10 10.0.19042 Build 19042

Logs

Web server error log

Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log

Insert your Nextcloud log here

Browser log

No client sided problem

Jan 10 '21 21:01 christianvw

We can see the same bug on our nextcloud instance for our university (Php 7.4, Nextcloud 20.0.5)

The solution provided above is working, thanks ! To quickly fix this issue on all group shared folders, we (with @jdorel) used the following SQL code :

UPDATE `oc_share` SET `accepted` = 1 WHERE `share_type` = 1;

accepted field for group folder shares should always be set to 1.

Jan 24 '21 22:01 larueli

Should it be? set to 1 when created? AFAIU, there's a listener that's supposed to accept shares on behalf of users being added to groups, over there:

https://github.com/nextcloud/server/blob/master/apps/files_sharing/lib/Listener/UserAddedToGroupListener.php#L50

While it has been suggested ( https://github.com/nextcloud/server/issues/19520#issuecomment-733913036 ) that NextCloud cron should take care of these, I've been digging and could only find a job that removes shares to users that no longer belong to a group ( https://github.com/nextcloud/server/blob/master/lib/private/Repair/OldGroupMembershipShares.php#L69 ) . The event listener above seems to be the one accepting on behalf of users joining a group.

Then again, looking at the SELECT * FROM oc_share; reported above, it looks like users did accept it ....

If this is the case, we may then fix this in https://github.com/nextcloud/server/blob/master/lib/private/Share20/DefaultShareProvider.php#L149 , adding some $qb->setValue('accepted', $qb->createNamedParameter(IShare::STATUS_ACCEPTED)); in the IShare::TYPE_GROUP elseif from the create method. See https://github.com/nextcloud/server/compare/master...Worteks:fix-sharewithgroup Still, ... shouldn't we see similar issues reported, that wouldn't involve ldap groups? Patching there would change behavior regardless of the backend our group came from.

@christianvw could you confirm that cc512224-e7c4-103a-9934-21e2415ffd4b we see in the share_with column is indeed the user ID for the account you did add to your group, after the share was initially created? Or were they two members in your group to begin with?

Feb 04 '21 14:02 faust64

@faust64

Still, ... shouldn't we see similar issues reported, that wouldn't involve ldap groups? Patching there would change behavior regardless of the backend our group came from.

I think there have been some without LDAP, e.g.

https://github.com/nextcloud/server/issues/19520
https://github.com/nextcloud/server/issues/24016
https://github.com/nextcloud/server/issues/20696
https://github.com/nextcloud/server/issues/19778
https://github.com/nextcloud/server/issues/18958

Apr 05 '21 19:04 coach1988

This issue seems to be pretty widespread. I have the same issue with the "Everyone" group, but just ran some tests and realized that it happens with all groups and users (all LDAPs in my case).

Here a similar issue reported for the "Everyone" group: https://github.com/icewind1991/group_everyone/issues/19#issue-837144773

Apr 07 '21 16:04 gabriele-semino

+1

Apr 24 '21 11:04 daha96

Group memberships are checked in an hourly background job which sends the corresponding events.

Apr 30 '21 08:04 blizzz

Group memberships are checked in an hourly background job which sends the corresponding events.

is there another workaround than the one described above (manual adjustment of the database)?

Apr 30 '21 10:04 scroom

Group memberships are checked in an hourly background job which sends the corresponding events.

is there another workaround than the one described above (manual adjustment of the database)?

nope

Apr 30 '21 11:04 blizzz

Group memberships are checked in an hourly background job which sends the corresponding events.

is there another workaround than the one described above (manual adjustment of the database)?

nope

Would it be problematic to change all entries for groups to accepted? And should not this be the default case for groups? Is this the actual bug as @christianvw mentioned?

Older group-shares have accepted set to 1. It might be a regression?

May 04 '21 07:05 scroom

Group memberships are checked in an hourly background job which sends the corresponding events.

is there another workaround than the one described above (manual adjustment of the database)?

nope

Would it be problematic to change all entries for groups to accepted? And should not this be the default case for groups? Is this the actual bug as @christianvw mentioned?

Older group-shares have accepted set to 1. It might be a regression?

It depends on the configuration, but by default the shares are accepted. Still, sharing need to be made aware of group membership changes. I cannot reproduce any misbehaviour.

May 04 '21 10:05 blizzz

Interestinly I see in my Nextcloud (21.0.2.1) installation the same isse.

In the oc_share table there is a row for every user who was in the LDAP group when sharing. I tried adding the new user into this table as well and the share popped up right away for the new user as well.

Maybe it is intended to keep the accept=0 for the group and just a syncer is missing, who checks for new users and adds a line per new LDAP user and share?

Jun 07 '21 12:06 pkolmann

For me this trigger is the workaround:

CREATE DEFINER=root@localhostTRIGGERoc_group_user_after_insertAFTER INSERT ONoc_group_user FOR EACH ROW BEGIN if (SELECT COUNT(id) FROM oc_share WHERE share_with = NEW.gid and share_type = 1)>0 then INSERT INTO oc_share(share_type,share_with,uid_owner,uid_initiator,parent,item_type,item_source,file_source,file_target,permissions,stime,accepted,expiration,note) SELECT 2, NEW.uid, uid_owner, uid_initiator, id, item_type, item_source, file_source, file_target, permissions, stime, 1, expiration, note FROM oc_share WHERE share_with = NEW.gid AND share_type = 1; END if; END

but its better when we solve that in code.

There are two things, the shares for a group must have accepted = 1 and new users in the group must inherit the group sharing like in my trigger above.

I've tested it with the SQL backends app and it works with new users, i think it works also with LDAP.

Jun 08 '21 05:06 leuedaniel

Small recap of the problem

LDAP users created (and added to the ldap group) after the share creation in nextcloud are not getting the share (because it does not get accepted by nextcloud).

Problem still there, reproducibility

I still see exactly the same behavior in Nextcloud 21.0.2 (Docker), even in a fresh installation.

For me, this misbehavior is 100% reproducible with the process described in the initial post. Mind you with Nextcloud as well as openLDAP in a docker container.

And considering the many competitors in these and other issues, it is also a problem for at least some others. Even new issues popped up in the last months, regarding the exact same problem.

I wonder how this problem doesn't seem to occur in large organizations that use Nextcloud in combination with LDAP, this issue should stand out.

Fixes in the database

A trigger in the database could fix the problem, of course (https://github.com/nextcloud/server/issues/25062#issuecomment-856459649).
Manual (and therefore recurring) adjustments in the database would fix the problem temporarily too, of course (https://github.com/nextcloud/server/issues/25062#issuecomment-766445043).

However, like @leuedaniel said, the true solution must be a fix in the Nextcloud application (a fix of the background job / syncer / group share creation / ...). Is there maybe even a fix coming? @blizzz

Possible (real) fixes in the application

@faust64 did a pull request (one liner, https://github.com/nextcloud/server/pull/26428) with the idea from my initial post in mind: Setting the accepted to 1 for group shares (directly on group share creation).

His change (in lib/private/Share20/DefaultShareProvider.php):

//...

} elseif ($share->getShareType() === IShare::TYPE_GROUP) {
    //Set the GID of the group we share with
    $qb->setValue('share_with', $qb->createNamedParameter($share->getSharedWith()));
    $qb->setValue('accepted', $qb->createNamedParameter(IShare::STATUS_ACCEPTED)); //<-- **NEW**
    
    //...

However @blizzz has been looking more at a background job. This would also be an alternative solution.

Nevertheless something does not work here as intended.

Jun 18 '21 19:06 christianvw

Any updates? Issue also present in Nextcloud 23

Dec 17 '21 08:12 Gianni154

Here is the behavior I get:

When creating a user and adding it to a group in ldap, indeed the share is not accepted automatically. It appears in pending shares, and I can accept it (no errors, accepting works). When doing the same but in nextcloud, the share is accepted automatically. The problem is specific to LDAP.

But, if I wait until the background job runs, the share is correctly accepted.

So, this is not a big problem as of 23 from what I can see, it’s a matter of waiting a few hours top, or manually accepting the pending share. It is annoying but it is not a major issue. If we want to fix this we need to find a way to trigger the UserAddedEvent event on the user for all groups it is a member of on first login, but we must be careful to avoid the background job emitting the event a second time, so it is not as trivial as it may sound.

Jan 03 '22 16:01 come-nc

You're right manually accepting works for me. Didn't know there is a workflow to accept pending shares.

Jan 12 '22 11:01 cybrwshl

It appears we have the very same problem with our Nextcloud 23.0.2 and openldap-2.4.40

Can someone please confirm that this throws log error messages like:

OC\Share20\Exception\ProviderException: Recipient not in receiving group

Workaround seems to have worked so far, thanks @christianvw for elaborating!

Feb 18 '22 10:02 schlagmichdoch

This still seems to be a problem. Any progress on fixong this?

Oct 09 '22 10:10 nilsmelchert

This still seems to be a problem. Any progress on fixong this?

This needs a refactoring of the way the background job is storing which group-member associations were already announced through the event, so that on login we can add a check that all groups the user is a member of has been announced without having a major performance impact. So the fix is complicated and the problem not considered blocking, so it will take a bit of time before it’s tackled.

Oct 10 '22 07:10 come-nc

However, I find it interesting, that it used to work just fine for me. Just 1 week ago, before updating, I was able to share folders between ldap groups. Could it be possible, that there is just a bug in the ldap configuration that can be easily fixed?

Oct 10 '22 07:10 nilsmelchert

However, I find it interesting, that it used to work just fine for me. Just 1 week ago, before updating, I was able to share folders between ldap groups. Could it be possible, that there is just a bug in the ldap configuration that can be easily fixed?

I am not sure you are talking about the same bug then. This ticket is about having to wait for the background job to sync groups before a newly added user to a group sees the shares with this group without having to accept them by hand. Sharing with an LDAP group works, it’s just this specific usecase where you have to either wait or accept manually.

Oct 10 '22 08:10 come-nc

Well how long would I have to wait? I have triggered the backgroundjob (cron.php right?) manually and nothing happened. When applying the fix presented above UPDATE `oc_share` SET `accepted` = 1 WHERE `share_type` = 1; everything works. The problem I have seems to be related dont you think so?

Oct 10 '22 08:10 nilsmelchert

In my case, the groups in http://localhost/settings/users/MY_GROUP_NAME are not showing their members. However when looking at specific users the group membership is shown.

Oct 10 '22 08:10 nilsmelchert

I have to take it back, my problem was not related to this Issue. Thanks for pointing out. I just misconfigured the ldap settings by chosing a wrong option for Group-Member association.

Oct 10 '22 17:10 nilsmelchert

Just for my understanding: Do we know, why the paradigm has changed from setting accepted=1 on the group share (as it was before), but letting it at 0 and have a background job do it for all users individually? Why has that changed?

Oct 12 '22 11:10 ruedigerkupper

Just for my understanding: Do we know, why the paradigm has changed from setting accepted=1 on the group share (as it was before), but letting it at 0 and have a background job do it for all users individually? Why has that changed?

I don’t know, all I can say it the background job is not especially designed for this usecase, it just fires the event signaling that users have been added to the group, which might trigger other things than accepting shares.

Oct 13 '22 08:10 come-nc

By the way if you do not want to use mangle with the database, then there is also an other workaround (other then simply using groupfolders ):

Create a Circle with only that group as Admin
Share the folder with that circle instead

This seems to work fine on our NC25 server.

Dec 23 '22 15:12 susnux

Hi, please update to 24.0.9 or better 25.0.3 and report back if it fixes the issue. Thank you!

My goal is to add a label like e.g. 25-feedback to this ticket of an up-to-date major Nextcloud version where the bug could be reproduced. However this is not going to work without your help. So thanks for all your effort!

If you don't manage to reproduce the issue in time and the issue gets closed but you can reproduce the issue afterwards, feel free to create a new bug report with up-to-date information by following this link: https://github.com/nextcloud/server/issues/new?assignees=&labels=bug%2C0.+Needs+triage&template=BUG_REPORT.yml&title=%5BBug%5D%3A+

Jan 23 '23 10:01 szaimen

@szaimen This is still valid, we have no mechanism to handle LDAP groups on first login before background job kicks in.

Mar 09 '23 15:03 come-nc