linstor-server remove resource failed

Hi, Resource has been marked as deleting, but with resource definition shows this error: linstor rd d vdd_rSc
SUCCESS:
Description:
Resource definition 'vdd_rSc' marked for deletion.
Details:
Resource definition 'vdd_rSc' UUID is: 1d92be2b-9fac-422c-b5ba-c870fc59f8c9
ERROR:
(Node: 'node01') An error occurred while cleaning up layer 'StorageLayer'

cat /var/log/linstor-controller/ErrorReport-6107FB64-00000-000003.log

Error message: Domain name cannot contain character '_'
Error context:
The given node name 'vdd_rSc' is invalid.

linstor r l
╭─────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ Node ┊ Port ┊ Usage ┊ Conns ┊ State ┊ CreatedOn ┊ ╞═════════════════════════════════════════════════════════════════════════════════╡ ┊ linstor_db ┊ node01 ┊ 7001 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2021-07-30 23:40:13 ┊ ┊ linstor_db ┊ node02 ┊ 7001 ┊ InUse ┊ Ok ┊ UpToDate ┊ 2021-07-30 23:40:13 ┊ ┊ linstor_db ┊ node03 ┊ 7001 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2021-07-30 23:40:13 ┊ ┊ linstor_db ┊ node04 ┊ 7001 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2021-07-30 23:40:13 ┊ ┊ vdd_rSc ┊ node01 ┊ 7000 ┊ ┊ Ok ┊ DELETING ┊ 2021-07-30 17:22:26 ┊ ┊ vdd_rSc ┊ node02 ┊ 7000 ┊ ┊ Ok ┊ DELETING ┊ 2021-07-30 17:26:00 ┊ ┊ vdd_rSc ┊ node03 ┊ 7000 ┊ ┊ Ok ┊ DELETING ┊ 2021-07-30 17:22:26 ┊ ┊ vdd_rSc ┊ node04 ┊ 7000 ┊ ┊ Ok ┊ DELETING ┊ 2021-07-30 17:22:26 ┊ ┊ vm-100-disk-1 ┊ node02 ┊ 7002 ┊ Unused ┊ Ok ┊ UpToDate ┊ 2021-08-02 16:23:39 ┊ ╰──────────────────────────────────────────────────────────────────────────

Can I clean up StorageAlayer manually? Or what should I do to completely delete the resource?

thanks

Aug 02 '21 15:08 Siya-Sharifi

Can you show us the actual ErrorReport? I am a bit confused that you are trying to delete a resource-definition but the error complains about an invalid node-name. What adds to my confusion is that apparently your nodes are called node01-node04, so no _ in any node name..

I also ran a quick test if there is a bug in deleting a resource-definition containing an _, but could not reproduce this right now.

Although what I just saw (and missed in my test) that your volumes are additionally resizing right now. That should also be investigated in order to get back to a normal state

Aug 02 '21 15:08 ghernadi

First, I tried to shrink a volume while a node was out of Cluster, this was Node03, so in node02 linstore-controler was active linstor vd set-size vdd_rSc 0 5G

Then I started NODE03. Later, a VM placed in Pve but When creating a VMS, Proxmox VM tries to create on any node. It failed. PVE shows: Trying to create diskful resource (vm-100-disk-1) on (node01). Diskfull assignment failed, let's autoplace it.

cat /etc/pve/storage.cfg dir: local path /var/lib/vz content iso,vztmpl,backup

lvmthin: local-lvm thinpool data vgname pve content rootdir,images

drbd: drbdstorage content images,rootdir controller 10.10.10.11,10.10.10.12,10.10.10.13,10.10.10.14 resourcegroup rScGrp_vdd preferlocal yes <<<--- i want to place vm on same node [ ErrorReport-6108162C-14A1C-000014.log ErrorReport-6108162C-14A1C-000013.log ErrorReport-6108162C-14A1C-000012.log ErrorReport-6108162C-14A1C-000011.log ErrorReport-6108162C-14A1C-000010.log ](url)

DRBD does not understand it and the created hard drive is synchronized.

If you try to create a VM again while DRBD is synchronized, an abnormality will occur. This time, a hard disk is also created in the Deflaut-Pool DFLTDISKressstorpool.

linstor v l ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ Node ┊ Resource ┊ StoragePool ┊ VolNr ┊ MinorNr ┊ DeviceName ┊ Allocated ┊ InUse ┊ State ┊
╞══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ node01 ┊ linstor_db ┊ vdd_pool ┊ 0 ┊ 1001 ┊ /dev/drbd1001 ┊ 204 MiB ┊ Unused ┊ UpToDate ┊
┊ node02 ┊ linstor_db ┊ vdd_pool ┊ 0 ┊ 1001 ┊ /dev/drbd1001 ┊ 204 MiB ┊ Unused ┊ UpToDate ┊
┊ node03 ┊ linstor_db ┊ vdd_pool ┊ 0 ┊ 1001 ┊ /dev/drbd1001 ┊ 204 MiB ┊ InUse ┊ UpToDate ┊
┊ node04 ┊ linstor_db ┊ vdd_pool ┊ 0 ┊ 1001 ┊ /dev/drbd1001 ┊ 204 MiB ┊ Unused ┊ UpToDate ┊
┊ node01 ┊ vdd_rSc ┊ vdd_pool ┊ 0 ┊ 1000 ┊ None ┊ ┊ ┊ Resizing, Unknown ┊
┊ node02 ┊ vdd_rSc ┊ vdd_pool ┊ 0 ┊ 1000 ┊ None ┊ ┊ ┊ Resizing, Unknown ┊
┊ node03 ┊ vdd_rSc ┊ DfltDisklessStorPool ┊ 0 ┊ 1000 ┊ None ┊ ┊ ┊ Resizing, Unknown ┊
┊ node04 ┊ vdd_rSc ┊ vdd_pool ┊ 0 ┊ 1000 ┊ None ┊ ┊ ┊ Resizing, Unknown ┊
┊ node01 ┊ vm-100-disk-1 ┊ vdd_pool ┊ 0 ┊ 1002 ┊ None ┊ ┊ ┊ Unknown ┊
┊ node02 ┊ vm-100-disk-1 ┊ vdd_pool ┊ 0 ┊ 1002 ┊ None ┊ ┊ ┊ Unknown ┊
┊ node03 ┊ vm-100-disk-1 ┊ vdd_pool ┊ 0 ┊ 1002 ┊ None ┊ ┊ ┊ Unknown ┊
┊ node04 ┊ vm-100-disk-1 ┊ vdd_pool ┊ 0 ┊ 1002 ┊ None ┊ ┊ ┊ Unknown ┊
┊ node01 ┊ vm-100-disk-2 ┊ vdd_pool ┊ 0 ┊ 1003 ┊ /dev/drbd1003 ┊ 10.00 GiB ┊ ┊ Unknown ┊
┊ node02 ┊ vm-100-disk-2 ┊ DfltDisklessStorPool ┊ 0 ┊ 1003 ┊ None ┊ ┊ ┊ Unknown ┊
┊ node03 ┊ vm-100-disk-2 ┊ vdd_pool ┊ 0 ┊ 1003 ┊ None ┊ ┊ ┊ Unknown ┊
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
An error occurred while cleaning up layer 'StorageLayer'","obj_refs":{"RscDfn":"vm-100-disk-3","RscGrp":"rScGrp_vdd"}},{"ret_code":21233667,"message":"Created resource 'vm-100-disk-3' on 'node02'","obj_refs":{"RscDfn":"vm-100-disk-3","RscGrp":"rScGrp_vdd"}}] at /usr/share/perl5/PVE/Storage/Custom/LINSTORPlugin.pm line 363.

more Info:

linstor sp l ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ StoragePool ┊ Node ┊ Driver ┊ PoolName ┊ FreeCapacity ┊ TotalCapacity ┊ CanSnapshots ┊ State ┊ SharedName ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ DfltDisklessStorPool ┊ node01 ┊ DISKLESS ┊ ┊ ┊ ┊ False ┊ Ok ┊ ┊ ┊ DfltDisklessStorPool ┊ node02 ┊ DISKLESS ┊ ┊ ┊ ┊ False ┊ Ok ┊ ┊ ┊ DfltDisklessStorPool ┊ node03 ┊ DISKLESS ┊ ┊ ┊ ┊ False ┊ Ok ┊ ┊ ┊ DfltDisklessStorPool ┊ node04 ┊ DISKLESS ┊ ┊ ┊ ┊ False ┊ Ok ┊ ┊ ┊ vdd_pool ┊ node01 ┊ LVM ┊ vg_vdd ┊ 79.79 GiB ┊ 100.00 GiB ┊ False ┊ Ok ┊ ┊ ┊ vdd_pool ┊ node02 ┊ LVM ┊ vg_vdd ┊ 79.79 GiB ┊ 100.00 GiB ┊ False ┊ Ok ┊ ┊ ┊ vdd_pool ┊ node03 ┊ LVM ┊ vg_vdd ┊ 99.80 GiB ┊ 100.00 GiB ┊ False ┊ Ok ┊ ┊ ┊ vdd_pool ┊ node04 ┊ LVM ┊ vg_vdd ┊ 99.80 GiB ┊ 100.00 GiB ┊ False ┊ Ok ┊ ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Aug 02 '21 16:08 Siya-Sharifi

The original error in this issue said:

Resource definition 'vdd_rSc' marked for deletion.
The given node name 'vdd_rSc' is invalid.

That looks like some piece of code mixing up resource definition names and node names, possibly because they were passed or stored as String objects. Wherever possible, code inside LINSTOR should handle object names using the various name objects (ResourceName, NodeName, etc.) instead of raw String objects, as those specialized name objects are designed to prevent any such mixups or the use of name checks that are unsuitable for the type of object that the name belongs to.

Sep 01 '21 15:09 raltnoeder