linstor-server icon indicating copy to clipboard operation
linstor-server copied to clipboard

Node evacuate does not work with single replica

Open kvaps opened this issue 3 years ago • 2 comments

If node is containing single replica of DRBD resource, then after evacuating it become to Outdated state.

kvaps avatar Aug 24 '22 08:08 kvaps

Can you elaborate on this issue? I could not reproduce this issue, even went back to v1.17.0 (first release of node evacuate):

linstor --no-utf8 --no-color r l -a
+-------------------------------------------------------------------------------+
| ResourceName | Node  | Port | Usage  | Conns |    State | CreatedOn           |
|===============================================================================|
| rsc          | bravo | 7000 | Unused | Ok    | UpToDate | 2022-09-07 14:22:44 |
+-------------------------------------------------------------------------------+

linstor n evacuate bravo

linstor --no-utf8 --no-color r l -a
+---------------------------------------------------------------------------------------------+
| ResourceName | Node    | Port | Usage  | Conns |                State | CreatedOn           |
|=============================================================================================|
| rsc          | bravo   | 7000 | Unused | Ok    | UpToDate, Evacuating | 2022-09-07 14:22:44 |
| rsc          | charlie | 7000 | Unused | Ok    |         Inconsistent | 2022-09-07 14:22:45 |
+---------------------------------------------------------------------------------------------+

sleep 1.0s
linstor --no-utf8 --no-color r l -a
+---------------------------------------------------------------------------------------------+
| ResourceName | Node    | Port | Usage  | Conns |                State | CreatedOn           |
|=============================================================================================|
| rsc          | bravo   | 7000 | Unused | Ok    | UpToDate, Evacuating | 2022-09-07 14:22:44 |
| rsc          | charlie | 7000 | Unused | Ok    |   SyncTarget(28.12%) | 2022-09-07 14:22:45 |
+---------------------------------------------------------------------------------------------+

...


linstor --no-utf8 --no-color r l -a
+---------------------------------------------------------------------------------+
| ResourceName | Node    | Port | Usage  | Conns |    State | CreatedOn           |
|=================================================================================|
| rsc          | charlie | 7000 | Unused | Ok    | UpToDate | 2022-09-07 14:22:45 |
+---------------------------------------------------------------------------------+

ghernadi avatar Sep 07 '22 12:09 ghernadi

Reproduced situation on actual version (linstor-controller 1.23.0, drbd 9.2.4) Volume with one replica | ubuntu18-0 | pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317 | lvm | 0 | 1015 | /dev/drbd1015 | 126.65 MiB | InUse | UpToDate | After evacuating

root@master0:~# linstor node evacuate ubuntu18-0
SUCCESS:
    Successfully set property key(s): StorPoolName
SUCCESS:
    Successfully set property key(s): StorPoolName
INFO:
    Tie breaker resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' created on DfltDisklessStorPool
INFO:
    Resource-definition property 'DrbdOptions/Resource/quorum' updated from 'off' to 'majority' by auto-quorum
INFO:
    Resource-definition property 'DrbdOptions/Resource/on-no-quorum' updated from 'off' to 'io-error' by auto-quorum
SUCCESS:
Description:
    New resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on node 'ubuntu22-0' registered.
Details:
    Resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on node 'ubuntu22-0' UUID is: ae6c1ac3-2ed7-4cb2-881e-bfc4900853d6
SUCCESS:
Description:
    Volume with number '0' on resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on node 'ubuntu22-0' successfully registered
Details:
    Volume UUID is: 0875cbbe-e200-454e-9a17-2fab3f8eb01f
SUCCESS:
    Added peer(s) 'ubuntu22-0' to resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'master0'
SUCCESS:
    Added peer(s) 'ubuntu22-0' to resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'ubuntu18-0'
SUCCESS:
    Created resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'ubuntu22-0'
SUCCESS:
Description:
    Resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'ubuntu22-0' ready
Details:
    Node(s): 'ubuntu22-0', Resource: 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317'
SUCCESS:
    Created resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'master0'
SUCCESS:
    Added peer(s) 'master0' to resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'ubuntu18-0'
SUCCESS:
    Added peer(s) 'master0' to resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'ubuntu22-0'
SUCCESS:
Description:
    Resource 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317' on 'master0' ready
Details:
    Node(s): 'ubuntu22-0', Resource: 'pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317'

root@master0:~# linstor v l
| master0    | pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317 | DfltDisklessStorPool |     0 |    1015 | /dev/drbd1015 |             | Unused | TieBreaker |
| ubuntu18-0 | pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317 | lvm                  |     0 |    1015 | /dev/drbd1015 |  126.65 MiB | InUse  |   UpToDate |
| ubuntu22-0 | pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317 | lvm                  |     0 |    1015 | /dev/drbd1015 |     315 KiB | Unused |   UpToDate |

No error reports or some events on nodes. Drbd status on new node

root@ubuntu22-0:/# drbdadm status
pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317 role:Secondary
  disk:UpToDate
  master0 connection:Connecting

On diskless node

root@master0:/# drbdadm status
pvc-a0f9bc71-0328-4cd9-8706-1dcac322d317 role:Secondary
  disk:Diskless quorum:no
  ubuntu22-0 connection:StandAlone

On old node

root@ubuntu18-0:/# drbdadm status
# No currently configured DRBD found.

If resource have two replicas - there is no such problem, evacuate works fine

duckhawk avatar Jul 03 '23 04:07 duckhawk