cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard
verified publisher icon apache

Failed add volume when instance running on VMware use VMware Paravirtual SCSI controller

Open leduyquy opened this issue 1 year ago • 4 comments

ISSUE TYPE
  • Bug Report
COMPONENT NAME
Instance, volume, VMware
CLOUDSTACK VERSION
 4.19.0.1
CONFIGURATION

VMware backend

OS / ENVIRONMENT

Ubuntu 22.04 for Management VMware 8.0

SUMMARY
STEPS TO REPRODUCE
Test case 1:
Set Root Disk Controller to pvscsi
Create instance from template
After create success add new volume

Test case 2:
Create Volume manual
Attach volume to instance
EXPECTED RESULTS
Volume attach to Instance
ACTUAL RESULTS
Failed to attach volume to instance

Log error:
2024-07-16 08:49:07,688 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-20:ctx-97d2789a job-22512) (logid:accc321a) Unexpected exception while executing org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd
com.cloud.utils.exception.CloudRuntimeException: Failed to attach volume new-volume to VM test-vm
        at com.cloud.storage.VolumeApiServiceImpl.sendAttachVolumeCommand(VolumeApiServiceImpl.java:4402)
        at com.cloud.storage.VolumeApiServiceImpl.orchestrateAttachVolumeToVM(VolumeApiServiceImpl.java:2304)
        at com.cloud.storage.VolumeApiServiceImpl.orchestrateAttachVolumeToVM(VolumeApiServiceImpl.java:4803)
        at jdk.internal.reflect.GeneratedMethodAccessor4425.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:4842)
        at jdk.internal.reflect.GeneratedMethodAccessor357.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at com.sun.proxy.$Proxy235.handleVmWorkJob(Unknown Source)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:654)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:602)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
2024-07-16 08:49:07,690 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-20:ctx-97d2789a job-22512) (logid:accc321a) Complete async job-22512, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to attach volume new-volume to VM test-vm"}

leduyquy avatar Jul 16 '24 09:07 leduyquy

@leduyquy what version of vmware 8 are you using?

DaanHoogland avatar Jul 18 '24 11:07 DaanHoogland

I using VMware ESXi, 8.0.0, 21203435 for node vSphere Client version 8.0.0.10200 for vCenter

leduyquy avatar Jul 22 '24 03:07 leduyquy

ok @leduyquy this should not be a problem.

I'll try an reproduce your problem to see what happens.

DaanHoogland avatar Jul 22 '24 09:07 DaanHoogland

I'm able to reproduce this in 4.19.1 env (VMware 8.0.0, Build: 21216066), seems attach data volume tried to use LSI Logic SCSI controller (as the data disk controller was osdefault) and failed, as it's not available. All four controllers created, are Paravirtual.

VMware-VM-PVSCSI-Controller 
2024-07-22 11:39:06,241 ERROR [c.c.s.r.VmwareStorageProcessor] (DirectAgent-296:ctx-f10157f9 10.0.33.11, job-37/job-38, cmd: AttachCommand) (logid:73329ae9) Failed to attach volume!
java.lang.IllegalStateException: Scsi disk controller of type VirtualLsiLogicController not found among configured devices.
	at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.getScsiDiskControllerKey(VirtualMachineMO.java:2468)
	at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.attachDisk(VirtualMachineMO.java:1427)
	at com.cloud.storage.resource.VmwareStorageProcessor.attachVolume(VmwareStorageProcessor.java:2113)
	at com.cloud.storage.resource.VmwareStorageProcessor.attachVolume(VmwareStorageProcessor.java:1998)
...
2024-07-22 11:39:06,242 INFO  [c.c.h.v.u.VmwareHelper] (DirectAgent-296:ctx-f10157f9 10.0.33.11, job-37/job-38, cmd: AttachCommand) (logid:73329ae9) [ignored]failed to get message for exception: Scsi disk controller of type VirtualLsiLogicController not found among configured devices.
2024-07-22 11:39:06,250 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-296:ctx-f10157f9 10.0.33.11, job-37/job-38, cmd: AttachCommand) (logid:73329ae9) AttachCommand failed due to [Exception: java.lang.IllegalStateException
Message: Scsi disk controller of type VirtualLsiLogicController not found among configured devices.
].
java.lang.IllegalStateException: Scsi disk controller of type VirtualLsiLogicController not found among configured devices.
	at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.getScsiDiskControllerKey(VirtualMachineMO.java:2468)
	at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.attachDisk(VirtualMachineMO.java:1427)
...
2024-07-22 11:39:06,260 DEBUG [c.c.s.VolumeApiServiceImpl] (Work-Job-Executor-5:ctx-1495a9dd job-37/job-38 ctx-3cf350e6) (logid:73329ae9) Volume: DATA01 failed to attach to VM: null
2024-07-22 11:39:06,267 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:ctx-1495a9dd job-37/job-38 ctx-3cf350e6) (logid:73329ae9) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to attach volume DATA01 to VM testvm02-pvscsi
2024-07-22 11:39:06,267 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:ctx-1495a9dd job-37/job-38 ctx-3cf350e6) (logid:73329ae9) Rethrow exception com.cloud.utils.exception.CloudRuntimeException: Failed to attach volume DATA01 to VM testvm02-pvscsi
2024-07-22 11:39:06,268 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:ctx-1495a9dd job-37/job-38) (logid:73329ae9) Done with run of VM work job: com.cloud.vm.VmWorkAttachVolume for VM 5, job origin: 37
2024-07-22 11:39:06,268 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:ctx-1495a9dd job-37/job-38) (logid:73329ae9) Unable to complete AsyncJobVO: {id:38, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkAttachVolume, cmdInfo: rO0ABXNyAB9jb20uY2xvdWQudm0uVm1Xb3JrQXR0YWNoVm9sdW1lB62v-WGH4hwCAAJMAAhkZXZpY2VJZHQAEExqYXZhL2xhbmcvTG9uZztMAAh2b2x1bWVJZHEAfgABeHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAAFdAAUVm9sdW1lQXBpU2VydmljZUltcGxwc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAAAAAG, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 32985617269719, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Jul 22 11:39:04 UTC 2024, removed: null}, job origin:37
com.cloud.utils.exception.CloudRuntimeException: Failed to attach volume DATA01 to VM testvm02-pvscsi
	at com.cloud.storage.VolumeApiServiceImpl.sendAttachVolumeCommand(VolumeApiServiceImpl.java:4598)
	at com.cloud.storage.VolumeApiServiceImpl.orchestrateAttachVolumeToVM(VolumeApiServiceImpl.java:2489)
	at com.cloud.storage.VolumeApiServiceImpl.orchestrateAttachVolumeToVM(VolumeApiServiceImpl.java:5017)

sureshanaparti avatar Jul 22 '24 11:07 sureshanaparti

Fixed by #9636

JoaoJandre avatar Sep 30 '24 13:09 JoaoJandre