one icon indicating copy to clipboard operation
one copied to clipboard

NUMA monitoring inconsistent after host hardware change

Open OpenNebulaSupport opened this issue 1 year ago • 1 comments

Description Error undefined method [] for nil:NilClass when running onehost show X after replacing the host hardware.

Presumably the new hardware has different NUMA capabilities but same hostname and SSH config.

To Reproduce Steps to reproduce the behavior:

  1. Create a new dummy host: onehost create 127.0.0.1 --im kvm --vm kvm
  2. Modify the host database body with: onedb update-body host --id <dummy-host-id>

Add the following body, making sure to adapt the ID to the ID of the dummy host:

New Body
<HOST>
  <ID>CHANGE ME!</ID>
  <NAME>XXX</NAME>
  <STATE>2</STATE>
  <PREV_STATE>2</PREV_STATE>
  <IM_MAD><![CDATA[kvm]]></IM_MAD>
  <VM_MAD><![CDATA[kvm]]></VM_MAD>
  <CLUSTER_ID>105</CLUSTER_ID>
  <CLUSTER>XXX</CLUSTER>
  <HOST_SHARE>
    <MEM_USAGE>0</MEM_USAGE>
    <CPU_USAGE>0</CPU_USAGE>
    <TOTAL_MEM>527460924</TOTAL_MEM>
    <TOTAL_CPU>12800</TOTAL_CPU>
    <MAX_MEM>511060804</MAX_MEM>
    <MAX_CPU>12800</MAX_CPU>
    <RUNNING_VMS>0</RUNNING_VMS>
    <VMS_THREAD>1</VMS_THREAD>
    <DATASTORES>
      <DISK_USAGE><![CDATA[0]]></DISK_USAGE>
      <FREE_DISK><![CDATA[450084]]></FREE_DISK>
      <MAX_DISK><![CDATA[455882]]></MAX_DISK>
      <USED_DISK><![CDATA[5799]]></USED_DISK>
    </DATASTORES>
    <PCI_DEVICES/>
    <NUMA_NODES>
      <NODE>
        <CORE>
          <CPUS><![CDATA[0:-1,20:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[0]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[1:-1,21:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[1]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[2:-1,22:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[2]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[3:-1,23:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[3]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[4:-1,24:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[4]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[5:-1,25:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[8]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[6:-1,26:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[9]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[7:-1,27:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[10]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[8:-1,28:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[11]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[9:-1,29:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[12]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[24:-1,88:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[24]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[21:-1,85:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[21]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[52:-1,116:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[52]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[14:-1,78:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[14]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[42:-1,106:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[42]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[32:-1,96:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[32]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[60:-1,124:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[60]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[58:-1,122:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[58]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[22:-1,86:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[22]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[50:-1,114:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[50]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[48:-1,112:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[48]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[40:-1,104:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[40]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[7:-1,71:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[7]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[38:-1,102:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[38]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[30:-1,94:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[30]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[28:-1,92:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[28]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[56:-1,120:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[56]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[20:-1,84:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[20]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[18:-1,82:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[18]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[46:-1,110:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[46]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[5:-1,69:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[5]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[36:-1,100:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[36]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[26:-1,90:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[26]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[54:-1,118:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[54]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[16:-1,80:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[16]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[44:-1,108:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[44]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[34:-1,98:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[34]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[62:-1,126:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[62]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[55:-1,119:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[55]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[45:-1,109:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[45]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[6:-1,70:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[6]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[35:-1,99:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[35]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[63:-1,127:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[63]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[25:-1,89:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[25]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[53:-1,117:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[53]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[15:-1,79:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[15]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[43:-1,107:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[43]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[33:-1,97:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[33]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[61:-1,125:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[61]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[59:-1,123:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[59]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[23:-1,87:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[23]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[51:-1,115:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[51]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[49:-1,113:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[49]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[13:-1,77:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[13]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[41:-1,105:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[41]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[39:-1,103:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[39]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[31:-1,95:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[31]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[29:-1,93:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[29]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[57:-1,121:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[57]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[19:-1,83:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[19]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[47:-1,111:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[47]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[37:-1,101:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[37]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[27:-1,91:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[27]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[17:-1,81:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[17]]></ID>
        </CORE>
        <HUGEPAGE>
          <PAGES><![CDATA[0]]></PAGES>
          <SIZE><![CDATA[1048576]]></SIZE>
          <USAGE><![CDATA[0]]></USAGE>
        </HUGEPAGE>
        <HUGEPAGE>
          <PAGES><![CDATA[0]]></PAGES>
          <SIZE><![CDATA[2048]]></SIZE>
          <USAGE><![CDATA[0]]></USAGE>
        </HUGEPAGE>
        <MEMORY>
          <DISTANCE><![CDATA[0]]></DISTANCE>
          <TOTAL><![CDATA[527460924]]></TOTAL>
          <USAGE><![CDATA[0]]></USAGE>
        </MEMORY>
        <NODE_ID><![CDATA[0]]></NODE_ID>
      </NODE>
      <NODE>
        <CORE>
          <CPUS><![CDATA[10:-1,30:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[0]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[11:-1,31:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[1]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[12:-1,32:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[2]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[13:-1,33:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[3]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[14:-1,34:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[4]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[15:-1,35:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[8]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[16:-1,36:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[9]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[17:-1,37:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[10]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[18:-1,38:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[11]]></ID>
        </CORE>
        <CORE>
          <CPUS><![CDATA[19:-1,39:-1]]></CPUS>
          <DEDICATED><![CDATA[NO]]></DEDICATED>
          <FREE><![CDATA[2]]></FREE>
          <ID><![CDATA[12]]></ID>
        </CORE>
        <HUGEPAGE>
          <PAGES><![CDATA[0]]></PAGES>
          <SIZE><![CDATA[1048576]]></SIZE>
          <USAGE><![CDATA[0]]></USAGE>
        </HUGEPAGE>
        <HUGEPAGE>
          <PAGES><![CDATA[0]]></PAGES>
          <SIZE><![CDATA[2048]]></SIZE>
          <USAGE><![CDATA[0]]></USAGE>
        </HUGEPAGE>
        <MEMORY>
          <DISTANCE><![CDATA[1 0]]></DISTANCE>
          <TOTAL><![CDATA[66051828]]></TOTAL>
          <USAGE><![CDATA[0]]></USAGE>
        </MEMORY>
        <NODE_ID><![CDATA[1]]></NODE_ID>
      </NODE>
    </NUMA_NODES>
  </HOST_SHARE>
  <VMS/>
  <TEMPLATE>
    <ARCH><![CDATA[x86_64]]></ARCH>
    <CGROUPS_VERSION><![CDATA[1]]></CGROUPS_VERSION>
    <CPUSPEED><![CDATA[0]]></CPUSPEED>
    <FENCE_IP><![CDATA[10.11.0.244]]></FENCE_IP>
    <HOSTNAME><![CDATA[eon17]]></HOSTNAME>
    <HYPERVISOR><![CDATA[kvm]]></HYPERVISOR>
    <IM_MAD><![CDATA[kvm]]></IM_MAD>
    <KVM_CPU_MODEL><![CDATA[EPYC-Milan]]></KVM_CPU_MODEL>
    <KVM_CPU_MODELS><![CDATA[486 pentium pentium2 pentium3 pentiumpro qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 qemu64 kvm64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Broadwell-noTSX Broadwell-noTSX-IBRS Skylake-Client-noTSX-IBRS Icelake-Client Icelake-Client-noTSX Opteron_G1 Opteron_G2 Opteron_G3 EPYC EPYC-IBPB EPYC-Rome EPYC-Milan Dhyana]]></KVM_CPU_MODELS>
    <KVM_MACHINES><![CDATA[pc-i440fx-rhel7.6.0 pc pc-q35-rhel8.6.0 pc-q35-rhel9.4.0 q35 pc-q35-rhel8.5.0 pc-q35-rhel8.3.0 pc-q35-rhel7.6.0 pc-q35-rhel8.4.0 pc-q35-rhel9.2.0 pc-q35-rhel8.2.0 pc-q35-rhel9.0.0 pc-q35-rhel8.0.0 pc-q35-rhel8.1.0]]></KVM_MACHINES>
    <MODELNAME><![CDATA[AMD EPYC 7713P 64-Core Processor]]></MODELNAME>
    <RESERVED_CPU><![CDATA[]]></RESERVED_CPU>
    <RESERVED_MEM><![CDATA[16400120]]></RESERVED_MEM>
    <VERSION><![CDATA[6.4.4]]></VERSION>
    <VM_MAD><![CDATA[kvm]]></VM_MAD>
  </TEMPLATE>
  <MONITORING/>
</HOST>
  1. Run onehost show <dummy-host-id>

The CLI output will be the following:

root@daniel-XPS-9320:/usr/lib/one/ruby/cli# onehost show 5
HOST 5 INFORMATION                                                              
ID                    : 5                   
NAME                  : XXX  
CLUSTER               : XXX     
STATE                 : MONITORED           
IM_MAD                : kvm                 
VM_MAD                : kvm                 
LAST MONITORING TIME  : 06/26 14:46:42      

HOST SHARES                                                                     
RUNNING VMS           : 0                   
MEMORY                                                                          
  TOTAL               : 31G                 
  TOTAL +/- RESERVED  : 15.3G               
  USED (REAL)         : 7.8G                
  USED (ALLOCATED)    : 0K                  
CPU                                                                             
  TOTAL               : 1600                
  TOTAL +/- RESERVED  : 1600                
  USED (REAL)         : 32                  
  USED (ALLOCATED)    : 0                   

MONITORING INFORMATION                                                          
ARCH="x86_64"
CGROUPS_VERSION="2"
CPUSPEED="0"
FENCE_IP="10.11.0.244"
HOSTNAME="daniel-XPS-9320"
HYPERVISOR="kvm"
IM_MAD="kvm"
KVM_CPU_FEATURES="vme,ds,acpi,ss,ht,tm,pbe,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,xtpr,pdcm,osxsave,f16c,rdrand,arat,tsc_adjust,clflushopt,clwb,intel-pt,sha-ni,umip,pku,ospke,waitpkg,gfni,vaes,vpclmulqdq,rdpid,movdiri,movdir64b,pks,fsrm,md-clear,serialize,arch-lbr,stibp,arch-capabilities,core-capability,ssbd,avx-vnni,xsaveopt,xsavec,xgetbv1,xsaves,pdpe1gb,abm,invtsc,rdctl-no,ibrs-all,skip-l1dfl-vmentry,mds-no,pschange-mc-no"
KVM_CPU_MODEL="Broadwell-noTSX-IBRS"
KVM_CPU_MODELS="486 pentium pentium2 pentium3 pentiumpro coreduo n270 core2duo qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 kvm64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge IvyBridge-IBRS SapphireRapids SapphireRapids-noTSX Opteron_G1"
KVM_MACHINES="pc-i440fx-jammy ubuntu pc-i440fx-impish-hpb pc-q35-5.2 pc-i440fx-2.12 pc-i440fx-2.0 pc-i440fx-xenial pc-i440fx-6.2 pc pc-q35-4.2 pc-i440fx-2.5 pc-i440fx-4.2 pc-i440fx-focal pc-i440fx-hirsute pc-q35-xenial pc-i440fx-jammy-hpb pc-i440fx-5.2 pc-i440fx-1.5 pc-q35-2.7 pc-q35-eoan-hpb pc-i440fx-zesty pc-i440fx-disco-hpb pc-q35-groovy pc-i440fx-groovy pc-q35-artful pc-i440fx-2.2 pc-i440fx-trusty pc-i440fx-eoan-hpb pc-q35-focal-hpb pc-q35-jammy-maxcpus pc-q35-bionic-hpb pc-i440fx-artful pc-i440fx-2.7 pc-q35-6.1 pc-i440fx-jammy-maxcpus pc-i440fx-yakkety pc-q35-2.4 pc-q35-cosmic-hpb pc-q35-2.10 x-remote pc-i440fx-1.7 pc-q35-5.1 pc-q35-2.9 pc-i440fx-2.11 pc-i440fx-jammy-hpb-maxcpus pc-q35-3.1 pc-i440fx-6.1 pc-q35-4.1 pc-q35-jammy ubuntu-q35 pc-i440fx-2.4 pc-i440fx-4.1 pc-q35-eoan pc-q35-jammy-hpb pc-i440fx-5.1 pc-i440fx-2.9 pc-i440fx-bionic-hpb isapc pc-i440fx-1.4 pc-q35-cosmic pc-q35-2.6 pc-i440fx-3.1 pc-q35-bionic pc-q35-disco-hpb pc-i440fx-cosmic pc-q35-2.12 pc-i440fx-bionic pc-q35-groovy-hpb pc-q35-disco pc-i440fx-cosmic-hpb pc-i440fx-2.1 pc-i440fx-wily pc-q35-impish pc-q35-6.0 pc-i440fx-impish pc-i440fx-2.6 pc-q35-impish-hpb pc-q35-hirsute pc-q35-4.0.1 pc-q35-hirsute-hpb pc-i440fx-1.6 pc-q35-5.0 pc-q35-2.8 pc-i440fx-2.10 pc-q35-3.0 pc-i440fx-6.0 pc-q35-zesty pc-q35-4.0 pc-q35-focal microvm pc-i440fx-2.3 pc-q35-jammy-hpb-maxcpus pc-i440fx-focal-hpb pc-i440fx-disco pc-i440fx-4.0 pc-i440fx-groovy-hpb pc-i440fx-hirsute-hpb pc-i440fx-5.0 pc-i440fx-2.8 pc-q35-6.2 q35 pc-i440fx-eoan pc-q35-2.5 pc-i440fx-3.0 pc-q35-yakkety pc-q35-2.11"
MODELNAME="13th Gen Intel(R) Core(TM) i7-1360P"
RESERVED_CPU=""
RESERVED_MEM="16400120"
TOTAL_ZOMBIES="1"
VERSION="6.8.0"
VM_MAD="kvm"
ZOMBIES="358"

undefined method `[]' for nil:NilClass

Expected behavior The monitoring system should be able to detect NUMA capabilities changes and merge them properly into the host information or, at least, log it.

Details

  • Affected Component: Monitoring, core, CLI Tools
  • Hypervisor: KVM
  • Version: development

Additional context The CLI error comes from the function merge_numa_monitoring in the onehost_helper.rb CLI helper.

    def merge_numa_monitoring(numa_nodes, monitoring)
        return if monitoring.nil?
        
        monitoring = [monitoring] if monitoring.class == Hash
        numa_nodes.each do |node|
            mon_node = monitoring.find {|x| x['NODE_ID'] == node['NODE_ID'] }
            
            node['MEMORY']['FREE'] = mon_node['MEMORY']['FREE']
            node['MEMORY']['USED'] = mon_node['MEMORY']['USED']

            node['HUGEPAGE'].each do |hp|
                mon_hp = mon_node['HUGEPAGE'].find {|x| x['SIZE'] == hp['SIZE'] }
                hp['FREE'] = mon_hp['FREE']
                
            end
        end
    end

The function parameters numa_nodes and monitoring have the following values: numa_nodes:

{"CORE"=>{"CORES"=>"-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- ", "FREE"=>128, "USED"=>0}, "HUGEPAGE"=>[{"PAGES"=>"0", "SIZE"=>"1048576", "USAGE"=>"0"}, {"PAGES"=>"0", "SIZE"=>"2048", "USAGE"=>"0"}], "MEMORY"=>{"DISTANCE"=>"0", "TOTAL"=>"32458772", "USAGE"=>"0"}, "NODE_ID"=>"0"}
{"CORE"=>{"CORES"=>"-- -- -- -- -- -- -- -- -- -- ", "FREE"=>20, "USED"=>0}, "HUGEPAGE"=>[{"PAGES"=>"0", "SIZE"=>"1048576", "USAGE"=>"0"}, {"PAGES"=>"0", "SIZE"=>"2048", "USAGE"=>"0"}], "MEMORY"=>{"DISTANCE"=>"1 0", "TOTAL"=>"66051828", "USAGE"=>"0"}, "NODE_ID"=>"1"}

monitoring:

{"HUGEPAGE"=>[{"FREE"=>"0", "SIZE"=>"2048"}, {"FREE"=>"0", "SIZE"=>"1048576"}], "MEMORY"=>{"FREE"=>"17756328", "USED"=>"14702444"}, "NODE_ID"=>"0"}

The monitoring parameter only includes information from a single NUMA node due to the hardware modification, causing the aforementioned inconsistency.

Progress Status

  • [ ] Code committed
  • [ ] Testing - QA
  • [ ] Documentation (Release notes - resolved issues, compatibility, known issues)

OpenNebulaSupport avatar Jun 26 '24 13:06 OpenNebulaSupport

Tested in kvm-ssh microenv version 6.10 the steps indicated:

# step 1: host creation
onehost create 127.0.0.1 --im kvm --vm kvm
ID: 2
# step 2: host update
onedb update-body host --id 2
# step 3: host information
onehost show 2

for the step 3, this is the output:

HOST 2 INFORMATION
ID                    : 2
NAME                  : XXX
CLUSTER               : XXX
STATE                 : MONITORED
IM_MAD                : kvm
VM_MAD                : kvm
LAST MONITORING TIME  : -

HOST SHARES
RUNNING VMS           : 0
MEMORY
  TOTAL               : 503G
  TOTAL +/- RESERVED  : 487.4G
  USED (REAL)         : 0K
  USED (ALLOCATED)    : 0K
CPU
  TOTAL               : 12800
  TOTAL +/- RESERVED  : 12800
  USED (REAL)         :
  USED (ALLOCATED)    : 0

MONITORING INFORMATION
ARCH="x86_64"
CGROUPS_VERSION="1"
CPUSPEED="0"
FENCE_IP="10.11.0.244"
HOSTNAME="eon17"
HYPERVISOR="kvm"
IM_MAD="kvm"
KVM_CPU_MODEL="EPYC-Milan"
KVM_CPU_MODELS="486 pentium pentium2 pentium3 pentiumpro qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 qemu64 kvm64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Broadwell-noTSX Broadwell-noTSX-IBRS Skylake-Client-noTSX-IBRS Icelake-Client Icelake-Client-noTSX Opteron_G1 Opteron_G2 Opteron_G3 EPYC EPYC-IBPB EPYC-Rome EPYC-Milan Dhyana"
KVM_MACHINES="pc-i440fx-rhel7.6.0 pc pc-q35-rhel8.6.0 pc-q35-rhel9.4.0 q35 pc-q35-rhel8.5.0 pc-q35-rhel8.3.0 pc-q35-rhel7.6.0 pc-q35-rhel8.4.0 pc-q35-rhel9.2.0 pc-q35-rhel8.2.0 pc-q35-rhel9.0.0 pc-q35-rhel8.0.0 pc-q35-rhel8.1.0"
MODELNAME="AMD EPYC 7713P 64-Core Processor"
RESERVED_CPU=""
RESERVED_MEM="16400120"
VERSION="6.4.4"
VM_MAD="kvm"

NUMA NODES

  ID CORES                                                                                                                                                                                            USED FREE
   0 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --  0    128
   1 -- -- -- -- -- -- -- -- -- --                                                                                                                                                                    0    20

NUMA MEMORY

 NODE_ID TOTAL    USED_REAL            USED_ALLOCATED       FREE
       0 503G     -                    0K                   -
       1 63G      -                    0K                   -

NUMA HUGEPAGES

 NODE_ID SIZE     TOTAL    FREE     USED
       0 1024M    0        -        0
       0 2M       0        -        0
       1 1024M    0        -        0
       1 2M       0        -        0

WILD VIRTUAL MACHINES

NAME                                                      IMPORT_ID  CPU     MEMORY

VIRTUAL MACHINES

  ID USER     GROUP    NAME                                                                                        STAT  CPU     MEM HOST                                                               TIME

As it is shown, we can see a numa section and not a Ruby exception

MiguelERuiz avatar Aug 23 '24 14:08 MiguelERuiz

Closing this as works for me. If needed we will reopen it

rsmontero avatar Nov 05 '24 09:11 rsmontero