The visualization window is empty

[wwdok]

I run the control_your_robot example, but got this:

[zhrli]

same here ，I found there is no camera object

[kassiokienitz]

me too, all looks to me good, but I get only a black screem. I'm testing on a Hyper-V type 2 VM running Ubuntu 22.

[zhrli]

I added a camera in this project,but the simulation is very slow. please try to fix it .

import numpy as np

import genesis as gs

########################## 初始化 ########################## gs.init(backend=gs.gpu)

########################## 创建场景 ########################## scene = gs.Scene( viewer_options = gs.options.ViewerOptions( camera_pos = (0, -3.5, 2.5), camera_lookat = (0.0, 0.0, 0.5), camera_fov = 30, res = (960, 640), max_FPS = 60, ), sim_options = gs.options.SimOptions( dt = 0.01, ), show_viewer = True, )

########################## 实体 ########################## plane = scene.add_entity( gs.morphs.Plane(), ) franka = scene.add_entity( gs.morphs.MJCF( file = 'xml/franka_emika_panda/panda.xml', ), )

cam = scene.add_camera( res = (640, 480), pos = (3.5, 0.0, 2.5), lookat = (0, 0, 0.5), fov = 30, GUI = False, ) ########################## 构建 ########################## scene.build()

jnt_names = [ 'joint1', 'joint2', 'joint3', 'joint4', 'joint5', 'joint6', 'joint7', 'finger_joint1', 'finger_joint2', ] dofs_idx = [franka.get_joint(name).dof_idx_local for name in jnt_names]

############ 可选：设置控制增益 ############

设置位置增益

franka.set_dofs_kp( kp = np.array([4500, 4500, 3500, 3500, 2000, 2000, 2000, 100, 100]), dofs_idx_local = dofs_idx, )

设置速度增益

franka.set_dofs_kv( kv = np.array([450, 450, 350, 350, 200, 200, 200, 10, 10]), dofs_idx_local = dofs_idx, )

设置安全的力范围

franka.set_dofs_force_range( lower = np.array([-87, -87, -87, -87, -12, -12, -12, -100, -100]), upper = np.array([ 87, 87, 87, 87, 12, 12, 12, 100, 100]), dofs_idx_local = dofs_idx, )

硬重置

for i in range(150): if i < 50: franka.set_dofs_position(np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), dofs_idx) elif i < 100: franka.set_dofs_position(np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), dofs_idx) else: franka.set_dofs_position(np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx)

scene.step()

PD控制

for i in range(1250): if i == 0: franka.control_dofs_position( np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), dofs_idx, ) elif i == 250: franka.control_dofs_position( np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), dofs_idx, ) elif i == 500: franka.control_dofs_position( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx, ) elif i == 750: # 用速度控制第一个自由度，其余的用位置控制 franka.control_dofs_position( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0])[1:], dofs_idx[1:], ) franka.control_dofs_velocity( np.array([1.0, 0, 0, 0, 0, 0, 0, 0, 0])[:1], dofs_idx[:1], ) elif i == 1000: franka.control_dofs_force( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx, ) # 这是根据给定控制命令计算的控制力 # 如果使用力控制，它与给定的控制命令相同 print('控制力:', franka.get_dofs_control_force(dofs_idx))

# 这是自由度实际经历的力
print('内部力:', franka.get_dofs_force(dofs_idx))

scene.step()
cam.set_pose(
    pos    = (3.0 * np.sin(i / 60), 3.0 * np.cos(i / 60), 2.5),
    lookat = (0, 0, 0.5),
)
cam.render()

cam.stop_recording(save_to_filename='../video/video.mp4', fps=60)

[zhouxian]

I added a camera in this project,but the simulation is very slow. please try to fix it .

import numpy as np

import genesis as gs

########################## 初始化 ########################## gs.init(backend=gs.gpu)

########################## 创建场景 ########################## scene = gs.Scene( viewer_options = gs.options.ViewerOptions( camera_pos = (0, -3.5, 2.5), camera_lookat = (0.0, 0.0, 0.5), camera_fov = 30, res = (960, 640), max_FPS = 60, ), sim_options = gs.options.SimOptions( dt = 0.01, ), show_viewer = True, )

########################## 实体 ########################## plane = scene.add_entity( gs.morphs.Plane(), ) franka = scene.add_entity( gs.morphs.MJCF( file = 'xml/franka_emika_panda/panda.xml', ), )

cam = scene.add_camera( res = (640, 480), pos = (3.5, 0.0, 2.5), lookat = (0, 0, 0.5), fov = 30, GUI = False, ) ########################## 构建 ########################## scene.build()

jnt_names = [ 'joint1', 'joint2', 'joint3', 'joint4', 'joint5', 'joint6', 'joint7', 'finger_joint1', 'finger_joint2', ] dofs_idx = [franka.get_joint(name).dof_idx_local for name in jnt_names]

############ 可选：设置控制增益 ############

设置位置增益

franka.set_dofs_kp( kp = np.array([4500, 4500, 3500, 3500, 2000, 2000, 2000, 100, 100]), dofs_idx_local = dofs_idx, )

设置速度增益

franka.set_dofs_kv( kv = np.array([450, 450, 350, 350, 200, 200, 200, 10, 10]), dofs_idx_local = dofs_idx, )

设置安全的力范围

franka.set_dofs_force_range( lower = np.array([-87, -87, -87, -87, -12, -12, -12, -100, -100]), upper = np.array([ 87, 87, 87, 87, 12, 12, 12, 100, 100]), dofs_idx_local = dofs_idx, )

硬重置

for i in range(150): if i < 50: franka.set_dofs_position(np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), dofs_idx) elif i < 100: franka.set_dofs_position(np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), dofs_idx) else: franka.set_dofs_position(np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx)

scene.step()

PD控制

for i in range(1250): if i == 0: franka.control_dofs_position( np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), dofs_idx, ) elif i == 250: franka.control_dofs_position( np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), dofs_idx, ) elif i == 500: franka.control_dofs_position( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx, ) elif i == 750: # 用速度控制第一个自由度，其余的用位置控制 franka.control_dofs_position( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0])[1:], dofs_idx[1:], ) franka.control_dofs_velocity( np.array([1.0, 0, 0, 0, 0, 0, 0, 0, 0])[:1], dofs_idx[:1], ) elif i == 1000: franka.control_dofs_force( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx, ) # 这是根据给定控制命令计算的控制力 # 如果使用力控制，它与给定的控制命令相同 print('控制力:', franka.get_dofs_control_force(dofs_idx))

# 这是自由度实际经历的力
print('内部力:', franka.get_dofs_force(dofs_idx))

scene.step()
cam.set_pose(
    pos    = (3.0 * np.sin(i / 60), 3.0 * np.cos(i / 60), 2.5),
    lookat = (0, 0, 0.5),
)
cam.render()

cam.stop_recording(save_to_filename='../video/video.mp4', fps=60)

What do you mean by slow? What’s the for if you turn off the viewer? Rendering is expected to be ~few hundred fps and much slower than simulation

[zhrli]

I added a camera in this project,but the simulation is very slow. please try to fix it . import numpy as np import genesis as gs ########################## 初始化 ########################## gs.init(backend=gs.gpu) ########################## 创建场景 ########################## scene = gs.Scene( viewer_options = gs.options.ViewerOptions( camera_pos = (0, -3.5, 2.5), camera_lookat = (0.0, 0.0, 0.5), camera_fov = 30, res = (960, 640), max_FPS = 60, ), sim_options = gs.options.SimOptions( dt = 0.01, ), show_viewer = True, ) ########################## 实体 ########################## plane = scene.add_entity( gs.morphs.Plane(), ) franka = scene.add_entity( gs.morphs.MJCF( file = 'xml/franka_emika_panda/panda.xml', ), ) cam = scene.add_camera( res = (640, 480), pos = (3.5, 0.0, 2.5), lookat = (0, 0, 0.5), fov = 30, GUI = False, ) ########################## 构建 ########################## scene.build() jnt_names = [ 'joint1', 'joint2', 'joint3', 'joint4', 'joint5', 'joint6', 'joint7', 'finger_joint1', 'finger_joint2', ] dofs_idx = [franka.get_joint(name).dof_idx_local for name in jnt_names] ############ 可选：设置控制增益 ############

设置位置增益

franka.set_dofs_kp( kp = np.array([4500, 4500, 3500, 3500, 2000, 2000, 2000, 100, 100]), dofs_idx_local = dofs_idx, )

设置速度增益

franka.set_dofs_kv( kv = np.array([450, 450, 350, 350, 200, 200, 200, 10, 10]), dofs_idx_local = dofs_idx, )

设置安全的力范围

franka.set_dofs_force_range( lower = np.array([-87, -87, -87, -87, -12, -12, -12, -100, -100]), upper = np.array([ 87, 87, 87, 87, 12, 12, 12, 100, 100]), dofs_idx_local = dofs_idx, )

硬重置

for i in range(150): if i < 50: franka.set_dofs_position(np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), dofs_idx) elif i < 100: franka.set_dofs_position(np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), dofs_idx) else: franka.set_dofs_position(np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx)

scene.step()

PD控制

for i in range(1250): if i == 0: franka.control_dofs_position( np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), dofs_idx, ) elif i == 250: franka.control_dofs_position( np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), dofs_idx, ) elif i == 500: franka.control_dofs_position( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx, ) elif i == 750: # 用速度控制第一个自由度，其余的用位置控制 franka.control_dofs_position( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0])[1:], dofs_idx[1:], ) franka.control_dofs_velocity( np.array([1.0, 0, 0, 0, 0, 0, 0, 0, 0])[:1], dofs_idx[:1], ) elif i == 1000: franka.control_dofs_force( np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), dofs_idx, ) # 这是根据给定控制命令计算的控制力 # 如果使用力控制，它与给定的控制命令相同 print('控制力:', franka.get_dofs_control_force(dofs_idx))

# 这是自由度实际经历的力
print('内部力:', franka.get_dofs_force(dofs_idx))

scene.step()
cam.set_pose(
    pos    = (3.0 * np.sin(i / 60), 3.0 * np.cos(i / 60), 2.5),
    lookat = (0, 0, 0.5),
)
cam.render()

cam.stop_recording(save_to_filename='../video/video.mp4', fps=60)

What do you mean by slow? What’s the for if you turn off the viewer? Rendering is expected to be ~few hundred fps and much slower than simulation

I was wondering whether I add this cam in this code in a right way.

[zhouxian]

It looks correct to me. Please share you log screen

[zhrli]

It looks correct to me. Please share you log screen

[zhouxian]

This is indeed abnormal… what’s you os, system and detailed setting? Are you doing local desktop?

[zhrli]

This is indeed abnormal… what’s you os, system and detailed setting? Are you doing local desktop? yes ,local desktop

sys infor: Linux ubuntu 6.2.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

what do you mean by detailed setting?

[ALS-eesoline]

I get a black window and a warning "[frontend_ir.cpp:begin_frontend_struct_for_on_snode@1678] ti.loop_config(serialize=True) does not have effect on the struct for. The execution order is not guaranteed." I thought it may cause the empty black window, however, it is not mentioned in many issues. Do you get this warning? or it is just my problem.

Ubuntu 2404, x86_64

[duburcqa]

Unfortunately I do not think this issue is actionable at this point. Can you provide more details about your setup ? Native linux or VM ? Intel CPU or AMD CPU ? ARM or x86 ? Discrete Nvidia GPU ? ...

[sofyaslk]

Has anyone found a solution for this?