Objects Disappear After Unity Simulation

[TREEANA]

I want to make changes directly to a scene loaded in Unity, but the objects disappear when the simulation ends. Is there a solution for this?

If I want to add movement or interaction to loaded scenes, how should I do it?

[YueYANG1996]

The ProcThor/Holodeck scenes only appear at running time. I'm not sure if it is possible to edit them in Unity. @AlvaroHG, do you have any comments?

[basteran]

Hi I'm interested too in this Topic. I'd like to move some objects around using the Unity interface, maybe take some pictures and then save the state of the world. Do you know if this is possible? How can we do it?

[zoeyliu1999]

@basteran Hi! I've managed to move objects to match an expected layout by adjusting the generated .json file. It can be a bit time-consuming, so if anyone has a more efficient method or tool for this process, please let us know!

Here's how you can do it:

1. Locate Object Data in the JSON File: Find the rotation and position of each piece of furniture under the "objects" section in the JSON file. It should look something like this:

"objects": [
    {
        "assetId": "Bed_9",
        "id": "bed-0 (bedroom)",
        "kinematic": true,
        "position": {
            "x": 1.2107633399963378,
            "y": 0.6772908568382263,
            "z": 1.5
        },
        "rotation": {
            "x": 0,
            "y": 90,
            "z": 0
        },
        "material": null,
        "roomId": "bedroom",
        "vertices": [
            [246.65266799926758, 41.439504623413086],
            [246.65266799926758, 258.5604953765869],
            [-4.5, 258.5604953765869],
            [-4.5, 41.439504623413086]
        ],
        "object_name": "bed-0",
        "layer": "Procedural0"
    }
]

2. Adjust Object Position and Rotation in Unity: After connecting to Unity, move the objects to the desired positions and rotations within the scene. For example, let's say you move the bed to:

"position": {
    "x": -1.2,
    "y": 0.6772908568382263,
    "z": 0.5
},
"rotation": {
    "x": 0,
    "y": 45,
    "z": 0
}

3. Update the JSON File: Go back to your JSON file and update the position and rotation fields with the new values. Save the changes.
4. Reload the JSON File in Unity: Reconnect to Unity with the updated JSON file. Check if the objects have been correctly moved to the new positions.

[basteran]

@basteran Hi! I've managed to move objects to match an expected layout by adjusting the generated .json file. It can be a bit time-consuming, so if anyone has a more efficient method or tool for this process, please let us know!

Here's how you can do it:

1. Locate Object Data in the JSON File: Find the rotation and position of each piece of furniture under the "objects" section in the JSON file. It should look something like this:

"objects": [
    {
        "assetId": "Bed_9",
        "id": "bed-0 (bedroom)",
        "kinematic": true,
        "position": {
            "x": 1.2107633399963378,
            "y": 0.6772908568382263,
            "z": 1.5
        },
        "rotation": {
            "x": 0,
            "y": 90,
            "z": 0
        },
        "material": null,
        "roomId": "bedroom",
        "vertices": [
            [246.65266799926758, 41.439504623413086],
            [246.65266799926758, 258.5604953765869],
            [-4.5, 258.5604953765869],
            [-4.5, 41.439504623413086]
        ],
        "object_name": "bed-0",
        "layer": "Procedural0"
    }
]

2. Adjust Object Position and Rotation in Unity: After connecting to Unity, move the objects to the desired positions and rotations within the scene. For example, let's say you move the bed to:

"position": {
    "x": -1.2,
    "y": 0.6772908568382263,
    "z": 0.5
},
"rotation": {
    "x": 0,
    "y": 45,
    "z": 0
}

3. Update the JSON File: Go back to your JSON file and update the position and rotation fields with the new values. Save the changes.
4. Reload the JSON File in Unity: Reconnect to Unity with the updated JSON file. Check if the objects have been correctly moved to the new positions.

Hi @zoeyliu1999 , thanks for your suggestions!

I have some issues with what you said:

1. I cannot connect to Unity: I tried to use a Virtual Machine with Ubuntu 24.04 and Unity won't even start. I downloaded Unity on my Windows computer, Unity starts but it fails to load the Scene from file (using the specified script from the guide) as the Unity script has many errors.
2. How can I get the new position and rotation of an object while Unity is still open in order to update the JSON file?
3. What if I want to add or remove an object to/from the scene?

Thank you, I hope you can give me more support. I think it could be interesting to provide this kind of action to a developer for tuning the scene generated with Holodeck. It's a great tool, but in my opinion, it is too heavily dependent on ChatGPT and its generation..

[zoeyliu1999]

Hi @basteran, regarding your issues:

1. You might consider asking the authors for advice. I'm using macOS, and connect_to_unity.py works for me.

2. After running the script to connect to Unity, you should be able to see the exact scene and move objects within Unity. Once you've adjusted the objects to your desired position, you can check the new position and rotation in Unity, as shown in this image.

3. Yes, you can remove objects from the scene by deleting their descriptions in the JSON file. However, I am still figuring out how to add objects to the scene.

Hope this helps!

[basteran]

Thanks @zoeyliu1999 , I still have some questions:

1. Ok, I'll consider opening a new Issue about it
2. What do I do with the new position and rotation? Should I manually create/update a "bed" object (as in your example) in the JSON and add the position and rotation from the Transform Tab from Unity?
3. What if I remove a "receptacle" object, such as a table, that holds some other objects on top of it? That becomes tricky..

Thanks again, I really appreciate your help!

[TREEANA]

I implemented a feature that allows saving and loading of objects even in play mode.

https://drive.google.com/file/d/16e79pdidW5lvNdB5c9tWOMrChry0YJ4N/view?usp=sharing.

But every time I load, the objects keep sinking. LOL

[basteran]

I implemented a feature that allows saving and loading of objects even in play mode.

https://drive.google.com/file/d/16e79pdidW5lvNdB5c9tWOMrChry0YJ4N/view?usp=sharing.

But every time I load, the objects keep sinking. LOL

I guess you're making some mistakes when computing the y value for every object, even those hung on a wall!

[YueYANG1996]

I implemented a feature that allows saving and loading of objects even in play mode.

https://drive.google.com/file/d/16e79pdidW5lvNdB5c9tWOMrChry0YJ4N/view?usp=sharing.

But every time I load, the objects keep sinking. LOL

It looks like you were setting each object's y value to 0. For floor objects, you need to add half the object's height as the offset for the y-axis.

[hyeonmin11]

@TREEANA How did you do this? Can you share the code?