Blender Cycles Fidelity Test

[vis-prime]

Fixes / Photos here : https://github.com/google/model-viewer/issues/4482

This script renders a exr file to the output directory. Opens it & uses PILLOW to do ACES tonemaping. Saves the final png & deletes the exr file.

The command :

blender -b -P test/renderers/blender-cycles/render.py -- {"scenario": {...}, "outputFile": "..."}

Command breakdown

• blender is path to the blender executable , How do I mention it in the commit ? (3.6.4 or 4.X.X daily build )
• -b is for background mode
• -P runs the python script
• test/renderers/blender-cycles/render.py is the path to the python file
• and last part is the config json ( accessible under sys.argv[5] )

Current settings

• 128 samples (default: 4096 )
• use GPU (default: CPU)
• Importance sampling/Adaptive Sampling/Noise Threshold enabled (default: Enabled)
• OpenImage Denoiser Enabled (default: Enabled)
• colorspace Standard (default: 'AgX' in 4.X & "Filmic" in 3.X)"
• exports a temp.exr (32bit ZIP codec) to the output dir

WSL on my windows system does not work on my old system, so npm install didn't work :-( So I've replicated the changes from the V-Ray commit :-) and to get outputs, I am running the script directly using the config.json

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[bhouston]

@vis-prime amazing! I'll try it out now. I think you need to side the Google CLA as mentioned above! It is super simple to do. :). I think it should use whatever Blender you have installed, so just the default Blender path I think is sufficient.

[bhouston]

It may be useful to have a README.md that describes how to set this up?

Are you using Blender directly, I guess you modify PYTHONPATH to point to somewhere inside of Blender, or are you compiling Blender as a Python Module? I assume as well you have to match the local Python version to Blender's python version, or are you using their python build directly?

This would help me reproduce.

[vis-prime]

alright! CLA done, will do a readme rehearsal here

1. Ensure you have Blender installed. You can either install the full version of Blender or extract the zip file to a location of your choice. (~350mb)

2. When running the script, provide the path to the Blender executable as the first argument.

3. The entire process utilizes Blender's internal Python module, so there should be no concerns about python version mismatches.

4. Note that PILLOW is not included in Blender's Python & gets automatically installed within the script.

So in my case with windows and blender 3.6

C:/Program Files/Blender Foundation/Blender 3.6/blender 
-b 
-P 
F:/net/model-viewer/packages/render-fidelity-tools/test/renderers/blender-cycles/render.py 
-- {"scenario": 
         {
             "lighting": "...", 
            "dimensions": {"width": 2048, "height": 1536}, 
            "target": {"x": 0, "y": 3.8, "z": 0}, 
            "orbit": {"theta": 0, "phi": 90,
             "radius": 20}, 
             "verticalFoV": 45, 
             "renderSkybox": true, 
             "name": "DirectionalLightTest", 
             "model": "....gltf", 
             "exclude": ["stellar"]
         }, 
         "outputFile": "..._cycles.png"
 }

extras

https://docs.blender.org/manual/en/latest/advanced/command_line/launch/index.html#command-line-launch-index

[vis-prime]

Previews (click to zoom in)

Image order: Cycles | Model Viewer | Three gpu pathtracer

[elalish]

Looking good! Mind signing our CLA and resolving the conflicts? Once these new goldens are uploaded we can look at merging this. Looking at CesiumMan and BrainStem it appears Blender is rendering these at their first animation frame instead of their resting pose - can you fix that in the python?

[vis-prime]

CesiumMan model notes

1. This and any models which have rigs/armature can be set into REST pose so it's no longer in frame 1 pose.

2. This model also has a icosphere mesh ,which automatically loads as hidden in the viewport but this mesh is still enabled for rendering so it shows up in the final render

(left: viewport | center: render output | right: node tree)

Should I hide this mesh in render too ? or leave it since this is the default behavior ?

[elalish]

I believe the sphere is a test, probably of unreferenced meshes and whether they get accidentally displayed or not. Since it loads as "hidden in viewport", it sounds like Blender is doing the right thing, so I think we can go ahead and use that as a sign to hide it in the Python render as well.

[vis-prime]

The ones which got fixed after

• setting the armatures to REST state
• hiding collections in render output which were hidden in viewport by default (the icospheres)

Brainstem is still messed up, I guess it lost its original/resting position when the frame 1 got applied

[vis-prime]

which one of these is correct ?

{
      "name": "blender-cycles",
      "description": "Blender's Cycles render engine",
      "command": {
        "executable": "python3",
        "args": [
          "blender",
          "-b",
          "-P",
          "test/renderers/blender-cycles/render.py"
        ]
      }
    }

or

{
     "name": "blender-cycles",
     "description": "Blender's Cycles render engine",
     "command": {
       "executable": "blender",
       "args": [
         "-b",
         "-P",
         "test/renderers/blender-cycles/render.py"
       ]
     }
   }

[elalish]

Hmm, I'm not sure about "correctness" if they both work, but

The entire process utilizes Blender's internal Python module, so there should be no concerns about python version mismatches.

makes me think that the second one is better, since the first might use your random system version of Python?

[vis-prime]

I haven't been able to test it with the npm run update-screenshots command

npm install fails cause WSL does not work (long story , need to buy a new laptop)

I've been just loading the config.json & running the script directly, so not sure if either of these commands are correct

@bhouston can you give it a go ?

[bhouston]

@vis-prime I will have a look!

[julienduroure]

I just fixed the sphere issue. Should be available in tomorrow's build.

https://github.com/KhronosGroup/glTF-Blender-IO/commit/fc2a7ae81df857cb1b9f19d64245a71f8d075757

The sphere is used for bone/joint shape, and should neither been seen in render

[bhouston]

The second method seems the most reasonable, run blender and give it a python script seems simplest. I may have suggested the first approach, but I think I was wrong earlier.

[vis-prime]

Yeah, fully executing the code directly via blender sounds good. Just have to make sure that all the file paths are still valid. (relative paths might not work)

Questions :

1. Should I remove my icosphere hiding code in favor of https://github.com/KhronosGroup/glTF-Blender-IO/commit/fc2a7ae81df857cb1b9f19d64245a71f8d075757 becoming live in blender 4.0 ?

2. and what are the next steps ? upload the goldens ?

3. I'm guessing there's no need to exclude any of the configs since it's good to have an visual feedback on unsupported features (like iridescence) ?

[elalish]

1. Yes
2. Yes
3. Correct, generally we exclude ones that crash or have some problem unrelated to glTF.

[vis-prime]

Some crucial changes are coming to GLTF & Principled v2 logic in the coming weeks (blender 4.0 releases in November)

Right now there are some material issues with using the 4.0 daily builds (https://github.com/KhronosGroup/glTF-Blender-IO/pull/2014#issuecomment-1752909459)

Is it better to wait for 4.0 release ? or Render the Golden's with the old principled bsdf from blender 3.6 and update it later ?

[elalish]

I'll defer to @bhouston since he's paying for the work, but since I don't see any particular time rush, I think it might be easier to wait for the 4.0 release.

[julienduroure]

Some crucial changes are coming to GLTF & Principled v2 logic in the coming weeks (blender 4.0 releases in November)

Changed are made. You should be able to get good render with 4.0 beta now.

[vis-prime]

awesome, looking good now!

I managed to get WSL2 working, there are still some puppeteer related errors when I run npm commands , but since this script does not need a browser, I'll try to run it via npm and see if the paths are valid/it works as expected (these images were rendered directly using config.json)

image order : Model Viewer | THREE-GPU-Pathtracer | Cycles

[elalish]

Looking great! I can already see some bugs it's catching in Blender. You'll probably want to wait for some more of @bhouston's PRs to land so that you're updating the right scenarios.

[elalish]

Is this ready for review? It's still marked as draft.

[vis-prime]

blender 4.0 is out ! to install on linux : sudo snap install blender --classic

Tried wsl2 , no luck

Only thing i wanted to confirm was to check if blender was getting the full paths as arg and not the relative paths (as blender's python env might not have access to it)

The error i got while testing with wsl2

prime@vis:~/dev/model-viewer/packages/render-fidelity-tools$ npm run update-screenshots khronos-BoomBox blender-cycles

> @google/[email protected] update-screenshots
> node --experimental-modules ./lib/workflows/update-screenshots.js ./test/config.json khronos-BoomBox blender-cycles

🆙 Updating screenshots
⏭  Skipping khronos-DragonAttenuation...
⏭  Skipping khronos-IridescentDishWithOlives...
⏭  Skipping khronos-ToyCar...
⏭  Skipping khronos-SheenChair...
⏭  Skipping khronos-MaterialsVariantsShoe...
blender-cycles: Rendering khronos-BoomBox...Error: Offline rendering process for Blender's Cycles render engine failed: [ALSOFT] (EE) Failed to set real-time priority for thread: Operation not permitted (1)

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: /snap/blender/4217/4.0/python/bin/python3.10 -m pip install --upgrade pip

    at updateScreenshots (file:///home/prime/dev/model-viewer/packages/render-fidelity-tools/lib/workflows/update-screenshots.js:130:31)
📋 Screenshot updates concluded
npm ERR! Lifecycle script `update-screenshots` failed with error: 
npm ERR! Error: command failed 
npm ERR!   in workspace: @google/[email protected] 
npm ERR!   at location: /home/prime/dev/model-viewer/packages/render-fidelity-tools 

[elalish]

I'm not sure, but I'll let @bhouston guide this one.

[vis-prime]

Installed npm packages with WSL2 then on windows ran npm run update-screenshots khronos-BoomBox blender-cycles

and after fixing the relative path issue by doing a some path surgery , it still shows the error message but the render output is created correctly !

(another note : none of the python print statements,render progress messages or error messages show up on the terminal when running the update-screenshot command )

(in the video i'm using 12 samples to get the file quickly)

https://github.com/google/model-viewer/assets/119810373/b6142e00-3a78-46cf-a33e-e9c526387ff0

[vis-prime]

on WSL2 ran the new npm run render-goldens -- --renderer=blender-cycles -m command

Image is generated correctly but there's an error at the end which stops the code, (so just ran the command multiple times to get the all the remaining goldens)

Image order : cycles | model-viewer

[elalish]

Sorry, I kind of lost track of this one. Should I go ahead and merge it? Appears to be complete now?

[vis-prime]

I'll just run/test the command in the latest blender version and let you know..

Can merge after that

Edit : should be okay to merge now