lua-scripts icon indicating copy to clipboard operation
lua-scripts copied to clipboard
verified publisher icon darktable-org

Added UltraHDR export plugin to generate UltraHDR JPEG images.

Open koto opened this issue 1 year ago • 6 comments

koto avatar Aug 23 '24 15:08 koto

I/m not sure how to use this. I read the header comment and it says I need to provide a jpg image and a gainmap image. I read the ultrahdr_app documentation and it says I need to supply an SDR image and an HDR image.

Could you please provide a workflow that I can use to test this?

Thanks

wpferguson avatar Aug 24 '24 17:08 wpferguson

I/m not sure how to use this. I read the header comment and it says I need to provide a jpg image and a gainmap image. I read the ultrahdr_app documentation and it says I need to supply an SDR image and an HDR image.

The script is using the API-4 encoding variant, i.e. one that expects a JPEG SDR and a JPEG gainmap on input, both of which can be created quite easily in darktable. In hindsight, I probably should make this clear in the UI as well.

Could you please provide a workflow that I can use to test this?

https://discuss.pixls.us/t/manual-creation-of-ultrahdr-images/45004/12

koto avatar Aug 25 '24 13:08 koto

I added a combobox in UI for choosing the encoding variant, so that user knows what the required inputs are. Adding new encoding variants in the future (e.g. SDR and HRD -> UltraHDR) will now be easier.

koto avatar Aug 26 '24 14:08 koto

I changed the plugin to appear as a lib (prevously it was an export storage plugin), cleaning up the UI, adding more descriptions, tooltips, and exposing preferences. I also added support for generating UltraHDR images from SDR + JPEG-XML HDR image pairs.

https://www.youtube.com/watch?v=q5hrx2E7BL4 demonstrates generating UltraHDR JPEGs from SDR + monochrome gainmap pairs.

The biggest challenge for plugin users is that is requires compiling ultrahdr_app from libultrahdr, but I suspect this might eventually change when the library makes it to package managers. Other required binaries (ffmpeg, exiftool) are trivial to obtain.

@wpferguson, LMK if this passes the bar for lua-scripts contrib. If not, I might release it in a separate repo for my own use case.

koto avatar Sep 19 '24 13:09 koto

I'm trying to run this in a completely new environment. I have an SDR raw and I duplicated it and created a gainmap. I select option 1 (SDR + Gainmap) and use original location. I hit the button to run it and it crashes in option 2.

    79.5329 LUA INFO: callback: using encoding variant 1 
    79.5646 LUA ERROR : ...ll/.config/dttest502_ultrahdr/lua/lib/dtutils/string.lua:305: bad argument #1 to 'match' (string expected, got nil)
stack traceback:
	[C]: in ?
	[C]: in function 'string.match'
	...ll/.config/dttest502_ultrahdr/lua/lib/dtutils/string.lua:305: in upvalue '_should_be_sanitized'
	...ll/.config/dttest502_ultrahdr/lua/lib/dtutils/string.lua:316: in function 'lib/dtutils.string.sanitize'
	(...tail calls...)
	...bill/.config/dttest502_ultrahdr/lua/lib/dtutils/file.lua:459: in function 'lib/dtutils.file.check_if_file_exists'
	...bill/.config/dttest502_ultrahdr/lua/contrib/ultrahdr.lua:150: in upvalue 'assert_settings_correct'
	...bill/.config/dttest502_ultrahdr/lua/contrib/ultrahdr.lua:352: in function <...bill/.config/dttest502_ultrahdr/lua/contrib/ultrahdr.lua:343>
	[C]: in ?

I hovered all the widgets looking for tooltips to tell me what is wrong or what I'm missing, but I didn't get enough information to figure it out.

wpferguson avatar Sep 20 '24 01:09 wpferguson

I fixed the crash, thanks! And many other errors that are obvious when running the plugin on a clean profile 😅

When generating UltraHDRs with gainmaps as inputs, the metadata.cfg file is also required. Previously you had to select one from libultrahdr/examples directory, but now I added the option of just generating it, which will happen on first run.

EDIT: The metadata.cfg file is no longer needed; instead the UI has appropriate widgets for all the params, and the file will be generated automatically.

Now the only required setting on fresh install on most distros should be to choose the ultrahdr_app executable.

koto avatar Sep 20 '24 09:09 koto

Saw your comment on pixls. I've been pretty swamped lately trying to get everything done for feature freeze. Since feature freeze I've been updating the docs and triaging issues.

If you think this is ready, push your changes and I'll go over it this week.

wpferguson avatar Nov 24 '24 20:11 wpferguson

No worries @wpferguson, I'm not pressed for time here. I am using this plugin already and consider it stable-enough (there's no bugs I'm aware of), though obviously it's hard to tell how universal the workflow(s) in the plugin would be for other users. I'll leave the decision on whether to merge to contrib/ to you, I'm fine either way.

koto avatar Nov 25 '24 16:11 koto

Did some testing...

When I test I clone the latest copy of darktable master and compile it. I pull the latest master from the lua scripts. Then I pull the PR into a test branch in the lua scripts directory.

First test:

  • Start darktable and go to script_manager to start ultrahdr.
  • Tried to page to the last page of contrib and script_manager wouldn't advance the page.
  • Checked the console (I was running with -d lua) and there were all kinds errors about a missing field.
  • Tried restarting and the error persisted.
  • Checked out the master branch of the lua-scripts and script_manager worked fine.
  • Went back to the test branch and got the field errors again.
  • Stopped darktable, deleted the ultrahdr script, then restarted and script_manager worked fine.
  • Stopped darktable, checked out ultrahdr, which restored it, then restarted and script_manager worked fine, letting me page forward and start ultrahdr.
  • I tried several tests to see if I could recreate the problem but couldn't.
  • Removed the whole environment and recreated it and everything worked, so I don't know what that was all about.

Second test:

  • Selected a CR2 file (huge sunflower from pixls.us play raw) and duplicated it.
  • Opened the dupe and made it grayscale for a crude tonemap.
  • Filled in the path with the suggested and tried to export. Message popped up saying couldn't export... I think this is probably because the exporter had never been opened so it wasn't initialized and when you try and get the settings you get some useless values.
  • Tried turning on logging, but it isn't set up properly. No log level was set so logging wouldn't produce any output. Also, log level needs to be set in each function since darktable lua is event driven. You can look at script_manager as an example of how to make it work.
  • Exported an image with the exporter, then tried ultrahdr again and I'm still getting the error about not being able to export.
  • Exported the gain map to a jpg, removed the dupe jpeg, imported the jpg gainmap and tried the pair again. Still could not export the CR2.

wpferguson avatar Dec 03 '24 03:12 wpferguson

Did some testing...

When I test I clone the latest copy of darktable master and compile it. I pull the latest master from the lua scripts. Then I pull the PR into a test branch in the lua scripts directory.

First test:

  • Start darktable and go to script_manager to start ultrahdr.
  • Tried to page to the last page of contrib and script_manager wouldn't advance the page.
  • Checked the console (I was running with -d lua) and there were all kinds errors about a missing field.

Was that version or get_substitution_tooltip? If yes, that is probably fixed in https://github.com/koto/lua-scripts/commit/87ca8bde27219034a6e250d19cea14f0be6f1ca6; I realized overnight that I should have rebased the PR branch, it was on top of an older master. This should be fixed now, I force-pushed the rebased branch.

If it's not that, IDK what happened :/

  • Tried restarting and the error persisted.
  • Checked out the master branch of the lua-scripts and script_manager worked fine.
  • Went back to the test branch and got the field errors again.
  • Stopped darktable, deleted the ultrahdr script, then restarted and script_manager worked fine.
  • Stopped darktable, checked out ultrahdr, which restored it, then restarted and script_manager worked fine, letting me page forward and start ultrahdr.
  • I tried several tests to see if I could recreate the problem but couldn't.
  • Removed the whole environment and recreated it and everything worked, so I don't know what that was all about.

I only have 4.8.1 here, but starting on that version afresh seems to work (up to the point of enabling the plugin without errors):

$ mkdir darktable.tmp
$ /Applications/darktable.app/Contents/MacOS/darktable -d lua --library /Users/koto/darktable.tmp/library --configdir /Users/koto/darktable.tmp
[[ install lua-scripts via UI and close DT ]]
$ cd darktable.tmp/lua
$ git fetch origin pull/502/head:ultrahdr
$ git checkout ultrahdr
$ /Applications/darktable.app/Contents/MacOS/darktable -d lua --library /Users/koto/darktable.tmp/library --configdir /Users/koto/darktable.tmp
[[ start contrib/ultrahdr script via script manager ]]

Second test:

  • Selected a CR2 file (huge sunflower from pixls.us play raw) and duplicated it.
  • Opened the dupe and made it grayscale for a crude tonemap.
  • Filled in the path with the suggested and tried to export. Message popped up saying couldn't export... I think this is probably because the exporter had never been opened so it wasn't initialized and when you try and get the settings you get some useless values.
  • Tried turning on logging, but it isn't set up properly. No log level was set so logging wouldn't produce any output. Also, log level needs to be set in each function since darktable lua is event driven. You can look at script_manager as an example of how to make it work.
  • Exported an image with the exporter, then tried ultrahdr again and I'm still getting the error about not being able to export.
  • Exported the gain map to a jpg, removed the dupe jpeg, imported the jpg gainmap and tried the pair again. Still could not export the CR2.

I can't reproduce this on 4.8.1/MacOS Sequoia 15.1.1 + Homebrew libultrahdr 1.3.0

  1. I imported CR2 from https://discuss.pixls.us/t/darktable-filmic-v5-vs-v6-sunflower-challenge/32503 to the fresh library.
  2. Created a duplicate image, dropped its exposure a bit, enabled monochrome module, added tone equalizer, kept only the highlights (that's only for visuals, it should not affect the export). No editing to the base image, history shows only the defaults for the scene-referred workflow.
  3. Switched to lighttable mode
  4. Set the ultrahdr output file path pattern to recommended $(FILE_FOLDER)/$(FILE_NAME)_ultrahdr, did not touch the export module at all.
  5. Selected both images.
  6. Selected SDR + gain map mode (it might have been the default, I don't remember)
  7. Pressed generate UltraHDR button.

File was generated with no errors in the UI, stdout shows only expected output. All the binaries were already in the path, so I didn't have to configure them.

580.5940 [pixelpipe_process CPU] buffer aligment problem: IN=0x13ecf8040 OUT=0x38be28000
   581.3681 [pixelpipe_process CPU] buffer aligment problem: IN=0x30ebf0040 OUT=0x38be28000
Warning: ICC_Profile deleted. Image colors may be affected - /private/var/folders/8j/0y62j51n1z59sbgvwjskkc9m0000gn/T/0L0A3314.jpg
    1 image files created
Warning: ICC_Profile deleted. Image colors may be affected - /private/var/folders/8j/0y62j51n1z59sbgvwjskkc9m0000gn/T/0L0A3314.CR2_gainmap.jpg
    1 image files updated
   705.6702 [pixelpipe_process CPU] buffer aligment problem: IN=0x14d978040 OUT=0x14e718000
   814.5790 [pixelpipe_process CPU] buffer aligment problem: IN=0x30ebf0040 OUT=0x38be28000
   848.0272 [pixelpipe_process CPU] buffer aligment problem: IN=0x13ecf8040 OUT=0x34f380000
   848.2606 [pixelpipe_process CPU] buffer aligment problem: IN=0x14c658040 OUT=0x14d030000

$ grep ultrahdr darktablerc
lua/executable_paths/ultrahdr_app=/opt/homebrew/bin/ultrahdr_app
lua/module_ultrahdr/copy_exif=FALSE
lua/module_ultrahdr/encoding_variant=1
lua/module_ultrahdr/gainmap_downsampling=0
lua/module_ultrahdr/hdr_capacity_max=6
lua/module_ultrahdr/hdr_capacity_min=1
lua/module_ultrahdr/import_to_darktable=FALSE
lua/module_ultrahdr/max_content_boost=6
lua/module_ultrahdr/min_content_boost=1
lua/module_ultrahdr/output_filepath_pattern=$(FILE_FOLDER)/$(FILE_NAME)_ultrahdr
lua/module_ultrahdr/overwrite_on_conflict=FALSE
lua/module_ultrahdr/quality=95
lua/module_ultrahdr/selection_type=1
lua/module_ultrahdr/target_display_peak_nits=10000
lua/script_manager/contrib/ultrahdr=TRUE
plugins/lighttable/1/module_ultrahdr_visible=TRUE
plugins/lighttable/module_ultrahdr/expanded=TRUE

I'll look into reproducing on prebuilt DT master + fixing the log outputs.

koto avatar Dec 03 '24 06:12 koto

I checked out release-4.8.1 and compiled that then tested. Much better, but now I'm wondering what's broken in master

   189.0564 LUA DEBUG: ultrahdr.lua: set_profile: 206: 26 11 1 10 
   189.7987 LUA DEBUG: ultrahdr.lua: set_profile: 206: 1 1 11 -10 
   189.8037 LUA DEBUG: ultrahdr.lua: set_profile: 206: 26 11 1 10 
   190.4006 LUA DEBUG: ultrahdr.lua: set_profile: 206: 1 1 11 -10 
   190.4009 LUA DEBUG: ultrahdr.lua: generate_ultrahdr: 473: Exported files: /tmp/0L0A3314.jpg, /tmp/0L0A3314.CR2_gainmap.jpg 
   190.4010 LUA DEBUG: ultrahdr.lua: execute_cmd: 390: /usr/bin/exiftool -all= /tmp/0L0A3314.jpg -o /tmp/0L0A3314.jpg.noexif 
Warning: ICC_Profile deleted. Image colors may be affected - /tmp/0L0A3314.jpg
    1 image files created
   190.4874 LUA DEBUG: ultrahdr.lua: execute_cmd: 390: /usr/bin/exiftool -all= /tmp/0L0A3314.CR2_gainmap.jpg -overwrite_original 
Warning: ICC_Profile deleted. Image colors may be affected - /tmp/0L0A3314.CR2_gainmap.jpg
    1 image files updated
   190.5622 LUA DEBUG: ultrahdr.lua: execute_cmd: 390: /usr/local/bin/ultrahdr_app -m 0 -i /tmp/0L0A3314.jpg.noexif -g /tmp/0L0A3314.CR2_gainmap.jpg -L 10000 -f /tmp/metadata.cfg -z /tmp/0L0A3314_ultrahdr.jpg 
unsupported option -L

## ultra hdr demo application.
Usage : /usr/local/bin/ultrahdr_app 
    -m    mode of operation. [0:encode, 1:decode] 

## encoder options : 
    -p    raw hdr intent input resource (10-bit), required for encoding scenarios 0, 1, 2, 3. 
    -y    raw sdr intent input resource (8-bit), required for encoding scenarios 1, 2. 
    -a    raw hdr intent color format, optional. [0:p010, 5:rgba1010102 (default)] 
    -b    raw sdr intent color format, optional. [1:yuv420, 3:rgba8888 (default)] 
    -i    compressed sdr intent input resource (jpeg), required for encoding scenarios 2, 3, 4. 
    -g    compressed gainmap input resource (jpeg), required for encoding scenario 4. 
    -w    input file width, required for encoding scenarios 0, 1, 2, 3. 
    -h    input file height, required for encoding scenarios 0, 1, 2, 3. 
    -C    hdr intent color gamut, optional. [0:bt709, 1:p3 (default), 2:bt2100] 
    -c    sdr intent color gamut, optional. [0:bt709 (default), 1:p3, 2:bt2100] 
    -t    hdr intent color transfer, optional. [0:linear, 1:hlg (default), 2:pq] 
    -q    quality factor to be used while encoding sdr intent, optional. [0-100], 95 : default.
    -e    compute psnr, optional. [0:no (default), 1:yes] 
    -R    color range of hdr intent, optional. [0:narrow-range (default), 1:full-range]. 
    -s    gainmap image downsample factor, optional. [integer values in range [1 - 128] (1 : default)]. 
    -Q    quality factor to be used while encoding gain map image, optional. [0-100], 95 : default. 
    -G    gamma correction to be applied on the gainmap image, optional. [any positive real number (1.0 : default)].
    -M    select multi channel gain map, optional. [0:disable, 1:enable (default)]. 
    -D    select encoding preset, optional. [0:real time, 1:best quality (default)]. 
    -k    min content boost recommendation, must be in linear scale, optional 
    -K    max content boost recommendation, must be in linear scale, optional 
    -x    binary input resource containing exif data to insert, optional. 

## decoder options : 
    -j    ultra hdr compressed input resource, required. 
    -o    output transfer function, optional. [0:linear, 1:hlg (default), 2:pq, 3:srgb] 
    -O    output color format, optional. [3:rgba8888, 4:rgbahalffloat, 5:rgba1010102 (default)] 
          It should be noted that not all combinations of output color format and output 
          transfer function are supported. 
          srgb output color transfer shall be paired with rgba8888 only. 
          hlg, pq shall be paired with rgba1010102. 
          linear shall be paired with rgbahalffloat. 
    -u    enable gles acceleration, optional. [0:disable (default), 1:enable]. 

## common options : 
    -z    output filename, optional. 
          in encoding mode, default output filename 'out.jpeg'. 
          in decoding mode, default output filename 'outrgb.raw'. 
    -f    gainmap metadata config file. 
          in encoding mode, resource from which gainmap metadata is read, required for encoding scenario 4. 
          in decoding mode, resource to which gainmap metadata is written, optional. 

## examples of usage :

## encode scenario 0 :
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -w 1920 -h 1080 -q 97 -a 0
    ultrahdr_app -m 0 -p cosmat_1920x1080_rgba1010102.raw -w 1920 -h 1080 -q 97 -a 5
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -w 1920 -h 1080 -q 97 -C 1 -t 2 -a 0
    ultrahdr_app -m 0 -p cosmat_1920x1080_rgba1010102.raw -w 1920 -h 1080 -q 97 -C 1 -t 2 -a 5

## encode scenario 1 :
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -y cosmat_1920x1080_420.yuv -w 1920 -h 1080 -q 97 -a 0 -b 1
    ultrahdr_app -m 0 -p cosmat_1920x1080_rgba1010102.raw -y cosmat_1920x1080_rgba8888.raw -w 1920 -h 1080 -q 97 -a 5 -b 3
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -y cosmat_1920x1080_420.yuv -w 1920 -h 1080 -q 97 -C 2 -c 1 -t 1 -a 0 -b 1
    ultrahdr_app -m 0 -p cosmat_1920x1080_rgba1010102.raw -y cosmat_1920x1080_rgba8888.raw -w 1920 -h 1080 -q 97 -C 2 -c 1 -t 1 -a 5 -b 3
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -y cosmat_1920x1080_420.yuv -w 1920 -h 1080 -q 97 -C 2 -c 1 -t 1 -e 1 -a 0 -b 1

## encode scenario 2 :
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -y cosmat_1920x1080_420.yuv -i cosmat_1920x1080_420_8bit.jpg -w 1920 -h 1080 -t 1 -o 3 -O 3 -e 1 -a 0 -b 1
    ultrahdr_app -m 0 -p cosmat_1920x1080_rgba1010102.raw -y cosmat_1920x1080_420.yuv -i cosmat_1920x1080_420_8bit.jpg -w 1920 -h 1080 -t 1 -o 3 -O 3 -e 1 -a 5 -b 1

## encode scenario 3 :
    ultrahdr_app -m 0 -p cosmat_1920x1080_p010.yuv -i cosmat_1920x1080_420_8bit.jpg -w 1920 -h 1080 -t 1 -o 1 -O 5 -e 1 -a 0
    ultrahdr_app -m 0 -p cosmat_1920x1080_rgba1010102.raw -i cosmat_1920x1080_420_8bit.jpg -w 1920 -h 1080 -t 1 -o 1 -O 5 -e 1 -a 5

## encode scenario 4 :
    ultrahdr_app -m 0 -i cosmat_1920x1080_420_8bit.jpg -g cosmat_1920x1080_420_8bit.jpg -f metadata.cfg

## encode at high quality :
    ultrahdr_app -m 0 -p hdr_intent.raw -y sdr_intent.raw -w 640 -h 480 -c <select> -C <select> -t <select> -s 1 -M 1 -Q 98 -q 98 -D 1

## decode api :
    ultrahdr_app -m 1 -j cosmat_1920x1080_hdr.jpg 
    ultrahdr_app -m 1 -j cosmat_1920x1080_hdr.jpg -o 3 -O 3
    ultrahdr_app -m 1 -j cosmat_1920x1080_hdr.jpg -o 1 -O 5

I'm guessing I probably need to update my ultrahdr_app copy.

wpferguson avatar Dec 03 '24 15:12 wpferguson

Yes, -L was added in 1.3.0 in https://github.com/google/libultrahdr/commit/e432c4d93ded76d0eaead27b9e4636a85e67672b. FWIW, nowadays libultrahdr is available directly through packages on some distros, and 1.3.0 bundles ultrahdr_app IIUC,

koto avatar Dec 03 '24 15:12 koto

Got it to work with updated libultrahdr and using 4.8.1.

Now I'm going to chase what's going on with export in master

wpferguson avatar Dec 03 '24 16:12 wpferguson

One potential culprit might be https://github.com/darktable-org/lua-scripts/pull/502/files#diff-964234d8155307277adf239627f3dc0d579d0c2b6669adb192e3d168cc3a9e17R419-R420, I remember some gboolean change wasn't completed in DT, there was some pending work. The symptoms would be indeed that a temporary file is correctly created, then the export succeeds, but the plugin reports it as failing, and a file gets deleted afterwards.

koto avatar Dec 03 '24 16:12 koto

My hunch was correct - it's https://github.com/darktable-org/darktable/issues/17528. I'll update the PR with the new logging code and an updated workaround condition. In 9.3.0 and 9.4.0, I'll treat ::write_image returning false as success.

koto avatar Dec 03 '24 18:12 koto

With the workaround code I can now generate UltraHDRs with the same reproduction steps on 4.9.0+1181~gfa54963782-dirty (still MacOS).

koto avatar Dec 03 '24 18:12 koto

I added an issue and a fix https://github.com/darktable-org/darktable/issues/17938, https://github.com/darktable-org/darktable/issues/17939. It's really LATE in the release cycle, so we'll see if it gets merged.

wpferguson avatar Dec 03 '24 19:12 wpferguson

The fix is in, so we can test for sane return values.

wpferguson avatar Dec 04 '24 21:12 wpferguson

Done. The workaround for write_image return value is now only applied in 9.3.0.

koto avatar Dec 04 '24 23:12 koto

@koto the only thing I'm waiting on is the changes I flagged in the code. Once those are done I'm ready to merge.

wpferguson avatar Dec 21 '24 18:12 wpferguson

I don't see any review comments in the PR, did you mean the changes with the log levels in each function? These are already in.

koto avatar Dec 22 '24 07:12 koto

I went through the file and commented on things that needed changed, such as not translating debug messages, but none of the changes I requested have been made. Are you looking at the PR on github (https://github.com/darktable-org/lua-scripts/pull/502), or just getting emails and responding to them?

wpferguson avatar Jan 02 '25 15:01 wpferguson

I went through the file and commented on things that needed changed, such as not translating debug messages, but none of the changes I requested have been made. Are you looking at the PR on github (#502), or just getting emails and responding to them?

On GitHub. I don't see any review comments, only this conversation :/ Can you double check the comments were submitted?

koto avatar Jan 02 '25 22:01 koto

My fault. I didn't hit the correct button.

wpferguson avatar Jan 03 '25 16:01 wpferguson

Looks like everything in addressed now then :)

koto avatar Jan 04 '25 08:01 koto

Could you write a page for the Lua scripts manual, https://docs.darktable.org/lua/stable/lua.scripts.manual/?

wpferguson avatar Jan 07 '25 18:01 wpferguson

https://github.com/darktable-org/luadocs/pull/16

koto avatar Jan 11 '25 10:01 koto