ynput
Folder batch publishing tool
Changelog Description
Batch Publisher can be opened from AYON launcher tray. It is a tool for batch publishing multiple files each as a separate product.
Testing notes:
- git clone the OpenPype repo
- checkout this branch "Feature/op 7692 folder batch publishing tool"
- Setup dev environment on AYON server and set OpenPype directory for bundle to directory of OpenPype
- Ask @bradenjennings for ayon internal testing data
- Make sure your AYON server has a project available. For example make sure you setup AYON server on AY_CG_demo project .
- Open Batch publisher for AYON launcher tray
- Choose the browse button and pick the root directory of test data, here C:\errolDotting
- The table view should show some fbx file paths
- Choose your folder (asset) to make the line "defined"
please note you probably only want to review the widgets.py file (it will soon be broken up into multiple files for model view etc).
The rest of the code is for publishing using Deadline "OpenPype" plugin, and isn't working yet. The whole approach might be wrong anyway.
@bradenjennings thanks for updating the testing notes. When you think the PR is a state to review, then mark it with the Ready for review button.
Batch Publisher can be opened from AYON launcher tray. It is a tool for batch publishing multiple files each as a separate product.
I wonder, isn't that somewhat what the Tray Publisher was intended to be for? If not, what in its design "gets in the way" and would that mean that maybe the issue is in the design of the tray publisher?
Could you share a screenshot of what it looks like?
I wonder if some work here might overlap with the work done here: https://github.com/ynput/OpenPype/pull/5795 since you describe that it shows the result of batch ingestion in a table, I wonder whether it could actually just generate a relevant CSV file and thus re-use the entry point from the CSV one. (or the other way around, have the CSV one use the 'batch ingest' entry point maybe being defined here)
Also Batch Publishing / Ingesting may have it overlap with this here: https://community.ynput.io/t/batch-publishing-ingest/840/18 - so just cross-referencing that as well @fabiaserra @mkolar
Also Batch Publishing / Ingesting may have it overlap with this here: https://community.ynput.io/t/batch-publishing-ingest/840/18 - so just cross-referencing that as well @fabiaserra @mkolar
I mean most of the code on this PR is a subset of mine haha @bradenjennings why didn't you tag me?
Also I think this misses the point of the opportunity we have with this of abstracting the parts that can be reused for other modules and continues with the OP bad practice of just copy pasting code all over the place...
Also Batch Publishing / Ingesting may have it overlap with this here: https://community.ynput.io/t/batch-publishing-ingest/840/18 - so just cross-referencing that as well @fabiaserra @mkolar
I mean most of the code on this PR is a subset of mine haha @bradenjennings why didn't you tag me?
I was pointed in the direction of your forum post and asked to implement something similar. We are actually not going to use the publishing code that was taken from your development (It's not working in AYON). It was just temporarily copied to get something up and running quickly. As noted above I really wanted the widgets.py module to be reviewed at this stage. Which contains the bulk of the front end UI and back end model.
Also I think this misses the point of the opportunity we have with this of abstracting the parts that can be reused for other modules and continues with the OP bad practice of just copy pasting code all over the place...
I'm starting from scratch with this development with the goal at first of supporting one client. Before getting to more of an abstraction I need to start with something
Here is a screenshot of Batch Publisher UI. A few details are about to change, such as removal of the variant column,
I was pointed in the direction of your forum post and asked to implement something similar. We are actually not going to use the publishing code that was taken from your development (It's not working in AYON). It was just temporarily copied to get something up and running quickly. As noted above I really wanted the widgets.py module to be reviewed at this stage. Which contains the bulk of the front end UI and back end model.
Well at the very minimum you need to credit the owner of the code... specially when it's a complete copy paste of mine haha
I was pointed in the direction of your forum post and asked to implement something similar. We are actually not going to use the publishing code that was taken from your development (It's not working in AYON). It was just temporarily copied to get something up and running quickly. As noted above I really wanted the widgets.py module to be reviewed at this stage. Which contains the bulk of the front end UI and back end model.
Well at the very minimum you need to credit the owner of the code... specially when it's a complete copy paste of mine haha
Sorry about that, and thanks for your development. Excuse me I'm new to ynput, just getting my head around things.
But please review the widgets.py module. That is the only code I need reviewing at this stage.
If it helps I can remove your publishing code from this PR.
Making that publish abstraction working in Ayon would be a very easy change honestly, the json file structure hasn't changed that much and it would just require to run through the Ayon plugin of Deadline instead. I'm planning on doing those changes myself to update our fork to Ayon
If it helps I can remove your code from this PR.
what's left on the PR if you remove my code? haha I don't understand why you guys didn't reach out to me first to grab the latest code of mine... Did you look at the widget of mine https://github.com/fabiaserra/OpenPype/blob/feature/batch_ingester/openpype/modules/ingest/tray/ingest_dialog.py and the rest of the code that I pushed?
what's left on the PR if you remove my code?
About 1000 lines of code
haha I don't understand why you guys didn't reach out to me first to grab the latest code of mine... Did you look at the widget of mine https://github.com/fabiaserra/OpenPype/blob/feature/batch_ingester/openpype/modules/ingest/tray/ingest_dialog.py and the rest of the code that I pushed?
Sorry I wasn't aware of that code, and already started on development with a new direction.
After studying conversations, community posts and all comments, I have some questions about this batch publishing tool: Does it support families such as model, rigs etc? It would be also great to have included some videos on testing notes to demonstrate how it works.
After studying conversations, community posts and all comments, I have some questions about this batch publishing tool: Does it support families such as model, rigs etc? It would be also great to have included some videos on testing notes to demonstrate how it works.
Currently a configuration needs to be provided. Only the families that are configured work out of the box.
The configuration might need to change for different project and client data,
I've added a function to publish.py called publish_version_pyblish, which now publishes locally. https://github.com/ynput/OpenPype/pull/6144/files#diff-3891d3e2ed1b2de8daa2d8cbe953e46a7e349ce194a968b82913c5d7641c6164R24
The Batch Publisher model populates first by running each glob string search named GLOB_SEARCH_TO_PRODUCT_INFO_MAP, within the top level directory you pick.
For example a glob statement is
"glob": "*/fbx/*.fbx"
it then uses the product type and representation that is contained within the same level of the dictionary.
https://github.com/ynput/OpenPype/pull/6144/files#diff-1ccdf8c93903740c855ee943a69ae58fc5184d636ca92487f1187b77cff08ea2R16
Next I walk the top level directory again and try to add any more product types. By checking the extension is in PRODUCT_TYPE_TO_EXT_MAP. This allows a more general purpose way of adding many product types to batch publisher. https://github.com/ynput/OpenPype/pull/6144/files#diff-1ccdf8c93903740c855ee943a69ae58fc5184d636ca92487f1187b77cff08ea2R49
representation name can come from GLOB_SEARCH_TO_PRODUCT_INFO_MAP otherwise similar to @fabiaserra code representation can come from PRODUCT_TYPE_TO_EXT_MAP https://github.com/ynput/OpenPype/pull/6144/files#diff-1ccdf8c93903740c855ee943a69ae58fc5184d636ca92487f1187b77cff08ea2R35
Did simple test on this ticket. It launches and publishes successfully in AYON, but it would be great if there is select/deselect all option for the files as It is quite painful for users to unclick each to publish when they want to publish some files from tons of versions. Not sure if the product type preference is preset or not. But it would be great to set product type accordingly with the chosen folder.
Did simple test on this ticket. It launches and publishes successfully in AYON, but it would be great if there is select/deselect all option for the files as It is quite painful for users to unclick each to publish when they want to publish some files from tons of versions
There is already a right click / context menu option to toggle selected rows between enabled or not
Not sure if the product type preference is preset or not. But it would be great to set product type accordingly with the chosen folder.
You can edit the product type per row, by double clicking the cell to activate the editor widget combo box.
The default product type is chosen by an internal dictionary mapping. This should be exposed to AYON settings before releasing it.