d3cod3
ofxPlayhead by @daandelange
Yes, this would be awesome to have within Mosaic !
Btw, are we talking Mosaic or ofxVP ?
Features
It would make sense to integrate the player in the main GUI so it can control Mosaic's frame processing. This will allow users to precisely control playback of patches : frame by frame execution (useful for debugging patches), slowmotion playback, have different play modes (fixed time step [offline], relative time-step [rt-Rel], and an experimental "cooking-realtime" one [rt-Abs]), the offline mode is ideal for exporting/rendering frames.
But it would also be useful to have the time Counters and Ramps variables exposed to be used for patching.
Integration
How could users access/connect to time variables if it's not in a nodeObject ? Remote connections ? An always present SystemNodeObject like the SystemAudioNodeObject that controls/queries the playhead ?
One moment I was thinking about "canvas pins" (current) and "gui pins" : pins that connect on one side in the GUI and on the other side on the Canvas; but that would make a spaghetti mess and feel very weird when navigating in the canvas.
Also, nodeObjects should be able to access time variables. There's already a singleton logic embedded, or do we prefer to pass the object as arguments like nodeObject::render(timeCounters); ?
I'd also suggest making this a "module" and not baking it too hard into Mosaic. (the audio object should become strippable, this one too).
Just for the notice : There are some small bugs to correct and settings/terminology to refine, but the current state is fine and works good, specially the "offline" and "rt-Rel" modes.
Evolution
Also, some future evolution notes: this clock is a slave of the openframeworks clock : it's serving time as OpenFrameworks orchestrates frames. The time reference is internal (std::chrono) and could be replaced/interchanged, and eventually we could use it later to orchestrate frames using a custom ofRunMainLoop().
Ideally to support simulation (cf: delta time) better : there might be some process() callbacks needed, which might require a different/additional API, but that will come later.
About the features, 100% agree with your proposals. Counters and ramps could be visible/accesible from within the teoretical module or object.
Integration, i would avoid canvas pins, instead we could make use of one of the last additions, the wireless send and receive objects, they works really well, ~~we could implement a system send/receive fixed variable objects to access time variables~~, having this solving the nodes object accessing time vars; then i would have the core implementation integrated in ofxVP, passing the object as an argument as you suggest: nodeObject::render(timeCounters);, then implementing the ofxPlayhead GUI module in Mosaic accessing the core variables from ofxVP engine.
With this we'll avoid another always present system object, and everything parked in it's right place, i think.
I'll share a simple scheme soon.
Just for clarification, all this will control the graphical thread only, the audio thread needs to remain independent.
In the meantime, let me know when the code is ready for me to fork it and try the first moves to integration.
More soon
I've considered better and implementing a system wireless send/receive fixed variable object would overcomplicate the code and it would not bring more benefit than a simple specific nodeObject ( system playhead for example )
So, i'm seeing it like this:
- ofxPlayhead core implementation integrated inside ofxVP engine ( Playhead engine )
- system playhead object added to ofxVP object list, linked to Playhead engine, serving all the time variables available
- Playhead GUI module integrated in Mosaic together with the other modules, linked to the ofxVP Playhead engine
What do you think?
How many time variables come with ofxPlayhead? It is a fixed number?
Sounds good to me, although it might not need to be a SystemObject, just a regular object. When the object is there it can access ofxVP's ofxPlayhead instance and serve them as a regular Object.
There are quite some variables available, not all are needed, or maybe serve the main ones with direct pins and the rest as an array output. They are fixed but determined by the plugin, more might be added. There's metronom data (elapsedBeats, elapsedNotes, elapsed) and counter data ( elapsedFrames, elapsedTime, tDelta, etc.) and a lot of "ramps" which are calculated from the metronom data. We mainly want the counter data, maybe the metronom data, and the ramps optionally (in an array?).
ofxVP integration note: there needs to be a time signature for the ramps/metronom data to work, but they are not mandatory for the playhead to work. Also, infinite duration : I haven't tested lately, it might need some fixes.
More widely, this could be a great opportunity to implement/enforce better composition settings within ofxVP patches : time signature, fps, duration, sound buffer sizes, default pixel resolution ?, etc.
Yes yes, i called it system playhead but i mean a regular object as you're saying.
I like the independent float pins with the most important variables, and then the ramps in an array, sounds good.
I'll wait for the time signature and infinite duration testing, no problem.
Completely agree, it's something i had in the list of thing to do from some time now, it could be the opportunity for a proper composition setting panel/menu! The audio part i'm not sure, we'll have to test it because when some parameter change it needs a restart/reload of the DSP engine.
Maybe three sections? :
-
Time ( Graphic thread )
- Time signature
- FPS
- Duration
-
Video/Image
- Defaul textures/pixels resolution
- ???
-
Audio ( maybe ) ( we already have this in Sound menu )
- Input Device
- Output Device
- BPM (TEMPO)
- Sample Rate
- Sound Buffer Size
It's a start
Not easy to think about setting namespaces. I think we need to think a bit further to define this.
FPS could be both in video and time. And BPM is also time-signature related, that could rather be in the time domain.
And Video and Audio sections/engines could have an "enabled flag" so we can disable one or both. (In the future. Yes, for now, the DSP engine will probably remain mandatory)
Ultimately, Mosaic should be architectured something like : ( also thinking about a future addressable TreeScene/Component System)
Suggestion 1:
<core-modules><!-- Childs: Always loaded (mandatory), settings can change without reloads -->
<time-orchestrator duration="-1" fps="60" bpm="120" num="4" denum="4" />
</core-modules>
<engines><!-- Childs: Optional, but changes (may?) require reloading the changed engine and all NodeObjects and Modules. -->
<engine-renderer-opengl width="800" height="600" vsync="1" />
<engine-audio-pd sample-rate="44000" buffer-size="256" device-in="XXX" device-out="YYY" />
</engines>
<modules />
<node-objects />
Or we could rather provide a global and minimal CompositionSettings that is used by the different engines, to unify even further and facilitate switching engines. Suggestion 2:
<core-modules>
<composition-settings duration="-1" fps="60" bpm="120" num="4" denum="4" width="800" height="600" />
<time-orchestrator currentPos="0" speed="1"/>
</core-modules>
<engines>
<engine-renderer-opengl vsync="1" />
<engine-audio-pd sample-rate="44000" buffer-size="256" device-in="XXX" device-out="YYY" />
</engines>
I see, let's think about it.
Both suggestions are good choices, i slightly like the second more, the only thing is that i want to maintain retro-compatibility with older patches ( starting from the 0.7.2 release ), in order to work in future releases with composition setting implemented and still be able to load older patches with no compisition setting data.
So we'll just need to write this with retro-compatibility in mind.
About the enabled flag, it's already implementerd right now for the DSP, it can be switched ON/OFF from the main menu
I don't know about disabling the opengl renderer, you mean the graphical objects rendering? because on the graphical thread basically runs everything, even if we disable globally the video objects rendering, the opengl engine will still be running with all the rest of Mosaic stuff. Anyway, we'll see.
About the parameters that need reloading, they are just these for the audio engine:
<engine-audio-pd sample-rate="44000" buffer-size="256" device-in="XXX" device-out="YYY" />
And about default texture resolution, it's not complicated, ther is a virtulal method already implemented in PatchObject class:
virtual void resetResolution(int fromID=-1, int newWidth=-1, int newHeight=-1) { unusedArgs(fromID,newWidth,newHeight); }
that i used for some specific objects like lua script or glsl shader that updates their textures resolution when connected to an output window object ( a single one or in cascade ), because output window object can set/change the output window resolution, hence the need to change the conneted generative graphics objects textures resolution.
We'll just need to override the method in every object that uses textures:
void resetResolution(int fromID, int newWidth, int newHeight) override;
About changing the complete XML structure, i think we'll leave this great step to when we'll switch to PugiXml, a lot of code must be changed for that, i'm not planning this for now. It would need two main thing implemented, all the new XML read/write adapted to PugiXml, obviously, and a retro-compatible mechanism that, detect an older patch created with ofxXMLSettings, convert it to the new PugiXml standard, save the converted patch and finally opens it. So a bunch of code.
But i'm going too further, let's stay on topic, i like suggestion 2, we can move from there
Future talk: I agree with all in general. Yes, the pugiXML switch and re-architecturing will probably come with the new parameters switch too. Retro-compatibility is do-able. (a test engine would be so awesome for this!) Disabling the OpenGL renderer would provide a headless mode, it is indeed hard to disable now. Once it's a module we'll see if it's possible.
On-topic: Yes I prefer solution 2 also, it makes more sense to centralise all composition-settings into 1 place.