Support Process Versioning

[0xc1c4da]

What Happens

When constructing a spec by adding multiple BPMN files with the same process id, but different version, BpmnParser raises ValidationException citing 'Duplicate process ID'

Expectation

The engine uses the previous process version for any active instances of those processes. The engine chooses the latest version of the process for any new instantiations of those processes.

All versions of the process exists in the spec

[danfunk]

Interesting. I think we were in this a few months before we realized the utility of versioning workflows. Both BPMN and DMN files. And your expectations are thoughtful ones. Very thoughtful.

When we encountered this problem previously, we felt

1. Deserialization should work without providing the original spec, so we made this true ( I believe we have made this true, but let me know if we failed to so in some circumstance?)
2. It would be good for the client to know what version of the specs were involved in the serialization of a running workflow and to take responsibility for it.
3. It should be the clients prerogative to decide precisely what version of the specs to use when building a new workflow. Not a choice between the latest and the "previous", but a choice between the latest, and the 12 or 20, or 50 versions prior. IE, I want to use version 20 of this primary process file, version 12 of this DMN, and version 6 of this sub-workflow.

Ideally, versions of the BPMN and DMN and other associated files could be managed well outside the library (nay git?) rather than some dictate of the library.

I would be happy to talk to you about how we have managed this version issue internally over the last few years. And I'm eager to find ways we could collaborate on a system that would handle versioning for everyone.

Dan

[0xc1c4da]

Thanks for the reply Dan

On point 1 and 2, yes I agree, this is what is implemented and this makes sense.

On point 3, what you described is exactly what I meant, for example in the Camunda Modeler, a CallActivity task defines a Binding which defaults to latest, but can also define deployed, version or versionTag. I am currently thinking to use semver for versioning

Ideally, versions of the BPMN and DMN and other associated files could be managed well outside the library (nay git?) rather than some dictate of the library.

Yes, I agree. There's 2 types of versioning that are going on. The processes that are managed in git, and then these processes are instantiated and are 'live' in the engines current state. I can foresee migration issues, where a new version of the process is introduced that is not backwards compatible with the active processes in the existing state.

For our use-case, our engine loop would look something like:

• build current_workflow_spec from git versioned files via add_bpmn_files_by_glob, add_bpmn_files or add_bpmn_file
• if previous workflow state exists in database, load last serialized workflow state & spec as previous_workflow_spec
• if current_workflow_spec != previous_workflow_spec, then merge current_workflow_spec into previous_workflow_spec as merged_workflow_spec
• load previous workflow state with merged_workflow_spec
  • any existing process instances would use the process version associated with it
  • any new processes would use the version associated with the CallActivity (in our case binding latest)
• call workflow.do_engine_steps()
• serialize workflow state and save to database

This would allow us to create an auditability trail of the specification and current state at any point in time, and maintain a running system whilst also able to update processes.

I had also contacted Sartography via the web form, as we would be interested in helping financially support some open source tooling around SpiffWorkflow.

[danfunk]

Oh my goodness. I hate that darn email web form. And my slow reaction to these posts. I'm going to yank that form off the site right now. Would love to talk to you about working together on Spiff. You can reach me at [email protected]

[danfunk]

Interesting.

We've thought a ton about versioning as well, as one of our projects has long running (3 to 12 months) processes, that will almost certainly change. In fact, we're busy re-thinking it from the ground up again right now.

While we use Camunda's web forms at the moment, we are trying hard to move away from any Camunda specific tags in the XML moving forward. The "deployed" and "version" options in the Camunda modeler are not a part of the BPMN 2.0 standard. The standard, from what I've read, does not seem to address versioning at all, which seems wise.

It seems very likely that someone could make a modification to a workflow specification that would prevent any type of successful "merge". Where we are at presently in our efforts is in the event changes to the spec MUST take place within a running workflow, then end users must start over from the beginning, and we then do our best to pre-populate any unser interactions with previous answers as they walk back through the process. I'm not sure what else to do at this level, because the changes that you can make in a workflow are so potentially complex - all the way up to deleting all of the tasks in the workflow and building something completely different.

[danfunk]

Just doing some clean up and closing out old tickets.