Allow for declaration of multiple co-existing AsyncAPIs

[Sheldoras]

Describe the feature request Currently Springwolf only allows for one AsyncAPI document to be provided by a service. The only limiting option about 'which' channels/operations etc go into said AsyncAPI specification is the base-package property.

Looking at other tools (Springdoc for OpenAPI spec generation for example) they allow for multiple OpenAPI specs to be declared in parallel (say one spec for version 1.0.0 and one for 2.0.0 which are simultaneously supported by a service). It is also possible to filter more finely which endpoints (in the case of OpenAPI) go into which OpenAPI specification.

Motivation If a service wants to support different AsyncAPI versions in parallel, it would be beneficial if two (or more) distinct specifications could be configured, each including only its relevant channels/operations/schemas.

Technical details With Springdoc and OpenAPI this declaration can look like so:

    @Bean
    fun version1(): GroupedOpenApi {
        val version = "1.0"
        return GroupedOpenApi.builder()
            .group(version)
            .displayName("API Version $version")
            .pathsToMatch("/public/v1/**")
            .pathsToMatch("/legacy/v1/**")
            .build()
    }

    @Bean
    fun version2(): GroupedOpenApi {
        val version = "2.0"
        return GroupedOpenApi.builder()
            .group(version)
            .displayName("API Version $version")
            .pathsToMatch("/public/v2/**")
            .pathsToMatch("/legacy/v2/**")
            .pathsToExclude("/public/v2/secret/**")
            .build()
    }

Something equivalent for providing multiple AsyncAPI specifications per service would be fantastic

[github-actions[bot]]

Welcome to Springwolf. Thanks a lot for reporting your first issue. Please check out our contributors guide and feel free to join us on discord.

[timonback]

Hi @Sheldoras,

Great suggestion.

A couple questions for understanding:

1. How do you plan to group the different endpoints (pathsToMatch in OpenAPI)? Is it only based on the channel name or do you actually intend to group by the actual operation (including the schema, so that you can differentiate between v1 and v2 payloads)?
2. Have you considered tags within the current document?
3. Do you wish to group into different documents (group in OpenAPI)?

[Sheldoras]

Hi @timonback,

thanks for considering this!

1. In our particular scenario grouping by channel name would be sufficient (considering all operations on an included channel part of the document). Additionally it would be great if only payloads (schemas) that are actually part of the operations on included channels would appear in the specification. So if I have a channel '/v1/somedata' to which I publish 'V1Data' payload and a '/v2/somedata' to which I publish 'V2Data' payload and my filter is like '/v1/**' then 'V2Data' should not appear in the specification.
2. Tagging all the relevant elements in a given document with their appropriate versions is an option in general, however not applicable to our case as we need mutiple documents (due to see 3.)
3. That is exactly what we would need as our services report their specifications (AsyncAPI and OpenAPI) to a catalog system which expects 'one document per version' for its internal catalogization.

If you have any other question I'm happy to answer them!

[github-actions[bot]]

The change is staged for release and will be part of the next release.

If you want to try and verify it in your application today, use the latest 1.X.0-SNAPSHOT build as described in our README.md > Testing SNAPSHOT version

Thank you for the report/contribution!

[timonback]

Hi @Sheldoras , we just merged a first preview of this feature. It is available as a SNAPSHOT.

The group can be configured in application.properties (example: https://github.com/springwolf/springwolf-core/blob/master/springwolf-examples/springwolf-kafka-example/src/main/resources/application.properties#L43).

More details about the configuration at https://github.com/springwolf/springwolf-core/blob/master/springwolf-core/src/main/java/io/github/springwolf/core/configuration/properties/SpringwolfConfigProperties.java#L202 (only one of the filters (regex pattern) can be used)

Also, the group can be selected as part of the ui via settings button in the header.

---

Does this address your use-case? What is still missing?

We are looking forward to your feedback.

[Sheldoras]

Hi @timonback, thank you so much for looking into this! 🐺 I'm currently trying it out from the SNAPSHOT and really like it!

One very prominent question from our specific use-case would be: Is it also possible to define those groups programmatically (as I did it in the Springdoc example in my initial post)? I currently did it via properties as it was done in your example and it works! However we also perform some customizations (as in adding extension fields on a per-group basis) in our OpenAPI group declarations and would need to do so as well with our AsyncAPI ones.

    @Bean
    fun version1(): GroupedOpenApi {
        val version = "1.0.0"
        return GroupedOpenApi
            .builder()
            .group(version)
            .displayName("My API $version")
            .pathsToMatch("/v1/**")
            .addOpenApiCustomizer { openApi ->
                run {
                    openApi.info.extensions["x-apitype"] = "public"
                    openApi.info.version = version
                }
            }.build()
    }

Springdoc offers this generic customizer hook here. Have you thought about adding something like this?

Thanks again for looking into this! I'll be sure to try it out some more!

[timonback]

Hi @Sheldoras, awesome!

Grouping is a feature that can develop a high complexity quickly while requiring stable apis. This is the reason we went with property based configuration first.

To understand further, do you declare group dynamically or dynamically modify a grouped AsyncApi? I could imagine a group specific AsyncAPI customizer as a public interface - in addition to the current interface.

[Sheldoras]

Hello @timonback ! The groups and their versions are hardcoded and don't change dynamically at runtime. So the properties setup would work for us (it's then just unfortunately inconsistent with the way we define our OpenAPI groups - which is happening as shown in the code snippet in my last post, so programatically).

What the current solution lacks that we would need is the ability to make these "arbitrary" changes to the spec (add extension fields, change the version field etc). So any possibility, as the one you already mentioned, to make group-specific adjustments to the spec would work for us!

Thanks!

[timonback]

Hi @Sheldoras, for the beginning we chose the easy option of allowing to overwrite the info object per group (instead of beans or adding a customizer).

We are still working on it (https://github.com/springwolf/springwolf-core/pull/1093), but here is the preview:

springwolf.docket.group-configs[0].group=Only Vehicles
springwolf.docket.group-configs[0].info.description=This group only contains endpoints that are related to vehicles.
springwolf.docket.group-configs[0].info.extension-fields.x-apitype=internal

Does this solve your request:

What the current solution lacks that we would need is the ability to make these "arbitrary" changes to the spec (add extension fields, change the version field etc).

[Sheldoras]

Hi @timonback, thanks for your work! Yes that would solve our current need for flexibility with the info object per group! :)

[github-actions[bot]]

The change is available in the latest release. 🎉

Thank you for the report/contribution and making Springwolf better!