springwolf-core Docs.yaml generated by SpringWolf seems not to be correct as per https://studio.asyncapi.com/ and https://editor.swagger.io/ validation

Describe the bug Using the https://www.springwolf.dev/docs/configuration/#actuator-support feature, I downloaded the docs.yaml from my back-end using url http://localhost:8450/actuator/springwolf/docs.yaml Then I used this docs.yaml in both https://studio.asyncapi.com/ and https://editor.swagger.io/ but validation errors occurs

Dependencies and versions used springwolf-core and springwolf-amqp 0.17.0

Code example No code example as I cannot disclose my asyncapi definition. One can use its own docs.yaml and follow the procedure describe above to reproduce this 'bug'

Stack trace and error logs No stacktrace, as the problem is not an exception, but rather, a possible issue in the generated docs.yaml

Question I noticed that asyncapi version in generated docs.yaml is 2.6.0. Maybe it would be better if it was 3.0.0 ? How can I make Springwolf generate a 3.0.0 asyncapi docs.yaml file?

Jan 10 '24 08:01 pdalfarr

Hi @pdalfarr Thank you for the report.

We are actively building on AsyncApi 3.0, which will resolve this.

AsyncApi 3.0 includes bigger changes, requiring also a significant time investments on our side to adapt Springwolf. Subscribe to #494 and #507 to get updates.

Jan 10 '24 18:01 timonback

Thank you for the report, the issue has been addressed in the new release.

Feel free to reopen this issue if there is still something missing.

Mar 02 '24 07:03 timonback

Hi,

I tested the generated asyncapi 3.0.0 generate yaml and it is pretty much OK, except for a few things I had to fix by hand. ( https://studio.asyncapi.com/ helped me to understand the errors thanks to the lower pane with messages warnings and errors ;-) )

I can not disclose neither the source code, nor the generated asyncapi file , but I can share the fix I made by hand:

add a missing "host" element in servers.amqp servers.amqp.host: localhost:5672
payload examples with date attribute: add missing double quotes: components.schemas...MyMessage.examples[0].activites[0].periodeDate.debut: 2015-07-20
-> components.schemas....MyMessage.examples[0].activites[0].periodeDate.debut: "2015-07-20"

same goes for createdOn: 2015-07-20T15:49:04-07:00 -> createdOn: "2015-07-20T15:49:04-07:00"

enum example payload are surrounded with '"..."' : single quote, then double quotes, then enum string value, then double quote and finally single quote. I had to remove these. components.schemas....MyMessage.examples[0].typeReference: '"ENUM_VALUE"' -> components.schemas....MyMessage.examples[0].typeReference: ENUM_VALUE
formatting problem for some entries in components.schemas (where does the leading '?' come from.. I have no clue , also the trailing ":" is on the next line..)

    ? my.package.MyMessage
    : type: object

so I fixed these like so:

    my.package.MyMessage:
      type: object

also occurring under operation:

operations:
  ? my-receiveCommunicationReply
  : action: receive

and I fixed it like so:

operations:
  my-receiveCommunicationReply:
    action: receive

"schemaFormat" and "schema" in messages. I had to remove these lines, and 'move up one level the $ref element

not working:

  messages:
    My.MessageEvent:
      payload:
        schemaFormat: application/vnd.aai.asyncapi+json;version=3.0.0
        schema:
          $ref: '#/components/schemas/My.MessageEvent'

I had to change it like so:

  messages:
    My.MessageEvent:
      payload:
        $ref: '#/components/schemas/My.MessageEvent'

Still, the asyncapi 3.0.0 it's a great step forward, and I really want to thank you (and all contributors) for this.

I hope my feedback is sufficient for you to investigate.

Regards,

Pascal

Mar 04 '24 06:03 pdalfarr

Additional note about "schemaFormat" and "schema" issue:

https://studio.asyncapi.com/ does NOT report the presence of these 2 elements as being an issue, BUT the IntelliJ AsyncAPI plugin https://plugins.jetbrains.com/plugin/15673-asyncapi does.

Mar 04 '24 08:03 pdalfarr

Thanks for the update. I updated the release notes -> migration guide to explicitly mention the host and protocol field.

I took a quick look at the springwolf-examples/springwolf-kafka-example to generate a yaml document. It generates the document at https://pastebin.com/k4JVsChx, which is valid according to AsyncApi Studio.

There: 2. Dates do have quotes (I had to modify the example, not included in the pastebin) 3. Enum do only have 1 pair of double quotes 4. Not sure where the ? and : is coming from 5. As you mention as well, it is fine in AsyncApi Studio. I checked the same kafka example using the Intellij Plugin. It displays errors on some messages, however not on all. This is surprising to me, as they all do have the same structure. -> Is this an issue in the Intellij plugin?

Can you adapt one of the springwolf examples (for example kafka) so that we can reproduce the issues there? Is probably sufficient to update one of the DTO objects https://github.com/springwolf/springwolf-core/blob/master/springwolf-examples/springwolf-kafka-example/src/main/java/io/github/springwolf/examples/kafka/dtos/ExamplePayloadDto.java

You can test the yaml output locally by running the ApiIntegrationTest (requires docker). Just change the url in https://github.com/springwolf/springwolf-core/blob/a3b4743b825efa79added6d186ea4e581dce0f2b/springwolf-examples/springwolf-kafka-example/src/test/java/io/github/springwolf/examples/kafka/ApiIntegrationTest.java#L35 to String url = "/springwolf/docs.yaml";

Mar 05 '24 19:03 timonback

Hi,

I am using Rabbit, not Kafka. Which Rabbit example can I use?

In the mean time, I'll try to debug on my side.

Mar 08 '24 13:03 pdalfarr

Hi @pdalfarr, check out the springwolf-amqp example project right next to the kafka example at https://github.com/springwolf/springwolf-core/blob/a3b4743b825efa79added6d186ea4e581dce0f2b/springwolf-examples/springwolf-amqp-example/src/main/java/io/github/springwolf/examples/amqp/dtos/ExamplePayloadDto.java

Mar 08 '24 13:03 timonback

Hi

Debugging com.fasterxml.jackson.core.io.SegmentedStringWriter gave a first insight:

One of my Rabbit messages classes is a generic:

public class ContexteMessage<T> {

    @Valid
    public T contexte;
}

and it seems yamlMapper has an hard time serializing this

cfr https://github.com/springwolf/springwolf-core/blob/a8f18eb7739d1df06be7a13c54f9de3bf2a1a6a4/springwolf-asyncapi/src/main/java/io/github/springwolf/asyncapi/v3/jackson/DefaultAsyncApiSerializerService.java#L31

Which eventually leads to

https://github.com/snakeyaml/snakeyaml/blob/master/src/main/java/org/yaml/snakeyaml/emitter/Emitter.java#L620

hence the "?" in the generated yaml

I keep digging.

Mar 08 '24 14:03 pdalfarr

Hi @pdalfarr, any update on what you have discovered?

Mar 24 '24 09:03 timonback

Hi @timonback ,

I did some tests and, starting with your test, I could only reproduce the "date format" issue reported by https://studio.asyncapi.com/ i.e. generated asyncapi.yaml contains:

localDate: 2015-07-20

and https://studio.asyncapi.com/ is reporting this as an issue. When adding double quotes, studio.asyncapi does not report an error anymore

localDate: "2015-07-20"

Important: Maybe the other issues are related to the fact that I am using SpringWolf with a SpringBoot 2 application.

PR available:

I created PR https://github.com/springwolf/springwolf-core/pull/669 with the modification I did in ApiIntegrationTest.java so you can have a look.

In this PR, I am willingly leaving ´localDate: "2015-07-20"´ which set the test in failure. Maybe this issue reported by studio.asyncapi is a "false positive". This could really be the case as studio.asyncapi does complain about missing double quotes for

 localDate: 2015-07-20

but it does NOT complain about missing double quotes for localDateTime ...

localDateTime: 2015-07-20T15:49:04-07:00

In the meantime, I would suggest that we close (put on hold ..?) this issue, as I can't tell for sure whether it's related or not to my 'old SpringBoot 2.x setup'.

Mar 24 '24 14:03 pdalfarr

Hi @pdalfarr, we believe that this is a bug in the async api parser and created an issue for it: https://github.com/asyncapi/parser-js/issues/980

Apr 05 '24 14:04 sam0r040

Hi @pdalfarr, the issue got a response indicating that springwolf generates a valid async api spec. The issue seems to be in the js-yaml library that is used by AsyncApi Studio.

As a workaround you should be able to specify your own example for LocalDate fields by using the @Schema annotation with escaped double quotes:

@Schema(example = "\"2000-01-01T02:00:00+02:00\"")
private OffsetDateTime offsetDateTimet;

@pdalfarr Please let us know if it works.

Apr 19 '24 13:04 sam0r040

Thanks for your suggestion.

Some tests I made to try to obtain debut: "2024-04-18" in yaml:

Adding 2024-04-18 in Java:

 @Schema(example = "2024-04-18")
 private LocalDate debut;

leads to this in yaml

    debut: Thu Apr 18 02:00:00 CEST 2024

-> the example attribute is taken into account as being a 'LocalDate' instance, but then, it gets 'reformatted' This is not what we want.

Adding "escaped double quoted 2024-04-18 this in Java:

 @Schema(example = "\"2024-04-18\"")
 private LocalDate debut;

leads to this in yaml

    debut: 2015-07-20

-> it seems the example is not taken into account and generated example use the '2015-07-20' default value (where does is come from?). And still no double quotes.

Question: Is the suggestion working on your side?

Thanks.

Apr 22 '24 10:04 pdalfarr

FYI, I think I have a better understanding of the cause regarding enums being between ' " ... " ' .

Let's consider this enum:

@Getter
@RequiredArgsConstructor
public enum MessageStatus {
    OK(200, "OK"),
    BAD_REQUEST(400, "Bad Request"),
    UNAUTHORIZED(401, "Unauthorized"),
    NOT_FOUND(404, "Not Found"),
    INTERNAL_SERVER_ERROR(500, "Internal Server Error");

    private final int value;
    private final String reasonPhrase;
}

generated yaml is:

    my.package.infra.remote_service_1.message.reply.MessageReply:
      type: object
      properties:
        errorMessage:
          type: string
        messageStatus:
          type: string
          enum:
            - OK
            - BAD_REQUEST
            - UNAUTHORIZED
            - NOT_FOUND
            - INTERNAL_SERVER_ERROR
        payload:
          $ref: '#/components/schemas/my.package.infra.remote_service_1.message.reply.Message'
      examples:
        - errorMessage: string
          messageStatus: OK
          payload:

No problem here: messageStatus: OK is correct, i.e., OK is NOT surrounded by single and double quote

If I add the annotation @Schema(enumAsRef = true) on the MessageStatusclass, then the generated yaml is:

    my.package.infra.remote_service_1.message.reply.MessageReply:
      type: object
      properties:
        errorMessage:
          type: string
        messageStatus:
          $ref: '#/components/schemas/my.package.infra.remote_service_1.message.reply.MessageStatus'
        payload:
          $ref: '#/components/schemas/my.package.infra.remote_service_1.message.reply.Message'
      examples:
        - errorMessage: string
          messageStatus: '"OK"'
          payload:

So, having enum being Component, seems to 'trigger' the bug (I guess this is a bug...?), i.e., turn

   messageStatus: OK

into

   messageStatus: '"OK"'

Which is NOT correct

now, If I change @Schema(enumAsRef = true) to @Schema(enumAsRef = true, example = "OK") the generated yaml is now:

   my.package.infra.remote_service_1.message.reply.MessageReply:
     type: object
     properties:
       errorMessage:
         type: string
       messageStatus:
         $ref: '#/components/schemas/my.package.infra.remote_service_1.message.reply.MessageStatus'
       payload:
         $ref: '#/components/schemas/my.package.infra.remote_service_1.message.reply.Message'
     examples:
       - errorMessage: string
         messageStatus: OK
         payload:

So, exactly the same as before EXCEPT FOR messageStatus: OK which does not have the ' " ... " ' around the enum anymore.

I could not find this bug being reported in https://github.com/asyncapi/parser-js/issues?q=is%3Aissue+is%3Aopen+enum I am not sure where / how to report this bug.

Can I kindly ask you to help me reporting this bug / initiate an inquiry ? I am not sure where I should send this info? Feel free to copy pate / reference this comment where needed.

Thanks.

Pascal

Apr 22 '24 12:04 pdalfarr

Hi @pdalfarr,

thanks a lot for putting so much effort in analyzing. I think we have multiple points here.

Thanks for your suggestion.

Some tests I made to try to obtain debut: "2024-04-18" in yaml:
Adding 2024-04-18 in Java:
 @Schema(example = "2024-04-18")
 private LocalDate debut;
leads to this in yaml
    debut: Thu Apr 18 02:00:00 CEST 2024
-> the example attribute is taken into account as being a 'LocalDate' instance, but then, it gets 'reformatted' This is not what we want.

I think this is a bug and I can reproduce it in core.

Adding "escaped double quoted 2024-04-18 this in Java:
 @Schema(example = "\"2024-04-18\"")
 private LocalDate debut;
leads to this in yaml
    debut: 2015-07-20
-> it seems the example is not taken into account and generated example use the '2015-07-20' default value (where does is come from?). And still no double quotes.

Question: Is the suggestion working on your side?

This was just a quick guess for a workaround. I had look at what happens when the example is generated and I can see that with the double quotes, there is no example in the schema at all. This is why the default date is returned. Without the escaped double quotes, the schema contains the example as one would expect. So unfortunately it is not possible to use it as a workaround.

Apr 26 '24 09:04 sam0r040

@pdalfarr we fixed the issue with LocalDate in https://github.com/springwolf/springwolf-core/pull/722

Apr 26 '24 12:04 sam0r040

Great news, thanks! Regarding the "enum issue", do you need additional info from my side? Do you foresee a bug fix? or maybe it's up to another project (asyncapi project?) to provide a bug-fix?

Apr 26 '24 14:04 pdalfarr

I will move the problem with the enums to a new issue to have the different issues separated and close this one. We currently do not need more information from you, but I will notify you in the new issue when I had the time to investigate. For me it's to early to say if it's an issue in Springwolf or in some other library.

Apr 26 '24 15:04 sam0r040