springwolf-core icon indicating copy to clipboard operation
springwolf-core copied to clipboard

Extend publishing api for kafka plugin to support publishing of kafka headers

Open sam0r040 opened this issue 3 years ago • 3 comments

This PR should be a starting point and base for discussion to implement https://github.com/springwolf/springwolf-core/issues/51

The SpringwolfKafkaController now expects a structred request body, that contains a payload field and an optional header field to pass the kafka headers. The headers and the payload will be converted into a ProducerRecord and then passed on to the configured KafkaTemplate. I also considers the other two obvious options (http headers, query parameters) but somehow felt that this solution might be the most straight forward.

I also changed how SpringWolfKafkaProducer and its KafkaTemplate work together to improve testability and keep the KafkaTemplate out of an applications spring context.

sam0r040 avatar Aug 26 '22 16:08 sam0r040

Thanks for the PR and sorry for the late response!

Unfortunately I did not have the time I would like to put into this project lately, but I will try to get to reviewing this in the coming weeks.

stavshamir avatar Sep 02 '22 09:09 stavshamir

No worries :). I would also try to add the headers to the client and generate header fields from Annotations, but I would definitely need some code pointers/ideas to start.

sam0r040 avatar Sep 09 '22 12:09 sam0r040

Hi @sam0r040, I finally got to reviewing this (sorry it took so long)! This looks like a great addition. I don't have any comments code-wise, but I do have some questions:

  1. The use case which raised this issue is producing a message for a topic with multiple possible payloads. Did you try to see if this PR solve this issue?
  2. It can take place in another PR, but did you consider generating the type headers automatically (adding them to the async api doc)
  3. We can also consider adding support for the header annotation for consumers (and manual consumer/producer) in a future PR.

Thanks for the nice work!

stavshamir avatar Sep 23 '22 18:09 stavshamir

We extended the ui to also send the headers in this PR https://github.com/springwolf/springwolf-ui/pull/26

sam0r040 avatar Oct 21 '22 13:10 sam0r040