Registering Sedona types and SQL functions on a standalone Flink instance

[radekaadek]

Hello,

I would like to ask if there is a way to register Sedona types and SQL functions in Flink without having to run SedonaContext.create()

For context, I'm using Flink through the SQL Gateway with a JDBC connection and I need a way to register Sedona without any Java or Python code.

I'm also open to make a contribution in this area if it's not a lot of work.

[github-actions[bot]]

Thank you for your interest in Apache Sedona! We appreciate you opening your first issue. Contributions like yours help make Apache Sedona better.

[jiayuasu]

@Imbruced Pawel, did you try SedonaFlink in this situation before?

[Imbruced]

@jiayuasu I haven't but I can take a look Sat/Sun on this one

[radekaadek]

Hi @Imbruced, any updates?

[Imbruced]

@radekaadek, sorry, I've been busy with the Apache Sedona book. I've tried to find an obvious solution, but I haven't found one. Can you explain precisely how you are running Apache Flink? It could help me find anything. Ofc, you are welcome to craft the solution yourself; I am happy to review it later.

[radekaadek]

I use the Apache Flink through a custom Docker image that I base on of the official image to which I add jar dependencies for the connectors and plugins I need to use. I then spin up this custom image and I interface with it through the SQL Gateway's REST API.

I had been looking for a solution to the problem and stumbled upon modules in Flink and the documentation for them explicitly states:

For example, users can define their own geo functions and plug them into Flink as built-in functions to be used in Flink SQL and Table APIs.

As the documentation later states, these modules can then loaded by calling LOAD MODULE in the SQL Client.

This seems very promising for me but the documentation doesn't state whether custom types/serializers can be loaded. However I found some documentation for third-party serializers here. I'm not very familiar with what Sedona needs in order to work correctly and to register its types but I suspect that the documentation on third-party serializers can help us.

[Imbruced]

Sedona needs to register the

• custom geometry types like Point (org.locationtech.jts.geom.Point), Polygon (org.locationtech.jts.geom.Polygon)
• custom index types like Quadtree (org.locationtech.jts.index.quadtree.Quadtree), STRtree (org.locationtech.jts.index.strtree.STRtree)
• custom functions

[radekaadek]

Ok, thanks, I'll try to come up with a solution on the weekend.

[radekaadek]

I have run into some issues and made a PR.