[BUG] Error generating models with types referencing to attributes types

[Sincasios]

Bug Report Checklist

• [X] Have you provided a full/minimal spec to reproduce the issue?
• [X] Have you validated the input using an OpenAPI validator (example)?

This API is valid, but it cannot be shown because it contains circular references

• [X] Have you tested with the latest master to confirm the issue still exists?

Yes, with openapi-generator-cli-7.0.0-20220719.043604-2.jar, valides says "OK" but generates crashes: Exception in thread "main" java.lang.RuntimeException: Could not process model 'EventFilter_exptUeBehav_expectedUmts_inner_geographicAreas_inner_anyOf_1'.Please make sure that your schema is correct!

• [X] Have you searched for related issues/PRs?
• [X] What's the actual output vs expected output?

A python-flask server that can be launched

• [ ] [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

I'm triying to generate a OpenAPI server from 3GPP 5G specs. I got this specs from this repository https://github.com/jdegre/5GC_APIs and trying to build the .yaml called: TS29520_Nnwdaf_AnalyticsInfo.yaml

openapi-generator-cli generate -g python-flask -i TS29520_Nnwdaf_AnalyticsInfo.yaml -o output

Then I tried to install dependencies:

cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/python -m openapi_server

And got the first error: ImportError: cannot import name 'escape' from 'jinja2' (~/openapi/5GC_APIs/output/lib/python3.8/site-packages/jinja2/__init__.py)

I searched for this and is for the flask version (1.1.2): https://stackoverflow.com/questions/71718167/importerror-cannot-import-name-escape-from-jinja2 I updated the Flask version:

bin/pip install Flask==2.1.0

And now it starts with the generated models problems:

bin/python -m openapi_server
ImportError: cannot import name 'NnwdafEventsSubscription' from partially initialized module 'openapi_server.models.nnwdaf_events_subscription' (most likely due to a circular import) (~/openapi/5GC_APIs/output/openapi_server/models/nnwdaf_events_subscription.py)

Reading some bugs opened here I found "swagger-cli bundle", I tried this:

swagger-cli bundle -o b_TS29520_Nnwdaf_AnalyticsInfo.yaml -r -t yaml TS29520_Nnwdaf_AnalyticsInfo.yaml

But "dereference" doesn't work because there is a "Circular $ref pointer", but if I remove this I can generate a single .yaml with all the spec:

swagger-cli bundle -o b_TS29520_Nnwdaf_AnalyticsInfo.yaml -t yaml TS29520_Nnwdaf_AnalyticsInfo.yaml

I uploaded the output .yaml as gziped file (to use github upload): b_TS29520_Nnwdaf_AnalyticsInfo.yaml.gz

Now repeat the code generation using this new bundled yaml:

rm -rf output
openapi-generator-cli generate -g python-flask -i b_TS29520_Nnwdaf_AnalyticsInfo.yaml -o output
cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/pip install Flask==2.1.0
bin/python -m openapi_server

And now the real question comes: It raises new errors based on genereated models from the yaml:

from openapi_server.models.network_area import NetworkArea
ModuleNotFoundError: No module named 'openapi_server.models.network_area'

If we look the spec we can see where this come from some attributes referencing other types:

              networkArea:
                $ref: '#/components/schemas/EventFilter/properties/networkArea'

And this adds the "import openapi_server.models.network_area" but this file doesn't exists. The real module is:

models/event_filter_network_area.py

Found here:

$ find -iname "*network_area*"
./openapi_server/models/event_filter_network_area_g_ran_node_ids_inner_gnb_id.py
./openapi_server/models/event_filter_network_area_g_ran_node_ids_inner.py
./openapi_server/models/event_filter_network_area_ecgis_inner.py
./openapi_server/models/event_filter_network_area_ncgis_inner.py
./openapi_server/models/event_filter_network_area_tais_inner.py
./openapi_server/models/event_filter_network_area.py  <------------------------------------

So this is my first problem or bug: When a type references a subtype like "#/components/schemas/EventFilter/properties/networkArea" code adds a import to "openapi_server.models.network_area" but this doesn't exists, it is "models.event_filter_network_area import EventFilterNetworkArea"

I tried to fix this case with a string replace: Search for "models.network_area import Network area" and replace this to "model.event_filter_network_area import EventFilterNetworkArea as NetworkArea": sed -i "s/models.network_area import NetworkArea/models.event_filter_network_area import EventFilterNetworkArea as NetworkArea/g" *.py

And it worked... but now I have a second situation, similar but different:

    from openapi_server.models.start import Start
ModuleNotFoundError: No module named 'openapi_server.models.start'

This is more funny, because there isn't any "start" model. What happens here is initially the same: Types referencing other types.

                  startTs:
                    $ref: '#/components/schemas/AnalyticsData/properties/start'

And if we go to "AnalyticsData" we can see the definition as a string with format date-time:

      properties:
        start:
          format: date-time
          type: string
          description: string with format 'date-time' as defined in OpenAPI.

The problem here is that "AnalyticsData" doesn't generates any class to contain this "start" property, it keeps the datetime directly:

        :param start: The start of this AnalyticsData.  # noqa: E501
        :type start: datetime
        .....
        self.openapi_types = {
            'start': datetime,

So no "analytics_data_start" module is generated and nothing can be imported. The openapi-generator should 2 one of this things:

• See that the referenced type is a basic type and inline it in all other models.
• Always generate a "model.*" for all attributes (or al least all referenced attributes)

In the same "AnalyticsData" reference we have the two situations: the start that defines that is a string with "format: date-time" and then "expiry" that references to start to define his type:

        :param start: The start of this AnalyticsData.  # noqa: E501
        :type start: datetime <---- Inlined datetime
        :param expiry: The expiry of this AnalyticsData.  # noqa: E501
        :type expiry: Start <---- Using inexistent "model.start" instead of use "datetime"

Is this a bug? Is a openapi-generator limitation? Maybe a incorrect spec? I'm not and OpenAPI expert, but reading the yaml I can see that OpenAPI has the information to generate the code.

openapi-generator version

$ openapi-generator-cli version
6.0.1

Downloaded using npm.

OpenAPI declaration file content or url

b_TS29520_Nnwdaf_AnalyticsInfo.yaml.gz

Generation Details

openapi-generator-cli generate -g python-flask -i b_TS29520_Nnwdaf_AnalyticsInfo.yaml -o output

Steps to reproduce

Described in "Description", but basically:

openapi-generator-cli generate -g python-flask -i b_TS29520_Nnwdaf_AnalyticsInfo.yaml -o output
cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/pip install Flask==2.1.0
bin/python -m openapi_server

Suggest a fix

For the first case: Don't import the "last slash" (or generate this model). It is importing the inexistent "NetworkArea" (generatedfrom /NetworkArea) when the real import is EventFilterNetworkArea.

For the second case:

• See that the referenced type is a basic type and inline it in all other models.
• Always generate a "model.*" for all attributes (or al least all referenced attributes)

[Sincasios]

I have a small OpenAPI yml with the 2 cases isolated:

openapi: 3.0.0
info:
  version: 1.0.0
  title: Error case
paths:
  /analytics:
    get:
      operationId: GetTest
      parameters:
        - name: TestObject
          in: query
          schema:
            $ref: '#/components/schemas/TestObject'
      responses:
        '200':
          description: OK

components:
  schemas:
    TestObject:
      description: Object with an attribute with type string - date-time + other attributes referencing it
      type: object
      properties:
        basictype:
          type: string
          description: A basic type

        composedType:
          description: The base composed type
          type: object
          properties:
            innertype:
              type: string
              description: The innertype

        refComposed:
          description: Reference to a composed field
          $ref: '#/components/schemas/TestObject/properties/composedType'

        refBasic:
          description: Referencing a basic type
          $ref: '#/components/schemas/TestObject/properties/basictype'

That I generate, prepare and execute with:

openapi-generator-cli generate -g python-flask -i error.yml -o output
cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/pip install Flask==2.1.0
bin/python -m openapi_server

The first problem is the basictype reference: ModuleNotFoundError: No module named 'openapi_server.models.basictype'

We can see in the code the import and the type mapping:

        self.openapi_types = {
            'basictype': str, <------------------- Base type
...
            'ref_basic': Basictype <----------- Referenced type, inexistent
        }

I can fix this removing imports and changing to 'str', now the problem is the reference to a composed subtype: ModuleNotFoundError: No module named 'openapi_server.models.composed_type'

Because the model is "test_object_composed_type" I can fix this in this test making a import rename: Remove: from openapi_server.models.composed_type import ComposedType

And add: from openapi_server.models.test_object_composed_type import TestObjectComposedType as ComposedType

And now the server works fine:

$ bin/python -m openapi_server
 * Serving Flask app '__main__' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses (0.0.0.0)
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://127.0.0.1:8080
 * Running on http://192.168.30.2:8080 (Press CTRL+C to quit)

I can patch the code of this small example, but for a real API like the provided by 3GPP is very tedious.