Bug: `advanced_auth` is being parsed even if the yaml predicate does not match

[luutuankiet]

As title states. Steps to reproduce are

1. Use the provided yaml below. Notice I declared SelectiveAuthenticator which we can choose to use token
2. Create a ticktick account & application credentials to test following this doc. (You can use this site I created to retrieve token)
3. Use the following snippet and add your token in TICKTICK_TOKEN
4. Observe pyairbyte complains as shown in log

Interestingly, if I commented out the whole advanced_auth block in the yaml file then the source will run.

import airbyte as ab
from pathlib import Path
from dotenv import load_dotenv; load_dotenv()
import os; os.chdir(Path(__file__).cwd())
import yaml


ticktick_yaml = Path('./ticktick_source.yaml')
config = {
    "authorization": {
        "auth_type": "Token",
        "bearer_token": f"{os.getenv("TICKTICK_TOKEN")}"
    }
}


source = ab.get_source(
    name='ticktick',
    config=config,
    source_manifest=ticktick_yaml,
    streams="*"
)

source.check()

custom connector yaml

version: 6.48.15

type: DeclarativeSource

description: Source for the ticktick openapi endpoint at https://developer.ticktick.com/

check:
  type: CheckStream
  stream_names:
    - projects

definitions:
  oauth_authenticator:
    type: OAuthAuthenticator
    client_id: "{{ config.authorization.client_id }}"
    client_secret: "{{ config.authorization.client_secret) }}"
    grant_type: client_credentials
    access_token_value: "{{ config.authorization.client_access_token }}"
    scopes:
      - "tasks:"
      - read
  bearer_token:
    type: BearerAuthenticator
    api_token: "{{ config.authorization.bearer_token }}"    


  base_requester:
    type: HttpRequester
    url_base: https://api.ticktick.com
    authenticator:
      type: SelectiveAuthenticator
      authenticator_selection_path: ["authorization", "auth_type"]
      authenticators:
        Oauth: "#/definitions/oauth_authenticator"
        Token: "#/definitions/bearer_token"      
    error_handler:
      type: DefaultErrorHandler
  streams:
    tasks:
      type: DeclarativeStream
      name: tasks
      retriever:
        type: SimpleRetriever
        decoder:
          type: JsonDecoder
        requester:
          $ref: "#/definitions/base_requester"
          path: /open/v1/project/{{ stream_partition['parent_id'] }}/data
          http_method: GET
        record_selector:
          type: RecordSelector
          extractor:
            type: DpathExtractor
            field_path:
              - tasks
          record_filter:
            type: RecordFilter
            condition: "{{ record  }}"
        partition_router:
          type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: id
              partition_field: parent_id
              stream:
                $ref: "#/definitions/streams/projects"
      primary_key:
        - id
      schema_loader:
        type: InlineSchemaLoader
        schema:
          $ref: "#/schemas/tasks"
    projects:
      type: DeclarativeStream
      name: projects
      retriever:
        type: SimpleRetriever
        decoder:
          type: JsonDecoder
        requester:
          $ref: "#/definitions/base_requester"
          path: /open/v1/project/
        record_selector:
          type: RecordSelector
          extractor:
            type: DpathExtractor
            field_path: []
          record_filter:
            type: RecordFilter
            condition: "{{ not record.closed }}"
          schema_normalization: Default
      primary_key:
        - id
      schema_loader:
        type: InlineSchemaLoader
        schema:
          $ref: "#/schemas/projects"

streams:
  - $ref: "#/definitions/streams/projects"
  - $ref: "#/definitions/streams/tasks"
    
spec:
  type: Spec
  advanced_auth:
    auth_flow_type: oauth2.0
    predicate_key:
      - authorization
      - auth_type
    predicate_value: Oauth
    oauth_config_specification:
      complete_oauth_output_specification:
        required:
          - access_token
        properties:
          access_token:
            type: string
            path_in_connector_config:
              - authorization
              - client_access_token
      oauth_connector_input_specification:
        scope: "tasks: read"
        consent_url: >-
          https://ticktick.com/oauth/authorize?scope={{scope_value |
          urlEncode}}&client_id={{client_id_value}}&state={{state}}&redirect_uri={{redirect_uri_value}}&response_type=code
        extract_output:
          - access_token
        access_token_url: https://ticktick.com/oauth/token
        access_token_params:
          code: "{{ auth_code_value }}"
          scope: "{{ scope_value | urlEncode }}"
          grant_type: authorization_code
          redirect_uri: "{{ redirect_uri_value }}"
        access_token_headers:
          Content-Type: application/x-www-form-urlencoded
          Authorization: >-
            Basic {{ (client_id_value ~ ':' ~ client_secret_value) | b64encode
            }}
      complete_oauth_server_input_specification:
        required:
          - client_id
          - client_secret
        properties:
          client_id:
            type: string
          client_secret:
            type: string
      complete_oauth_server_output_specification:
        required:
          - client_id
          - client_secret
        properties:
          client_id:
            type: string
            path_in_connector_config:
              - authorization
              - client_id
          client_secret:
            type: string
            path_in_connector_config:
              - authorization
              - client_secret
  connection_specification:
    type: object
    $schema: http://json-schema.org/draft-07/schema#
    properties:
      authorization:
        type: object
        title: Authentication Type
        oneOf:
        - title: OAuth2
          type: object
          required:
          - auth_type
          - client_id
          - client_secret
          properties:
            auth_type:
              type: string
              const: Oauth
              order: 0
            client_id:
              title: Client ID
              type: string
              description: The client ID of your Ticktick application.
                Read more <a href="https://developer.ticktick.com/api#/openapi?id=getting-started">here</a>.
              airbyte_secret: true
            client_secret:
              title: Client Secret
              type: string
              description: The client secret of of your Ticktick application.
                application. Read more <a href="https://developer.ticktick.com/api#/openapi?id=getting-started">here</a>.
              airbyte_secret: true
            client_access_token:
              title: Access Token
              type: string
              description: Access token for making authenticated requests; filled after complete oauth2 flow.
              airbyte_secret: true
        - type: object
          title: Bearer Token (from Oauth2)
          required:
          - auth_type
          - bearer_token
          properties:
            auth_type:
              type: string
              const: Token
              order: 0
            bearer_token:
              title: Bearer Token
              type: string
              description: Access token for making authenticated requests; filled after complete oauth2 flow.
              airbyte_secret: true
    additionalProperties: true

metadata:
  assist: {}
  testedStreams:
    tasks:
      hasRecords: true
      streamHash: 614b09c66bbaf39cfd22bf6fbc3b5c8d45bbe19d
      hasResponse: true
      primaryKeysAreUnique: true
      primaryKeysArePresent: true
      responsesAreSuccessful: true
    projects:
      hasRecords: true
      streamHash: f302a05535246c68b860f6b24d368272c73ddb28
      hasResponse: true
      primaryKeysAreUnique: true
      primaryKeysArePresent: true
      responsesAreSuccessful: true
  autoImportSchema:
    tasks: true
    projects: true

schemas:
  tasks:
    type: object
    $schema: http://json-schema.org/schema#
    required:
      - id
    properties:
      id:
        type: string
      desc:
        type:
          - string
          - "null"
      etag:
        type:
          - string
          - "null"
      kind:
        type:
          - string
          - "null"
      tags:
        type:
          - array
          - "null"
        items:
          type:
            - string
            - "null"
      items:
        type:
          - array
          - "null"
        items:
          type:
            - object
            - "null"
          properties:
            id:
              type:
                - string
                - "null"
            title:
              type:
                - string
                - "null"
            status:
              type:
                - number
                - "null"
            isAllDay:
              type:
                - boolean
                - "null"
            timeZone:
              type:
                - string
                - "null"
            sortOrder:
              type:
                - number
                - "null"
      title:
        type:
          - string
          - "null"
      status:
        type:
          - number
          - "null"
      content:
        type:
          - string
          - "null"
      dueDate:
        type:
          - string
          - "null"
      columnId:
        type:
          - string
          - "null"
      isAllDay:
        type:
          - boolean
          - "null"
      priority:
        type:
          - number
          - "null"
      timeZone:
        type:
          - string
          - "null"
      projectId:
        type:
          - string
          - "null"
      sortOrder:
        type:
          - number
          - "null"
      startDate:
        type:
          - string
          - "null"
      repeatFlag:
        type:
          - string
          - "null"
    additionalProperties: true
  projects:
    type: object
    $schema: http://json-schema.org/schema#
    required:
      - id
    properties:
      id:
        type: string
      kind:
        type:
          - string
          - "null"
      name:
        type:
          - string
          - "null"
      color:
        type:
          - string
          - "null"
      closed:
        type:
          - boolean
          - "null"
      groupId:
        type:
          - string
          - "null"
      viewMode:
        type:
          - string
          - "null"
      sortOrder:
        type:
          - number
          - "null"
      permission:
        type:
          - string
          - "null"
    additionalProperties: true

api_budget:
  type: HTTPAPIBudget
  policies:
    - type: MovingWindowCallRatePolicy
      rates:
        - limit: 3
          interval: PT1S
        - limit: 60
          interval: PT1M
      matchers:
        - method: GET
          url_path_pattern: .*
  status_codes_for_ratelimit_hit:
    - 503
    - 500
    - 429

log

Exception has occurred: AirbyteConnectorCheckFailedError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Connector check failed. (AirbyteConnectorCheckFailedError)
------------------------------------------------------------
AirbyteConnectorCheckFailedError: Connector check failed.
    Please review the log file for more information.
    Connector Name: 'ticktick'
------------------------------------------------------------
Caused by: Connector failed. (AirbyteConnectorFailedError)
------------------------------------------------------------
AirbyteConnectorFailedError: Connector failed.
    Please review the log file for more information.
    Connector Name: 'ticktick'
------------------------------------------------------------
Caused by: 'str' object has no attribute 'value'
    Log file: /tmp/airbyte/logs/ticktick/ticktick-log-K397H8YD3.log
    Log file: /tmp/airbyte/logs/ticktick/ticktick-log-K397H8YD3.log
  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte/_connector_base.py", line 469, in _execute
    try:

                    message: AirbyteMessage = AirbyteMessage.model_validate_json(json_data=line)

                    if progress_tracker and message.record:

                        stream_name = message.record.stream

                        progress_tracker.tally_bytes_read(

                            bytes_read=len(line) - envelope_size - len(stream_name),

                            stream_name=stream_name,

                        )

                    self._peek_airbyte_message(message)

                    yield message



                except Exception:

                    # This is likely a log message, so log it as INFO.

                    self._print_info_message(line)
  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte/_executors/declarative.py", line 115, in execute
    yield from source_entrypoint.run(parsed_args)
  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte_cdk/entrypoint.py", line 168, in run
    source_spec: ConnectorSpecification = self.source.spec(self.logger)
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 409, in spec
    self._spec_component.generate_spec() if self._spec_component else super().spec(logger)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/spec/spec.py", line 57, in generate_spec
    self.advanced_auth.auth_flow_type = self.advanced_auth.auth_flow_type.value  # type: ignore # We know this is always assigned to an AuthFlow which has the auth_flow_type field
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'value'

During handling of the above exception, another exception occurred:

  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte/_connector_base.py", line 338, in check
    if msg.type == Type.CONNECTION_STATUS and msg.connectionStatus:

                        if msg.connectionStatus.status != Status.FAILED:

                            rich.print(

                                f"Connection check succeeded for `{self.name}`.",

                                file=sys.stderr,

                            )

                            log_connector_check_result(

                                name=self.name,

                                state=EventState.SUCCEEDED,

                            )

                            return



                        log_connector_check_result(

                            name=self.name,

                            state=EventState.FAILED,

                        )

                        # Give logs a chance to be flushed.

                        sleep(1)

                        raise exc.AirbyteConnectorCheckFailedError(

                            connector_name=self.name,

                            help_url=self.docs_url,

                            context={

                                "failure_reason": msg.connectionStatus.message,

                            },

                        )

                raise exc.AirbyteConnectorCheckFailedError(
  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte/_connector_base.py", line 496, in _execute
    connector_name=self.name,

                log_text=self._last_log_messages,

                original_exception=e,

            ) from None
airbyte.exceptions.AirbyteConnectorFailedError: Connector failed. (AirbyteConnectorFailedError)
------------------------------------------------------------
AirbyteConnectorFailedError: Connector failed.
    Please review the log file for more information.
    Connector Name: 'ticktick'
------------------------------------------------------------
Caused by: 'str' object has no attribute 'value'
    Log file: /tmp/airbyte/logs/ticktick/ticktick-log-K397H8YD3.log

During handling of the above exception, another exception occurred:

  File "/home/ubuntu/dev/ticktick_dbt/.venv/lib/python3.12/site-packages/airbyte/_connector_base.py", line 370, in check
    connector_name=self.name,

                    original_exception=ex,

                ) from None
  File "/home/ubuntu/dev/ticktick_dbt/EL/main.py", line 26, in <module>
    source.check()
  File "/usr/lib/python3.12/runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.12/runpy.py", line 198, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
airbyte.exceptions.AirbyteConnectorCheckFailedError: Connector check failed. (AirbyteConnectorCheckFailedError)
------------------------------------------------------------
AirbyteConnectorCheckFailedError: Connector check failed.
    Please review the log file for more information.
    Connector Name: 'ticktick'
------------------------------------------------------------
Caused by: Connector failed. (AirbyteConnectorFailedError)
------------------------------------------------------------
AirbyteConnectorFailedError: Connector failed.
    Please review the log file for more information.
    Connector Name: 'ticktick'
------------------------------------------------------------
Caused by: 'str' object has no attribute 'value'
    Log file: /tmp/airbyte/logs/ticktick/ticktick-log-K397H8YD3.log
    Log file: /tmp/airbyte/logs/ticktick/ticktick-log-K397H8YD3.log