Improvement-Backend: Secrets Management

[ayush-shah]

Currently we manage the secrets by adding them manually into secrets factory(ClassConverterFactory.java) on backend, for each connector we have different pattern in configs. reduction of manual addition to the factory and have automated masking logic for different config would help in dynamic masking and less dependency of manual addition to the code.

[sushi30]

Using anyOf in OM jsonschemas

We might be abusing jsonschema's ability to express validation when in fact, in OM, it is used for code generation. So although jsonschema supports validation using arbitrary structures "anyOf", it is not considered a friendly feature of strongly typed languages like Java. This can be demonstrated in the dynamic type inference we use in python. Even in this (python) case, we are observing an abuse of the system because when we are using the if isinstance directive and by doing so giving up on the static type checking model and deferring it to Runtime. Nothing stops a developer from checking against types that are not part of the composition tree and no static type checking can validate this decision (because anyOf can literally be anything). This idea is summarized pretty will in this discussion in jsonschema2pojo.

Example solution

There can be multiple solutions that come to mind. IMO for OM's use-case an exclusive OR can be applied where a field can be resolved to multiple types and the nested fields will be instantiated as concrete classes (or null if they are not on the resolved path) instead of loosely-typed (implementation example). This way when using the parent class one use more detailed business logic (such as cascading switch/case of if/elif).

In the GCP credentials case this will be manifested as this josn schema:

"properties": {  
  "gcpCredentialsFile": {  
    "title": "GCP Credentials File",  
    "description": "Pass the path of file containing the GCP credentials info",  
    "$ref": "#/definitions/gcpCredentialsPath"  
  },  
  "gcpCredentialsValues": {  
    "title": "GCP Credentials Values",  
    "description": "Pass the values of GCP credentials",  
    "$ref": "gcpValues.json"  
  },
  ...
},  
"oneOf": [  
  {  
    "required": [  
      "gcpCredentialsFile"  
    ]  
  },  
  {  
    "required": [  
      "gcpCredentialsValues"  
    ]  
  }  
]

With the concrete class being built

public class GCPCredentials {  
	// ...
    private String gcpCredentialsFile;  
    // ...
    private GCPValues gcpCredentialsValues;  
    // ...
}

Summary

The actual implementation is much more nuanced because of the way these definitions are handled throughout the application across the different modules but generally it seems like there are two categories of solutions:

1. Removing the use of anyOf in any OM property definitions.
2. Writing a custom processor that generates deterministic classes inferred from the using anyOf.