starbase icon indicating copy to clipboard operation
starbase copied to clipboard

Integration with graph-google-cloud failure on .env generation

Open lemontreeran opened this issue 3 years ago • 2 comments

When we are doing yarn starbase run to run the Google Cloud(graph-google-cloud) integration with Starbase, the .env in .integrations/graph-google-cloud is not generated correctly. Therefore nothing is imported into the Neo4J database. Would you please have a look at this issue and supply the fix?

Here is the config.xml in different format that I tested.

  1. Use filename for config.yaml[1] in Starbase
integrations:
  - name: graph-google-cloud
    instanceId: testInstanceId
    directory: ./.integrations/graph-google-cloud
    gitRemoteUrl: https://github.com/JupiterOne/graph-google-cloud.git
    config:
      SERVICE_ACCOUNT_KEY_FILE: sa-private-key.json
      PROJECT_ID: "PROJECT_ID"
      ORGANIZATION_ID: "ORGANIZATION_ID"
      CONFIGURE_ORGANIZATION_PROJECTS: false
      FOLDER_ID: ""
storage:
  engine: neo4j
  config: 
    username: neo4j
    password: devpass
    uri: bolt://localhost:7687
  • Here is the .env generated .integrations/graph-google-cloud. According to docs[2], this .env is expecting only string. The json file name is treated as a string which is not getting any correct connections with GCP.
SERVICE_ACCOUNT_KEY_FILE=sa-private-key.json
PROJECT_ID=PROJECT_ID
ORGANIZATION_ID=ORGANIZATION_ID
CONFIGURE_ORGANIZATION_PROJECTS=false
FOLDER_ID=
    1. Use the flattened GSA private key json string for config.yaml[1] in Starbase
integrations:
  - name: graph-google-cloud
    instanceId: testInstanceId
    directory: ./.integrations/graph-google-cloud
    gitRemoteUrl: https://github.com/JupiterOne/graph-google-cloud.git
    config:
      SERVICE_ACCOUNT_KEY_FILE: {"type":"service_account","project_id":"project_id","private_key_id":"private_key_id","private_key":"-----BEGIN PRIVATE KEY-----\n<encrypted string>\n-----END PRIVATE KEY-----\n","client_email":"client_email","client_id":"client_id","auth_uri":"auth_uri","token_uri":"https://oauth2.googleapis.com/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_x509_cert_url":"client_x509_cert_url"}
      PROJECT_ID: "PROJECT_ID"
      ORGANIZATION_ID: "ORGANIZATION_ID"
      CONFIGURE_ORGANIZATION_PROJECTS: false
      FOLDER_ID: ""
storage:
  engine: neo4j
  config: 
    username: neo4j
    password: devpass
    uri: bolt://localhost:7687
  • Here is the .env generated .integrations/graph-google-cloud. According to docs[2], this .env is expecting only string. The json object is just converted into [object Object] which is not a json object anymore.
SERVICE_ACCOUNT_KEY_FILE=[object Object]
PROJECT_ID=PROJECT_ID
ORGANIZATION_ID=ORGANIZATION_ID
CONFIGURE_ORGANIZATION_PROJECTS=false
FOLDER_ID=

Referece: [1] https://github.com/JupiterOne/starbase/blob/main/README.md?plain=1#L170 [2] https://github.com/JupiterOne/graph-google-cloud/blob/main/docs/development.md?plain=1#L229

lemontreeran avatar Aug 18 '22 00:08 lemontreeran

Hi there!

In the second configuration I think the only needed update is to surround the entire SERVICE_ACCOUNT_KEY_VALUE in single quotes, like:

SERVICE_ACCOUNT_KEY_FILE: '{"type":"service_account","project_id":"project_id",...}'

That should let Starbase see the value as a flattened string value instead of an object. Long term, we can look at better handling when Starbase sees config values as objects like this, but this should hopefully be a workaround for you to move forward now.

adam-in-ict avatar Aug 18 '22 12:08 adam-in-ict

Thanks @adam-in-ict ! This workaround works. I will try this solution for the time being.

lemontreeran avatar Aug 20 '22 05:08 lemontreeran