Connecting to a local DynamoDB instance

[sventhebarbarian]

When running tests locally using an offline/local DynamoDB instance, we need to change the AWS region and endpoint to localhost and the local DynamoDB.

Using powertools I have not been able to work out a way to do this, it seems to normally be set when initiating AWS.DynamoDB.DocumentClient, but this is hardcoded inside the powertools dynamo client code.

I tried env variables but Dynamo threw an error the region was invalid, maybe switching aws credential files but that seems dangerous and messy.

There is probably an easy way to do this I am not thinking of, another option would be a method in the dynamo client that can reset the region/endpoint?

[mtimbs]

The AWS SDK documentation makes it look like you cant set the config of the DocumentClient after instantiation and digging through the source code we can see they've not allowed a public API for updating config on the class.

// aws-sdk/lib/document_client.js

  constructor: function DocumentClient(options) {
    var self = this;
    self.options = options || {};
    self.configure(self.options);
  },

  /**
   * @api private
   */
  configure: function configure(options) {
    var self = this;
    self.service = options.service;
    self.bindServiceObject(options);
    self.attrValue = options.attrValue =
      self.service.api.operations.putItem.input.members.Item.value.shape;
  },

If they did we could do something as simple as...

// DynamoDBClient.ts

import { ClientConfiguration } from 'aws-sdk/clients/dynamodb';
import DynamoDB from '@dazn/lambda-powertools-dynamodb-client';

const config: ClientConfiguration = {
  region: process.env.AWS_REGION,
  accessKeyId: process.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_KEY,
};

if (['test'].includes(process.env.NODE_ENV)) {
  config.endpoint = 'http://localhost:8000';
  config.sslEnabled = false;
}

export default DynamoDB.configure(config);

The library could probably be tweaked so that we could change the export line above to take a parameter

e.g. export default DynamoDB(configuration)

That way we could easily use this plugin online, importing the DynamoDBClient into any functions that need to interact with Dynamo, and have environment variables control the connection type.

I'm 99.9% sure this can't be changed without introducing a breaking change unfortunately 😞

If you don't need any of the context and tracing for offline tests you could do the following.....

// DynamoDBClient.ts
import { DocumentClient } from 'aws-sdk/clients/dynamodb';
import DynamoDB from '@dazn/lambda-powertools-dynamodb-client';

export default ['test'].includes(process.env.NODE_ENV)
  ? new DocumentClient({
    region: process.env.AWS_REGION,
    accessKeyId: process.env.AWS_ACCESS_KEY,
    secretAccessKey: process.env.AWS_SECRET_KEY,
    endpoint: 'http://localhost:8000',
    sslEnabled: false,
  })
  : DynamoDB;

[sventhebarbarian]

I hacked it this way:

file: dazn-lambda-powertools/packages/lambda-powertools-dynamodb-client/index.js

----------added near top

let options = {};
if (process.env.IS_OFFLINE) {
  options = {
    region: 'localhost',
    endpoint: 'http://localhost:8000',
  };
}

then changed client initialization to: const client = new DynamoDB.DocumentClient(options)

The region and endpoint should be set by env variables rather than hardcoded, and process.env.IS_OFFLINE could be changed to something else (was playing with Serverless offline which automatically adds this)

not sure if this breaks the design but might be a way to code it in, or for others who need to hack it

[mtimbs]

That is one option I did not consider.... allowing the library itself to access environment variables.... It would certainly mean no breaking changes, but unfortunately it also relies on a specific variables being set which makes things tricky to generalise.

[sventhebarbarian]

Yes it seems a bit messy to add arbitrary env's.

Another option, but it would change the way the dazn module is programmed, is to not initialize DocumentClient in the modules global scope, I believe this would mean abstracting the dynamodb client into its own class with DocumentClient as a member, rather than extending DocumentClient directly and returning that derived class.

I understand you could then add the connection options as eg an argument to the classes constructor (or default to empty).

[cdelgadob]

Hey @theburningmonk , any updates on this?

I'm happy to submit a PR in the line of this proposed solution:

I hacked it this way:

file: dazn-lambda-powertools/packages/lambda-powertools-dynamodb-client/index.js

----------added near top

let options = {};
if (process.env.IS_OFFLINE) {
  options = {
    region: 'localhost',
    endpoint: 'http://localhost:8000',
  };
}

then changed client initialization to: const client = new DynamoDB.DocumentClient(options)

The region and endpoint should be set by env variables rather than hardcoded, and process.env.IS_OFFLINE could be changed to something else (was playing with Serverless offline which automatically adds this)

not sure if this breaks the design but might be a way to code it in, or for others who need to hack it