Amazon DynamoDB support

Applies to: Dataedo 23.x versions, Article available also for: 24.x (current), 10.x
You are looking at documentation for an older release.
Switch to the documentation for Dataedo 24.x (current).

DynamoDB is a key-value and document database that is based on a single-table model. Working with DynamoDB is more about access patterns, data usage, and denormalization rather than the traditional approach of highly normalized, relational databases modeled in accordance with business needs. Such an approach results in much faster times of queries for tables containing billions of rows.

Dataedo provides a native connector to DynamoDB which documents primary key and attributes of the table.

Supported elements and metadata

Dataedo reads following metadata from DynamoDB databases:

  • Tables
  • Attributes
    • Name
    • Primitve Types
    • Complex Types
  • Primary Keys
    • Partition Key
    • Sort Key

Not supported metadata:

  • Secondary indexes

Configure AWS

Connecting Dataedo to DynamoDB instance requires you to create an IAM (Identity and Access Management) account, which is basically a programmatic user that has its' own Access Key (username) and Secret Key (password). Authenticating with this account is a recommended way to access AWS services, since it's much more secure than using a root account. Following section describes configuring IAM account which will be used to conncect Dataedo to DynamoDB.

Create an IAM user

Dataedo connects to AWS Athena with an IAM user, which is a default authentication method for programmatic access. Account for Dataedo will require the following permissions:

  • AmazonDynamoDBReadOnlyAccess

To create an IAM user, go to IAM service main window and open the Users tab and click the Add user button. Give a user a name and select Access Key – Programmatic access in the Select AWS access type section. Click Next.

create-iam-user

In Permissions tab, select Attach existing policies directly, and look for AmazonDynamoDBReadOnlyAccess (you can use filter policies search bar). Click Next.

dynamodb-policies

In next step, you can set tags, optionally. Click next, review settings and create a user. Save the IAM credentials, as you will need them later to authenticate Dataedo.

Get AWS region

Open DynamoDB service home page. You will find DynamoDB region list in upper right corner. Look for highlighted region, which is the region where DynamoDB tables reside.

DynamoDB region

Connect Dataedo to Amazon DynamoDB

Add new connection

To connect to Amazon DynamoDB with Athena create new documentation by clicking Add documentation and choosing Database connection.

Add connection

On the Add documentation window choose Amazon DynamoDB:

dynamodb-source

Connection details

Provide connection details. To obtain them, follow steps described in Configure AWS section of this article:

  • AWS Region - AWS region in which DynamoDB reside,
  • Access Key - IAM user access key ID,
  • Secret Key - IAM user secret key,
  • Sample Size - number of documents/rows which will be scanned to get the attributes.

dynamodb-conn-details

Click Connect, and select objects to import. Click Next and set the documentation title. After succesfull import click Finish.

Outcome

Your DynamoDB database metadata has been imported to new documentation in the repository.

dynamodb-outcome