Amazon Athena support

Dataedo 10.2 introduces native connector to Amazon Athena. Amazon Athena is a query engine on Amazon Web Services. It is unique in a way that it provides access to AWS Glue Data Catalog, which holds metadata from various AWS services.

Using Athena connector you can extract metadata from:

• AWS Glue Data Catalog
• Amazon S3
• Amazon DocumentDB
• Amazon DynamoDB
• Amazon RDS
• Amazon Redshift

Connector

Supported schema elements and metadata

• Tables
  • Columns
    • Data type
    • Position
    • Nullable
    • Description
    • Default value
• Views
  • Definition
  • Columns
    • Data type
    • Position
    • Nullable

Supported versions

Dataedo Athena connector was tested with Athena Engine version 2. Athena Engine version 1 is not officially supported.

Data Profiling

Dataedo does not support data profiling in the Amazon Athena.

Tested data catalogs

• S3 - Aws Glue Data Catalog
• Amazon DynamoDB
• Amazon DocumentDB
• Amazon CloudWatch
• Amazon CloudWatch Metrics

Configure AWS Services

Set up AWS Athena

You do not need to explicitly activate Athena as by default it is enabled. Although, if you have never used it in a selected region, you need to select an S3 bucket for storing query results.

To configure this service for the first time, find Athena Service in an AWS console. If it is the first launch in a region, you will see an Athena home screen. Click the Explore the query editor button, open settings tab and click the Manage button.

Open a list of available buckets by clicking Browse S3 and select bucket in which query results will be saved. IMPORTANT: If you cannot find a bucket, make sure it was created in the same region as currently selected Save settings.

Now you need to configure a custom Workgroup. Open the Workgroups tab and click the Create workgroup button.

Set the name for the workgroup and select the S3 bucket (can be the very same bucket as previously selected for Athena). Additionally, you can uncheck Publish query metrics to AWS CloudWatch.

Create an IAM user

Dataedo connects to AWS Athena with an IAM user, which is a default authentication method for programmatic access. Account for Dataedo will require the following permissions:

• AWSQuicksightAthenaAccess – to read metadata with Athena
• AmazonS3FullAccess – to save query results in an S3 bucket
• AWSLambdaRole – to run the Lambda function

First, create an IAM group with the required permission. Find IAM Service in AWS console, open the User Groups tab, and click Create Group button.

Give your group a distinctive name, and add the aforementioned permissions in Attach permissions policies section.

Go back to IAM service main window and open the Users tab and click the Add user button. Give a user a name and select Access Key – Programmatic access in the Select AWS access type section.

Go next and add the user to the group created in the previous step. Other options can be left default.

In the last step, AWS will provide you with an Access key ID and Secret access key. These are credentials to your IAM account which you will later use to connect to Athena with Dataedo. Store them safely (we recommend saving these values in an encrypted password manager file).

Connect Dataedo to AWS Athena

Add new connection

To connect to AWS Athena create new documentation by clicking Add documentation and choosing Database connection.

On the Add documentation window choose Amazon Athena:

Connection details

Provide connection details:

• AWS Region - AWS region in which Athena reside.
• Access Token - IAM user access key ID.
• Secret Token - IAM user secret key.
• Data Catalog - Data catalog (athena data source) you want to connect to.
• Workgroup - Athena workgroup
• Database - Athena database.

Importing Metadata

When connection was successful Dataedo will read objects and show a list of objects found. You can choose which objects to import. You can also use advanced filter to narrow down list of objects.

Confirm list of objects to import by clicking Next.

Next screen allow you to change default name of the documentation under which it will be visible in Dataedo repository.

Click Import to start the import.

When done close import window with Finish button.

Outcome

Your Athena database schema has been imported to new documentation in the repository.