How to list table columns in Athena database

In this article, I'll share with you a couple of ways to list all database columns and columns for a specific table in Amazon Athena.

SHOW COLUMNS command

This approach is excellent for quick investigation but is very limited. It returns only column names and outputs string instead of a table.

You can get a list of all columns within a table using this statement.

SHOW COLUMNS IN <TABLE_NAME>

Example

SHOW COLUMNS IN cities_csv
latd
latm
lats
ns
lond
lonm
lons
ew
city
state

Querying AWS Glue Data Catalog

Querying AWS Glue Data Catalog can provide you with full columns' metadata in a tabular format.

All table columns

To select all columns within Athena enviroment you can use this SQL query.

SELECT 
    table_catalog,
    table_schema AS database_name,
    table_name,
    column_name,
    ordinal_position,
    column_default,
    CASE
        WHEN is_nullable = 'YES' THEN TRUE
        ELSE FALSE
    END AS is_nullable,
    data_type
FROM information_schema.columns

Columns in a specific database

Below query helps you list columns in a specific database.

SELECT 
    table_catalog,
    table_schema AS database_name,
    table_name,
    column_name,
    ordinal_position,
    column_default,
    CASE
        WHEN is_nullable = 'YES' THEN TRUE
        ELSE FALSE
    END AS is_nullable,
    data_type
FROM information_schema.columns
WHERE table_schema = '<DATABASE_NAME>' 

Columns in a specific table

Below query lets you list columns in a specific table.

SELECT 
    table_catalog,
    table_schema AS database_name,
    table_name,
    column_name,
    ordinal_position,
    column_default,
    CASE
        WHEN is_nullable = 'YES' THEN TRUE
        ELSE FALSE
    END AS is_nullable,
    data_type
FROM information_schema.columns
WHERE table_schema = '<DATABASE_NAME>' 
    AND table_name = '<TABLE_NAME>'

Example

SELECT 
    table_catalog,
    table_schema AS database_name,
    table_name,
    column_name,
    ordinal_position,
    column_default,
    CASE
        WHEN is_nullable = 'YES' THEN TRUE
        ELSE FALSE
    END AS is_nullable,
    data_type
FROM information_schema.columns
WHERE table_schema = 'default' 
    AND table_name = 'elb_logs'

Response

Image title

Using AWS CLI

To get list of all tables and columns inside them you can also use official AWS CLI.

aws athena list-table-metadata \
    --catalog-name <CATALOG_NAME> \
    --database-name <DATABASE_NAME \
    --max-items 2

Example

aws athena list-table-metadata \
    --catalog-name AwsDataCatalog \
    --database-name geography \
    --max-items 2

Response

{
    "TableMetadataList": [
        {
            "Name": "country_codes",
            "CreateTime": 1586553454.0,
            "TableType": "EXTERNAL_TABLE",
            "Columns": [
                {
                    "Name": "country",
                    "Type": "string",
                    "Comment": "geo id"
                },
                {
                    "Name": "alpha-2 code",
                    "Type": "string",
                    "Comment": "geo id2"
                },
                {
                    "Name": "alpha-3 code",
                    "Type": "string",
                    "Comment": "state name"
                },
                {
                    "Name": "numeric code",
                    "Type": "bigint",
                    "Comment": ""
                },
                {
                    "Name": "latitude",
                    "Type": "bigint",
                    "Comment": "location (latitude)"
                },
                {
                    "Name": "longitude",
                    "Type": "bigint",
                    "Comment": "location (longitude)"
                }
            ],
"list-table-metadata is a paginated operation. Multiple API calls may be issued in order to retrieve the entire data set of results. You can disable pagination by providing the --no-paginate argument." ~AWS docs

You can read more on this method in the official docs.

Dataedo

You can also import AWS Athena columns using Dataedo. You can download Dataedo here and proceed with these instructions to import metadata from Athena.

Final result

Image title

Comments are only visible when the visitor has consented to statistics cookies. To see and add comments please accept statistics cookies.
0
There are no comments. Click here to write the first comment.