Apache Spark SQL support (ODBC)

Applies to: 8.x versions, Article available also for: 9.x (current), 7.x
You are looking at documentation for an older release.
Switch to the documentation for Dataedo 9.x (current).

Apache Spark SQL is currently not officially supported. Connection is possible with generic ODBC driver. Metadata returned depends on driver version and provider.

We have tested and successfully connected to and imported metadata from Apache Spark SQL with ODBC drivers listed below. It is highly likely it will work with other drivers as well.

Tested ODBC Driver and environments:

We have tested and successfully connected to and imported metadata in following distributions:

  • Cloudera

  • Hortonworks

Hive ODBC version: 2.6.1 (64 bit)

Supported schema elements and metadata

Dataedo reads following metadata from Apache Spark SQL.

  • Tables
    • Columns
      • Data type with length
      • Nullable
  • Views (displayed as a table)
    • Columns
      • Data type with length
      • Nullable

ODBC Driver

Download Hive ODBC driver

ODBC configuration

Learn more

Connect to Apache Spark SQL

Found issue with this article? Comment below
0
There are no comments. Click here to write the first comment.