Skip to content

Latest commit

 

History

History
104 lines (81 loc) · 4.5 KB

QUICKSTART.md

File metadata and controls

104 lines (81 loc) · 4.5 KB
Raw

Dremio and Arrow Flight Quickstart

Documentation

  • Documentation is available here.

  • QUICKSTART at Dremio Here.

1 Query data using Flight clients

This process is the same if you launched the Dremio locally or via docker.

1.1 Query your datasets with Python

1.1.1 Getting Started

  1. Install Python 3
  2. Download and install the dremio-flight-endpoint whl file
    • python -m pip install <PATH TO WHEEL>
  3. Create a local folder to store the client file and config file.
    1. Create a file named example.py in created folder. Copy the contents of arrow-flight-client-examples/python/example.py into example.py.
    2. Create a file named config.yaml in created folder. Copy the contents of arrow-flight-client-examples/python/config_template.yaml into config.yaml.
    3. Uncomment the options in config.yaml as needed, appending the argument after the key. ie. username: my_username. You can either delete the options that are not being used or leave them commented.
      • e.g. If you are connecting to a local instance of Dremio, your config file would look like:
      username: my_username
password: my_password
query: SELECT 1
  4. Run the Python Arrow Flight Client by navigating to the created folder in the previous step and running the command python3 example.py.

1.2 Query your dataset with arrow flight client in java

This lightweight Java client application connects to the Dremio Arrow Flight server endpoint. It requires the username and password for authentication. Developers can use admin or regular user credentials for authentication. Any datasets in Dremio that are accessible by the provided Dremio user can be queried. By default, the hostname is localhost and the port is 32010. Developers can change these default settings by providing the hostname and port as arguments when running the client. Moreover, the tls option can be provided to establish an encrypted connection.

1.2.1 Prerequisites

  • Java 8
  • Maven 3.5 or above

1.2.2 Build the Java sample application

  • Clone this repository.
  • Navigate to arrow-flight-client-examples/java.
  • Build the sample application on the command line with:
    • mvn clean install -DskipTests

1.2.3 Instructions on using this Java sample application

  • By default, the hostname is localhost and the port is 32010.
  • Run the Java sample application:
    • java -jar target/java-flight-sample-client-application-1.0-SNAPSHOT-shaded.jar -query <QUERY> -host <DREMIO_HOSTNAME> -user <DREMIO_USER> -pass <DREMIO_PASSWORD>
  • The application has a demo mode that runs an end-to-end demonstration without any arguments required, use the -demo flag to run the demo:
    • java -jar target/java-flight-sample-client-application-1.0-SNAPSHOT-shaded.jar -demo
    • To run the demo, you must have a running Dremio instance at the specified host and port.
    • The Dremio instance must also have services.flight.auth.mode: "arrow.flight.auth2" set in the dremio.conf file.
  • Learn more about different command line options with the help menu:
    • java -jar target/java-flight-sample-client-application-1.0-SNAPSHOT-shaded.jar -h

1.2.4 Usage

usage: java -jar target/java-flight-sample-client-application-1.0-SNAPSHOT-shaded.jar -query <QUERY> -host <DREMIO_HOSTNAME> -port <DREMIO_PORT> -user <DREMIO_USER> -pass <DREMIO_PASSWORD>

optional arguments:
  -h, --help
    show this help message and exit
  -port, --flightport
    Dremio flight server port
    Default: 32010
  -host, --hostname
    Dremio co-ordinator hostname
    Default: localhost
  -kstpass, --keyStorePassword
    The jks keystore password
  -kstpath, --keyStorePath
    Path to the jks keystore
  -pass, --password
    Dremio password
    Default: dremio123
  -demo, --runDemo
    A flag to to run a demo of querying the Dremio Flight Server Endpoint.
    Default: false
  -query, --sqlQuery
    SQL query to test
  -tls, --tls
    Enable encrypted connection
    Default: false
  -user, --username
    Dremio username
    Default: dremio
  -projectId, --projectId
    Dremio Cloud project to connect to.
    Default: default project for organization

Forums

If you have any questions, click here to join our forums.