External Data Store

External Data Stores allow you to push information from Chakra to your own data warehouse / data lake or any sort of datastore. This push happens in realtime and is useful for syncing Chakra data state to your warehouse when the size of data is large.

Currently the following datastores are supported

• AWS Kinesis
• Apache Kafka

The List of datastores can be accessd from the External Data Store Dashboard

NOTE: These pages are only usable by Admin users.

AWS Kinesis

Please add the following details for a Kinesis Data Store to enable a connection

• Region - AWS Region
• Stream Name - The kinesis stream name to which you want to connect
• Access Key Id - The AWS accessKeyId
• Secret Access Key - The AWS secretAccessKey

Test

To Test if the connection is valid or not please click Test Connection

If the test is successful you will see a prompt like this:

If the test fails you will see a prompt like this:

Apache Kafka

To configure Apache kafka as an External Data Store You need to provide the following details:

• Broker (required) - List of Kafka brokers (min 1)
• SSL Enabled - should ssl be enabled. if enabled the SSL specific configurations also need to be completed
• SSL Reject Unauthorized - Should ssl reject unauthorized connections
• SSL CA - SSL Certifying Authority (CA) stores. Contents should be in utf8
• SSL Key - SSL key usually like client-key.pem. Contents should be in utf8
• SSL Cert - SSL cert usually like client-cert.pem. Contents should be in utf8

Test

To Test if the connection is valid or not please click Test Connection

If the test is successful you will see a prompt like this:

If the test fails you will see a prompt like this:

Postgres

To configure Postgres as an External Data Store You need to provide the following details:

• Hostname (required) - hostname of the db instance
• Port (required) - port of the db instance
• Username (required) - username for the db
• Password (required) - password for the db
• Database Name (required) - name of the database with which to connect

Test

Test connection

To Test if the connection is valid or not please click Test Connection

If the test is successful you will see a prompt like this:

If the test fails you will see a prompt like this:

Test sending a data event

You can also test sending a data event and writing to a test table in the postgres database.

First you need to create a chakra_connection_test table.

create table "public"."chakra_connection_test" (
    "id" uuid not null,
    "createdAt" int8,
    "data" jsonb 
);

Then enter a json test data into the json test data box. Press "Send Test Data" to test sending the data event.