Data Source Connection Management

Key Concepts

For Sisense to retrieve data from a data source, you must create a connection between Sisense and the data source.

Note the following terms:

  • Connector - A connector is a standalone software or function used to connect any application to a database system. A connector has the information required to connect to a particular data source type. Multiple connections can be based on each connector.

  • Connection - A connection is based on a connector and the information required to connect to a specific data source instance. The instance information required is the data source's:

    • URL (connection string)

    • Credentials

Introduction to Connection Management

Connection Management enables efficient management and governance of all data connections in one place. It enables seamless connectivity between Sisense and various data sources, ensuring that connections are reusable, secure, and sharable across multiple assets such as ElastiCubes, Live Models, and Notebooks.

Traditionally, each data model required its own separate connection, leading to potential duplication, difficulty in governance, and manual updates. With Connection Management, this process is streamlined by allowing a single connection to be shared and reused across different models and assets, reducing redundancy and ensuring consistency.

From a technical perspective, Connection Management acts as a repository for all connections. Each connection is treated as an atomic entity within the system, identified by its own OID (Object Identifier), which remains consistent across different environments - just like Data Models or Dashboards.

Key Use Cases

  • Reuse Across Multiple Assets: A single connection can be reused across multiple data models and asset types (ElastiCubes, Live Models, Notebooks), streamlining connection management.

  • Secure Sharing: Connections can be shared without the need to expose sensitive data, such as login credentials or passwords, ensuring security across teams and users.

  • Centralized Management: Centralized management and governance enable administrators to control and monitor all connections from a single location, simplifying oversight.

  • Instant Updates: Any changes made to a connection are instantly applied to all dependent assets, eliminating the need for manual updates and ensuring consistency across the system.

  • Encourages Connection Sharing: The system encourages the sharing of existing connections to reduce duplication and maintain a streamlined connection structure.

  • Visibility and Dependency Management: Gain visibility into all assets that depend on a connection, with the ability to manage dependencies, ensuring that any connection changes are carefully handled.

Since the release of Sisense version L2024.3, Connection Management has become the unified and single solution for managing all connections to data sources in Sisense.

Creating New Connections

Creating a connection is an integrated step within the asset creation wizard - whether you are designing a Notebook, ElastiCube, or Live Data model. During the creation process, you can either choose an existing connection that is available to you or create a new one.

To create a new connection:

  1. Click Create New.

  2. Select the desired data source provider.

  3. Fill in the required details, such as the connection name, user name, and password.

Testing of the connection is part of the Save/Continue validation process to ensure that the connection details are correct and functional before proceeding.

Creating new connections is available for roles higher than Designer > Admin, Data Admin, and Data Designer.

Sharing Connections

Connections can be shared during the creation flow or afterward. To share a connection while creating it:

  1. Once it is saved, click Share.

  2. Select the user or group with whom you want to share the connection.

For existing connections:

  1. Find the connection in the list.

  2. Click the 3-dot context menu and select Share.

Connections can be shared with the following permission levels:

  • Can Use Connection: Allows the user to use the connection for creating data assets, but without access to credentials

  • Can Edit Connection: Allows the user to edit the connection details

Admin and Data Admin roles can share any connection within the instance, while Data Designers can only share connections they own.

From a technical perspective, connections have four types of permissions:

  • OWNER: Defines the owner of the connection

  • EDIT: Enables editing of any connection details

  • USE: Allows users to create data assets based on the connection, without having access to the credentials

  • READ: Granted by default to all users, enabling them to read data from assets such as dashboards and widgets

To unshare a connection, open the Share window, find the user in the list, and click the trashcan icon to remove their access.

Note:

Sharing a connection in the Notebooks tab is currently not supported and should be performed using the PATCH​/connections​/{connectionId}​/shares​/update endpoint.

Editing Connections

To edit an existing connection:

  1. Find the connection in the list.

  2. Click the 3-dot context menu and select Edit.

Important:

  • Any changes made to a connection are instantly applied to all assets dependent on it, such as Notebooks, ElastiCubes, or Live Data models. Ensure that the updates will not disrupt any assets using the connection.

  • Saving an edited connection requires testing it, which is included out-of-the-box when clicking Save.

Checking Dependencies

To see which assets are using a particular connection:

  1. Click the 3-dot context menu of the connection and select Dependencies.

    A list of data models that depend on the connection is displayed.

  2. If the list of dependent assets is too large, or if you need a complete array of all dependencies, click Download list.

Note:

Always check the dependencies before editing or deleting a connection to avoid unintended disruptions to assets such as Notebooks, ElastiCubes, or Live Data models.

Deleting Connections

To delete a connection:

  1. Select the connection from the list.

  2. Click the 3-dot context menu of the connection and select Delete.

If there are assets that depend on the connection, you must select a new connection to replace it before proceeding.

Note:

Once a connection is deleted, it cannot be restored, and all dependent assets will be affected. Ensure you check the dependencies beforehand to prevent unintended disruptions to assets like Notebooks, ElastiCubes, or Live Data models.

Connection Utilization Scope

While we aim to minimize duplicated connections, there are some limitations due to differences in connection structure between Notebooks and data models:

  • Connections created in Notebooks: These connections require a “Default Database” and can only be used within Notebooks

  • Connections created in Data Models: These connections can be used in both ElastiCubes and Live Data models, provided the connector is supported in Live Data models (e.g., CSV is not compatible with Live Data models)

From a technical perspective, the accessibility of a connection for a specific asset type is defined by the “supportedModelTypes” parameter. The available options are:

  • EXTRACT > ElastiCube

  • LIVE > Live Data Model

  • NOTEBOOK > Notebook

To modify the default permissions, for example, limiting the connection to ElastiCubes only, you can use the PATCH /connections/{connectionId} endpoint.

Multitenancy Considerations

Connections are isolated within the tenant they are created from and cannot be accessed by users from other tenants.

Centralized Connections Hub

To view all available connections, navigate to the new data model creation wizard by clicking from within the data model. For more detailed information about each connection, such as its description and creation date, switch to the List view using the toggle in the top-right corner.

The List view also enables filtering and sorting of connections for easier management.

Roles:

  • Admins and Data Admins: Can view all connections within their tenant scope

  • Data Designers: Can only view connections they own or those shared with them

Note:

A centralized connection list, accessible directly from the data page (without needing to enter the data model wizard), will be available for all users in the next release, further simplifying access.

Importing Connections to the System

Currently, there is no dedicated option to export connections individually, such as Perspectives or Data Security. However, connections can still be imported into the system via two main methods: Importing Data Models and Promoting via Git Integration.

Importing Data Models

When a data model is exported, the connection parameters are encrypted and included as part of the exported .smodel or .sdata files. Upon importing the data model back into the system, these connection parameters are automatically converted into a managed connection, unless a connection with the same OID already exists in the instance.

Important:

If the imported data model contains a modified connection with an OID that already exists in the system, the existing connection will remain unaffected. The system will simply fetch the current connection details from the database using the OID from the data model.

Example:

If an exported data model has a connection with OID 123 based on the connection string jdbc:snowflake:ABC, but the connection string was later changed to jdbc:snowflake:DEF, upon importing the data model, the system will use the current connection details from the database (i.e., jdbc:snowflake:DEF), not the one from the imported data model.

The ability to duplicate connections during the import of data models will be introduced in a future release.

Git Integration Promotion

When pulling a data model from a remote repository, such as GitHub or Azure, into the system, an empty connection is generated, unless a connection with the same OID already exists in the instance.

Since sensitive data like connection parameters are excluded from tracked files in the Git project, for the data model to function properly, you must update the connection with the relevant credentials under the static connection OID.

For example:

Dev Server:

  • Data Model: Sample E-Commerce

  • Connection OID: 123

  • User: Developer

  • Password: xxxxxx

Stage Server:

  • Data Model: Sample E-Commerce

  • Connection OID: 123

  • User: QA

  • Password: xxxxxx

Prod Server:

  • Data Model: Sample E-Commerce

  • Connection OID: 123

  • User: Admin

  • Password: xxxxxx

This way, the same data model will use the same connection OID across environments, while pointing to the appropriate database for each environment.

Note:

To prevent merge conflicts and simplify the flow, for Git imported data models, the option to switch connections is disabled. You can only update the imported one. Once it is updated, all of the options are the same as in regular data models, including switching.

API Endpoints

Endpoints to work with the feature > 2.0 Connections Management:

  • GET: retrieves connection

    • all connections

    • single connection

  • POST: creates new connection

    • connector’s manifest can be accessed in sisense.dev

  • PATCH: updates existing connection

    • updating full connection or specific properties

    • adjust shares - grant/revoke access

  • DELETE: removes connection

    • has validation on the dependencies, which must be addressed in the UI

    • dedicated endpoints for retrieving dependencies and replacing connection planned for latter phases

Enablement and Migration

Starting with Sisense version L2024.3 and newer, Connection Management becomes the unified and standard solution for managing connections to data sources for all customers. The migration to this new system is part of the system upgrade and involves several key steps:

  • Conversion of Old Connections: All legacy connections are converted into the new managed connections format.

  • Setting “supportedModelTypes”: The connection type is determined by the asset it was converted from:

    • NOTEBOOK: If the connection was converted from a Notebook

    • EXTRACT/LIVE: If the connection was converted from a Data Model

  • Deduplication of Connections: Keeping only unique connection aiming to prevent duplicates. If there were multiple connections with identical connection details but owned by different users, they are converted into two separate managed connections.

  • Ensuring Backward Compatibility: All assets that previously used these connections will continue to function seamlessly, maintaining full backward compatibility.

This migration process ensures a smooth transition to the new Connection Management system without disrupting existing workflows or assets.