Replicating the Application Database

When you install Sisense, an application database is installed locally on your machine that contains metadata for ElastiCubes, dashboards, and users. If you have several application databases across multiple servers, you can combine them to form a replica set. The goal of a replica set is to ensure that each application database has the same metadata about ElastiCubes, dashboards, and filters. If one database fails on one query server, the next available database can replace it. To maintain consistency throughout your configuration and support high availability, each database must be identical with the other databases in your configuration.

There are two types of nodes within a replica set. The primary node, which supports write operations and the secondary nodes, which replicate the primary database's operations log and syncs the data to reflect that of the primary. The primary and secondary nodes can be hosted on the build or query nodes in a high availability configuration on servers with or without Sisense installed.

There are four steps to support high availability in Sisense:

  1. Setting Up Replica Nodes: Each database must be configured to connect to other instances in the replica set and share a KeyFile that is used for authentication.

  2. Creating a Replica Set: After setting up your nodes, you need to set one as the primary, which has write access to the other nodes in the replica set.

  3. Connecting to the Replica Set: Once you have created the replica set, connect it to Sisense.

  4. Connecting Additional Web Servers to Your Replica Set: Your replica set can support your Sisense. To do so, you need to connect it to the replica set instead of the localhost that it is connected to be default. This way, if your server's database fails, the Sisense is supported by another database within the replica set.

This topic describes how to set up your databases in a replica set and connect it to Sisense.

Prerequisites

  1. Servers with port 27018 open.
  2. RoboMongo installed on a machine that has access to those servers on port 27018 (any node can be on one of the servers).
  3. MongoDB installed on the machine. By default, this is installed when Sisense is installed. If Sisense is not installed, download the MongoDB version 3.4.5 zip file on the secondary servers from here.

Setting Up Replica Nodes

Note:

For each machine that you have installed Sisense, you must stop the Sisense-related services from running to prepare the primary database that will be replicated to the other secondary nodes.

Each node in your replica set must have a MongoDB configuration file. The configuration file contains information about the database such as its open port and the name of the replica set the application database belongs to. When Sisense is installed on the node, this configuration file is created for you. For nodes in which Sisense is not installed, you must create this configuration file as described in the procedure below.

In addition, each database must share a KeyFile used for authentication. This KeyFile must be the same on each instance, whether Sisense is installed or not.

This procedure describes how to set up a replica node on a Sisense server or a server where Sisense is not installed.

To set up a replica node:

  1. Create a KeyFile. See Creating a KeyFile for more information.
  2. Create a MongoDB user with "write access".

  3. When Sisense is installed:

    1. Stop the Sisense.Repository service.
    2. Copy the KeyFile you prepared above as described in Step 1 and add it to the directory ...\Program Files\Sisense\Infra\MongoDB.
    3. In the ...\Program Files\Sisense\Infra\MongoDB\mongodbconfig.conf directory, backup the MongoDB configuration and save the backup copy in another directory.

    4. In the MongoDB directory, open the mongodbconfig.conf file and uncomment the commented lines, keyFile, replication and replSetName, by removing # from the beginning of the row.

      storage:
      dbPath: "C:/ProgramData/Sisense/PrismWeb/Repository/DB"
      journal:
      enabled: true
      systemLog:
      destination: file
      path: "C:/ProgramData/Sisense/PrismWeb/Repository/Logs/sisenseRepository.log"
      net:
      bindIp: 127.0.0.1
      port: 27018
      # ssl:
      # mode: requireSSL
      # PEMKeyFile: keyFile path
      # PEMKeyPassword: keyFile password
      security:
      authorization: "enabled"
      #keyFile: "C:/Program Files/Sisense/Infra/MongoDB/keyfile"
      # replication:
      # replSetName: replica set name             

  4. Bind each of your servers in your high availability configuration together by adding their IP addresses and separating them with a comma:

    For Sisense V6.7 and earlier:
    net:
    bindIp: [127.0.0.1, XX.XX.XX.XX, XX.XX.XX.XX]


    For Sisense V6.7.1 and later
    net:
    bindIp: 127.0.0.1,XX.XX.XX.XX

    Note:

    The value of bindIp has been changed from an array to a string. As a string, the IP addresses should be separated by a comma with no space in between. In addition, the only IP addresses to be included in the string are 127.0.0.1 and the public IP of your local server.

    1. Under security, define the location of your KeyFile created in Step 1.

      security:
      authorization: "enabled"
      keyFile: "C:/Program Files/Sisense/Infra/MongoDB/keyfile"

    2. Set a name for the replica set.

      replication:
      replSetName: replicasetname

    3. Save and close the mongodbconfig.conf file.
    4. Start the Sisense.Repository service.

  5. When Sisense is not installed:

    1. Copy the KeyFile you prepared above as described in Step 1 to your MongoDB's configuration file folder.
    2. In the MongoDB configuration folder, open the mongodbconfig.conf file.

    3. Bind each of your servers in your high availability configuration together by adding their IP addresses and separating them with a comma:

      net:
      bindIp: [127.0.0.1, XX.XX.XX.XX, XX.XX.XX.XX]

      Note:

      You can add your PC's IP address as well to allow you connect to the MongoDB with an application such as RoboMongo.

    4. Under security, define the location of your KeyFile created in Step 1.

      security:
      authorization: "enabled"
      keyFile: "C:/MongoDB/keyfile"

    5. Set a name for the replica set.
      replication:
      replSetName: <replicasetname>

    6. Save and close the mongodbconfig.conf file.

Creating a Replica Set

After you have prepared the MongoDB configuration file for each of your nodes, you must create a replica set with these nodes. With a replica set, each member has the same Sisense data but is otherwise independent. If the primary becomes unavailable, the replica set holds an election to select a new primary from one of the secondary nodes.

You should perform this procedure on your primary node. It sets the server as the primary node, and connects the secondary nodes to it.

To create replica set:

  1. Open Robomongo with a WriteUser account and select Open Shell to connect to the primary node.

  2. In the new shell, run the command rs.initiate() to create a replica set.

  3. Run the following commands:

    cfg = rs.conf()
    cfg.members[0].host = "XX.XX.XX.XX:27018" (Enter your IP address of the primary node and the MongoDB port, by default 27018).
    rs.reconfig(cfg)

  4. Run the command rs.add("XX.XX.XX.XX:27018"), where the XX is the IP address of your secondary node. Run this command to add the unique IP address for each secondary node.

    Example:

    rs.initiate()
    cfg = rs.conf()
    cfg.members[0].host = "10.50.1.199:27018"
    rs.reconfig(cfg)
    rs.add("10.20.1.196:27018")
    rs.add("10.20.1.197:27018")

  5. Run the command rs.status() to verify that the replica set was configured properly. Run this command every couple of minutes to check the progress until stateStr of the secondaries service displays SECONDARY.

Connecting Sisense to the Replica Set

After you have created the replica set, you must connect the primary web application server to your replica set to support your high availability configuration.

To modify the primary server's configuration files:

  1. In your browser, open the Configuration Manager located at localhost:3030.

  2. Under Application Database, modify the value of Connection String to:

    mongodb://AppUser:we6jBUYGoOrh0K6l+XpTmA==@localhost:27018,XX.XX.XX.XX:27018,XX.XX.XX.XX:27018/?replicaSet=&readPreference=primaryPreferred
    replacing '@localhost:27018/' with the IP addresses of each of your nodes in the replica set separated by a comma and replacing with the name of your replica set.

    Example:

  3. Click Save to save your changes.

  4. Click Yes to restart the relevant services.

Connecting Additional Web Servers to Your Replica Set

To add additional nodes to support your primary web server, each additional server must be configured with the IP address of the primary and its AppUser credentials. The AppUser defines who can access the application database.

This procedure describes how to configure additional nodes for high availability.

To configure your nodes:

  1. Open the Configuration Manager in your browser located at localhost:3030.

  2. Under Application Database, modify the value of the Connection String and Encryption Key fields to be the same as in your primary Sisense web server.

  3. Click Save to save the changes.

  4. Click Yes to restart the relevant services.

Defining the Application database's Read Preferences

Read preferences specify where each member of a replica set should direct its read operations.

If you have implemented high availability with replica sets to support your Sisense implementation, and your application databases are geographically spread out, you may want to define different read preferences to different nodes within the set.

Defining your own read preferences is useful in cases such as preventing a secondary node which has fallen behind a primary node from giving old data.

You can define the following read preferences for the application database:

primary: This preference routes all reads to the replica set primary. If the primary is unavailable for some reason a read operation would produce an error or exception. This preference is useful to never return stale data.

(Sisense Default) primaryPrefered: Reads are sent to the primary, but if it is unavailable, the reads come from secondary members instead.

secondary: Reads are only allowed on secondary members of the replica set. When no secondaries are available, an error or exception is returned.

secondaryPrefered: Reads are routed to a secondary, but if no secondary is available read operations are sent to the primary.

Nearest: Reads are performed on the nearest available node, disregarding if it is a primary or secondary member. The nearest node is determined by sending pings to all members and measuring the response time.
For additional information about read preferences, click here.

To modify the application database's read preferences:

  1. Open the Configuration Manager in your browser located at localhost:3030.

  2. Under Application Database, modify the value of the Connection String and change the readPreference value (primaryPreferred as default) to be your preference type.

    Example:
    "mongodb://AppUser:we6jBUYGoOrh0K6l+XpTmA==@XX.XX.XX.XX:27018,XX.XX.XX.XX:27018,XX.XX.XX.XX:27018/?replicaSet= &readPreference="