Build Log Walkthrough

Configurations Involved

Below are the configurations that are used throughout the build process to manage connectivity, system resource utilization, and other functionality.

Config Section Type Parameter Stated As Description When to Adjust Type (HOT/COLD)
Note: Hot means that the configuration is applied dynamically, without requiring application components to be restarted. Cold means at least one application component restart is required.
Build Connectivity Build Service to MonetDB (BLD Pod) SimpleCommandTimeout.value Milliseconds Determines how long the build service will wait for a result set to be returned from a query executed against MonetDB. If the time elapses, the query will be considered to have failed.
This timeout will be used on queries that are classified as "Simple Commands".
If the build process fails to perform a query against MonetDB, consider increasing the time set for this parameter. Cold
Build Connectivity Build Service to MonetDB (BLD Pod) ComplexCommandTimeout.value Milliseconds Determines how long the build service will wait for a results set to be returned from a query executed against MonetDB. If the time elapses, the query will be considered to have failed.
This timeout will be used on queries that are classified as "Complex Commands".
If the build process fails to perform a query against MonetDB, consider increasing the time set for this parameter. Cold
Build Connectivity Build Service to MonetDB (BLD Pod) BaseTableTimeout.value Milliseconds Cold
Build Connectivity Build Service to MonetDB (BLD Pod) CustomerColumnTimeout.value Milliseconds Cold
Build Connectivity Build Service to MonetDB (BLD Pod) CustomerTableTimeout.value Milliseconds Cold
Base Connectivity Build Service to MonetDB (BLD Pod) queryOnFailureRetries.value Number The number of retries to attempt to query MonetDB in the event of a query failure. Hot
Base Connectivity Build Service to MonetDB (BLD Pod) queryOnFailureInterval.value Seconds The time delay between the query attempts. Hot
Build Concurrency BasetTableMaxThreads.value Number
Build Concurrency CustomTableMaxthreads.value Number
Build Concurrency AllowConcurrentBaseAndCustomElements.value

Column Data Type List

Type Code
Big Int 0
Binary 1
Bit 2
Char 3
DateTime 4
Decimal 5
Float 6
Image 7
Int 8
Money 9
NChar 10
NText 11
NVarChar 12
Real 13
UniqueIdentifier 14
SmallDateTime 15
SmallInt 16
SmallMoney 17
Text 18
Timestamp 19
TinyInt 20
VarBinary 21
VarChar 22
Variant 23
XML 25
UDT 29
Structured 30
Date 31
Time 32
DateTime2 33
DateTimeOffset 34
None -1

MongoDB

be >> Collections >> ecmModel

# Fields Description Example
1 _id CubeID - The ID assigned to the cube
2 _title/value <Cube Name> - The name of the cube
3 schemaID <SchemaID> - The ID assigned to the cube in MonetDB
4 server
5 lastBuildTime
6 lastUpdated
7 datasets Contains the data structure
>
8 relations

be >> Collections >> buildLogData

# Fields Description Example
1 _id CubeID - The ID assigned to the cube
2 _title/value <Cube Name> - The name of the cube
3 schemaID <SchemaID> - The ID assigned to the cube in MonetDB
4 server
5 lastBuildTime
6 lastUpdated

be >> Collections >> physicalSchema

This collection contains the physical schema of all the cubes. This can be used in order to understand the MonetDB schema, including the tables, fields and their definitions.

# Fields Description Example
1 _id CubeID e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
2 name CubeTitle aTestMySQL
3 tables/Table ID TableID d5279f72-f5ed-43ef-b18a-5c3d543f58a0
> indexed aEmployees
>> columns
oid Column ID 7a807ddb-91fb-452a-a225-6c9a1153fdaf
name Colum name aEmployeeID
type Column Type (INT, VARCHAR, etc.) INT
generated ??? True/False

External Factors Affecting the Build

# Process Performed By Description Log Entry Investigating
1 Disk Space Monitor Management Service This process is run by the Management Service and continuously monitors disk space availability.

Disk space utilization is calculated by running a shell script to check both the Inodes and Iused utilization %. The folder checked is the "/tmp" directory.

The threshold is set in "Service Configuration >> Management" via the "MaxDiskUsageThreshold.value" parameter. The value is set in %, but as a real number. Example: 0.95 is the default value, designating that up to 95% of the disk can be utilized before considering the storage disk space as insufficient to continue operating (e.g., builds get canceled).
BE#082210 Inodes usage exceeds threshold. inodes use <Utilized Space for Inodes>, threshold:<MaxDiskUsageThreshold.value>ORBE#251800 Disk space usage exceeds threshold. space use <Utilized Space for Inodes>, threshold: <MaxDiskUsageThreshold.value> To check to see what in fact is currently utilized, do the following in an ssh session:

To check the disk space usage that will be compared against the threshold:

df <path of the storage>

For example:

df /opt/sisense

The returned value will show the utilization. In the example below, it is 35%.

Filesystem 1K-blocks Used Available Use%
Mounted on/dev/mapper/sisense_vg-root 146595780 48305752 90773664 35% /


The returned value of 35% is converted to a real number, in this case 0.35, and cannot be bigger than the threshold number set.

To check the Iused utilization that will be compared against the threshold:

df -i <path of the storage>

For example:

df -i /opt/sisense

The returned value shows the utilization. In the example below, it is 15%.

Filesystem Inodes IUsed IFree IUse% Mounted on/dev/mapper/sisense_vg-root 9379840 1361677 8018163 15% /

The returned value of 15% is converted to a real number, in this case 0.15, and cannot be bigger than the threshold number set.
2 Safe Mode EC BLD Pod

Build Logs Walkthrough

The following section contains the logs entries that reflect the build process progression in detail. It reflects a successful simple build process, outlining the various phases of the build process and their successful completion.

A build request receives a unique Request ID that can be used to follow through the logs and trace the progression of the build process.

The values used for this example are:

# Description Example
1 RequestID The ID assigned to the request that is processed by the build/management pods. X-request:df0e52b1-8090-4735-bd1f-cfaca2b25ea8
2 CubeID The ID assigned to the cube (also called source ID throughout the logs). e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
3 CubeTitle The name of the cube. TestMySQL
4 SchemaID The ID assigned to the cube in MonetDB. 0545564c-1341-3920-96eb-e19021de1b02
5 UserID The user that submits the request to build.
6 UserRole Super

BUILD SERVICE

Example of an Elastic Cube build (stored in MonetDB):

configuration.

Class

Message #

Description

Successful Log entry

Example

Possible Error Messages

Exception Handling/

Potential Root Cause

Workaround/

Solution

PHASE: INITIALIZATION
LOG: build.log
The following describe the initial processing of the build request, which includes establishing the cube footprint on the storage and other initialization steps.
RequestFilterBE#683109A request is received to run a build on the given {TBC} from the API Gateway. Receive a request which is processed asynchronously. The build starts with a REST call to the build API, followed by verifying that the data source exists, and that the user is allowed to perform the build.BE#683109 Start handling request - url=<API URL request initiating the request>, userId=<User ID>, userRole=<UserRole>The user <UserID> = MongoDB >> prismWebDB >> Collections >> Users >> _id<User Role> = MongoDB >> prismWebDB >> Collections >> Users >> _roleId assign to the given user and the name of the role is in MongoDB >> prismConfig >> Collections >> complied_roles >> nameBE#683109 Start handling request - url=http://build:8086/api//v2/internal/builds/, userId=6124c5dc2b42f5001ad4541a, userRole=super
BuildServiceBE#398437Calls the identity pod to verify that the logged in user/token is authenticated and authorized to run the build. The user needs permission on the cube to allow "EditModel ''; otherwise the build will fail.

However, if a permissioned user defined a scheduled build (cronjob), the job will run successfully as an unauthenticated user and, therefore, will not fail due to the permission issue.
If cube is being built for the first time, the general user permission to build a cube is going to be checked:BE#895641 enforcePermissionByDataSource() - dataSourceId is null (possible reason: cube was not built yet)

OR

If existing cube, than user needs to have permission to the given cube:

BE#398437 Validate permission. Data Source Title: <CubeTitle>
BE#398437 Validate permission. Data Source Title: TestMySQLBE#471044 User is unauthorized for this operationBE#002491 User is unauthorized for this operationBE#755909 Data source id not found. <CubeTitle>BE#584681 Validate permission unexpected status: <status description>Exceptions:BE#274082 User is unauthorized for this operationThe user is not authorized to perform this operation.Assign the proper permission.
BuildServiceBE#916742If a new cube is being built for the first time, no Cube ID has been assigned yet. In this step, an ID will be generated.BE#916742 Data source not found, assuming it is new: <CubeID>BE#916742 Data source not found, assuming it is new: b203e437-723f-4fd3-9879-ce9c5fe6039dBE#196278 failed to get sessionId for dataSourceId: <CubeID>BE#538665 failed to get sessionId for dataSourceId: <CubeID><, Error: <Error # and description>
BuildServiceBE#895641If there are no permissions associated with the user requesting the build, the permission required is derived from the CubeID.BE#895641 enforcePermissionByDataSource() - dataSourceId is null (possible reason: cube was not built yet) BE#895641 enforcePermissionByDataSource() - dataSourceId is null (possible reason: cube was not built yet) BE#486159 DatasourceId not found, Failed to enforce permissions, return empty list.
RequestFilterBE#587104Indicates that the process of receiving the request is complete.

Note that this log entry can appear further down the log, as the request is processed in parallel with the build process being initiated.
BE#587104 Finished handling request - url=<API URL request initiating the request>, BE#587104 Finished handling request - http://10.50.65.129:30845/api/v1/elasticubes/servers/next/LocalHost/EC%20Test/isBuildingn/a
BuildServiceBE#392556The ElastiCube data model (schema) is retrieved from MongoDB and the information is processed.

Started handling a build process, the OID is the ID of the cube being built.

MonetDB Schema ID is yet to be assigned at this point.
BE#392556 doBuild started Oid: <CubeID>, Schema Id: <SchemaID>MongoDB collection path:CubeID = be >> Collections >> ecmModel >> _idSchemaID = be >> Collections >> ecmModel >> schemaIdORBE#301302 Trigger Build called. Data Source Id: <CubeID>BE#392556 doBuild started Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, Schema Id: nullBE#297509 Failed to run build. Data Source Title: <Cube Name>, ECM Model Oid: <Cube ID>, error: <cause of failure>, <error message>Potential causes of failure:
- Disconnection from MongoDB
- An invalid or non-existent model
BuildServiceBE#040383Initiate the build process for the cube. This step sends a request to the management pod for the status of the platform and readiness to initiate the build.BE#040383 StartBuildingCube DataSourceId <CubeID>BE#040383 StartBuildingCube DataSourceId e51d6ac3-819c-48f7-b1c4-7003f8cdff7eBE#786938 Start BuildingCube failed DataSourceId: <CubeID>The following error message will contain the reason for the system not being ready to initiate the build:BE#983868 <status description> ,<BuildStatus>, <CubeID>BE#130359 service not found, serviceName: Management ServiceThe following are management pod reasons that identify why the build request cannot be initiated:
  1. The cube is not assigned with a data group
  2. The system is in a down or restore mode
  3. The system is in a backup mode
  4. System disk resources are insufficient to initiate
  5. The cube is busy
This can occur if there is a problem with ZooKeeper and/or the management service is inaccessible.
Check the platform to determine whether any of the conditions specified are the cause, and rectify the problem.
BuildServiceBE#766306Start creation of the build plan for the cube.BE#766306 Start Planning Build. Oid: <CubeID>, Schema Id: <SchemaID>BE#766306 Start Planning Build. Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, Schema Id: nullBE#375008 Failed to create models for build. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#136035 Failed to run build. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>Cube Name = CubeID = MongoDB >> be >> Collections >> ecmModel >> title >> _valueCombined Error:Failed to set data source model for build
  1. Unable to connect and get information from MongoDB
  2. CubeID does not exist in MongoDB
EcmModelBE#974336 Generate a new SchemaID that will be assigned to the schema of the cube. In a case where a new cube is being built:BE#974336 Calculated schema id <SchemaID>Or in a case even if ID exists, for a new ID:BE#000735 Calculated a new schema id <SchemaID> in the case a change in the model is recognized. Old schema id <OldSchemaID> ignoredOr no schema id is being calculated:BE#105635 Ignoring from calculating a new schema id BE#974336 Calculated schema id '0545564c-1341-3920-96eb-e19021de1b02'
BuildServiceBE#869430Create models for the cube, based on the definitions for the cube.BE#869430 Creating Models. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#869430 Creating Models. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7eBE#063661 exception was received while saving the cube models, Failed to create modelsMongoDB is not accessible to save the cube metadata model.
LogicalServiceImplBE#656221Create the logical DB schema to be used for the creation of the cube data schema.BE#656221 Creating Logical Schema. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#656221 Creating Logical Schema. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7eBE#345777 EcmModel.schemaId is nullException:"Build general error"This will occur when the Cube ID does not exist.
MonetPhysicalServiceImplBE#617290 Create the physical schema from the logical schema that will be used for the creation of the schema (e.g., by MonetDB when creating the cube data schema).BE#617290 Creating Physical Schema. Data Source Title: <Cube Name>, ECM Model Oid: <Cube ID>BE#617290 Creating Physical Schema. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
DbSchemaLogicalToMonetBE#522685The index size setting for the data group, which can be SHORT or LONG.

This determines how the INT is set in the data.

It assigns the index size that is used to determine how integer values are stored in MonetDB.
BE#522685 ElastiCube: <Cube Name> datagroup: <Data Group> index size: <Data Group Index Set to SHORT or LONG>BE#522685 ElastiCube: EC Test datagroup: Default index size: SHORTBE#074977 Couldn't get datagroup for ElastiCube <CubeTitle>. Assuming index size is SHORT"If the cube is not assigned to a data group, the default group is used for the cube.
DbSchemaLogicalToMonetBE#610733Converts the data structure outlined in the cube metadata from MongoDB to the destination data structure - in this case, MonetDB.

This includes defining the dimension tables that need to be populated during the data import into MonetDB.

Each table and its columns will be defined.
BE#610733 Scanning for tables. Data Source Title: <Schema Name>BE#610733 Scanning for tables. Data Source Title: EC Test
DbSchemaLogicalToMonetBE#028991 Verify that the relationship is valid in terms of column types. That is, a relationship can be set between columns that have the same data type (e.g., int cannot have a relationship with string). Also, define relationship dimension tables if needed.BE#028991 Scanning for relationships. Data Source Title: <Cube Name>BE#028991 Scanning for relationships. Data Source Title: EC TestBE#544724 Columns in relationship have different types: <ColumnType> != <Relationship Column Type>Exception:BE#403079 Columns in relationship have different types: <ColumnType> != <Relationship Column Type>Relationships are set between columns that have different data types (e.g., INT and CHAR columns).

The build will fail.
Remove relationships if they are not needed, or convert the types to be compatible.
DbSchemaLogicalToMonetBE#647721For the cube, create dimension tables that will be created in the destination database, in this caseMonetDB.BE#647721 Scanning for dimension tables. Data Source Title: <CubeTitle>BE#647721 Scanning for dimension tables. Data Source Title: EC Test
LogicalServiceImplBE#920165Creating a logical ETLT (extract, transform, load, transform) plan for the build.BE#920165 Creating Logical Etlt Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#920165 Creating Logical Etlt. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7eBE#975363 Base dataset must contain connection. dataset name: <data set name>BE#535740 Base dataset must contain connection. dataset name: <data set name>
MonetPhysicalServiceImplBE#820116 Creating a physical ETLT (extract, transform, load, transform) plan for the build.BE#820116 Creating Physical Etlt. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#820116 Creating Physical Etlt. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
EtltModelLogicalToMonetBE#304408Send a request to the translator to generate an SQl statement for the destination database (e.g., MonetDB) which will be used to generate the custom elements.

Note: If there is an issue here, setting the log level to debug will add a DEBUG entry showing the generated SQL statement, which can be analyzed and debugged.
BE#304408 Requesting for Custom Elements SQL Expressions from TranslatorBE#304408 Requesting for Custom Elements SQL Expressions from TranslatorBE#813129 Failed to get custom elements SQL expressions from translator, error: Failed to get Custom Elements SQL Expressions
TranslationRestProviderBE#813055Send a request to generate the dependencies.BE#813055 Start Generating Custom Elements SQL. translationServiceType: TranslationService, Payload: <Cube Name>BE#813055 Start Generating Custom Elements SQL. translationServiceType: TranslationService, Payload: TestMySQLBE#694874 Failed to generate custom elements SQL, error: <error returned from translator>BE#348900 Failed to generate custom elements' SQLs, schemaEtlPayload:{}", schemaEtlPayload, <translator error generated>BE#130359 service not found, serviceName: Translation ServiceException:BE#096400 Failed to generate custom elements' SQLsBE#654510 Failed to generate custom elements' SQLs
Translator - ETLT - Generate Custom Elements - START (translator.log)
TranslationService.RestServer This call was made by the build service to the translator in order to generate an SQL statement that will be used against the target database (e..g, MonetDB) to generate the custom data setup in the custom elements setup of the model.

The x-request-id value can be used to match against the build service and management pod specific build request.

The build service is calling the translator REST API for "generateCustomElementsSql".

The build request ID is indicated.
[X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"path":"/api/translation/generateCustomElementsSqls","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\" <-xrequest ID>\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-12T12:45:21.989Z","Log_Thread":"Worker(41)","appTypeName":null} ] [X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"path":"/api/translation/generateCustomElementsSqls","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"a5d6efa8-a839-41e9-8822-3dfcc9792182\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-12T12:45:21.989Z","Log_Thread":"Worker(41)","appTypeName":null}
TranslationService.RestServer.Controllers Pass the payload (which is the cube model setup) to generate the conversion plan for the custom element.

This example is for MonetDB as the destination and, therefore, the utilization of the MonetDB services.
[X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"schemaEtlPayload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: <MondetDBSchema ID>\n SchemaName: <Cube Title/Logical Schema name>\n SchemaPhysicalName: <phyiscal database name>\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering GenerateCustomElementsSqls","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-12T12:45:22.020Z","Log_Thread":"Worker(41)","appTypeName":null} ] [X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"schemaEtlPayload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering GenerateCustomElementsSqls","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-12T12:45:22.020Z","Log_Thread":"Worker(41)","appTypeName":null} ]
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema Perform the ETLT translation, creating the "space" in which the process will take place.[X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"SchemaName":<Logical database name>","SchemaId":"<MondetDBSchema ID>","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-12T12:45:22.022Z","Log_Thread":"Worker(41)","appTypeName":null} ][X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"SchemaName":<Logical database name>","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-12T12:45:22.022Z","Log_Thread":"Worker(41)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema Close the "space" used for the translation.[X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"SchemaName":" <Logical schema name>","SchemaId":"<SchemaID>","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-12T12:45:22.028Z","Log_Thread":"Worker(41)","appTypeName":null} ] [X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-12T12:45:22.028Z","Log_Thread":"Worker(41)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema Close the "space" used for the translation.[[X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"SchemaName":" <Logical schema name>","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-12T12:45:22.028Z","Log_Thread":"Worker(41)","appTypeName":null} [X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-12T12:45:22.028Z","Log_Thread":"Worker(41)","appTypeName":null}
TranslationService.RestServer Indicates that the request has been processed successfully. [X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"path":"/api/translation/generateCustomElementsSqls","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-12T12:45:22.071Z","Log_Thread":"Worker(41)","appTypeName":null} ] [X-Req:37a553fb-5ce1-4e81-a6e3-007759f603cc]:[{"path":"/api/translation/generateCustomElementsSqls","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-12T12:45:22.071Z","Log_Thread":"Worker(41)","appTypeName":null} ]
Translator - ETLT - Generate Custom Elements - END (translator.log)
EtltModelLogicalToMonetBE#048055A request to the translator to generate dependencies for the build plan. The dependencies are used to know when and how to process the custom elements.

Note: Setting the log level to debug will provide you the details about the dependencies that are in place, in the message [DEBUG] "BE#220560 Finish Generating Dependencies. Response:"

"For example:

One Custom Table "CustTableTest" with a dependency on one base table called "Locations", but has no other dependency on Custom Table, Custom Code, or Custom Column.(customTables=[CustomTableDependencies(logicalTableName=CustTableTest, dependencies=CustomElementDependencies(baseTables=[Locations], customTables=[], customCodeTables=[], customColumns=[]))], One Custom Column called "LocationName" inside table "Employees" which has a dependency on base Table "Locations", but has no other dependency on Custom Table, Custom Code, or Custom Column.customColumns=[CustomColumnDependencies(logicalTableName=Employees, logicalColumnName=LocationName, dependencies=CustomElementDependencies(baseTables=[Locations, Employees], customTables=[], customCodeTables=[], customColumns=[]))])]
BE#048055 Requesting for Dependencies from TranslatorBE#048055 Requesting for Dependencies from TranslatorBE#068969 Failed to get dependencies from translator, error: <translator error generated>
TranslationRestProviderBE#795211Received the response from the translator.BE#795211 Start Generating Dependencies. translationServiceType: TranslationService, Payload: <CubeTitle>BE#795211 Start Generating Dependencies. translationServiceType: TranslationService, Payload: TestMySQLBE#538964 Failed to generateDependencies, schemaEtlPayload: <CubeTitle>Exceptions:BE#626253 Failed to generate dependencies.
Translator - ETLT - Generate Dependencies - START (translator.log)
TranslationService.RestServern/aCall to the translator API to generate a dependency map for the cube build.

The build request ID is indicated.
[X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"<x-request ID>\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.647Z","Log_Thread":"Worker(44)","appTypeName":null} [X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.647Z","Log_Thread":"Worker(44)","appTypeName":null}
TranslationService.RestServer.Controllersn/aPass the payload to generate the dependencies for the model build.[X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: <schema ID>\n SchemaName: <logical db name>\n SchemaPhysicalName: <physical db name>\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:51.651Z","Log_Thread":"Worker(44)","appTypeName":null} [X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:51.651Z","Log_Thread":"Worker(44)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchemn/aPerform the physical schema translation, creating the "space" in which the process will take place.[X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"SchemaName":"<;Logical schema name>","SchemaId":"<Schema ID>","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.652Z","Log_Thread":"Worker(44)","appTypeName":null} [X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.652Z","Log_Thread":"Worker(44)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbScheman/aClose the "space" used for the translation.[X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"SchemaName":"<Logical schema name>","SchemaId":"<Schema ID>","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.652Z","Log_Thread":"Worker(44)","appTypeName":null} [X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.652Z","Log_Thread":"Worker(44)","appTypeName":null}
TranslationService.RestServern/aThe dependency map creation is complete. [X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.679Z","Log_Thread":"Worker(44)","appTypeName":null} [X-Req:fedd8824-a07a-4b1f-a9d3-de64fc7dc87f]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.679Z","Log_Thread":"Worker(44)","appTypeName":null}
TranslationService.RestServern/aA call to the translator API to generate a dependency map for the cube build. The build request ID is indicated. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"<x-request ID>\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.954Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.954Z","Log_Thread":"Worker(58)","appTypeName":null}
TranslationService.RestServer.Controllers Pass on the payload (the cube's metadata received by the gateway via the build service) to generate the dependencies for the model build. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbScheman/aPerform the ETLT translation, creating the "space" in which the process will run. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbScheman/aClose the "space" used for the translation. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null}
TranslationService.RestServern/aExecution of the dependency map creation is complete.[translation-56dddb98d5-6lbz2]:[6]:[58] [] [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.957Z","Log_Thread":"Worker(58)","appTypeName":null} [translation-56dddb98d5-6lbz2]:[6]:[58] [] [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.957Z","Log_Thread":"Worker(58)","appTypeName":null}
Translator - ETLT - Generate Dependencies - END (translator.log)
BuildServiceBE#578789Store the new model into MongoDB.

If the previous build failed on the cube, meaning that the build did not finish successfully, there might be a partially built model stored and, therefore, it will be deleted from MongoDB for the cube.

A notification message is sent to the storage services to save the models metadata of the cube. Each record will first be deleted and then a new record will be created in MongoDB.

The list of metadata types transactions performed will include:
  1. DataSourceInfo
  2. EcmModel
  3. PhysicalName2DataSourceId
  4. LogicalDbSchema
  5. DataSchema
  6. PhysicalLiveDbSchema
  7. PhysicalMonetDbSchema
  8. PhysicalEtltModel
  9. PhysicalName2DataSourceId
Also, the following files will be saved in the farm:
  • ElastiCube.json
  • ElastiCubePhysicalDbSchema.json
  • ElastiCubeInfo.json
BE#578789 Storing Models. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#578789 Storing Models. Data Source Title: TestMySQL, ECM model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7eAn exception will be returned to the build from the MongoDB client. Furthermore, the following failures when transacting files could occur:

File 1:BE#449244 Failed to write ElastiCube.json
BE#785588 Failed to store ECM model file

File 2:BE#456740 Failed to write ElastiCubePhysicalDbSchema.json
BE#156327Failed to store Physical DB Schema file

File 3:BE#108980 Failed to write ElastiCubeInfo.json
BE#143359 Failed to store cube info file
A potential connection failure could occur as data is being stored to the storage. Another potential issue could be with the size of the JSON file that has a limitation of 4MB.
LogicalSchemaConverterBE#859311Create a logical schema for the cube. The logical schema is created based on the cube model defined.BE#859311 Creating Logical Schema. Data Source Title: <CubeTitle, ECM Model Oid: <CubeID>BE#859311 Creating Logical Schema. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
CubeModelsServiceBE#295973Shows the index size setting for the data group, which can be SHORT or LONG.
Assigns the index size that will be used to determine how integer values will be stored in MonetDB.
BE#295973 ElastiCube: <CubeTitle> datagroup: Default index size: <Index size defined in data group>BE#295973 ElastiCube: TestMySQL datagroup: Default index size: SHORTBE#557625 Couldn't get datagroup for ElastiCube {}. Assuming index size is SHORT", <CubeTitle>
PhysicalSchemaConverterBE#404542Create the physical schema for the model.BE#404542 Creating Physical Schema. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#404542 Creating Physical Schema. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
TranslatorServiceImplBE#498652Generate dependencies for the plan to create the cube, meaning custom elements are dependent on what needs to be build first.BE#498652 Requesting for dependencies from TranslatorBE#498652 Requesting for dependencies from TranslatorBE#276087 Failed to get dependencies from Translator for cube + "<CubeName>" , error: <translator error returned>
TranslationRestProviderBE#795211Generate the dependencies.BE#795211 Start Generating Dependencies. translationServiceType: TranslationService, Payload: <CubeTitle>BE#795211 Start Generating Dependencies. translationServiceType: TranslationService, Payload: TestMySQLBE#130359 service not found, serviceName: Translation ServiceIn case translation service is not accessible or a problem with ZooKeeper.
Translator - Convert to Physical - Generate Dependencies - START (translator.log)
TranslationService.RestServern/aCall the translator API to generate a dependency map for the cube build. The build request ID is indicated. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"<x-request ID>\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.954Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.954Z","Log_Thread":"Worker(58)","appTypeName":null}
TranslationService.RestServer.Controllersn/aPass the payload to generate the dependencies for the model build.[X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbScheman/aPerform the physical conversion translation, creating the "space" in which the process will run.[X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"<Logical schema name>","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbScheman/aClose the "space" used for the translation. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:51.955Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServern/aExecution of the dependency map creation is complete. [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.957Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:b79e7c3d-8375-47a0-8fee-f5ae32ef40a2]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:51.957Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Translator - Convert to Physical - Generate Dependencies - END(translator.log)
TranslatorServiceImplBE#602533Request the custom elements defined for the model from the translator.BE#602533 Requesting for Custom Elements SQL Expressions from TranslatorBE#602533 Requesting for Custom Elements SQL Expressions from TranslatorBE#115834 Failed to get custom elements SQL expressions from Translator for cube "<CubeTitle>", error: <translator error returned>
TranslationRestProviderBE#81305Generate custom elements.BE#813055 Start Generating Custom Elements SQL. translationServiceType: TranslationService, Payload: <CubeTitle>BE#813055 Start Generating Custom Elements SQL. translationServiceType: TranslationService, Payload: TestMySQLBE#348900 Failed to generate custom elements' SQLs, <schema details>, <translator error returned>
Translator - Convert to Physical - Generate Custom Elements - START (translator.log)
TranslationService.RestServern/aThis call was made by the build service to the translator, to generate an SQL statement that will be used against the target database (e..g, MonetDB) to generate the custom data setup in the model's custom elements setup.

The x-request-id value can be used to match against the build service and management pod specific build request.

The build service calls the translator REST API for "generateCustomElementsSql". The build request ID is indicated.
[X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"path":"/api/translation/generateCustomElementsSqls","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:52.364Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"path":"/api/translation/generateCustomElementsSqls","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:52.364Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServer.Controllers Pass the payload that is the cube model setup, to generate the conversion plan for the custom element.

This example is for MonetDB as the destination and, therefore, the utilization of the MonetDB services.
[X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"schemaEtlPayload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering GenerateCustomElementsSqls","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:52.365Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"schemaEtlPayload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering GenerateCustomElementsSqls","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:20:52.365Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema Perform the ETLT translation, creating the "space" in which the process will run. [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:52.365Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:52.365Z","Log_Thread":"Worker(58)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema Close the "space" used for the translation. [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:52.366Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:20:52.366Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServer Indicates that the request has been processed successfully. [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"path":"/api/translation/generateCustomElementsSqls","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:52.369Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:3c6bad24-8213-44e1-9d7e-3e2561422033]:[{"path":"/api/translation/generateCustomElementsSqls","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:20:52.369Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServer Call the API to execute the translation. [X-Req:1762ebee-8af7-437d-92d7-fc3cfde7a23c]:[{"path":"/paths","headers":"{\"Connection\":[\"close\"],\"Host\":[\"translation.sisense:8070\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:13.595Z","Log_Thread":"Worker(44)","appTypeName":null} ] [X-Req:1762ebee-8af7-437d-92d7-fc3cfde7a23c]:[{"path":"/paths","headers":"{\"Connection\":[\"close\"],\"Host\":[\"translation.sisense:8070\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:13.595Z","Log_Thread":"Worker(44)","appTypeName":null} ]
TranslationService.RestServer Indicates that the request has been processed successfully. [X-Req:1762ebee-8af7-437d-92d7-fc3cfde7a23c]:[{"path":"/paths","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:13.597Z","Log_Thread":"Worker(44)","appTypeName":null} ] [X-Req:1762ebee-8af7-437d-92d7-fc3cfde7a23c]:[{"path":"/paths","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:13.597Z","Log_Thread":"Worker(44)","appTypeName":null} ]
Translator - Convert to Physical - Generate Custom Elements - END (translator.log)
n/an/aAn update of the DataSchema is sent to MongoDB. Note: This action is only shown in DEBUG level. A failure to access MongoDB will be raised if there is a problem to acess it.
BuildServiceBE#920558Initiate the build request (state machine).BE#920558 continueAsyncBuild Build called. Data Source Id: <Cube ID>BE#920558 continueAsyncBuild Build called. Data Source Id: e57083521d6ac3-819c-48f7-b1c4-7003f8cdff7e
BuildTrackerBE#594038Queue the cube's build. BE#594038 Trying to add data source id <Cube ID> to running initiate setBE#594038 Trying to add data source id e51d6ac3-819c-48f7-b1c4-7003f8cdff7e to running initiate setBE#708352 Build already running for cube <CubeID>BE#906122 Build not yet listed - exit quietlyBE#128238 Build {} already removed - exit quietlyBE#669279 Build {} was cancelled - exit quietly The build process will be force failed if there is already another build running for this cube.These are cancellation reasons that would be outlined.
BuildFAiledACtionBE#357871This message will appear when the build is being canceled due to a failure or build request duplication. BE#357871 Cleanup and log build failure: <BuildID>BE#357871 Cleanup and log build failure: e51d6ac3-819c-48f7-b1c4-7003f8cdff7en/a
BuildTrackerBE#060207 Confirm that a request has been added to build the cube.BE#060207 Added data source id <CubeID> to running initiate setBE#060207 Added data source id e51d6ac3-819c-48f7-b1c4-7003f8cdff7e to running initiate set
BuildServiceBE#621989Queueing the state machine. The build ID is associated with the cube. BE#621989 Enqueueing the state machine prepare: <BuildID>BE#621989 Enqueueing the state machine prepare: 27364933-0325-41c3-8bbe-e28c79f19244
BuildServiceBE#135750 Planning for the build is complete.BE#135750 Finished Planning Build. Oid: <Cube ID>, Schema Id: <Schema ID>BE#135750 Finished Planning Build. Oid: b203e437-723f-4fd3-9879-ce9c5fe6039d, Schema Id: 958fb6ee-44e3-354e-9dad-3458f9a86ea8
BuildServiceBE#712274Initiate the build state machine.BE#712274 Preparing build state machine: <BuildID>BE#712274 Preparing build state machine: 27364933-0325-41c3-8bbe-e28c79f19244
PrepareBuildActionBE#660892This step retrieves models from the cube in MongoDB prior to starting the build, even if the models were created in the previous steps (by builds running in parallel).BE#660892 Retrieve Models. Data Source Id: <CubeID>BE#660892 RetrieveModels. Data Source Id: e51d6ac3-819c-48f7-b1c4-7003f8cdff7eBE#170093 Failed to retrieve data source info for ID <CubeID>BE#306199 Failed to retrieve ecm model for data source <CubeID> Failed to get ECM model <Cube Title>Failure could be the result of no access to storage for the retrieve.
PrepareBuildActionBE#415124Indicates some build details, including the build type (e.g., full) and the build destination (e.g., MonetDB).BE#415124 Data source id: <CubeID>, build id: <BuildID>BE#415124 Data source id: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, build id: BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244), build type: full, build originator: NextECM, build destination: MonetDb
PrepareBuildActionBE#261800This log provides the setting for the General Build (meaning that it applies to all cubes).

Settings are in the configuration manager >> service configuration >> build section (available via 5 clicks on the Sisense logo):
  • Base Table Max Threads (integer) - The number of threads configured to run base table data import.
  • CustomTableMaxThreads.value (integer) - The number of threads that will process custom table creation in parallel.
  • AllowConcurrentBaseAndCustomElements.value (true/false) - Enables or disables the ability to run base table import and custom table creation in parallel. If set to true, the thread allocation setting will be used.
BE#261800 Setting build throttling: BuildThrottling(baseTableThread=<Base Table Max Threads>, customTableColumnThreads=<CustomTableMaxThreads.value>, parallelBaseAndCustom=<AllowConcurrentBaseAndCustomElements.value>BE#261800 Setting build throttling: BuildThrottling(baseTableThread=4, customTableColumnThreads=1, parallelBaseAndCustom=false)
n/an/aThis log entry contains the type of build (e.g., full, accumulative, change schema) being performed for the cube. {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"StartBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:53:58Z","Log_Thread":"builds-prepare-queue-0","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"<Build Type>","cubeID":"<Cube ID?","cubeName":"<Cube Title>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"<Build ID>","buildStep":"buildCubeStart"} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"StartBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:53:58Z","Log_Thread":"builds-prepare-queue-0","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","buildStep":"buildCubeStart"}
BaseWorkPlanBuilderBE#434609Create a sanity work plan. The sanity work plan is geared towards assessing whether the selected build type can be executed on the cube.BE#434609 Creating Sanity Work Plan. Id: <Build D>BE#434609 Creating Sanity Work Plan. Id: 27364933-0325-41c3-8bbe-e28c79f19244
BaseFullWorkPlanBuilderOr BaseAccumulativeWorkPlanBuilderOrBaseSchemaChangesWorkPlanBuilderBE#156994BE#760783BE#172962Execute a sanity work plan.

The following sanity test is applied to each build type:

Full Build:
  • Connection parameters exist to base tables
Accumulative:
  • A cube for the build exists
  • The build destination exists
  • Base tables exist
  • Models are valid
  • Valid connection parameters exist to base tables
Schema Change:
  • Cube for the build already exists
  • Build destination exists
  • Model(s) valid and have been changed in order to trigger a schema change build.
  • Valid connections parameters exists to base tables
BE#156994 Added [Initialize] tasks. Duration: <time elapsed to generate plan>BE#156994 Added [Initialize] tasks. Duration: PT0.012522S(Warning only)BE#422679 Build type for cube <CubeTitle> will be changed from: <existing build type> to <new build type>BE#999123 Failed to read ElastiCube.json.File not found error: <location in farm>/ElastiCube.json (No such file or directory); Failed to store ECM model file.BE#311890 - Failed to complete sanity check for build processBE#094445 Sanity check failed to cube <Cube ID> because of <exception>BE#064503 - "Exception in sanity check for build processBE#462345 <Exception>If it was determined that an accumulative build could not run on the cube, the build will run as a full build.

For example, if a cube has been imported and, for some reason, a cumulative build is initiated, it will be converted to a full build.

This indicates that the file that contains the existing model metadata for some reason is not accessible or does not exist. In order to resolve this, a full build could be run.

A sanity test fails to fully execute, resulting in a build failure. The sanity check failure will appear here.
TasksCoordinatorImplBE#000801Execute the sanity test work plan.BE#000801 Running coordinate group. Id: <BuildID>BE#000801 Running coordinate group. Id: 27364933-0325-41c3-8bbe-e28c79f19244
TaskGroupBE#289057The log displays the build process that is about to be executed using a DOT (Graphviz) formatted diagram. Note: You can use the https://dreampuf.github.io/GraphvizOnline site to convert the DOT format below to a visual representation of the process.BE#289057 ------- Dumping Task Group ------------------ DOT Start -----------strict digraph G {<the work plan in DOT format>}------- DOT END -----------]------- DOT Start -----------strict digraph G {1 [ label="AFTER_BUILD_INIT" fillcolor="#E6E6E6" shape="octagon" style="filled" ];2 [ label="Initializing" fillcolor="azure" style="filled" ];3 [ label="AFTER_SANITY_CHECK" fillcolor="#E6E6E6" shape="octagon" style="filled" ];4 [ label="BEFORE_SANITY_CHECK" fillcolor="#E6E6E6" shape="octagon" style="filled" ];5 [ label="BEFORE_BUILD_INIT" fillcolor="#E6E6E6" shape="octagon" style="filled" ];6 [ label="BuildFlow" fillcolor="azure" style="filled" ];1 -> 2;3 -> 4;4 -> 1;6 -> 5;2 -> 6;}------- DOT END -----------]
TaskGroupBE#345155Set the state of the work plan to run.BE#345155 Group <Build ID> workplan state set to RunningBE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to Runningn/a
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [BEFORE_BUILD_INIT]BE#125123 Spawned task [BEFORE_BUILD_INIT]n/a
AbstractTaskBE#539145Run the task.BE#539145 Starting task [BEFORE_BUILD_INIT] under group <BuildID> BE#539145 Starting task [BEFORE_BUILD_INIT] under group 27364933-0325-41c3-8bbe-e28c79f19244n/a
AbstractTaskBE#029473Status that the task completed successfully.BE#029473 Task [BEFORE_BUILD_INIT] result status: COMPLETEDBE#029473 Task [BEFORE_BUILD_INIT] result status: COMPLETEDBE#029473 Task [BEFORE_BUILD_INIT] result status: FAILEDBE#507776 Task [BEFORE_BUILD_INIT] is cancelled, thrown: <exception string>
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [BEFORE_BUILD_INIT] completed. Duration: <duration>BE#960343 Task [BEFORE_BUILD_INIT] completed. Duration: 00:00:00.017BE#197825 Task [BEFORE_BUILD_INIT] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_BUILD_INIT] failed, exception object is null. Duration: <duration in milliseconds>You might receive a "warn" type message:BE#048944 Task [BEFORE_BUILD_INIT] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.BE#109647 End execution of [BEFORE_BUILD_INIT] failed: <error string>A build failure will occur if one mandatory task fails.
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [BuildFlow]BE#125123 Spawned task [BuildFlow]
Abstract TaskBE#539145Run the task.

An indication is sent to the GUI/API that the build has begun.
[BE#539145 Starting task [BuildFlow] under group <Build ID> [BE#539145 Starting task [BuildFlow] under group 27364933-0325-41c3-8bbe-e28c79f19244
BuildLoggerBE#353798 This is a workflow logger entry, indicating that the workflow for the build is at the given stage. This indicates that a notification is sent to the UI that the build has begun.BE#353798 Logger: (BUILD_FLOW) Build flowBE#353798 Logger: (BUILD_FLOW) Build flow
Abstract TaskBE#029473Indicates that the task completed successfully.BE#029473 Task [BuildFlow] result status: <status>BE#029473 Task [BuildFlow] result status: COMPLETEDBE#507776 Task [BuildFlow] is cancelled, thrown: <exception string>
TasksCoordinatorImplBE#960343The duration of the task.[BE#960343 Task [BuildFlow] completed. Duration: <run time for the step of the work plan>[BE#960343 Task [BuildFlow] completed. Duration: 00:00:00.006BE#197825 Task [BuildFlow] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BuildFlow] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BuildFlow] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue runningBE#109647 End execution of [BuildFlow] failed: <error string>.
TasksCoordinatorImplBE#125123Initiate the initialization task.BE#125123 Spawned task [Initializing]BE#125123 Spawned task [Initializing]
Abstract TaskBE#539145Execute task for the build ID.BE#539145 Starting task [Initializing] under group <Build ID> BE#539145 Starting task [Initializing] under group 27364933-0325-41c3-8bbe-e28c79f19244
BuildLoggerBE#353798 BE#353798 Logger: (ECM_ECBSTART) Waiting in queueBE#353798 Logger: (ECM_ECBSTART) Waiting in queue
Abstract TaskBE#029473The initialization task completed successfully.BE#029473 Task [Initializing] result status: <status>BE#029473 Task [Initializing] result status: COMPLETEDBE#507776 Task [Initializing] is cancelled, thrown: <exception string>
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [Initializing] completed. Duration: <run time for the step of the work plan>BE#960343 Task [Initializing] completed. Duration: 00:00:00.001BE#197825 Task [Initializing] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Initializing] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Initializing] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.BE#109647 End execution of [Initializing] failed: <error string>.
TasksCoordinatorImplBE#125123The create build initialization task completed.BE#125123 Spawned task [AFTER_BUILD_INIT]BE#125123 Spawned task [AFTER_BUILD_INIT]
AbstractTaskBE#539145Execute the task under the build ID.BE#539145 [AFTER_BUILD_INIT] under group <Build ID> BE#539145 [AFTER_BUILD_INIT] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicate whether the task was successful or not.BE#029473 Task [AFTER_BUILD_INIT] result status: <status>BE#029473 Task [AFTER_BUILD_INIT] result status: COMPLETEDBE#507776 Task [AFTER_BUILD_INIT] is cancelled, thrown: <exception string>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_BUILD_INIT] completed. Duration: <duration>BE#960343 Task [AFTER_BUILD_INIT] completed. Duration: 00:00:00.000BE#197825 Task [AFTER_BUILD_INIT] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_BUILD_INIT] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_BUILD_INIT] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.BE#109647 End execution of [AFTER_BUILD_INIT] failed: <error string>
TasksCoordinatorImplBE#125123Create a task indicating that sanity checks are about to be performed.BE#125123 Spawned task [BEFORE_SANITY_CHECK]]BE#125123 Spawned task [BEFORE_SANITY_CHECK]]
AbstractTaskBE#539145Execute the task under the Build ID.BE#539145 Starting task [BEFORE_SANITY_CHECK] under group <Build ID> BE#539145 Starting task [BEFORE_SANITY_CHECK] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicate if the task was successful or not.BE#029473 Task [BEFORE_SANITY_CHECK] result status: COMPLETEDBE#029473 Task [BEFORE_SANITY_CHECK] result status: COMPLETEDBE#507776 Task [BEFORE_SANITY_CHECK] is cancelled, thrown: <exception string>
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [BEFORE_SANITY_CHECK] completed. Duration: <duration>BE#960343 Task [BEFORE_SANITY_CHECK] completed. Duration: 00:00:00.001BE#197825 Task [BEFORE_SANITY_CHECK] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_SANITY_CHECK] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BEFORE_SANITY_CHECK] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.BE#109647 End execution of [BEFORE_SANITY_CHECK] failed: <error string>
TasksCoordinatorImplBE#125123Create a task to perform model validations. This task assesses whether the model setup is valid for the selected build behavior.BE#125123 Spawned task [Ecm Model Validation]BE#125123 Spawned task [Ecm Model Validation]
AbstractTaskBE#539145Execute the task under the build ID.BE#539145 Starting task [Ecm Model Validation] under group <Build ID> BE#539145 Starting task [EcmModel Validation] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates whether the task was successful or not.BE#029473 Task [Ecm Model Validation] result status: <Status>BE#029473 Task [Ecm Model Validation] result status: COMPLETEDBE#029473 Task [Ecm Model Validation] result status: FAILED
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [Ecm Model Validation] completed. Duration: <duration>BE#960343 Task [Ecm Model Validation] completed. Duration: 00:00:00.000BE#197825 Task [Ecm Model Validation] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Ecm Model Validation] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Ecm Model Validation] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.BE#109647 End execution of [BEFORE_SANITY_CHECK] failed: <error string>
TasksCoordinatorImplBE#125123Create a task to perform base table validations.BE#125123 Spawned task Base Tables Validation]BE#125123 Spawned task [Base Tables Validation]
AbstractTaskBE#539145 Execute the task under the build ID.BE#539145 Starting task [Base Tables Validation] under group <Build ID> BE#539145 Starting task [Base Tables Validation] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates whether the task was successful or not.BE#029473 Task [Base Tables Validation] result status: <status>BE#029473 Task [Base Tables Validation] result status: COMPLETEDBE#029473 Task [Base Tables Validation] result status: FAILEDBE#327169 Task [Ecm Model Compare] has failed: Build 'By Table' was requested but all tables are marked as 'Replace'.
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [Base Tables Validation] completed. Duration: <duration>BE#960343 Task [Base Tables Validation] completed. Duration: 00:00:00.001BE#197825 Task [Base Tables Validation] failed with exception: <exception>Duration: 00:00:00.002 Error will be raised in the case a build request for "By Table" is placed, while all tables are assigned a full build.The build however will continue as full build.
TasksCoordinatorImplBE#125123 Create a task to compare models.BE#125123 Spawned task [Ecm Model Compare]BE#125123 Spawned task [Ecm Model Compare]
AbstractTaskBE#539145Execute the task under the build ID.BE#539145 Starting task [EcmModel Compare] under group <Build ID> BE#539145 Starting task [Ecm Model Compare] under group fc4ad47f-38fb-43b7-a1d1-1575c58bec7c
AbstractTaskBE#029473Indicates whether the task was successful or not.BE#029473 Task [Ecm Model Compare] result status: <status>BE#029473 Task [Ecm Model Compare] result status: COMPLETEDBE#029473 Task [Ecm Model Compare] result status: FAILEDBE#327169 Task [Ecm Model Compare] has failed: Build 'By Table' with changes in schema detected. Please perform a 'Schema Changes' build first Error will appear if the cube model schema was changed, which would require that a schema changes build will take place.
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [Ecm Model Compare] completed. Duration: <duration>BE#960343 Task [Ecm Model Compare] completed. Duration: 00:00:00.014BE#197825 Task [Ecm Model Compare] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [[Ecm Model Compare] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [[Ecm Model Compare] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
AbstractTaskBE#029473A checkpoint indicating that the sanity test completed.BE#029473 Task [AFTER_SANITY_CHECK] result status: <status>BE#029473 Task [AFTER_SANITY_CHECK] result status: COMPLETEDBE#029473 Task [AFTER_SANITY_CHECK] result status: FAILEDBE#327169 Task [Ecm Model Compare] has failed: <error message>
TasksCoordinatorImplBE#960343The duration of the task.BE#960343 Task [AFTER_SANITY_CHECK] completed. Duration: <duration>BE#960343 Task [AFTER_SANITY_CHECK] completed. Duration: 00:00:00.000BE#197825 Task [AFTER_SANITY_CHECK] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_SANITY_CHECK] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_SANITY_CHECK] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TaskGroupBE#345155Build Sanity Work Plan completedBE#345155 Group <Build ID> workplan state set to CompletedBE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to CompletedBE#345155 Group <Build ID> workplan state set to Failed
TaskGroupBE#345155The state is set to finalizing.BE#345155 Group <Build ID> workplan state set to FinalizingBE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to Finalizing
TasksCoordinatorImplBE#936003Executing post tasks.BE#936003 Group <Build ID> executing post tasksBE#936003 Group 27364933-0325-41c3-8bbe-e28c79f19244 executing post tasks A task failed to finalize a successful work plan: Setting the work plan state to fail.

Post task failures may update the state, but they do not stop execution of the subsequent post tasks.
>
Subsequent post tasks, if they exist, will be aware of the new state.

In this case, an error will be captured:"Could not finalize the process: "<TASKNAME>" possible partial state.
TaskGroupBE#345155The build work plan completed.

There could be another condition in which the work plan has been queued and, therefore, a different message will be received.
BE#345155 Group <Build ID> workplan state set to CompletedORBE#807413 Group <BuildID> workplan state is already <Status>BE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to Completed
BuildServiceBE#451341Inserting the work plan in the queue.BE#451341 Enqueueing the state machine and build plan: <Build ID>BE#451341 Enqueueing the state machine and build plan: 27364933-0325-41c3-8bbe-e28c79f19244
TasksCoordinatorImplBE#753071The state of the work plan for the build is Completed.BE#753071 Execution DONE: groupId=<Build ID>, state=CompletedBE#753071 Execution DONE: groupId=27364933-0325-41c3-8bbe-e28c79f19244, state=Completed BE#193546 FAILED TO STOP EXECUTION: groupId=<Build ID>An error occurs when the build is forced to stop and execution of the task fails.
TasksCoordinatorImplBE#010801Optional

If the build is canceled, a request to cancel the overall task group for the build will be invoked.
BE#010801 Cancelled task group <Build ID>: <Status>BE#010801 Cancelled task group 27364933-0325-41c3-8bbe-e28c79f1924: DONE
BuildServiceBE#710254Initiate the state machine for the build.BE#710254 Running the state machine: <Build ID>ORBE#599941 Build plan was triggered, waiting: <Build ID>BE#710254 Running the state machine: 27364933-0325-41c3-8bbe-e28c79f19244
n/an/aThe steps for the state machine that maintains the progression of the build.

The stage will be performed in the following order:
  1. MARK_STARTED
  2. CREATE_CUBE
  3. RUN_PRE_BUILD_PLUGIN
  4. STORE_DATA
  5. CREATE_PROCESS
  6. START_BUILD
  7. END
started END CREATE_PROCESS STORE_DATA MARK_STARTED START_BUILD CREATE_CUBE RUN_PRE_BUILD_PLUGIN / MARK_STARTED /uuid=620437aa-17c0-4a5c-b5be-baa90ceb828c / id=<CubeID>started CREATE_CUBE STORE_DATA START_BUILD RUN_PRE_BUILD_PLUGIN CREATE_PROCESS END MARK_STARTED / MARK_STARTED /uuid=620437aa-17c0-4a5c-b5be-baa90ceb828c / id=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
BuildStateMachineConfigurationBE#949662Marks the work plan as Started in the state machine.BE#949662 Move from:null to:MARK_STARTEDBE#949662 Move from:null to:MARK_STARTEDIf event is not accepted, an error will show:BE#987875 Event not accepted: <EventName>
BuildLoggerBE#353798The cube is being created.BE#353798 Logger: (PREPARING_CUBE) Starting build processBE#353798 Logger: (PREPARING_CUBE) Starting build processThe cube creation by the management pod has failed. Investigate the management pod and/or (if the build is to MonetDB) the EC BLD log to further investigate the root cause. Potential causes could be anything that interrupts the process (e.g., a backup process being initiated at same time the build is executed, Management Pod failure/restarts, etc., lack of system resources or breaching a certain configuration threshold (e.g., the limit of the number of cubes deployed).
CreateCubeActionBE#322635Prepare a request to be sent to the management pod to bring up the bld-pod.BE#322635 Requesting cube from Management. Data Source Title: <Cube Title>BE#322635 Requesting cube from Management. Data Source Title: TestMySQL
InternalRequestProxyBE#166586Indicates that a request is sent and waiting for a response from the management pod. Management will respond with its success or failure to bring up the pod.BE#166586 Sent async request id=<request id>, waiting...]BE#166586 Sent async request id=30ecc294-fd8e-4bc5-8825-6b14cb42dea0 (getCubeForBuild), waiting...BE#130359 service not found, serviceName: Management ServiceBE#919403 Async request id=<request ID> timeoutBE#766092 Error from async request id=<requestid> onget CubeFor.BuildBE#150935 Timeout on Get cube for build.BE#468714 Failed to create cube for build processBE#994601 build to <Build to Destination DB Type> is not supported for provider <Build to Destination DB Provider>; Build to destination is not supported with old connectorZookeeper is down or the management service is not found. on the request to the management pod. The timeout duration is based on the configuration in Configuration Manager >> Base configuration >> serviceCallTimeout.value

Check management.log to further investigate what happened to this request. In most cases, this designates that the management pod is taking a while to bring up a new pod for the build.

This error will show the error message when the management responds to the request in time, but with an error.
RequestFilterBE#683109Send an asynchronous request to the management pod.BE#683109 Start handling request - url=http://ip:8086/internal/asyncResponse, userId=<UserID>, userRole=<UserRole>BE#683109 Start handling request - url=http://10.233.100.140:8086/internal/asyncResponse, userId=6124c5dc2b42f5001ad4541a, userRole=super
InternalRestControllerBE#541125Receive confirmation from the management pod that the request has been received.BE#541125 got async response for id <request id>BE#541125 got async response for id 30ecc294-fd8e-4bc5-8825-6b14cb42dea0BE#948724 Management Service returned an error: <returned error message>BE#468714 Management response: <any of the following errors>
  1. ElastiCube failed to start
  2. failed to detect the EC
  3. Failed to detect the connector
  4. Pod not reached;
  5. Insufficient Memory
  6. Too many ElastiCubes already running
  7. Cube is already running
  8. Cube is already starting
  9. Crash loop back off
  10. Failed to get existing deployment
  11. Failed to create deployment
  12. Failed to create pod
  13. Failed to get data group
  14. Internal Error
Indicates a failure on the management side when processing the request to bring up a new BLD Pod.
RequestFilterBE#587104The request is completed.BE#587104 Finished handling request - <URL for request>BE#587104 Finished handling request - http://10.233.100.140:8086/internal/asyncResponse]
CreateCubeActionBE#459006Record the turnaround time it took for the request to be processed. At this point, the management pod has confirmed bld=pod is up.BE#459006 Requesting cube from Management returned in <request processing time by management cube> msBE#459006 Requesting cube from Management returned in 15753 ms
The following will be written when the connector is configured for the new framework. Note: When a new framework is used, the connection to the external data source platform will be established in the initialization phase.
DataSourceClientFactoryImplBE#673011This indicates the datasource that the build connects to.BE#673011 Get data source client - connectorId=<data source provider>, scope=<context of the connection, for example as a result of build or Live connection>BE#673011 Get data source client - connectorId=MySql, scope=BUILD]BE#445373 mysql-connector-java-8.0.13.jar is a different version of mysql-connector-java-8.0.25.jar and will not be loaded
BaseDataSourceClientBE#666905A connection is established via a pre-existing connection pool (instead of establishing another connection). In this case, if configured as such, the connection count to a specified external data source will not increase.BE#666905 Got connection from poolBE#666905 Got connection from pool
BaseDataSourceClientBE#881429The connection to the pool is open.

OR

If the connection has been closed, a new connection will be established.
BE#881429 Connection is openORBE#503556 Connection is closed creating new connectionBE#881429 Connection is openORBE#503556 Connection is closed creating new connection
BaseDataSourceClientBE#976792ORBE#028731If the connection type requires connection validation, it will be validated. Otherwise, validation is skipped.BE#976792 Using connection from connection pool without validatingORBE#028731 About to valid connection from connection poolBE#976792 Using connection from connection pool without validatingORBE#028731 About to valid connection from connection poolBE#916022 connection is not valid creating new connection"Connection validation failed and a new connection will be opened to the data source.
BaseDataSourceClientBE#819683Establish the connection to the configured source.BE#819683 Creating new connection for data source <data source provider>BE#819683 Creating new connection for data source MySql
BaseDataSourceClientBE#395627The connection to the data source was successful.BE#395627 Connection to data source <data source provider>l establishedBE#395627 Connection to data source MySql establishedBE#732669 Failed to connect. The driver of connector: <Connector> is null. Could not load driver.BE#324945 The method beforeConnection failed with the following error: <exception>Could not establish connection using the provided parametersBE#239292 The method afterConnection failed with the following error <error message>BE#611992 Failed to connect to data source <Connector> with the following error: <returned error from connector>For example:BE#611992 Failed to connect to data source <Connector> with the following error: CJ communications error: Communications link failureBE#253325 Query was canceledBE#181265 <Indicate the SQL exception captured>The connection does not have a proper driver in place.

Failed to initialize the driver (meaning that a failure occurred before a connection was attempted).

Configured connection details are invalid or the connection cannot be established with them.

The physical connection is made, however additional steps in establishing the connection have failed.

The root cause of the failures of the connector exception returned.

Points at connectivity issues with the connection system, which might still be connectivity or wrong connection details.

The query request to the data source is canceled if the connection is not established properly.

Capture the SQL error returned.
Follow troubleshooting methods for connectivity to data sources to find the actual root cause.
BaseDataSourceClientBE#177047The driver that is configured for the connector in the configuration manager (new connector framework). BE#177047 Driver Version is: <driver for the given connection> (Revision: <revision number>) BE#177047 Driver Version is: mysql-connector-java-8.0.13 (Revision: 66459e9d39c8fd09767992bc592acd2053279be6)
BaseDataSourceClientBE#149955Load the configuration for the connector.BE#149955 Executing get driver metadata for connector - connectorId=<data source provider>BE#149955 Executing get driver metadata for connector - connectorId=mysqlBE#139161 Failed to get driver metadata for connector - connectorId=<data source provider>, reason=Failed to connectORBE#583000 Failed to get driver metadata for connector", <exception details>This will occur if the configuration data for the connection cannot be loaded (e.g., from MongoDB).

Double-check whether the connection is available to MongoDB (e.g., see if the GUI displays the connection details).
BaseDataSourceClientBE#673011 BE#673011 Get data source client - connectorId=<data source provider>, scope=DISCOVERYBE#673011 Get data source client - connectorId=MySql, scope=DISCOVERY
BaseDataSourceClientBE#819683Establish the connection to the data source.BE#819683 Creating new connection for data source <data source provider>BE#819683 Creating new connection for data source MySqlBE#732669 Failed to connect. The driver of connector: <Connector> is nullBE#239292 The method afterConnection failed with the following error <error message>BE#611992 Failed to connect to data source <Connector> with the following error: <returned error from connector>For example:BE#611992 Failed to connect to data source MySql with the following error: CJ communications error: Communications link failure
SisenseConnectionBE#985922
BaseDataSourceClientBE#395627The connection to the data source was established successfully.BE#395627 Connection to data source <data source provider> establishedBE#395627 Connection to data source MySql established
BaseDataSourceClientBE#177047Indicates the driver used to connect to the data source.BE#177047 Driver Version is: <driver for the given connection> (Revision: <revision number>) BE#177047 Driver Version is: mysql-connector-java-8.0.13 (Revision: 66459e9d39c8fd09767992bc592acd2053279be6)
BaseDataSourceClientBE#675750Indicates the connection pool ID created as part of the connection, or the existing pool ID that was used. BE#675750 Return connection to pool - conn=<connection id>BE#675750 Return connection to pool - conn=com.mysql.cj.jdbc.ConnectionImpl@7488a913BE#692718 Not reusing connection due to query execute errorBE#672530 Failed to close connection <exception>If the query fails, it will close out the pool, meaning any connection established will be closed. If the pool cannot be closed, this error will appear.
n/an/aIndicates the connection string used to connect to the data source, and some behavior that will be used, including:
  1. ConnectTimeout = the duration a connection attempt is allowed before it is timed out.
{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"ConnectorsConnectionsMetrics","Log_Level":"INFO","Log_Component":"connectors","Log_DateTime":"2021-06-03T09:07:48Z","Log_Thread":"builds-queue-0","appTypeName":"openConnection","Log_TransactionID":"ef09938d-0beb-479f-938b-8e298d0c2e29","metricName":"ConnectorsConnectionsMonitorMessage_count","connectionId":"MySql_jdbc:MySql://10.50.65.129/SISENSE_TEST? useLegacyDatetimeCode=false& allowPublicKeyRetrieval=true&tinyInt1isBit=false& useJDBCCompliantTimezoneShift=true& zeroDateTimeBehavior=convertToNull& connectTimeout=8000& serverTimezone=UTC&_5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8_false_false_dc76e9f0c0006e8f919e0c515c66dbba3982f785_7cb6efb98ba5972a9b5090dc2e517fe14d12cb04","connector":"MySql","connectionType":"DISCOVERY","result":{"SUCCESS":1},"meta":{"trustCert":"false","clientCert":"false"}} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"ConnectorsConnectionsMetrics","Log_Level":"INFO","Log_Component":"connectors","Log_DateTime":"2021-06-03T09:07:48Z","Log_Thread":"builds-queue-0","appTypeName":"openConnection","Log_TransactionID":"ef09938d-0beb-479f-938b-8e298d0c2e29","metricName":"ConnectorsConnectionsMonitorMessage_count","connectionId":"MySql_jdbc:MySql://10.50.65.129/SISENSE_TEST? useLegacyDatetimeCode=false& allowPublicKeyRetrieval=true&tinyInt1isBit=false& useJDBCCompliantTimezoneShift=true& zeroDateTimeBehavior=convertToNull& connectTimeout=8000& serverTimezone=UTC&_5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8_false_false_dc76e9f0c0006e8f919e0c515c66dbba3982f785_7cb6efb98ba5972a9b5090dc2e517fe14d12cb04","connector":"MySql","connectionType":"DISCOVERY","result":{"SUCCESS":1},"meta":{"trustCert":"false","clientCert":"false"}}
n/an/aTBD". "Log_Type":"Structured","Log_Version":"2.0","Log_Message":"ConnectorsConnectionsMetrics","Log_Level":"INFO","Log_Component":"connectors","Log_DateTime":"2021-06-03T09:07:48Z","Log_Thread":"builds-queue-0","appTypeName":"openConnection","Log_TransactionID":"ef09938d-0beb-479f-938b-8e298d0c2e29","metricName":"ConnectorsConnectionsMonitorMessage_count","connectionId":"MySql_jdbc:MySql://10.50.65.129/SISENSE_TEST? useLegacyDatetimeCode=false& allowPublicKeyRetrieval=true&tinyInt1isBit=false& useJDBCCompliantTimezoneShift=true& zeroDateTimeBehavior=convertToNull& connectTimeout=8000& serverTimezone=UTC&_5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8_false_false_dc76e9f0c0006e8f919e0c515c66dbba3982f785_7cb6efb98ba5972a9b5090dc2e517fe14d12cb04","connector":"MySql","connectionType":"BUILD","result":{"SUCCESS":1},"meta":{"trustCert":"false","clientCert":"false"}} "Log_Type":"Structured","Log_Version":"2.0","Log_Message":"ConnectorsConnectionsMetrics","Log_Level":"INFO","Log_Component":"connectors","Log_DateTime":"2021-06-03T09:07:48Z","Log_Thread":"builds-queue-0","appTypeName":"openConnection","Log_TransactionID":"ef09938d-0beb-479f-938b-8e298d0c2e29","metricName":"ConnectorsConnectionsMonitorMessage_count","connectionId":"MySql_jdbc:MySql://10.50.65.129/SISENSE_TEST? useLegacyDatetimeCode=false& allowPublicKeyRetrieval=true&tinyInt1isBit=false& useJDBCCompliantTimezoneShift=true& zeroDateTimeBehavior=convertToNull& connectTimeout=8000& serverTimezone=UTC&_5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8_false_false_dc76e9f0c0006e8f919e0c515c66dbba3982f785_7cb6efb98ba5972a9b5090dc2e517fe14d12cb04","connector":"MySql","connectionType":"BUILD","result":{"SUCCESS":1},"meta":{"trustCert":"false","clientCert":"false"}}
BaseDataSourceClientBE#675750The connection pool ID that was established as part of the connection, or that was used.BE#675750 Return connection to pool - conn=<connection pool ID>BE#675750 Return connection to pool - conn=com.mysql.cj.jdbc.ConnectionImpl@156db6b9BE#949662BE#692718 Not reusing connection due to query execute errorBE#672530 Failed to close connection
BuildStateMachineConfigurationBE#949662Move the state machine to start creating the cube.BE#949662 Move from:MARK_STARTED to:CREATE_CUBEBE#949662 Move from:MARK_STARTED to:CREATE_CUBEBE#987875 Event not accepted: <event name>
RunPreBuildPluginActionBE#982374OrBE#896881Check if a pre-build plugin is configured for the cube. If there is, run the prebuild plugin.

Note: At this point, this option is not yet available.
BE#982374 No pre-build plugin availableORBE#896881 Run pre build plugin for <BuildID>BE#982374 No pre-build plugin availableBE#833692 Build plugins are not yet available for B2DBE#571675 Plugin changed the ECM model - not supportedBE#567342 Plugin handling failed: Build cancelled by pluginBE#363706 ECM model modified - restarting...
BuildStateMachineConfigurationBE#949662Move the state machine to the stage of running the pre-build plugins.BE#949662 Move from:CREATE_CUBE to:RUN_PRE_BUILD_PLUGINBE#949662 Move from:CREATE_CUBE to:RUN_PRE_BUILD_PLUGIN
BuildLoggerBE#353798Save the cube's metadata into the farm, to later be used by the process that will interact with the data source platform (e.g., EC BLD pod for MonetDB).BE#353798 Logger: (STORE_DATA) Storing data to diskBE#353798 Logger: (STORE_DATA) Storing data to diskBE#449244 Failed to write .ElastiCube.json. Failed to store ECM model file.BE#785588 Failed to write: Failed to store ECM model fileThis error indicates an inability to write the ElastiCube metadata, meaning that there is a problem writing to the storage.
BuildStateMachineConfigurationBE#949662Move the state machine to store data to the farm.BE#949662 Move from:RUN_PRE_BUILD_PLUGIN to:STORE_DATABE#949662 Move from:RUN_PRE_BUILD_PLUGIN to:STORE_DATA
BuildLoggerBE#353798 Write a log that the build workplan is being created.BE#353798 Logger: (PREPARING_WORK_PLAN) Creating work-planBE#353798 Logger: (PREPARING_WORK_PLAN) Creating work-plan
BaseWorkPlanBuilderBE#491492Create the build workplan.BE#491492 Creating Build Work PlanBE#491492 Creating Build Work Plan
BaseWorkPlanBuilderBE#790911The work plan will be executed for the Build ID.BE#790911 Build Work Plan Id: <Build ID>BE#790911 Build Work Plan Id: 27364933-0325-41c3-8bbe-e28c79f19244
n/an/a 2021-06-03T12:07:50.366, sisense, build, {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Metrics SchemaMetricType","Log_Level":"INFO","Log_Component":"translation","Log_DateTime":"2021-06-03T09:07:50Z","Log_Thread":"MetricsCollector","appTypeName":"translation","DatasourcesCount_avg":2.0,"DatasourcesCount_count":1,"DatasourcesCount_max":2.0,"DatasourcesCount_min":2.0,"RelationshipsCount_avg":0.0,"RelationshipsCount_count":1,"RelationshipsCount_max":4.9E-324,"RelationshipsCount_min":0.0,"SchemaSize_avg":3487.0,"SchemaSize_count":1,"SchemaSize_max":3487.0,"SchemaSize_min":3487.0,"TablesCount_avg":4.0,"TablesCount_count":1,"TablesCount_max":4.0,"TablesCount_min":4.0,"ViewsCount_avg":2.0,"ViewsCount_count":1,"ViewsCount_max":2.0,"ViewsCount_min":2.0} 2021-06-03T12:07:50.366, sisense, build, {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Metrics SchemaMetricType","Log_Level":"INFO","Log_Component":"translation","Log_DateTime":"2021-06-03T09:07:50Z","Log_Thread":"MetricsCollector","appTypeName":"translation","DatasourcesCount_avg":2.0,"DatasourcesCount_count":1,"DatasourcesCount_max":2.0,"DatasourcesCount_min":2.0,"RelationshipsCount_avg":0.0,"RelationshipsCount_count":1,"RelationshipsCount_max":4.9E-324,"RelationshipsCount_min":0.0,"SchemaSize_avg":3487.0,"SchemaSize_count":1,"SchemaSize_max":3487.0,"SchemaSize_min":3487.0,"TablesCount_avg":4.0,"TablesCount_count":1,"TablesCount_max":4.0,"TablesCount_min":4.0,"ViewsCount_avg":2.0,"ViewsCount_count":1,"ViewsCount_max":2.0,"ViewsCount_min":2.0}
BuildServiceBE#151137 The below tasks will be added to the work plan.BE#151137 Calculating Explain Build. Data Source Title: <CubeTitle>BE#151137 Calculating Explain Build. Data Source Title: TestMySQL
The following adds the tasks to the work plan of the build type, and these tasks are specific for MonetDB:
BE#820694 - Full Build
BE#977187 - Accumulative Build
BE#927709 - Schema Change Build
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add a task that will determine dependencies in the build, so the build order is established correctly for all elements.BE#820694 Added [InitDependencies] tasks. Duration: <duration>BE#820694 Added [InitDependencies] tasks. Duration: 00:00:00.012
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add a task for creating the physical schema.BE#820694 Added [Schema] tasks. Duration: <duration>BE#820694 Added [Schema] tasks. Duration: 00:00:00.007
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add the task for executing the Base Table phase of the build.BE#820694 Added [Base Table] tasks. Duration: <duration>BE#820694 Added [BaseTable] tasks. Duration: 00:00:00.006
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add the task for executing the custom columns part of the custom elements phase.BE#820694 Added [Custom Columns] tasks. Duration: <duration>BE#820694 Added [Custom Columns] tasks. Duration: 00:00:00.010
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add the task for executing the custom tables part of the custom elements phase.BE#820694 Added [Custom Table] tasks. Duration: <duration>BE#820694 Added [Custom Table] tasks. Duration: 00:00:00.010
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add the task for executing the custom code part of the custom elements phase.BE#820694 Added [Custom Code Table] tasks. Duration: <duration>BE#820694 Added [Custom Table] tasks. Duration: 00:00:00.010
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709The task's duration.BE#820694 Added [Finalizing] tasks. Duration: <duration>BE#820694 Added [Custom Table] tasks. Duration: 00:00:00.010
MonetFullWorkPlanBuilderBE#820694ORBE#977187ORBE#927709Add the build finish task.BE#820694 Added [Finish] tasks. Duration: <duration>BE#820694 Added [Finish] tasks. Duration: 00:00:00.010
n/an/a {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-06-03T09:07:52Z","Log_Thread":"builds-queue-0","appTypeName":"build","Log_TransactionID":"ef09938d-0beb-479f-938b-8e298d0c2e29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"b203e437-723f-4fd3-9879-ce9c5fe6039d","cubeName":"EC Test","controlServerIP":"10.233.8.68","controlServerHostname":"build-5c49c6fd8b-tp8pd","source":"NextECM","startBuildTimeStamp":"2021-06-03T09:07:29.904307","BuildId":"67c3afd8-0cff-4ceb-92c6-0a3553eb24b3","dataServerIP":"10.233.100.191","dataServerHostName":"ec-ec-test-bld-33e277fd-7f90-9-7798c7c8fc-jsrlq","buildStep":"Initialization completed","endTimeStamp":"2021-06-03T09:07:52.267332","duration":22.363} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-06-03T09:07:52Z","Log_Thread":"builds-queue-0","appTypeName":"build","Log_TransactionID":"ef09938d-0beb-479f-938b-8e298d0c2e29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"b203e437-723f-4fd3-9879-ce9c5fe6039d","cubeName":"EC Test","controlServerIP":"10.233.8.68","controlServerHostname":"build-5c49c6fd8b-tp8pd","source":"NextECM","startBuildTimeStamp":"2021-06-03T09:07:29.904307","BuildId":"67c3afd8-0cff-4ceb-92c6-0a3553eb24b3","dataServerIP":"10.233.100.191","dataServerHostName":"ec-ec-test-bld-33e277fd-7f90-9-7798c7c8fc-jsrlq","buildStep":"Initialization completed","endTimeStamp":"2021-06-03T09:07:52.267332","duration":22.363}
BuildStateMachineConfigurationBE#949662Move the state machine to the cube creation process.BE#949662 Move from:STORE_DATA to:CREATE_PROCESSBE#949662 Move from:STORE_DATA to:CREATE_PROCESS
BuildsTrackerBE#605645Initiate the build.BE#605645 Trying to add build info to data source id <CubeID> to mapsBE#605645 Trying to add build info to data source id e51d6ac3-819c-48f7-b1c4-7003f8cdff7e to mapsBE#780497 Build was not marked to run <CubeID>BE#934333 Build already running for cube <CubeID>Cannot set the cube's status to run, since it is marked not to be run.

Cannot set the cube's status to run, as it is already running.
BuildsTrackerBE#146799Add the build to the map (containing the list of builds queued to run).BE#146799 Adding build id BuildID(buildId=<BuildID>) from mapsBE#146799 Adding build id BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244) from maps
BuildsTrackerBE#770490The build was successfully added to the map (containing the list of builds queued to run).BE#770490 Build id BuildID(buildId=<BuildID>) was added to mapsBE#770490 Build id BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244) was added to maps
BuildsTrackerBE#885182Trigger the build.BE#885182 End trigger build for data source id: <CubeTitle>BE#885182 End trigger build for data source id: TestMySQL
TasksCoordinatorImplBE#000801Initiate the work plan for the Build ID.BE#000801 Running coordinate group. Id: <BuildID>BE#000801 Running coordinate group. Id: 27364933-0325-41c3-8bbe-e28c79f19244
TaskGroupBE#289057The following DOT graph lays out the work plan for the rest of the build. You can graph this online at:https://dreampuf.github.io/GraphvizOnlineBE#289057 ------- Dumping Task Group -----------strict digraph G {1 [ label="AFTER_BASE_TABLE" fillcolor="#E6E6E6" shape="octagon" style="filled" ];2 [ label="Base table addresses_db" fillcolor="azure" style="filled" ];3 [ label="Base table Employees" fillcolor="azure" style="filled" ];4 [ label="Base table consumable_products" fillcolor="azure" style="filled" ];5 [ label="AFTER_BUILD_FINISHED" fillcolor="#E6E6E6" shape="octagon" style="filled" ];6 [ label="Run post build plugin" fillcolor="azure" style="filled" ];7 [ label="AFTER_CREATE_SCHEMA" fillcolor="#E6E6E6" shape="octagon" style="filled" ];8 [ label="Create schema TestMySQL" fillcolor="azure" style="filled" ];9 [ label="AFTER_CUSTOM_CODE_TABLE" fillcolor="#E6E6E6" shape="octagon" style="filled" ];10 [ label="BEFORE_CUSTOM_CODE_TABLE" fillcolor="#E6E6E6" shape="octagon" style="filled" ];11 [ label="AFTER_CUSTOM_COLUMN" fillcolor="#E6E6E6" shape="octagon" style="filled" ];12 [ label="BEFORE_CUSTOM_COLUMN" fillcolor="#E6E6E6" shape="octagon" style="filled" ];13 [ label="AFTER_CUSTOM_TABLE" fillcolor="#E6E6E6" shape="octagon" style="filled" ];14 [ label="BEFORE_CUSTOM_TABLE" fillcolor="#E6E6E6" shape="octagon" style="filled" ];15 [ label="AFTER_INIT_DEPENDENCIES" fillcolor="#E6E6E6" shape="octagon" style="filled" ];16 [ label="Init dependencies TestMySQL" fillcolor="azure" style="filled" ];17 [ label="BEFORE_BASE_TABLE" fillcolor="#E6E6E6" shape="octagon" style="filled" ];18 [ label="BEFORE_BUILD_FINISHED" fillcolor="#E6E6E6" shape="octagon" style="filled" ];19 [ label="BEFORE_CREATE_SCHEMA" fillcolor="#E6E6E6" shape="octagon" style="filled" ];20 [ label="BEFORE_INIT_DEPENDENCIES" fillcolor="#E6E6E6" shape="octagon" style="filled" ];21 [ label="Dump Basebats" fillcolor="azure" style="filled" ];22 [ label="Finalizing" fillcolor="azure" style="filled" ];1 -> 2;1 -> 3;1 -> 4;5 -> 6;7 -> 8;9 -> 10;11 -> 12;13 -> 14;15 -> 16;17 -> 7;18 -> 9;18 -> 11;18 -> 13;19 -> 15;10 -> 1;12 -> 1;14 -> 1;3 -> 17;2 -> 17;4 -> 17;8 -> 19;21 -> 22;22 -> 18;16 -> 20;6 -> 21;}
TaskGroupBE#345155This will set the work plan to running mode (or indicate that the work plan is already running).BE#345155 Group <BuildID> workplan state set to RunningORBE#807413 Group <BuildID> workplan state is already Running BE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to RunningBE#807413 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state is already Running
TasksCoordinatorImplBE#125123Checkpoint task.BE#125123 Spawned task [BEFORE_INIT_DEPENDENCIES]BE#125123 Spawned task [BEFORE_INIT_DEPENDENCIES]
BuildStateMachineConfigurationBE#949662The state machine was moved to start the build.BE#949662 Move from:CREATE_PROCESS to:START_BUILDBE#949662 Move from:CREATE_PROCESS to:START_BUILD
BuildStateMachineConfigurationBE#949662State machine moved to end of the build.BE#949662 Move from:START_BUILD to:ENDBE#949662 Movefrom:START_BUILD to:END
LifecycleObjectSupportm/a stopped END CREATE_CUBE CREATE_PROCESS START_BUILD STORE_DATA RUN_PRE_BUILD_PLUGIN MARK_STARTED / / uuid=620437aa-17c0-4a5c-b5be-baa90ceb828c / id=<Build ID>stopped END CREATE_CUBE CREATE_PROCESS START_BUILD STORE_DATA RUN_PRE_BUILD_PLUGIN MARK_STARTED / / uuid=620437aa-17c0-4a5c-b5be-baa90ceb828c / id=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
TaskGroupBE#599941Execute the build workflow plan.BE#599941 Build plan was triggered, waiting: <BuildID>E#599941 Build plan was triggered, waiting: 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#539145Initiate the task to check that the dependencies are met, in order for the build to run properly.BE#539145 Starting task [BEFORE_INIT_DEPENDENCIES] under group <BuildID>BE#539145 Starting task [BEFORE_INIT_DEPENDENCIES] under group 27364933-0325-41c3-8bbe-e28c79f19244
TasksCoordinatorImplBE#029473Indicates that the task was executed successfully.BE#029473 Task [BEFORE_INIT_DEPENDENCIES] result status: COMPLETEDBE#029473 Task [BEFORE_INIT_DEPENDENCIES] result status: COMPLETEDBE#029473 Task [BEFORE_INIT_DEPENDENCIES] result status: FAILEDBE#327169 Task [BEFORE_INIT_DEPENDENCIES] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [BEFORE_INIT_DEPENDENCIES] completed. Duration: <duration>BE#960343 Task [BEFORE_INIT_DEPENDENCIES] completed. Duration: 00:00:00.001BE#197825 Task [BEFORE_INIT_DEPENDENCIES] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_INIT_DEPENDENCIES] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BEFORE_INIT_DEPENDENCIES] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
BuildStateMachineConfigurationBE#949662Move the state machine to the END of the plan.BE#949662 Move from:START_BUILD to:ENDBE#949662 Move from:START_BUILD to:END
TasksCoordinatorImplBE#125123Instantiate the task.E#125123 Spawned task [Init dependencies <CubeTitle>]E#125123 Spawned task [Init dependencies TestMySQL]
AbstractTaskBE#539145Execute the task that checks that all dependencies are met, in order for the build to run.BE#539145 Starting task [Init dependencies <CubeTitle>] under group <BuildID>BE#539145 Starting task [Init dependencies TestMySQL] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [Init dependencies <BuildTitle>] result status: COMPLETEDBE#029473 Task [Init dependenciesTestMySQL] result status: COMPLETEDBE#029473 Task [Init dependencies <BuildTitle>] result status: FAILEDBE#327169 Task [Init dependencies <BuildTitle>] has failed: <error message>
TasksCoordinatorImplBE#960343 The task's duration.BE#960343 Task [Init dependencies <CubeTitle>] completed. Duration: <duration>BE#960343 Task [Init dependencies TestMySQL] completed. Duration: <duration>BE#197825 Task [Init dependencies] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Init dependencies] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Init dependencies] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [AFTER_INIT_DEPENDENCIES]BE#125123 Spawned task [AFTER_INIT_DEPENDENCIES]
AbstractTaskBE#539145Start the task, indicating that dependencies have been processed.BE#539145 Starting task [AFTER_INIT_DEPENDENCIES] under group <BuildID>BE#539145 Starting task [AFTER_INIT_DEPENDENCIES] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [AFTER_INIT_DEPENDENCIES] result status: COMPLETEDBE#029473 Task [AFTER_INIT_DEPENDENCIES] result status: COMPLETEDBE#029473 Task [[AFTER_INIT_DEPENDENCIES] result status: FAILEDBE#327169 Task [AFTER_INIT_DEPENDENCIES] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_INIT_DEPENDENCIES] completed. Duration: <duration>BE#960343 Task [AFTER_INIT_DEPENDENCIES] completed. Duration: 00:00:00.00BE#197825 Task [IAFTER_INIT_DEPENDENCIES] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_INIT_DEPENDENCIES] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_INIT_DEPENDENCIES] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [BEFORE_CREATE_SCHEMA]BE#125123 Spawned task [BEFORE_CREATE_SCHEMA]
AbstractTaskBE#539145Start the task, indicating that the schema creation is about to begin.BE#539145 Starting task [BEFORE_CREATE_SCHEMA] under group <BuildID>BE#539145 Starting task [BEFORE_CREATE_SCHEMA] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [BEFORE_CREATE_SCHEMA] result status: COMPLETEDBE#029473 Task [BEFORE_CREATE_SCHEMA] result status: COMPLETEDBE#029473 Task [BEFORE_CREATE_SCHEMA] result status: FAILEDBE#327169 Task [BEFORE_CREATE_SCHEMA] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [BEFORE_CREATE_SCHEMA] completed. Duration: <duration>BE#960343 Task [BEFORE_CREATE_SCHEMA] completed. Duration: 00:00:00.000BE#197825 Task [IAFTER_INIT_DEPENDENCIES] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_INIT_DEPENDENCIES] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_INIT_DEPENDENCIES] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [Create schema <CubeTitle>]BE#125123 Spawned task [Create schemaTestMySQL]
AbstractTaskBE#539145Start the task, indicating that the cube schema is about to be created, based on the data model.

Send execution requests to create the schema to the EC BLD pod in the case of a build to MonetDB, or to the destination when building to a database.
BE#539145 Starting task [Create schema <CubeTitle>] under group <Build ID>BE#539145 Starting task [Create schema TestMySQL] under group27364933-0325-41c3-8bbe-e28c79f19244
MonetDBClientn/aFirst, the schema's existence is checked by sending a query request to MonetDB as follows, and expect it to return a single integer value:

select count (*) from schemas where name = <PhysicalSchemaName>.

If the schema does not exist, send a request to the BLD pod to create the schema. The command sent to MonetDB will be:

START TRANSACTION; CREATE SCHEMA \ <PhysicalSchemaName>\; COMMIT;

The step will include a connection to MonetDB. Establishing a connection will be attempted a set number of times, with a wait/sleep interval between each attempt, as configured:

"Base Configuration" under "MonetDBCOnnectionRetries.value" and "SleepInterval" in milliseconds.
BE#539884 Failed to execute single value <Exception returned>The query fails due to a connectivity timeout or issue with the MonetDB (or for some reason no result set is returned).BE#334827 Failed to check if schema exists. database=<PhysicalSchemaName>.BE#931517 Failed to create schema: <Cube Title>BE#061353 Failed to create schema: <Cube Title>BE#168468 executeQuery failed: java.sql.SQLNonTransientConnectionException: connection timed outBE#360395 ExecuteQuery failed: java.sql.SQLNonTransientConnectionException: connection timed out, retriesLimit: <>, retry <The retry number>BE#220425 executeQuery failed for query: The query failed to execute in the time allowed. Query is "Simple Command" and is based on the "SimpleCommandTimeout.value" config.

The query failed to execute in the time allowed. The query is "Simple Command" and is based on the "SimpleCommandTimeout.value" config.

The query is timing out from the MonetDB based on the"SimpleCommandTimeout.value" configuration.

The query execution will be retried, based on the number of attempts set in the configuration "queryOnFailureRetries.value", and each retry will be delayed by the number set in "queryOnFailureInterval.value".

Indicates a logical failure of the query execution returned by MonetDB.
Increase the value of the timeout and/or number of retries.

Investigate the BLD pod load at the time of query timeout.
BuildLoggerBE#353798Log an entry that schema creation is about to begin.BE#353798 Logger: (APPLY_SCHEMA) Applying schemaBE#353798 Logger: (APPLY_SCHEMA) Applying schema
MonetDbClientBE#786539(Debug Only)Establishing a connection to MonetDB is attempted.

Establishing the connection is attempted up to the number of retries set in the configuration, with a wait between each attempt. This is based on: "Base Configuration" under "MonetDBCOnnectionRetries.value" and "SleepInterval" in milliseconds.
BE#786539 Connected to "<Physical Database name>"BE#786539 Connected to 'aMySQLTest'BE#400378 Connecting to <physical schema> failed on attempt <attempt number>: <configured MonetDBConnectionRetries.valuey>BE#131341 Monet connection attempt interrupted. <exception>A connection to MonetDB cannot be established.

A disconnection occurs with MonetDB.
Try to increase the connectivity parameters, to try to provide more time and ability to connect to MonetDB, and execute the query.
MonetDbClientBE#281644(Debug Only)The command to create a table is sent to the BLD pod's MonetDB. The command sent to MonetDB contains the table creation and all of the columns for the table. The create table command is:
ecents.create_table("<table physical name>");
BE#281644 Running query (fetchSize: 1): _1:lng := ecents.create_table("<physical table name>");_2:lng := ecents.create_column("<Column 1>","<column type>");ecents.attach_column_to_table(_1, _2);_2 := nil;_2:lng := ecents.create_column("<Column 2>","<column type>");ecents.attach_column_to_table(_1, _2);_2 := nil;_2:lng := ecents.create_column("<mongodb oid of column>","oid");ecents.attach_column_to_table(_1, _2);_2 := nil;_1 := nil;BE#281644 Running query (fetchSize: 1): _1:lng := ecents.create_table("<aEmployeesIAAaKAAa2KQAa>");_2:lng := ecents.create_column("<aFirstAAaName>","varchar");ecents.attach_column_to_table(_1, _2);_2 := nil;_2:lng := ecents.create_column("<aLastIAAaName>","varchar");ecents.attach_column_to_table(_1, _2);_2 := nil;_2:lng := ecents.create_column("<df5d11d6bc36_459b_907b_050e4d103647n>","oid");ecents.attach_column_to_table(_1, _2);_2 := nil;_1 := nil;BE#539884 Failed to execute single value <Exception returned>BE#168468 executeQuery failed: java.sql.SQLNonTransientConnectionException: connection timed outBE#360395 ExecuteQuery failed: java.sql.SQLNonTransientConnectionException: connection timed out, retriesLimit: <>, retry <The retry number>BE#220425 executeQuery failed for query: <reason>BE#183814 Failed to create table: <table physical name> <returned exception>The query fails due to a connectivity timeout or issue with the MonetDB (or for some reason no result set is returned).

The query times out from MonetDB, due to the "SimpleCommandTimeout.value" configuration. The query execution will be retried based on the number of attempts set in "queryOnFailureRetries.value" and each retry will be delayed by the duration set in "queryOnFailureInterval.value".

Indicates a logical failure of the query execution, returned by MonetDB.

Table creation fails.
There might be an overload on MonetDB. Try to increase the connectivity parameters to allow for successful query execution.
n/an/aA log entry indicating that the schema creation process has begun for the Build ID/cube.

Build Step = "Creating Schema".
{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:08Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:08.831988","buildStep":"Creating Schema"}
n/an/aThis log will be written once the schema creation process for the cube has completed. {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:12Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"<Cube Title>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"<Build ID>","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:08.831988","buildStep":"Schema created successfully","endTimeStamp":"2021-10-26T08:54:12.683613","duration":3.851} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:12Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:08.831988","buildStep":"Schema created successfully","endTimeStamp":"2021-10-26T08:54:12.683613","duration":3.851}
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [Create schema <CubeTitle>] result status: COMPLETEDBE#029473 Task [Create schema TestMySQL] result status: COMPLETEDBE#029473 Task [Create schema <CubeTitle>] result status: FAILEDBE#327169 Task [Create schema <CubeTitle>] has failed: <error message>
TasksCoordinatorImplBE#960343Indicates the task's duration.BE#960343 Task [Create schema<Cube Title>] completed. Duration: <Duration>BE#960343 Task [Create schemaTestMySQL] completed. Duration: 00:00:03.870BE#197825 Task [Create schema] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Create schema] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Create schema] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running. A failure here means that the schema at the destination database/MonetDB failed since the transaction to create the schema/database failed. A returned error should indicate a failure string from the destination database.
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [AFTER_CREATE_SCHEMA]BE#125123 Spawned task [AFTER_CREATE_SCHEMA]
AbstractTaskBE#539145Start the task, indicating that the build plan has completed creating the schema.BE#539145 Starting task [AFTER_CREATE_SCHEMA] under group <BuildID>BE#539145 Starting task [AFTER_CREATE_SCHEMA] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [AFTER_CREATE_SCHEMA] result status: COMPLETEDBE#029473 Task [AFTER_CREATE_SCHEMA] result status: COMPLETEDBE#029473 Task [AFTER_CREATE_SCHEMA] result status: FAILEDBE#327169 Task [AFTER_CREATE_SCHEMA] has failed: <error message>
TasksCoordinatorImplBE#960343Indicates the task's duration.BE#960343 Task [AFTER_CREATE_SCHEMA] completed. Duration: 00:00:00.001BE#960343 Task [AFTER_CREATE_SCHEMA] completed. Duration: 00:00:00.001BE#197825 Task [AFTER_CREATE_SCHEMA] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_CREATE_SCHEMA] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_CREATE_SCHEMA] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
BuildLoggerBE#353798Indicates that the initialization phase for the build is complete.BE#353798 Logger: (INIT_END) Initialization finished successfullyBE#353798 Logger: (INIT_END) Initialization finished successfully
BUILD - INITIALIZATION - END
BUILD - BASE TABLES - START
TasksCoordinatorImplBE#125123Instantiate the task.BE#125123 Spawned task [BEFORE_BASE_TABLE]BE#125123 Spawned task [BEFORE_BASE_TABLE]
AbstractTaskBE#539145Start the task, indicating that the build plan completed the schema creation.BE#539145 Starting task [BEFORE_BASE_TABLE] under group <BuildID>BE#539145 Starting task [BEFORE_BASE_TABLE] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [BEFORE_BASE_TABLE] result status: COMPLETEDBE#029473 Task [BEFORE_BASE_TABLE] result status: COMPLETEDBE#029473 Task [BEFORE_BASE_TABLE] result status: FAILEDBE#327169 Task [BEFORE_BASE_TABLE] has failed: <error message>
TasksCoordinatorImplBE#960343Indicates the task's duration.BE#960343 Task [BEFORE_BASE_TABLE] completed. Duration: <duration>BE#960343 Task [BEFORE_BASE_TABLE] completed. Duration: 00:00:00.001BE#197825 Task [BEFORE_BASE_TABLE] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_BASE_TABLE] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BEFORE_BASE_TABLE] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
For every base table, the following will be done:
TasksCoordinatorImplBE#125123Create the base table task that will import data from data sources.BE#125123 Spawned task [Base table <Baste Table Name>]BE#125123 Spawned task [Base table Employees]
AbstractTaskBE#539145Instantiate the task that imports data from the base table.BE#539145 Starting task [Base table <Base Table Name>] under group <BuildID>BE#539145 Starting task [Base table Employees] under group 27364933-0325-41c3-8bbe-e28c79f19244
BuildLoggerBE#353798Write an entry for the base table.BE#353798 Logger: (BASE_TABLE_START) BuildingBE#353798 Logger: (BASE_TABLE_START) Building
n/an/aIndicates the metadata information for the build. "Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:17Z","Log_Thread":"task-coordinator-1-LEVEL2","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"<BuildType>","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"<CubeTitle>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"<BuildId>?","dataServerIP":"10.233.101.216","dataServerHostName":"<CubeID>","startTimeStamp":"2021-10-26T08:54:17.172125","cloudTable":"<BaseTable Name>","provider":"<Connector Used>","connectorArch":"<Old or New Architecture for connector>","columnsCount":6,"buildStep":"buildTableImportStart"} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:17Z","Log_Thread":"task-coordinator-1-LEVEL2","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:17.172125","cloudTable":"Employees","provider":"MySql","connectorArch":"Old","columnsCount":6,"buildStep":"buildTableImportStart"}
QueryBuilderHandlerORstatement.QueryBuilderHandlerBE#316300Indicates, based on how the system is configured, whether the translator query build or build query builder is used to generate a query.

The configuration is located in "Service Configuration > Build" under "UseTranslatorQueryBuilder.value". When switched on, the Translator query builder will be used.
BE#316300 Using translator query builderORBE#471928 Using build query builderBE#316300 Using translator query builderORBE#471928 Using build query builder
TranslatorQueryBuilderORBuildQueryBuilderBE#109101Indicates, based on how the system is configured, whether the translator query build or build query builder is used to generate a query.BE#109101 Generating the query using TranslatorOR BE#184212 Generating the query using BuildBE#109101 Generating the query using TranslatorOR BE#184212 Generating the query using BuildBE#497507 Error in creating query: <details of the error>
BuildQueryBuilderBE#084427List the columns included in the queryBE#084427 table name is: Locations columns: [<Column names>]BE#084427 table name is: Locations columns: [Location Name, Active, LocID]
TranslatorQueryBuilderBE#730414Call the translator to generate an SQL statement for the destination database where the build will occur (e.g., MonetDB).

This query is used to get the data that is needed for an accumulative (By Column or Append) type build.

For a full build or schema change, this is not required.
BE#730414 Calling to Translator generateBE#730414 Calling to Translator generate"BE#401221 " + <exception>If the build for B2D (build to destination) uses a connector that interacts with a file, metadata (and not query) is created for the connector to know how to interact with the data file. In this case a JSON exception can be raised.
ConnectorsDialectLoaderBE#736216Will load a library that will be used to perform the SQL translation, where the library is located in the specified path.BE#736216 Loading dialect for connector <provider> from path: <path of where dialects are deployed>BE#736216 Loading dialect for connector MySql from path: /opt/sisense/storage/connectors/framework/MySql/dialectsBE#585591 Dialects directory <path of where dialects are deployed>, for connector <provider>, not found This means that no plugging for translation is configured for this provider.
ConnectorsDialectLoaderBE#813824If the dialects directory is found, the specific class will be loaded.BE#813824 Loading dialect from file: <file name for class>BE#813824 Loading dialect from file: opt/sisense/storage/connectors/framework/MySql/dialects/<library file>BE#502330 Failed to find classes in dialect dir, connector <provider>, path <path of where dialects are deployed>BE#293793 Failed to create class loader to load plugin <provider>If the dialects directory is found, and no classes are available for the translator, it will indicate that the class is not found.

Or the class failed to load.
ConnectorsDialectLoaderBE#143931Indicates that the library was loaded successfully.BE#143931 Successfully loaded dialect <dialect class constructor>, from path <path of where dialects are deployed>, for connector <provider>BE#143931 Successfully loaded dialect <dialect class constructor>, from path <path of where dialects are deployed>, for connector <provider>BE#877362 Failed to load dialect for connector <provider>BE#870567 Failed to load, and initiate dialect, for connector <provider>This will appear if the class loading failed for the connector.
DialectsFactoryBE#132961ORBE#918883Will indicate if a library (dialect) was loaded for the translation, or if none is configured/needed.BE#132961 Dialect loaded from dialect directoryORBE#918883 No dialect to loadBE#132961 Dialect loaded from dialect directoryORBE#918883 No dialect to load
TranslationRestProviderBE#644291 Send the set query (or custom query) to the translator, to translate the query statement for the destination database (e.g, MonetDB).

The SQL statement build is geared toward retrieving existing data from the cube that is being built, to determine how to execute the build behavior. Therefore, the SQL statement will be constructed, based on the build behavior selected for the table.
BE#644291 Entering translateSqlToPhysical. type [<Destination Database>], schemaId [CubeID],sql [<sql statement in the logical format>], server [<server where the storage is on>], rowLimit [<limit rows returneD>], offset [<record offset to start the fetch from>], sample [<will be indicated here if sample data was selected for the build>], translationServiceType [<Service used to translate>]BE#644291 Entering translateSqlToPhysical. type [MonetDb], schemaId [2f38d114-c060-3db0-9ea1-832d492c24e7], sql [select count(*) from "Employees"], server [localhost], rowLimit [-1], offset [0], sample [0], translationServiceType [TranslationService]Full Build/Append /Change Schema

The query will only retrieve the number of records that are currently in the table in the destination database (e.g., MonetDB).

select count(*) from "Employees"Accumulative Build

We first fetch the highest value for the column that was set to be "Accumulate By" that exists in the destination database (e.g., MonetDB). This value, in turn, is used to select from the source data system the data from the record indicated by this query result.

select max("Employees"."AccuIndicator") from "Employees"]

We also perform a query to get the full record set in the destination database (e.g., MonetDB) for the table.

select count(*) from "Employees"
BE#640531 Failed to translateSqlToPhysical, returned body: <payload sent to translator>BE#547315 Running table expression: <expression sent to translator>BE#000794 Running table expression failed. Payload: <payload sent to translator>Failed to translate custom query
TranslationRestProviderBE#046037The returned query string for the destination database (for the example shown, MonetDB). The SQL will now be in the format that is used to communicate with the destination database.BE#046037 Exiting translateSqlToPhysical. sqlQueryString <query statement>BE#046037 Exiting translateSqlToPhysical. sqlQueryString [QUERY STATEMENT]Translated Full/Append/Change Only Build (example MonetDB as the destination database): SELECT COUNT(*) AS "aColumn" FROM (SELECT "aEmployees"."df5d11d6bc36_459b_907b_050e4d103647" AS "df5d11d6bc36_459b_907b_050e4d103647" FROM "aTestMySQL"."aEmployees" AS "aEmployees" ) AS "aEmployees"Translated Accumulative Build Query (example MonetDB as the destination database):SELECT MAX("aEmployees"."aAccuIndicator") AS "aColumn" FROM (SELECT "aEmployees"."aAccuIndicator" AS "aAccuIndicator" , "aEmployees"."df5d11d6bc36_459b_907b_050e4d103647" AS "df5d11d6bc36_459b_907b_050e4d103647" FROM "aMySQLTestIAAa"."aEmployees" AS "aEmployees" ) AS "aEmployees"]BE#922123 + <error returned from translator>Failed to get translation responseThe translation call failed to run, or no response was received.
MonetDbClientBE#281644The following is an example of a DEBUG only log entry.

This shows the instruction sent to MonetDB, to perform the query against the existing data (e.g., to get the MAX value for the accumulative by column).
BE#281644 Running query (fetchSize: 1): sql.init()The log entry shows some key values that are passed to the call to MonetDB.For Accumulative Build:
  1. type - shows the type of build.
  2. logical column - indicates that column that was set as the "Accumulate By".
  3. expectedValueLong - the max values that were returned in the query before indicating the last record that exists in the destination database, so we know where to select the records from the external data source system (meaning starting from the next record for the accumulative build).
  4. table - base table being processed
  5. Server - ip for the source data platform.
  6. Provider Type - the external datasource connector used.
BE#281644 Running query (fetchSize: 1): sql.init();{\"schema\":\"aMySQLTestIAAa\",\"logicalColumnName\":\"AccuIndicator\",\"ip\":\"127.0.0.1\",\"transport\":\"FRAMED\",\"type\":\"accumulativeSync\",\"platform\":\"JVM\",\"operator\":\"2\",\"serialization\":\"PROTOBUF\",\"protocol\":\"BINARY\",\"port\":\"9001\",\"expectedValueLong\":\"10\",\"logical_schema\":\"MySQLTest 2\",\"table\":\"aEmployees\",\"logical_table\":\"Employees\"}","{\"AdditionalParameters\":\"\",\"ApiVersion\":\"2\",\"Table\":\"Employees\",\"Server\":\"10.50.6.164\",\"Schema\":\"Sisense\",\"userName\":"****",\"userId\":\"61d5d1c42382f8001a364395\",\"userGroups\":\"61d5d1c22382f8001a364393,61d5d1c22382f8001a364394\",\"password\":"****",\"SslSupport\":\"false\",\"Database\":\"Sisense\",\"ProviderType\":\"MySql\",\"BuildMode\":\"BUILD_TO_MONET\",\"IsLive\":\"true\"}","{\"columnmapping\":[{\"srcname\":\"ID\",\"dstname\":\"aID\",\"logicalColumnType\":\"INT\"},{\"srcname\":\"First Name\",\"dstname\":\"aFirstIAAaName\",\"logicalColumnType\":\"TEXT\"},{\"srcname\":\"Last Name\",\"dstname\":\"aLastIAAaName\",\"logicalColumnType\":\"TEXT\"},{\"srcname\":\"AccuIndicator\",\"dstname\":\"aAccuIndicator\",\"logicalColumnType\":\"INT\"},{\"srcname\":\"Group Assigned\",\"dstname\":\"aGroupIAAaAssigned\",\"dTable\":\"dTable_aEmployees_aGroupIAAaAssigned\",\"dTableRefCount\":1,\"logicalColumnType\":\"TEXT\"},{\"srcname\":\"EmpLocID\",\"dstname\":\"aEmpLocID\",\"logicalColumnType\":\"INT\"}]} BE#400378 Connecting to <physical schema> failed on attempt <attempt number>: <configured MonetDBConnectionRetries.valuey>BE#131341 Monet connection attempt interrupted. <exception>BE#539884 Failed to execute single value <Exception returned>BE#168468 executeQuery failed: java.sql.SQLNonTransientConnectionException: connection timed outBE#360395 ExecuteQuery failed: java.sql.SQLNonTransientConnectionException: connection timed out, retriesLimit: <>, retry <The retry number>BE#220425 executeQuery failed for query: <reason>A connection to MonetDB cannot be established.

A disconnection occurs with MonetDB.

The query fails due to a connectivity timeout or issue with the MonetDB (or for some reason no result set is returned).

The query is timing out from MonetDB, based on the "SimpleCommandTimeout.value" configuration.

The query execution will be retried, based on the number of attempts set in the "queryOnFailureRetries.value" configuration, and each retry will be delayed by the number set in "queryOnFailureInterval.value".

This indicates a logical failure of the query execution returned by MonetDB.
Increase the value of the timeout and/or number of retries.

Investigate the BLD pod load at the time of the query timeout.
AbstractBaseTableBuildTaskBE#757046If MondetDB is a destination DB, send a request to connect and copy records from an external data source provider.

Indicates the number of records imported and written to the base table.

The SQL function running is:

(fetchSize: 1): sql.init();_1:str := ecci.copy_into_base_table(");

The command type here is "Base Table".
BE#757046 Writing records to table done. Total records: <# of records written>, Table: <Baste Table Name>BE#757046 Writing records to table done. Total records: 9, Table: EmployeesBE#964736 Provider do not exist in providers map BE#023237 Provider do not exist in providers mapBE#431789 Progress report cannot run. Result set is null. Base table creation continues.BE#837486 Unable to connect to MonetBE#246979 Unable to connect to Monet"BE#385552 <exception returned from MonetDB>BE#879028 Failed to close ResultSet: <exception>BE#642084 Request timeout <elapsed time> seconds). Failed to get connector <ConnecotName>BE#636134 Build failed: base table "<Base Table Name> was not completed as expectedBE#026807 <Error written to the combined log>BE#853480 <status string>BE#640720 build failed: base table "<base table>" was not completed as expected.Failed to read row: <row #>, Connector <description of returned error message>For example, the following is a returned string from an import from Excel spreadsheet.BE#384596 Reading from the data source failed for 'Sheet1'.Connectors: BE#903574 Error from Microsoft Excel.These errors indicate that the connection to the requested data source platform does not exist in the list loaded.

An error occurred in recording the progress of the base table process. The build will continue to work.

Indicates that MonetDB cannot be accessed or connected to.

Indicates a failure to execute the insert of records to the base table in MonetDB.

Indicates that the connection to the results returned by MonetDB cannot be cleared/closed.

The connector returned a timeout for the attempt to connect to the data source platform.

Indicates that the base table processing did not complete successfully.

An error indicating that the query failed, and an exception will be returned.

This is the general error returned, indicating the base table that failed and the reason.
Potential returned errors could indicate a connection issue to MonetDB or a connection issue to the external data source provider.

Connection failure errors returned can include:
  1. The connection timed out
  2. The connection attempt failed
  3. Could not establish the connection
  4. Could not connect to the DB
  5. Failed to connect to the data reader
  6. Failed to connect to the data source
  7. The connection was refused
  8. The connection is closed
  9. Unable to access the database
In all scenarios, follow the guidelines for analyzing connection issues.
BE#594189 Workflow cancellation reported on flowId=<Build ID>BE#594189 Workflow cancellation reported on flowId=<Build ID>BE#966010 Workflow was cancelled:
AbstractTaskBE#029473 The task completed successfully.BE#029473 Task [Base table <Base Table Name>] result status: COMPLETEDBE#029473 Task [Base table Employees] result status: BE#029473 Task [Base table Employees] result status: COMPLETEDBE#029473 Task [Base table <Base Table Name>] result status: FAILEDBE#327169 Task [Base table <Base Table Name> ] has failed: <proceeding with a build error description>For example:Build failed: base table "<name of table>" was not completed as expectedFailed to connect to data source: TException - service has thrown: InvalidRequestException(why=No connection could be made because the target machine actively refused it Failed to connect to data source: TException - service has thrown: InvalidRequestException(why=No connection could be made because the target machine actively refused it <ip>, trace=<null><ip>, trace=<null> Designates a failure in processing the base table, due to:
  1. An inability to connect to the data source platform
  2. The wrong name of a table or the table does not exist in the data source platform
BuildLoggerBE#353798 Log an entry indicating the successful processing of the base table.BE#353798 Logger: (BASE_TABLE_END) Successfully builtBE#353798 Logger: (BASE_TABLE_END) Successfully builtBE#607939 Logger: (BUILD_FAILED) <Error Category> <Error Description>Connectivity to external data platform platformConnection failed. Check that the data source is online, that your data source user credentials are correct, and that you have sufficient Internet speed.Problem accessing MonetDBElastiCube crashedProvide a detailed error for the interaction with the base table, from connectivity to failure to process the table.

There is a connection problem to MonetDB, or it crashed. Check the status of the EC BLD pod.
n/an/aIndicates for the base table/cube ID:
  1. The connector provider used
  2. The build behavior
    1. "Replace" (for full)
    2. "Append"
    3. "Accumulate" (for Accumulate by Table)
  3. The connector architecture used (Old vs New Framework)
  4. The duration of the query execution
  5. The number of rows fetched and processed by the query.
The entry here for "Build Step" is "buildTableImportEnd", indicating that a base table was processed.
{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:20Z","Log_Thread":"task-coordinator-1-LEVEL2","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"<build type>","cubeID":"<Cube ID>","cubeName":"<Cube Title>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:17.172125","cloudTable":"<Base Table Name>","provider":"<Connector provider>","connectorArch":"<ConnectorFramework>","columnsCount":6,"buildStep":"buildTableImportEnd","endTimeStamp":"<Base Table End>","duration":3.776,"rowsAfterFetch":<Rows ,"totalRows":<Total rows processed>,"buildBehavior":"<build behavior>","isSampleDataBuild":<SampleDataSwitch>} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:20Z","Log_Thread":"task-coordinator-1-LEVEL2","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:17.172125","cloudTable":"Employees","provider":"MySql","connectorArch":"Old","columnsCount":6,"buildStep":"buildTableImportEnd","endTimeStamp":"2021-10-26T08:54:20.948506","duration":3.776,"rowsAfterFetch":9,"totalRows":9,"buildBehavior":"replace","isSampleDataBuild":false}
TasksCoordinatorImpBE#960343The task's duration.BE#960343 Task [Base table Table Name>] completed. Duration: <duration>BE#960343 Task [Base tableEmployees] completed. Duration: 00:00:08.257BE#197825 Task [Base table Table Name] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Base table Table Name] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Base table Table Name] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123The task is a checkpoint task.BE#125123 Spawned task [AFTER_BASE_TABLE]BE#125123 Spawned task [AFTER_BASE_TABLE]
AbstractTaskBE#539145Start the task, indicating that the build plan completed the schema creation.BE#539145 Starting task [AFTER_BASE_TABLE] under group <BuildID>BE#539145 Starting task [AFTER_BASE_TABLE] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473The task completed successfully.BE#029473 Task [AFTER_BASE_TABLE] result status: COMPLETEDBE#029473 Task [AFTER_BASE_TABLE] result status: COMPLETED
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_BASE_TABLE] completed. Duration: <duration>BE#960343 Task [AFTER_BASE_TABLE] completed. Duration: 00:00:00.001BE#197825 Task [AFTER_BASE_TABLE] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_BASE_TABLE] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_BASE_TABLE] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
BASE TABLES = END
CUSTOM ELEMENTS = START
TasksCoordinatorImplBE#125123The task is a checkpoint task.BE#125123 Spawned task [BEFORE_CUSTOM_COLUMN]BE#125123 Spawned task [BEFORE_CUSTOM_COLUMN]
AbstractTaskBE#539145Start the task, indicating that the build plan completed the schema creation.BE#539145 Starting task [BEFORE_CUSTOM_COLUMN] under group <BuildID>BE#539145 Starting task [BEFORE_CUSTOM_COLUMN] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473The task completed successfully.BE#029473 Task [BEFORE_CUSTOM_COLUMN] result status: COMPLETEDBE#029473 Task [BEFORE_CUSTOM_COLUMN] result status: COMPLETEDBE#029473 Task [BEFORE_CUSTOM_COLUMN] result status: FAILEDBE#327169 Task [BEFORE_CUSTOM_COLUMN] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [BEFORE_CUSTOM_COLUMN] completed. Duration: <duration>BE#960343 Task [BEFORE_CUSTOM_COLUMN] completed. Duration: 00:00:00.001BE#197825 Task [BEFORE_CUSTOM_COLUMN] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_CUSTOM_COLUMN] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BEFORE_CUSTOM_COLUMN] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123The task is a checkpoint task.BE#125123 Spawned task [BEFORE_CUSTOM_TABLE]BE#125123 Spawned task [BEFORE_CUSTOM_TABLE]
AbstractTaskBE#539145Start the task, indicating that the build plan completed the schema creation.BE#539145 Starting task [BEFORE_CUSTOM_TABLE] under group <BuildID>BE#539145 Starting task [BEFORE_CUSTOM_TABLE] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473The task completed successfully.BE#029473 Task BEFORE_CUSTOM_TABLE] result status: COMPLETEDBE#029473 Task BEFORE_CUSTOM_TABLE] result status: COMPLETEDBE#029473 Task [BEFORE_CUSTOM_TABLE] result status: FAILEDBE#327169 Task [BEFORE_CUSTOM_TABLE] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [BEFORE_CUSTOM_TABLE] completed. Duration: <duration>BE#960343 Task [BEFORE_CUSTOM_TABLE] completed. Duration: 00:00:00.000BE#197825 Task [BEFORE_CUSTOM_TABLE] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_CUSTOM_TABLE] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 BEFORE_CUSTOM_TABLE[BEFORE_CUSTOM_COLUMN] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123 The task is a checkpoint task.BE#125123 Spawned task [BEFORE_CUSTOM_CODE_TABLE]BE#125123 Spawned task [BEFORE_CUSTOM_CODE_TABLE]
AbstractTaskBE#539145Start the task, indicating that the build plan completed the schema creation.BE#539145 Starting task [BEFORE_CUSTOM_CODE_TABLE] under group <BuildID>BE#539145 Starting task [BEFORE_CUSTOM_CODE_TABLE] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473The task completed successfully.BE#029473 Task [BEFORE_CUSTOM_CODE_TABLE] result status: COMPLETEDBE#029473 Task [BEFORE_CUSTOM_CODE_TABLE] result status: COMPLETEDBE#029473 Task [BEFORE_CUSTOM_CODE_TABLE] result status: FAILEDBE#327169 Task [BEFORE_CUSTOM_CODE_TABLE] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [BEFORE_CUSTOM_CODE_TABLE] completed. Duration: <duration>BE#960343 Task [BEFORE_CUSTOM_CODE_TABLE] completed. Duration: 00:00:00.000BE#197825 Task [BEFORE_CUSTOM_CODE_TABLE] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_CUSTOM_CODE_TABLE] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BEFORE_CUSTOM_CODE_TABLE] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
Create and execute a Custom Column task for each table/custom column defined.
Note thet since tasks are performed in parallel, tasks will intermix in the log.
TasksCoordinatorImplBE#125123Create a task to create a custom column.BE#125123 Spawned task [Custom column <base table.custom column>]BE#125123 Spawned task [Custom column Employees.ImportDateTime]
AbstractTaskBE#539145Start the task for the custom column.BE#539145 Starting task [Custom column <base table.custom column>] under group <BuildID>BE#539145 Starting task [Custom column Employees.ImportDateTime] under group 27364933-0325-41c3-8bbe-e28c79f19244
CustomBuilderAlso need to document CustomBuilderv2)BE#975557OrBE#284615Indicates the custom column being prepared, in this case "ImportDateTime".This step takes the logical setup for the custom column and translates it to a MonetDB MAL command. The results are converted to JSON format.BE#975557 Start preparing custom column. Column: <Custom Column name> Table: <Base Table name>BE#975557 Start preparing custom column. Column: ImportDateTime, Table: EmployeesBE#270257 Fail to prepare json for mal <exception>BE#794548 Fail to prepare json for mal <exception>For some reason, the MAL command created does not have a valid JSON format.
CustomBuilderBE#283073Each custom column will be processed and data will be enriched.

In this case, the Employees base table is being handled for custom elements.
BE#283073 Start building custom element. Table: <Base Table name>BE#283073 Start building custom element. Table: Employees
BE#860722Finished preparing the information to create the custom column.BE#860722 End preparing custom column. Column: LocationName, Table: <Base Table Name>]BE#860722 End preparing custom column. Column: LocationName, Table: Employees]
BuildLoggerBE#353798Log that the custom column action is being performed.BE#353798 Logger: (CUSTOM_COLUMN_START) BuildingBE#353798 Logger: (CUSTOM_COLUMN_START) Building
CustomBuilderBE#623101Build to EC - Interaction with MonetDB:

The SQL statement that is set will be run against MonetDB to enrich the custom column.

This is an example of a column called "ImportDateTime", which uses the Now() function to store the datetime at the time of the build.

The query will be performed if there is a successful connection to the destination database, in this example MonetDB.

Several parameters are used to manage the connection:

Connection Retry - Configure under "Base Configuration" under "MonetDBConnectionRetries.value". The number set determines the number of retry attempts.



Connection Timeout -
The following parameters are configured under "Service Configuration >> Build". Each timeout parameter will be used for the interaction with MonetDB. For example, the timeout used when the Base Table operation is being performed vs. when a custom column or custom table is created.
  • SimpleCommandTime.value - The timeout used when the operation calls for a simple SQL execution.
  • ComplexCommandTimeout.value - The timeout value used when the operation calls for a complex SQL execution.
  • BaseTableTimeout.value - The timeout value used when building data into a base table.
  • CustomColumnTimeout.value - The timeout value used when creating a custom column.
  • CustomTableTimeout.value - The timeout value used when creating a custom table.


MonetDB Command Execution


SQl command execution against MonetDB will be attempted, based on the configured parameters that include the following under "Service Configuration >> Base"
  • queryOnFailureRetries.value - The number of attempts to try to execute the command against MonetDB
  • queryOnFailureInterval.value - The wait period between retries, based on the configured number (in seconds).
BE#623101 Custom Command is 'sql.init()<MonetDB Statement>sql.exit();"BE#623101 Custom Command is 'sql.init()ecci.write_custom("{\"columnattributes\":[{\"33\":\"1\",\"34\":\"1\",\"35\":\"0\",\"14\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/07/740.tail\",\"15\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/03/345.tail\",\"37\":\"0:lng\",\"16\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1051.tail\",\"49\":\"0\",\"17\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1036.tail\",\"18\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1052.tail\",\"19\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1040.tail\",\"columnname\":\"aImportDateTime\",\"0\":\"6\",\"1\":\"2\",\"4\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1040.tail\",\"5\":\"0\",\"50\":\"aEmployees\",\"51\":\"0\",\"52\":\"2\",\"32\":\"0\"}]}", "SELECT Now() AS \"aImportDateTime\" FROM (SELECT \"aEmployees\".\"df5d11d6bc36_459b_907b_050e4d103647\" AS \"df5d11d6bc36_459b_907b_050e4d103647\" FROM \"aTestMySQL\".\"aEmployees\" AS \"aEmployees\" ) AS \"aEmployees\" ORDER BY \"aEmployees\".\"df5d11d6bc36_459b_907b_050e4d103647\";");sql.exit();"BE#767456/BE#815158/BE#998787 Write custom table was not completed as expected. server: <servername>, database: <MonetDB physical name>, Status: <Status>MonetDB Connection IssuesBE#400378 Connecting to <physical schema> failed on attempt <attempt number>: <configured MonetDBConnectionRetries.valuey>BE#456396 Unable to connect to cube: <physical database name> server <server running MonetDB>MonetDB Connection Issues/Failure to Execute CommandBE#360395 ExecuteQuery failed: java.sql.SQLNonTransientConnectionException: Connection reset (mserver5 still alive?), retrying, retriesLimit: <Configured queryOnFailureRetries.value>, retry <retry attempt>]BE#249779 ExecuteQuery failed, retriesLimit reached <Configured >BE#168468 executeQuery failed: <exception>BE#220425 executeQuery failed for query: <query string>BE#147157 Execute Monet command failed <MAL command sent to Monet that faileD>BE#356215/BE#521691 <Indicates the SQL statement exception against MonetDB>For example:
  • SQL non transient connection: Unable to connect (<BLD Pod instance Name>.sisense:50000): Connection refused (Connection refused)
  • SQL non transient connection error: Connection reset (mserver5 still alive?).]
BE#716357 Single results query returns with more than one resultsBE#140709 Failed to get resultsORBE#426713 Failed to get resultsBE#307467 Failed to get monet providerORBE#247168 Failed to get monet provider
Check the SQL written in the log for any issues. Analyze which of the columns, tables, build dbfarm might have changed.

It is possible that MonetDB is not accessible, or was interrupted during the process (e.g., a crash/restart occurred).

This entry will appear if the connection to MonetDB fails, and the retry mechanism to connect is used, based on the configured number of retries. The connection attempt fails, based on the configured timeout parameters.

Unable to access MonetDB to execute the query. Retries are attempted per the configuration. This could be due to system resource issues or an unresponsive MonetDB. For example, a memory spike in the BLD process that might lead to safe mode or a BLD pod crash that severs connectivity.

The retry attempts to connect to the BLD pod MonetDB has reached the limit and is unable to connect, resulting in a failed build.

A wrong or faulty query causes this error to be raised.

Indicates an error returned from MonteDB.

A wrong query string command will result in this error.

Indicates faulty data, since only one record set was expected to be returned from the query.

The query timed out or failed to execute the command against MonetDB.

The problem occurs when the MonetDB client cannot be instantiated.
MonetDbClientBE#786539(Debug Only)An attempt is made to connect to MonetDB.The number of retries allowed and wait between each retry is set in the configuration:"Base Configuration" under "MonetDBCOnnectionRetries.value" and "SleepInterval" in milliseconds.BE#786539 Connected to "<Physical Database name>"BE#786539 Connected to 'aMySQLTest' A connection to MonetDB cannot be established.

A disconnection from MonetDB occurs.
AbstractTaskBE#029473Indicates that the task is completed for the custom column.BE#029473 Task [Custom column <base table.custom column>] result status: COMPLETEDBE#029473 Task [Custom column Employees.ImportDateTime] result status: COMPLETED
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [Custom column <base table.custom column>] completed. Duration: <duration>BE#960343 Task [Custom column Employees.ImportDateTime] completed. Duration: 00:00:00.174
BuildLoggerBE#353798Log that the custom column action completed successfully.BE#353798 Logger: (CUSTOM_COLUMN_END) Successfully builtBE#353798 Logger: (CUSTOM_COLUMN_END) Successfully built
n/an/an/a {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:23Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"<Cube Title>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"<Build Start Time>","BuildId":"<Build ID>","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2<Stat time>","cloudTable":"<Base Table>","cloudColumn":"<Custom Column>","buildStep":"buildCustomColumnEnd","endTimeStamp":"<end time>","duration":0.171} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:23Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-11-10T12:51:44.391265","BuildId":"30623abd-6779-41f8-b3de-d3d8cbdd7afc","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-11-10T12:52:23.821709","cloudTable":"Employees","cloudColumn":"ImportDateTime","buildStep":"buildCustomColumnEnd","endTimeStamp":"2021-11-10T12:52:23.992988","duration":0.171}
n/an/an/a {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:23Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"<Buid Type>","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"<Cube Title>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":<start build time time>","BuildId":"<Build ID>","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"<start time>","cloudTable":"<Base Table>","cloudColumn":"<Custom Column>","buildStep":"buildCustomColumnStart"} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:23Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-11-10T12:51:44.391265","BuildId":"30623abd-6779-41f8-b3de-d3d8cbdd7afc","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-11-10T12:52:23.821709","cloudTable":"Employees","cloudColumn":"ImportDateTime","buildStep":"buildCustomColumnStart"}
Create and execute a custom table task for each table/custom column defined.
Note: Since tasks are performed in parallel, tasks will intermix in the log.
TasksCoordinatorImplBE#125123Create a task for the custom table.BE#125123 Spawned task [Custom table <Custom Table Name>]BE#125123 Spawned task [Custom table ActiveLocations]
AbstractTaskBE#539145Start the task for the custom table.BE#539145 Starting task [Custom table <Custom Table Name>] under group <BuildID>BE#539145 Starting task [Custom table ActiveLocations] under group 27364933-0325-41c3-8bbe-e28c79f19244
CustomBuilder(Also need to document CustomBuilderv2)BE#975557Indicates each custom column being created for the custom table. In this case two columns, Location ID and Location, are created for the ActiveLocations custom table.

This step takes the logical setup for the custom column in the custom table and translates it to a MonetDB MAL command. The results are converted to JSON format.
BE#975557 Start preparing custom column. Column: <Custom Column>, Table: <Custom Table Name>]Start preparing custom column. Column: <Custom Column>, Table: <Custom Table Name>]Start preparing custom column. Column: <Column ID>BE#975557 Start preparing custom column. Column: LocationID, Table: ActiveLocations]Start preparing custom column. Column: Location, Table: ActiveLocations]Start preparing custom column. Column: df5d11d6bc36_459b_907b_050e4d103647BE#270257 Fail to prepare json for mal <exception>For some reason, the MAL command created is not in valid JSON format.
CustomBuilderBE#283073Start building the custom table.BE#283073 Start building custom element. Table: <CustomTableName>BE#283073 Start building custom element. Table: ActiveLocations
BuildLoggerBE#353798Log that the custom table action is being performed.BE#353798 Logger: (CUSTOM_TABLE_START) BuildingBE#353798 Logger: (CUSTOM_TABLE_START) Building
CustomBuilderBE#623101The SQL statement that is set will be run against MonetDB, to enrich the custom column.

This is an example of a custom table being created with a custom query against the location's base table.

The query retrieves the locations that are set to active:

Select LocationID, Location from [Locations] where [Locations].Active = 'Y' and, with the results, create the custom table.
BE#623101 Custom Command is 'sql.init()<MonetDB Statement>sql.exit();"BE#623101 Custom Command is 'sql.init();ecci.write_custom("{\"columnattributes\":[{\"33\":\"1\",\"34\":\"1\",\"35\":\"0\",\"37\":\"0:lng\",\"49\":\"0\",\"columnname\":\"aLocationID\",\"0\":\"0\",\"1\":\"0\",\"2\":\"0\",\"3\":\"0\",\"4\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1057.tail\",\"5\":\"0\",\"50\":\"aActiveLocations\",\"51\":\"0\",\"52\":\"2\",\"32\":\"0\"},{\"22\":\"4\",\"49\":\"0\",\"29\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/07/743.tail\",\"50\":\"aActiveLocations\",\"51\":\"0\",\"30\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/07/743.theap\",\"52\":\"2\",\"31\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1066.tail\",\"53\":\"1\",\"10\":\"32\",\"32\":\"0\",\"54\":\"dTable_aActiveLocations_aLocation\",\"11\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/07/743.theap\",\"33\":\"1\",\"12\":\"0:lng\",\"34\":\"1\",\"13\":\"1\",\"35\":\"0\",\"36\":\"0:lng\",\"38\":\"0:lng\",\"39\":\"0:lng\",\"19\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1066.tail\",\"columnname\":\"aLocation\",\"0\":\"3\",\"1\":\"6\",\"4\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/07/743.theap\",\"5\":\"0\",\"40\":\"2048:lng\",\"41\":\"2048:lng\",\"20\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1060.tail\",\"21\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/07/743.tail\"},{\"33\":\"1\",\"34\":\"1\",\"35\":\"0\",\"37\":\"0:lng\",\"49\":\"0\",\"columnname\":\"df5d11d6bc36_459b_907b_050e4d103647\",\"0\":\"0\",\"1\":\"0\",\"2\":\"10\",\"3\":\"10\",\"4\":\"/opt/sisense/storage/farms/aTestMySQL_2021.11.10.12.51.44.798/dbfarm/aTestMySQL/bat/10/1061.tail\",\"5\":\"0\",\"50\":\"aActiveLocations\",\"51\":\"0\",\"52\":\"2\",\"32\":\"1\"}]}", "SELECT \"aLocations\".\"aLocationID\" AS \"aLocationID\",\"aLocations\".\"aLocation\" AS \"aLocation\" FROM (SELECT \"aLocations\".\"aLocationID\" AS \"aLocationID\" , \"aLocations\".\"df5d11d6bc36_459b_907b_050e4d103647\" AS \"df5d11d6bc36_459b_907b_050e4d103647\" , \"dTableAlias_aLocation\".\"value\" AS \"aLocation\" , \"dTableAlias_aActive\".\"value\" AS \"aActive\" FROM \"aTestMySQL\".\"aLocations\" AS \"aLocations\" INNER JOIN \"aTestMySQL\".\"dTable_aLocations_aLocation\" \"dTableAlias_aLocation\" ON \"aLocations\".\"aLocation\" = \"dTableAlias_aLocation\".\"key\" INNER JOIN \"aTestMySQL\".\"dTable_aLocations_aActive\" \"dTableAlias_aActive\" ON \"aLocations\".\"aActive\" = \"dTableAlias_aActive\".\"key\" ) AS \"aLocations\" WHERE \"aLocations\".\"aActive\" = 'Y';");sql.exit();"BE#767456/BE#815158/BE#998787 Write custom table was not completed as expected. server: <servername>, database: <MonetDB physical name>, Status: <Status>MonetDB Connection IssuesBE#400378 Connecting to <physical schema> failed on attempt <attempt number>: <configured MonetDBConnectionRetries.valuey>BE#456396 Unable to connect to cube: <physical database name> server <server running MonetDB>MonetDB Connection Issues/Failure to Execute CommandBE#360395 ExecuteQuery failed: java.sql.SQLNonTransientConnectionException: Connection reset (mserver5 still alive?), retrying, retriesLimit: <Configured queryOnFailureRetries.value>, retry <retry attempt>]BE#249779 ExecuteQuery failed, retriesLimit reached <Configured >BE#168468 executeQuery failed: <exception>BE#220425 executeQuery failed for query: <query string>BE#147157 Execute Monet command failed <MAL command sent to Monet that faileD>BE#356215/BE#521691 <Indicates the SQL statement exception against MonetDB>For example:
  • SQL non transient connection: Unable to connect (<BLD Pod instance Name>.sisense:50000): Connection refused (Connection refused)
  • SQL non transient connection error: Connection reset (mserver5 still alive?).]
BE#716357 Single results query returns with more than one resultsBE#140709 Failed to get resultsORBE#426713 Failed to get resultsBE#307467 Failed to get monet providerORBE#247168 Failed to get monet provider
Check the SQL written in the log for any issues. Analyze which of the columns, tables, or build dbfarm might have changed. It is also possible that MonetDB is not accessible or was interrupted during the process (e.g., a crash/restart occurred).

An entry will appear if the connection to MonetDB fails, and the retry mechanism to connect is used, based on the configured number of retries. The connection attempt fails based on the configured timeout parameters.

Unable to access MonetDB to execute the query. Retries are attempted per the configuration. It could be due to system resource issues or an unresponsive MonetDB. For example, a memory spike in the BLD process might lead to safe mode or a BLD pod crash severs connectivity.

The retry attempts to connect to the BLD pod MonetDB has reached a limit and is not able to connect, leading to a failed build.

A wrong or faulty query will result in this error being raised. It indicates an error returned from MonteDB.

A wrong query string command will result in this error.

Indicates faulty data, since only one record set was expected to be returned from the query.

The query timed out or failed to execute the command against MonetDB. The problem occurs when the MonetDB client is not able to be instantiated.
AbstractTaskBE#029473Indicates that the task for the custom table has completed.BE#029473 Task [Custom table <Custom Table Name>] result status: COMPLETEDBE#029473 Task [Custom table ActiveLocations] result status: COMPLETEDBE#756689 Failed to build custom table: <Custom Table Name>BE#029473 Task [Custom table <Custom Table Name>] result status: FAILEDBE#327169 Task [Custom table <Custom Table Name>] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [Custom column <Custom table Name>] completed. Duration: <duration>BE#960343 Task [Custom table Active Locations] completed. Duration: 00:00:00.174BE#197825 Task [Custom table <Custom Table Name>] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Custom table <Custom Table Name>] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Custom table <Custom Table Name>] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
BuildLoggerBE#353798Log the end of the custom table processing.BE#353798 Logger: (CUSTOM_TABLE_END) Successfully builtBE#353798 Logger: (CUSTOM_TABLE_END) Successfully built
n/an/an/an/a {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:24Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-11-10T12:51:44.391265","BuildId":"30623abd-6779-41f8-b3de-d3d8cbdd7afc","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-11-10T12:52:24.079276","cloudTable":"ActiveLocations","columnsCount":2,"buildStep":"buildCustomTableEnd","endTimeStamp":"2021-11-10T12:52:24.219380","duration":0.14,"rowsAfterFetch":4,"totalRows":4,"buildBehavior":"replace","isSampleDataBuild":false} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:24Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-11-10T12:51:44.391265","BuildId":"30623abd-6779-41f8-b3de-d3d8cbdd7afc","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-11-10T12:52:24.079276","cloudTable":"ActiveLocations","columnsCount":2,"buildStep":"buildCustomTableStart"}
n/an/an/an/a {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:24Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-11-10T12:51:44.391265","BuildId":"30623abd-6779-41f8-b3de-d3d8cbdd7afc","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-11-10T12:52:24.079276","cloudTable":"ActiveLocations","columnsCount":2,"buildStep":"buildCustomTableStart"} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-11-10T12:52:24Z","Log_Thread":"task-coordinator-0-LEVEL4","appTypeName":"build","Log_TransactionID":"886c5835-d115-455c-9a89-cddd3f77bf29","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-11-10T12:51:44.391265","BuildId":"30623abd-6779-41f8-b3de-d3d8cbdd7afc","dataServerIP":"10.233.99.96","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-11-10T12:52:24.079276","cloudTable":"ActiveLocations","columnsCount":2,"buildStep":"buildCustomTableStart"}
Create and execute a custom code task for the code defined.
Note: Since tasks are performed in parallel, tasks will intermix in the log.
INSERT TASKS
TasksCoordinatorImplBE#125123 The task is a checkpoint task.BE#125123 Spawned task [AFTER_CUSTOM_COLUMN]BE#125123 Spawned task [AFTER_CUSTOM_COLUMN]
AbstractTaskBE#539145Start the task.BE#539145 Starting task [AFTER_CUSTOM_COLUMN] under group <BuildID>BE#539145 Starting task [AFTER_CUSTOM_COLUMN] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [AFTER_CUSTOM_COLUMN] result status: COMPLETEDBE#029473 Task [AFTER_CUSTOM_COLUMN] result status: COMPLETEDBE#029473 Task [AFTER_CUSTOM_COLUMN] result status: FAILEDBE#327169 Task [AFTER_CUSTOM_COLUMN] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_CUSTOM_COLUMN] completed. Duration: <duration>BE#960343 Task [AFTER_CUSTOM_COLUMN] completed. Duration: 00:00:00.001BE#197825 Task [AFTER_CUSTOM_COLUMN] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_CUSTOM_COLUMN] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_CUSTOM_COLUMN] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123The task is a checkpoint task.BE#125123 Spawned task [AFTER_CUSTOM_TABLE]BE#125123 Spawned task [AFTER_CUSTOM_TABLE]
AbstractTaskBE#539145Start the task.BE#539145 Starting task [AFTER_CUSTOM_TABLE] under group <BuildID>BE#539145 Starting task [AFTER_CUSTOM_TABLE] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [AFTER_CUSTOM_TABLE] result status: COMPLETEDBE#029473 Task [AFTER_CUSTOM_TABLE] result status: COMPLETEDBE#029473 Task [AFTER_CUSTOM_TABLE] result status: FAILEDBE#327169 Task [AFTER_CUSTOM_TABLE] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_CUSTOM_TABLE] completed. Duration: 00:00:00.002BE#960343 Task [AFTER_CUSTOM_TABLE] completed. Duration: 00:00:00.002BE#197825 Task [AFTER_CUSTOM_TABLE] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_CUSTOM_TABLE] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_CUSTOM_TABLE] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Task is a checkpoint task.BE#125123 Spawned task [AFTER_CUSTOM_CODE_TABLE]BE#125123 Spawned task [AFTER_CUSTOM_CODE_TABLE]
AbstractTaskBE#539145Start the task.BE#539145 Starting task [AFTER_CUSTOM_CODE_TABLE] under group <BuildID>BE#539145 Starting task [AFTER_CUSTOM_CODE_TABLE] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [AFTER_CUSTOM_CODE_TABLE] result status: COMPLETEDBE#029473 Task [AFTER_CUSTOM_CODE_TABLE] result status: COMPLETEDBE#029473 Task [AFTER_CUSTOM_CODE_TABLE] result status: FAILEDBE#327169 Task [AFTER_CUSTOM_CODE_TABLE] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_CUSTOM_CODE_TABLE] completed. Duration: <duration>BE#960343 Task [AFTER_CUSTOM_CODE_TABLE] completed. Duration: 00:00:00.003BE#197825 Task [AFTER_CUSTOM_CODE_TABLE] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_CUSTOM_CODE_TABLE] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_CUSTOM_CODE_TABLE] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
BUILD - CUSTOM ELEMENTS END (build.log)
BUILD - FINALIZE START (build.log)
TasksCoordinatorImplBE#125123Start task.BE#125123 Spawned task [BEFORE_BUILD_FINISHED]BE#125123 Spawned task [BEFORE_BUILD_FINISHED]
AbstractTaskBE#539145Task is a checkpoint task.BE#539145 Starting task [BEFORE_BUILD_FINISHED] under group <BuildID>BE#539145 Starting task [BEFORE_BUILD_FINISHED] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [BEFORE_BUILD_FINISHED] result status: COMPLETEDBE#029473 Task [BEFORE_BUILD_FINISHED] result status: COMPLETEDBE#029473 Task [BEFORE_BUILD_FINISHED] result status: FAILEDBE#327169 Task [BEFORE_BUILD_FINISHED] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [BEFORE_BUILD_FINISHED] completed. Duration: <duration>BE#960343 Task [BEFORE_BUILD_FINISHED] completed. Duration: 00:00:00.001BE#197825 Task [BEFORE_BUILD_FINISHED] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [BEFORE_BUILD_FINISHED] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [BEFORE_BUILD_FINISHED] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Start the task.BE#125123 Spawned task [Finalizing]BE#125123 Spawned task [Finalizing]
AbstractTaskBE#539145This task sends a notification to the GUI indicating that the build is being finalized.BE#539145 Starting task [Finalizing] under group <BuildID>BE#539145 Starting task [Finalizing] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [Finalizing] result status: COMPLETEDBE#029473 Task [Finalizing] result status: COMPLETEDBE#029473 Task [Finalizing] result status: FAILEDBE#327169 Task [Finalizing] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [Finalizing] completed. Duration: <duration>BE#960343 Task [Finalizing] completed. Duration: 00:00:00.003BE#197825 Task [Finalizing] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Finalizing] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Finalizing] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Start a task.BE#125123 Spawned task [Dump Basebats]BE#125123 Spawned task [Dump Basebats]
AbstractTaskBE#539145This task calls MonetDB to dump all the database files that were transacted. The call is to the following MonetDB function, and passes the farm location and file to be transacted:

ecents.dump_all_basebats_info("<farm location>/basebats.txt"

The output of the returned query is written to the file.

This query against MonetDB is considered to be a "Simple Command" and therefore will be timed out using the "SimpleCommandTimeout.value".
BE#539145 Starting task [DumpBasebats] under group <BuildID>BE#539145 Starting task [DumpBasebats] under group 27364933-0325-41c3-8bbe-e28c79f19244BE#223097 Unable to connect to MonetBE#801369 <status of execution against MonetDB>BE#168468 executeQuery failed: java.sql.SQLNonTransientConnectionException: connection timed outBE#360395 ExecuteQuery failed: java.sql.SQLNonTransientConnectionException: connection timed out, retriesLimit: <>, retry <The retry number>BE#220425 executeQuery failed for query:Unable to connect to MonetDB in order to process the data.

The query timed out from MonetDB, based on the "SimpleCommandTimeout.value" configuration. The query execution will be retried, based on the number of attempts set in the "queryOnFailureRetries.value" configuration, and each retry will be delayed by the duration set in "queryOnFailureInterval.value".

Indicates a logical failure of the query execution returned by MonetDB.
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [Dump Basebats] result status: COMPLETEDBE#029473 Task [Dump Basebats] result status: COMPLETEDBE#029473 Task [DumpBasebats] result status: FAILEDBE#327169 Task [DumpBasebats] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [Dump Basebats] completed. Duration: <duration>BE#960343 Task [Dump Basebats] completed. Duration: 00:00:00.021BE#197825 Task [DumpBasebatse] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [DumpBasebats] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [DumpBasebats] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:20Z","Log_Thread":"task-coordinator-1-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"<Build Type>","cubeID":"<Cube ID>","cubeName":"<Cube Title>","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"<Build Start Time> ","BuildId":"<Build ID>","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:20.969505","buildStep":"Finalization Completed","endTimeStamp":"2021-10-26T08:54:20.978979","duration":0.009} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Append","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:20Z","Log_Thread":"task-coordinator-1-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","startTimeStamp":"2021-10-26T08:54:20.969505","buildStep":"Finalization Completed","endTimeStamp":"2021-10-26T08:54:20.978979","duration":0.009}
BuildLoggerBE#353798Indicates that the task completed successfully.BE#353798 Logger: (DUMP_DATA) Dumping logsBE#353798 Logger: (DUMP_DATA) Dumping logs
TasksCoordinatorImplBE#125123Start the task.BE#125123 Spawned task [Run post build plugin]BE#125123 Spawned task [Run post build plugin]
AbstractTaskBE#539145The task accesses a post-build plugin to customize the build.BE#539145 Starting task [Run post build plugin] under group <BuildID>BE#539145 Starting task [Run post build plugin] under group 27364933-0325-41c3-8bbe-e28c79f19244
RunPostBuildPluginTaskBE#850934Indicates whether a post-build plugin is configured.BE#850934 No post-build plugin availableBE#850934 No post-build plugin availableBE#245236 Failed to translate <> for post-build plugin: <specify plugin error>
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [Run post build plugin] result status: COMPLETEDBE#029473 Task [Run post build plugin] result status: COMPLETEDBE#029473 Task [Run post build plugin] result status: FAILEDBE#327169 Task [Run post build plugin] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [Run post build plugin] completed. Duration: <duration%gt;BE#960343 Task [Run post build plugin] completed. Duration: 00:00:00.023BE#197825 Task [Run post build plugin] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [Run post build plugin] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [Run post build plugin] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TasksCoordinatorImplBE#125123Start task.BE#125123 Spawned task [AFTER_BUILD_FINISHED]BE#125123 Spawned task [AFTER_BUILD_FINISHED]
AbstractTaskBE#539145This is a checkpoint task.BE#539145 Starting task [AFTER_BUILD_FINISHED] under group <BuildID>BE#539145 Starting task [AFTER_BUILD_FINISHED] under group 27364933-0325-41c3-8bbe-e28c79f19244
AbstractTaskBE#029473Indicates that the task completed successfully.BE#029473 Task [AFTER_BUILD_FINISHED] result status: COMPLETEDBE#029473 Task [AFTER_BUILD_FINISHED] result status: COMPLETEDBE#029473 Task [AFTER_BUILD_FINISHED] result status: FAILEDBE#327169 Task [AFTER_BUILD_FINISHED] has failed: <error message>
TasksCoordinatorImplBE#960343The task's duration.BE#960343 Task [AFTER_BUILD_FINISHED] completed. Duration: <duration>BE#960343 Task [AFTER_BUILD_FINISHED] completed. Duration: 00:00:00.001BE#197825 Task [AFTER_BUILD_FINISHED] failed with exception: <ERROR>. Duration: <duration in milliseconds>BE#697137 Task [AFTER_BUILD_FINISHED] failed, exception object is null. Duration: <duration in milliseconds>BE#048944 Task [AFTER_BUILD_FINISHED] failed. Duration: <duration in milliseconds>. Task is not mandatory - continue running.
TaskGroupBE#345155Indicates whether the work plan completed (or already has a status of completed).BE#345155 Group <BuildID> workplan state set to CompletedORBE#807413 Group <BuildID> workplan state is already Completed BE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to CompletedORBE#807413 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to Completed
TaskGroupBE#345155Set the workplan state machine to finalizing.BE#345155 Group <BuildID> workplan state set to FinalizingBE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to Finalizing
TasksCoordinatorImplBE#936003 BE#936003 Group <BuildID>executing post tasksBE#936003 Group 27364933-0325-41c3-8bbe-e28c79f19244executing post tasksCould not finalize the process: <task> failed - possible partial state
AbstractTaskBE#539145Start the build completion task.BE#539145 Starting task [Complete build] under group <BuildID>BE#539145 Starting task [Complete build] under group 27364933-0325-41c3-8bbe-e28c79f19244
BuildLoggerBE#353798A log entry that indicates that the cube files are being stored to the storage/farm and the cube is being finalized.BE#353798 Logger: (COMPLETE_BUILD) Saving files and attaching cubeBE#353798 Logger: (COMPLETE_BUILD) Saving files and attaching cubeBE#420733 completeBuild failed, error: <returned error>BE#566730 Failed to send a message to RabbitMQ for finish build
BuildServiceBE#123162ORBE#449981 The Build service is ready to send a request to the management pod to start the finalization phase. A request will not be sent if no completion is required.BE#123162 sending complete build to ManagementORBE#449981 complete build not required - no instance ID BE#123162 sending complete build to ManagementBE#449981 complete build not required - no instance IDBE#145888 exception was received while waiting for the call to management.completeBuild to return a response.Failed to complete build.This happens for a build failure by the management pod.
InternalRequestProxyBE#166586Send a request to the nanagement pod to finalize the build pod (in this case EC-BLD) and shut down the build pod, as it is no longer needed.BE#166586 Sent async request id=<RequestID> (stopCubeForBuild), waiting...BE#166586 Sent async request id=6d10e195-0a43-4fea-b822-3c1cad9ac957 (stopCubeForBuild), waiting...BE#130359 service not found, serviceName: Management ServiceBE#919403 Async request id=<request ID> timeoutBE#150935 Timeout on Stop Cube for BuildThis happens if Zookeeper is down or the management service is not found. This will designate a timeout on the request to the management pod. The timeout duration is based on the serviceCallTimeout.value parameter configuration in configuration manager in "Base configuration".

Check management.log to further investigate what happened to this request. In most cases, this designates that the management pod is taking a while to bring up a new pod for the build.
RequestFilterBE#683109Send request.BE#683109 Start handling request - url=http://10.233.101.198:8086/internal/asyncResponse, userId=<UserID>, userRole=<UserRole>BE#683109 Start handling request - url=http://10.233.101.198:8086/internal/asyncResponse, userId=6124c5dc2b42f5001ad4541a, userRole=super
InternalRestControllerBE#541125Received confirmation from the management pod that the request is being processed.BE#541125 got async response for id <RequestID>BE#541125 got async response for id 6d10e195-0a43-4fea-b822-3c1cad9ac957
RequestFilterBE#587104Handling of the sent request has finished.BE#587104 Finished handling request - http://10.233.101.198:8086/internal/asyncResponseBE#587104 Finished handling request - http://10.233.101.198:8086/internal/asyncResponse
InternalRequestProxyBE#166586Send a request to the management pod to bring up the new query pod for MonetDB. The request attaches the query pod to the newly created cube.BE#166586 Sent async request id=<RequestID> (attachNewlyBuiltCube), waiting...BE#166586 Sent async request id=1a7cc8eb-f7ba-4944-8eef-068b3d6a5606 (attachNewlyBuiltCube), waiting...BE#130359 service not found, serviceName: Management ServiceBE#919403 Async request id=<request ID> timeoutBE#150935 Timeout on Attach newly built cubeIf Zookeeper is down or the management service is not found, this will designate a timeout for the request to the management pod. The timeout duration is based on the configuration in Configuration Manager in "Base configuration" with parameter:serviceCallTimeout.value.

Check management.log to further investigate what happened to this request. In most cases, this designates that the management pod is taking a while to bring up a new pod for the build.
RequestFilterBE#683109Save the build metadata containing the row count for each table built and the duration it took to build. This information is stored in the MongoDB >> BE >> buildMetaDatal collection.BE#683109 Start handling request - url=http://build:8086/internal/, userId=<UserID>, userRole=superBE#683109 Start handling request - url=http://build:8086/internal/, userId=6124c5dc2b42f5001ad4541a, userRole=super
RequestFilterBE#587104Request handling has completed.BE#587104 Finished handling request - http://build:8086/internal/]BE#587104 Finished handling request - http://build:8086/internal/]
BuildServiceBE#869430 BE#869430 Creating Models. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#869430 Creating Models. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e These models are written to MongoDB and an error could result from the inability to access it.
LogicalServiceImplBE#656221 BE#656221 Creating Logical Schema. Data Source Title: TestMySQL, ECM Model Oid: <CubeID>BE#656221 Creating Logical Schema. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e These models are written to MongoDB and an error could result from the inability to access it.
LogicalServiceImplBE#920165 BE#920165 Creating Logical Etlt. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#920165 Creating Logical Etlt. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e These models are written to MongoDB and an error could result from the inability to access it.
MonetPhysicalServiceImplBE#820116 BE#820116 Creating Physical Etlt. Data Source Title: <CubeTitle>, ECM Model Oid: <CubeID>BE#820116 Creating Physical Etlt. Data Source Title: TestMySQL, ECM Model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e These models are written to MongoDB and an error could result from the inability to access it.
EtltModelLogicalToMonetBE#304408 BE#304408 Requesting for Custom Elements SQL Expressions from TranslatorBE#304408 Requesting for Custom Elements SQL Expressions from Translator An issue needs to be investigated on the translator side.
TranslationRestProviderBE#813055 BE#813055 Start Generating Custom Elements SQL. translationServiceType: TranslationService, Payload: <CubeTitle>BE#813055 Start Generating Custom Elements SQL. translationServiceType: TranslationService, Payload: TestMySQLBE#947936 Too large json to store. Type: <key> key: <physical database name>, length: <length of cube metadata>BE#130359 service not found, serviceName: TranslationServiceAn error can occur when the data is read from the JSON file and it is too large (e.g., a cube with many elements, such as tables/custom elements.)

An error will occur when the translation service cannot be accessed. The iIssue could also be the result of a problem with Zookeepr.
Translator - Generate Custom Elements - START(translator.log)
TranslationService.RestServer [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"path":"/api/translation/generateCustomElementsSqls","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"<x-request>\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.230Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"path":"/api/translation/generateCustomElementsSqls","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.230Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServer.Controllers [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"schemaEtlPayload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: <Logical schema name>\n SchemaPhysicalName: <physical db name>\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering GenerateCustomElementsSqls","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:21:44.233Z","Log_Thread":"Worker(58)","appTypeName":null} ][X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"schemaEtlPayload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering GenerateCustomElementsSqls","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:21:44.233Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"SchemaName":"<Logical schema name>","SchemaId":"<schema ID>","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.233Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.233Z","Log_Thread":"Worker(58)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.233Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.233Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServer [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"path":"/api/translation/generateCustomElementsSqls","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.238Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:c40937e9-67fb-4286-9fd4-202bce4f5e1d]:[{"path":"/api/translation/generateCustomElementsSqls","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.238Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Translator - Generate Custom Elements - END (translator.log)
BE#048055 BE#048055 Requesting for Dependencies from TranslatorBE#048055 Requesting for Dependencies from Translator
BE#795211 BE#795211 Start Generating Dependencies. translationServiceType: TranslationService, Payload: <CubeTitle>BE#795211 Start Generating Dependencies. translationServiceType: TranslationService, Payload: TestMySQL
Translator - Generate Dependencies - START (translator.log)
TranslationService.RestServer [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"<x-request ID>\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.246Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"path":"/api/translation/dependencymap","headers":"{\"Connection\":[\"Keep-Alive\"],\"Content-Type\":[\"application/json\"],\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Host\":[\"translation.sisense:8070\"],\"User-Agent\":[\"unirest-java/3.1.00\"],\"Content-Length\":[\"7160\"],\"x-request-id\":[\"49812f18-aae3-43a1-a904-1266d4199d2e\"],\"x-userId\":[\"61d5d1c42382f8001a364395\"],\"x-userRole\":[\"super\"],\"x-tenantId\":[\"61d5d1c52382f8001a3643a2\"],\"x-allowedTenants\":[\"[61d5d1c52382f8001a3643a2]\"]}","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"starting API execution","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.246Z","Log_Thread":"Worker(58)","appTypeName":null} ]
TranslationService.RestServer.Controllers X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: <Schema ID>\n SchemaName: <Logical schema name>\n SchemaPhysicalName: <Phyical db name>\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:21:44.248Z","Log_Thread":"Worker(58)","appTypeName":null} X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"payload":"class SchemaEtlPayload {\n CustomTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomTable]\n CustomColumns: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.CustomColumn]\n MonetDbSchema: class MonetDbSchema {\n SchemaId: 2271a858-81d3-3647-8737-b4cfe9905097\n SchemaName: MySQLTest\n SchemaPhysicalName: aMySQLTest\n Tables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDataTable]\n Relationships: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetRelationship]\n DimensionTables: System.Collections.Generic.List`1[Prism.Web.AppServices.ServiceProviders.Model.MonetDimensionTable]\n}\n\n}\n","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"entering DependencyDictionary","Log_Level":"Info","Log_Component":"TranslationService.RestServer.Controllers","Log_DateTime":"2022-01-17T12:21:44.248Z","Log_Thread":"Worker(58)","appTypeName":null}
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"SchemaName":"<Logical schema name>","SchemaId":"Schema ID","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.248Z","Log_Thread":"Worker(58)","appTypeName":null} ] [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Entered CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.248Z","Log_Thread":"Worker(58)","appTypeName":null} ]
Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.249Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"SchemaName":"MySQLTest","SchemaId":"2271a858-81d3-3647-8737-b4cfe9905097","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"Leaving CreateCloud()","Log_Level":"Info","Log_Component":"Datasource.SchemaToCloudConverters.DbSchemaToCloud.MonetDbSchema","Log_DateTime":"2022-01-17T12:21:44.249Z","Log_Thread":"Worker(58)","appTypeName":null}
TranslationService.RestServer [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.252Z","Log_Thread":"Worker(58)","appTypeName":null} [X-Req:3ded7a1d-c935-4873-bc4d-93046d8adbd9]:[{"path":"/api/translation/dependencymap","Log_Type":"Structured","Log_Version":"1.0","Log_Message":"API execution finished successfully","Log_Level":"Info","Log_Component":"TranslationService.RestServer","Log_DateTime":"2022-01-17T12:21:44.252Z","Log_Thread":"Worker(58)","appTypeName":null}
Translator - Generate Dependencies - END (translator.log)
BuildServiceBE#578789Store the new model in MongoDB.

If the previous build failed on the cube, meaning that the build did not finish successfully, a partially built model might be stored. Therefore, it will be deleted from MongoDB for the cube.

A notification message is sent to the storage services, to save the model's metadata of the cube. Each record is first deleted and then a new record is created in MongoDB.

The list of metadata includes:
  1. DataSourceInfo
  2. EcmModel
  3. PhysicalName2DataSourceId
  4. LogicalDbSchema
  5. DataSchema
  6. PhysicalLiveDbSchema
  7. PhysicalMonetDbSchema
  8. PhysicalEtltModel
  9. PhysicalName2DataSourceId
In addition, the following files are saved in the farm:
  • ElastiCube.json
  • ElastiCubePhysicalDbSchema.json
  • ElastiCubeInfo.json
BE#578789 Storing Models. Data Source Title: <CubeTitle>, ECM model Oid: <CubeID>BE#578789 Storing Models. Data Source Title: TestMySQL, ECM model Oid: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e These models are written to MongoDB and an error can result if it is inaccessible.
StorageDataServiceImplBE#801091Indicates the size of the data for the record transacted in MongoDB.
BE#801091 PhysicalName2DataSourceId - <Database Physical Name>length is <length of data in bytes>BE#801091 PhysicalName2DataSourceId - aTestMySQL length is 38BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might occur if the metadata retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
StorageDataServiceImplBE#801091Indicates the size of the data for the record transacted in MongoDB. document _Id to transact (which is the CubeID) BE#801091 DataSourceInfo - <CubeID> length is <length of data in bytes>BE#801091 DataSourceInfo - e51d6ac3-819c-48f7-b1c4-7003f8cdff7e length is 447BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might occur if the metadata retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
StorageDataServiceImplBE#801091Transact data to ecmModel in MongoDB using the key of the Schema ID to transact. This information will be stored in the MongoDB >> BE >> ecmModel collection using the schemaID to transact.BE#801091 EcmModel - <Schema ID> length is <length of data in bytes>BE#801091 EcmModel - e0605e68-92c0-3f47-8f06-b8c5f52018a7 length is 2719BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might occur if the metadata retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
StorageDataServiceImplBE#801091Transact data to MongoDB using the key of the Schema ID to transact. This information will be stored in the MongoDB >> BE >> logicalSchema collection using the document _Id to transact. BE#801091 DataSchema - <Schema ID> length is <length of data in bytes>BE#801091 DataSchema - e0605e68-92c0-3f47-8f06-b8c5f52018a7 length is 8981BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might occur if the metadata retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
StorageDataServiceImplBE#801091Indicates size of the data for the record transacted. Transact data to MongoDB using the key of the Schema ID to transact.

This information will be stored in the MongoDB >> BE >> logicalSchema collection using the document _Id to transact, which is the CubeID.
BE#801091 LogicalDbSchema - <CubeID> length is <length of data in bytes>BE#801091 LogicalDbSchema - e51d6ac3-819c-48f7-b1c4-7003f8cdff7e length is 1992BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might occur if the metadata retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
StorageDataServiceImplBE#801091Indicates the size of the data for the record transacted.

This information will be stored in the MongoDB >> BE >> physicalSchema collection using the document _Id to transact, which is the CubeID.
BE#801091 PhysicalMonetDbSchema - <CubeID> length is <length of data in bytes>BE#801091 PhysicalMonetDbSchema - e0605e68-92c0-3f47-8f06-b8c5f52018a7 length is 5134BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might appear if the meta data retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
StorageDataServiceImplBE#801091Indicates the size of the data for the record transacted.

This information will be stored in the MongoDB >> BE >> physicalEtlt collection using the Db Physical Name.
BE#801091 PhysicalEtltModel - <Database Physical Name> length is <length of data in bytes>BE#801091 PhysicalEtltModel - aTestMySQL length is 15599BE#947936 Too large json to store. Type: PhysicalName2DataSourceId key: <physical database name>, length: <length of cube metadata>BE #901516 <exception error>This is a WARN that might appear if the meta data retrieved for the cube is very long, when a cube contains a lot of metadata configuration information (e.g., lots of base tables, lots of column details, etc.) The length of data is considered large when it exceeds 3MB.
RequestFilterBE#587104Finished handling the request.BE#587104 Finished handling request - <URL>BE#587104 Finished handling request - http://build:8086/internal/
RequestFilterBE#683109Send a request to the management pod to start the finalization phase, meaning that the build service processing has completed.BE#683109 Start handling request - url=http://10.233.101.198:8086/internal/asyncResponse, userId=<UserID>, userRole=<UserRole>BE#683109 Start handling request - url=http://10.233.101.198:8086/internal/asyncResponse, userId=6124c5dc2b42f5001ad4541a, userRole=super
InternalRestControllerBE#541125Received acknowledgement from the management pod.BE#541125 got async response for id <RequestID>BE#541125 got async response for id 1a7cc8eb-f7ba-4944-8eef-068b3d6a5606BE#130359 service not found, serviceName: Management ServiceBE#919403 Async request id=<request ID> timeoutThis happens if Zookeeper is down or the management service is not found.

This designates a timeout on the request to the management pod. The timeout duration is based on the serviceCallTimeout.value configuration parameter in Configuration Manager in "Base configuration".

Check the management.log to investigate further what happened to this request. In most cases, this indicates that the management pod is taking a while to bring up a new pod for the build.
BuildServiceBE#777304Confirm that management completed the finalization phase of the build successfully, indicating how long it took.BE#777304 complete build returned from Management in <duration> msBE#777304 complete build returned from Management in 15459 ms
AbstractTaskBE#029473Finalize the Complete build task.BE#029473 Task [Complete build] result status: COMPLETEDBE#029473 Task [Complete build] result status: COMPLETED
TasksCoordinatorImplBE#492760The task completed successfully.BE#492760 Task [Complete build] ended, was pending for <duration includes time waiting for management pod>, execution took <time spend in build service>BE#492760 Task [Completebuild] ended, was pending for 00:00:09.732, execution took 00:00:01.427
RequestFilterBE#587104The request was handled.BE#587104 Finished handling request - http://10.233.101.198:8086/internal/asyncResponseBE#587104 Finished handling request - http://10.233.101.198:8086/internal/asyncResponse
AbstractTaskBE#539145Start the finalization task for the build.BE#539145 Starting task [Finish] under group 27364933-0325-41c3-8bbe-e28c79f19244BE#539145 Starting task [Finish] under group 27364933-0325-41c3-8bbe-e28c79f19244
ConnectorsDiscoveryImplBE#519221Close connectivity to the external data provider platform. (This includes closing all provider connections involved in this build.)BE#519221 Connector service <connector entry ID>4 provider closedBE#519221 Connector service MySql_27364933-0325-41c3-8bbe-e28c79f19244 provider closed BE#546839 Failed to close provider. serviceName = <name of the services used for the connection>BE#978913 Connector service {} is not registered as a providerClosing the connection failed (e.g., providers are not available, or there is an issue with the connector).
ConnectorsDiscoveryImplBE#947442Remove the connection entry from the pool of connections.BE#947442 Removed connector entry: <connector entry ID>ORBE#763476 Connector entry no longer exists: <connector entry ID>BE#947442 Removed connector entry: MySql_27364933-0325-41c3-8bbe-e28c79f19244BE#536896 Failed to close provider. serviceName = <name of the services used for the connection> <Exception>BE#203616 Failed to remove connector entry: <connector entry ID><Exception>
AbstractTaskBE#029473The task completed successfully.BE#029473 Task [Finish] result status: COMPLETEDBE#029473 Task [Finish] result status: COMPLETED
BuildLoggerBE#353798Indicates that the build finished successfully.BE#353798 Logger: (BUILD_SUCCEEDED) Build finished successfullyBE#353798 Logger: (BUILD_SUCCEEDED) Build finished successfullyBE#353798 Logger: (BUILD_CANCELLED) Build cancelled: Low disk space]This error appears when the Management Service has proactively canceled all active builds due to insufficient disk space in the storage for the cube to be built.Refer to the investigating storage space section of this document.
AbstractTaskBE#539145Initiate the task to upload build data usage analytics.BE#539145 Starting task [Analytics Upload] under group 27364933-0325-41c3-8bbe-e28c79f19244BE#539145 Starting task [Analytics Upload] under group 27364933-0325-41c3-8bbe-e28c79f19244
CubeModelsServiceBE#680675Send Build data to the analytics cube that is used for build analysis. This upload will only occur when it is configured to be on.

This is configured in the "Service Configuration >> Build" section of the configuration management via the "uploadLogicalSchemaAnalytics.value" parameter.
BE#680675 analytics: com.sisense.build.v2.models.CubeModelsStoreBE#680675 analytics: com.sisense.build.v2.models.CubeModelsStoreBE#929418 Analytics upload failed: <Exception>Issue can occur when sending the analytics data to the cube that is used to store analytics.
AbstractTaskBE#029473The task completed successfullyBE#029473 Task [Analytics Upload] result status: COMPLETEDBE#029473 Task [Analytics Upload] result status: COMPLETEDBE#029473 Task [Analytics Upload] result status: FAILEDBE#327169 Task [Analytics Upload] has failed: <error message>BE#707661 Got an error while uploading analytics using new cube models <exception>
This log entry shows the analytics data. It contains the model data, schema, and data providers used. {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Analytics:AnalyticsData","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"build-analytics-uploading-0","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","logicalSchema":{"oid":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","name":"TestMySQL","schemaId":"4830f677-d1b3-3a95-9888-91fcacca2b20","tables":{"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c":{"oid":"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c","name":"addresses_db","hidden":false,"custom":false,"columns":{"85f500c3-b3b3-4ea7-9136-90ed95f1e945":{"oid":"85f500c3-b3b3-4ea7-9136-90ed95f1e945","name":"Column 1","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true}}},"d5279f72-f5ed-43ef-b18a-5c3d543f58a0":{"oid":"d5279f72-f5ed-43ef-b18a-5c3d543f58a0","name":"Employees","hidden":false,"custom":false,"columns":{"7a807ddb-91fb-452a-a225-6c9a1153fdaf":{"oid":"7a807ddb-91fb-452a-a225-6c9a1153fdaf","name":"EmployeeID","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true},"a9a342f9-2c02-4dcd-8745-56d720edc713":{"oid":"a9a342f9-2c02-4dcd-8745-56d720edc713","name":"First Name","type":"TEXT","size":65535,"precision":65535,"scale":0,"hidden":false,"custom":false,"indexed":false},"0f9fbafa-6c6f-4f52-9c1c-803868d69d52":{"oid":"0f9fbafa-6c6f-4f52-9c1c-803868d69d52","name":"Last Name","type":"TEXT","size":65535,"precision":65535,"scale":0,"hidden":false,"custom":false,"indexed":false},"a151c32d-a9cc-4214-8f3e-e9b58ad23a80":{"oid":"a151c32d-a9cc-4214-8f3e-e9b58ad23a80","name":"Date","type":"DATE","size":10,"precision":10,"scale":0,"hidden":false,"custom":false,"indexed":true},"3ff382e2-b1b9-4f1f-a875-5a6291fcc053":{"oid":"3ff382e2-b1b9-4f1f-a875-5a6291fcc053","name":"Desc","type":"TEXT","size":50,"precision":50,"scale":0,"hidden":false,"custom":false,"indexed":true},"f529d5f5-1c23-4f77-a1e6-fac21cc8bfaf":{"oid":"f529d5f5-1c23-4f77-a1e6-fac21cc8bfaf","name":"Test","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true}}},"e1d22688-59f9-4733-8553-906d2393ab2b":{"oid":"e1d22688-59f9-4733-8553-906d2393ab2b","name":"consumable_products","hidden":false,"custom":false,"columns":{"81c057d5-9492-4788-8e45-cbd70cde11b0":{"oid":"81c057d5-9492-4788-8e45-cbd70cde11b0","name":"Column 1","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true}}}}},"logicalEtlt":{"oid":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","connections":{"4015b408-adf6-3460-80ce-0e75d27f40e4":{"oid":"4015b408-adf6-3460-80ce-0e75d27f40e4","provider":"MySql","schema":"SISENSE","timeout":300,"parameters":{"Database":"SISENSE"}}},"baseTables":{"e1d22688-59f9-4733-8553-906d2393ab2b":{"oid":"e1d22688-59f9-4733-8553-906d2393ab2b","connectionOid":"4015b408-adf6-3460-80ce-0e75d27f40e4","dataSourceSchemaName":"SISENSE","dataSourceTableName":"consumable_products","columnMapping":{"81c057d5-9492-4788-8e45-cbd70cde11b0":"Column 1"},"buildBehavior":{"type":"sync"},"optimize":false},"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c":{"oid":"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c","connectionOid":"4015b408-adf6-3460-80ce-0e75d27f40e4","dataSourceSchemaName":"SISENSE","dataSourceTableName":"addresses_db","columnMapping":{"85f500c3-b3b3-4ea7-9136-90ed95f1e945":"Column 1"},"buildBehavior":{"type":"sync"},"optimize":false},"d5279f72-f5ed-43ef-b18a-5c3d543f58a0":{"oid":"d5279f72-f5ed-43ef-b18a-5c3d543f58a0","connectionOid":"4015b408-adf6-3460-80ce-0e75d27f40e4","dataSourceSchemaName":"SISENSE","dataSourceTableName":"Employees","columnMapping":{"7a807ddb-91fb-452a-a225-6c9a1153fdaf":"EmployeeID","a9a342f9-2c02-4dcd-8745-56d720edc713":"First Name","0f9fbafa-6c6f-4f52-9c1c-803868d69d52":"Last Name","a151c32d-a9cc-4214-8f3e-e9b58ad23a80":"Date","3ff382e2-b1b9-4f1f-a875-5a6291fcc053":"Desc","f529d5f5-1c23-4f77-a1e6-fac21cc8bfaf":"Test"},"buildBehavior":{"type":"sync"},"optimize":false}}}} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Analytics: AnalyticsData","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"build-analytics-uploading-0","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","logicalSchema":{"oid":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","name":"TestMySQL","schemaId":"4830f677-d1b3-3a95-9888-91fcacca2b20","tables":{"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c":{"oid":"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c","name":"addresses_db","hidden":false,"custom":false,"columns":{"85f500c3-b3b3-4ea7-9136-90ed95f1e945":{"oid":"85f500c3-b3b3-4ea7-9136-90ed95f1e945","name":"Column 1","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true}}},"d5279f72-f5ed-43ef-b18a-5c3d543f58a0":{"oid":"d5279f72-f5ed-43ef-b18a-5c3d543f58a0","name":"Employees","hidden":false,"custom":false,"columns":{"7a807ddb-91fb-452a-a225-6c9a1153fdaf":{"oid":"7a807ddb-91fb-452a-a225-6c9a1153fdaf","name":"EmployeeID","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true},"a9a342f9-2c02-4dcd-8745-56d720edc713":{"oid":"a9a342f9-2c02-4dcd-8745-56d720edc713","name":"First Name","type":"TEXT","size":65535,"precision":65535,"scale":0,"hidden":false,"custom":false,"indexed":false},"0f9fbafa-6c6f-4f52-9c1c-803868d69d52":{"oid":"0f9fbafa-6c6f-4f52-9c1c-803868d69d52","name":"Last Name","type":"TEXT","size":65535,"precision":65535,"scale":0,"hidden":false,"custom":false,"indexed":false},"a151c32d-a9cc-4214-8f3e-e9b58ad23a80":{"oid":"a151c32d-a9cc-4214-8f3e-e9b58ad23a80","name":"Date","type":"DATE","size":10,"precision":10,"scale":0,"hidden":false,"custom":false,"indexed":true},"3ff382e2-b1b9-4f1f-a875-5a6291fcc053":{"oid":"3ff382e2-b1b9-4f1f-a875-5a6291fcc053","name":"Desc","type":"TEXT","size":50,"precision":50,"scale":0,"hidden":false,"custom":false,"indexed":true},"f529d5f5-1c23-4f77-a1e6-fac21cc8bfaf":{"oid":"f529d5f5-1c23-4f77-a1e6-fac21cc8bfaf","name":"Test","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true}}},"e1d22688-59f9-4733-8553-906d2393ab2b":{"oid":"e1d22688-59f9-4733-8553-906d2393ab2b","name":"consumable_products","hidden":false,"custom":false,"columns":{"81c057d5-9492-4788-8e45-cbd70cde11b0":{"oid":"81c057d5-9492-4788-8e45-cbd70cde11b0","name":"Column 1","type":"INT","size":11,"precision":11,"scale":0,"hidden":false,"custom":false,"indexed":true}}}}},"logicalEtlt":{"oid":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","connections":{"4015b408-adf6-3460-80ce-0e75d27f40e4":{"oid":"4015b408-adf6-3460-80ce-0e75d27f40e4","provider":"MySql","schema":"SISENSE","timeout":300,"parameters":{"Database":"SISENSE"}}},"baseTables":{"e1d22688-59f9-4733-8553-906d2393ab2b":{"oid":"e1d22688-59f9-4733-8553-906d2393ab2b","connectionOid":"4015b408-adf6-3460-80ce-0e75d27f40e4","dataSourceSchemaName":"SISENSE","dataSourceTableName":"consumable_products","columnMapping":{"81c057d5-9492-4788-8e45-cbd70cde11b0":"Column 1"},"buildBehavior":{"type":"sync"},"optimize":false},"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c":{"oid":"4f9a1cbc-c3f2-4abc-9744-fe3c7e3be18c","connectionOid":"4015b408-adf6-3460-80ce-0e75d27f40e4","dataSourceSchemaName":"SISENSE","dataSourceTableName":"addresses_db","columnMapping":{"85f500c3-b3b3-4ea7-9136-90ed95f1e945":"Column 1"},"buildBehavior":{"type":"sync"},"optimize":false},"d5279f72-f5ed-43ef-b18a-5c3d543f58a0":{"oid":"d5279f72-f5ed-43ef-b18a-5c3d543f58a0","connectionOid":"4015b408-adf6-3460-80ce-0e75d27f40e4","dataSourceSchemaName":"SISENSE","dataSourceTableName":"Employees","columnMapping":{"7a807ddb-91fb-452a-a225-6c9a1153fdaf":"EmployeeID","a9a342f9-2c02-4dcd-8745-56d720edc713":"First Name","0f9fbafa-6c6f-4f52-9c1c-803868d69d52":"Last Name","a151c32d-a9cc-4214-8f3e-e9b58ad23a80":"Date","3ff382e2-b1b9-4f1f-a875-5a6291fcc053":"Desc","f529d5f5-1c23-4f77-a1e6-fac21cc8bfaf":"Test"},"buildBehavior":{"type":"sync"},"optimize":false}}}}
AbstractTaskBE#539145Start the task to update cube models in MongoDB.BE#539145 Starting task [Storing New Cube Models] under group 27364933-0325-41c3-8bbe-e28c79f19244BE#539145 Starting task [Storing New Cube Models] under group 27364933-0325-41c3-8bbe-e28c79f19244
RequestFilterBE# 683109Send a request to update MongoDB with additional data. BE# 683109 Start handling request - url=http://build:8086/internal/, userId=, userRole=BE# 683109 Start handling request - url=http://build:8086/internal/, userId=, userRole=
RequestFilterBE#587104Finished handling the request.BE#587104 Finished handling request - http://build:8086/internal/BE#587104 Finished handling request - http://build:8086/internal/
AbstractTaskBE#029473The task executed successfully.BE#029473 Task [Storing New Cube Models] result status: COMPLETEDBE#029473 Task [Storing New Cube Models] result status: COMPLETED
RpcCallerBE#194580BE#194580 RPC call (HTTP), url="http://oxygen:8457/license/get", status=200 "OK" took 7 msBE#194580 RPC call (HTTP), url="http://oxygen:8457/license/get", status=200 "OK" took 7 ms
DPAAnalyticsDataSecurityClassifierBE#563355BE#563355 License hasDPA=trueBE#563355 License hasDPA=true
StorageDataServiceImplBE#801091Update the data schema in MongoDB.BE#801091 DataSchema - <Cube ID>length is <data size>BE#801091 DataSchema - e51d6ac3-819c-48f7-b1c4-7003f8cdff7e length is 8982
AbstractTaskBE#539145Start the task that will store the metadata into the file in the farm of the built cube. This contains the summary of how many records were processed into the cube for each table.BE#539145 Starting task [Store metadata] under group 27364933-0325-41c3-8bbe-e28c79f19244BE#539145 Starting task [Store metadata] under group 27364933-0325-41c3-8bbe-e28c79f19244BE#131601 Build is not successfully completed, no need to save metadataBE#036852 Failed to write BuildMetaData.jsonBE#798519 Failed to write Failed to store Build metadata fileIf the build fails, metadata will not be stored in the storage.

This indicates that there was a failure writing the build metadata to the storage. In this case, it was the BuildMetaData.json file.
AbstractTaskBE#029473Save the model's metadata in the storage.BE#029473 Task [Store metadata] result status: COMPLETEDBE#029473 Task [Store metadata] result status: COMPLETED
TaskGroupBE#345155Set the build workplan to complete.BE#345155 Group <BuidID> workplan state set to CompletedBE#345155 Group 27364933-0325-41c3-8bbe-e28c79f19244 workplan state set to CompletedBE#345155 Group <BuidID> workplan state set to Failed
BuildsTrackerBE#312940The build has completed and will therefore be removed from the queue. This action is performed if the build finished successfully or was manually/automatically canceled.BE#312940 cleaning builds tracker for data source id: e51d6ac3-819c-48f7-b1c4-7003f8cdff7eBE#312940 cleaning builds tracker for data source id: e51d6ac3-819c-48f7-b1c4-7003f8cdff7e
BuildsTrackerBE#434466Remove the Build ID from the memory map that maintains the list of queued builds.BE#434466 Removing build id BuildID(buildId=<Build ID>) from mapsBE#434466 Removing build id BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244) from maps
BuildsTrackerBE#986898Remove the Cube ID from the cube maps that are being processed for the build.BE#986898 Removing data source id <Cube ID> from running initiate mapBE#986898 Removing data source id e51d6ac3-819c-48f7-b1c4-7003f8cdff7e from running initiate map
BuildServiceBE#195426Release the build plan because it has completed.BE#195426 Releasing build plan from queue: <BuildID>, CompletedBE#195426 Releasing build plan from queue: 27364933-0325-41c3-8bbe-e28c79f19244, Completed
Indicates the end of the build and provides some details and performance statistics about the build. {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"EndBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","buildStep":"buildCubeEnd","endBuildTimeStamp":"2021-10-26T08:54:36.804415","duration":38.066} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"EndBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"buildCubeProcess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","buildStep":"buildCubeEnd","endBuildTimeStamp":"2021-10-26T08:54:36.804415","duration":38.066}
{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"FinishBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"finishBuildCube","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","endBuildTimeStamp":"2021-10-26T08:54:36.804415","duration":38.066,"dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","status":"Completed","lastBuildTime":"2021-10-26T08:54:07.724194Z","concurrentBase":4,"concurrentCustom":1,"parallelBaseAndCustom":false,"Num of Tables":3,"isSampleDataBuild":false} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"FinishBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"finishBuildCube","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","endBuildTimeStamp":"2021-10-26T08:54:36.804415","duration":38.066,"dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","status":"Completed","lastBuildTime":"2021-10-26T08:54:07.724194Z","concurrentBase":4,"concurrentCustom":1,"parallelBaseAndCustom":false,"Num of Tables":3,"isSampleDataBuild":false}
{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"SummarizeBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"ecs_buildsuccess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","endBuildTimeStamp":"2021-10-26T08:54:36.804415","duration":38.066,"dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","ecid":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","countTables":3,"countDimensionTables":0,"ecbcntcols":8,"ecbcntcc":0,"ecbcbtct":0,"cloudinfo":{"tableInfo":{"base":{"addedRows":9,"maxRows":9,"totalRows":9,"medianNumberOfRows":0,"count":3}},"datasources":{"uniqueProviders":"MySql","countedProviders":[{"providerName":"MySql","connectorArch":"Old","count":3}]}},"dur":38.066,"ecbinitialphysicalsize":807057,"ecbbuildtype":"full","ecbschematype":"schema-changes","ecbbuildnor":-1,"ecbramsize":0,"ecbfreeram":0,"ecbhddsize":0,"ecbfreehdd":0,"trk_id":"8877aafd-5f4f-4a08-ad53-b7c49c23dc6d","prod":"dataserver","cat":"ecb_end"} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"SummarizeBuild","Log_Level":"INFO","Log_Component":"build","Log_DateTime":"2021-10-26T08:54:36Z","Log_Thread":"task-coordinator-4-LEVEL3","appTypeName":"build","Log_TransactionID":"df0e52b1-8090-4735-bd1f-cfaca2b25ea8","action":"ecs_buildsuccess","buildPublishType":"full","cubeID":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","cubeName":"TestMySQL","controlServerIP":"10.233.47.118","controlServerHostname":"build-5cc98bfbfc-sx4rf","source":"NextECM","startBuildTimeStamp":"2021-10-26T08:53:58.737714","BuildId":"27364933-0325-41c3-8bbe-e28c79f19244","endBuildTimeStamp":"2021-10-26T08:54:36.804415","duration":38.066,"dataServerIP":"10.233.101.216","dataServerHostName":"ec-testmysql-bld-376cc277-ccbf-5","ecid":"e51d6ac3-819c-48f7-b1c4-7003f8cdff7e","countTables":3,"countDimensionTables":0,"ecbcntcols":8,"ecbcntcc":0,"ecbcbtct":0,"cloudinfo":{"tableInfo":{"base":{"addedRows":9,"maxRows":9,"totalRows":9,"medianNumberOfRows":0,"count":3}},"datasources":{"uniqueProviders":"MySql","countedProviders":[{"providerName":"MySql","connectorArch":"Old","count":3}]}},"dur":38.066,"ecbinitialphysicalsize":807057,"ecbbuildtype":"full","ecbschematype":"schema-changes","ecbbuildnor":-1,"ecbramsize":0,"ecbfreeram":0,"ecbhddsize":0,"ecbfreehdd":0,"trk_id":"8877aafd-5f4f-4a08-ad53-b7c49c23dc6d","prod":"dataserver","cat":"ecb_end"}
TasksCoordinatorImplBE#753071Indicates successful completion of the build.BE#753071 Execution DONE: groupId=<Build ID>, state=CompletedBE#753071 Execution DONE: groupId=27364933-0325-41c3-8bbe-e28c79f19244, state=Completed
TasksCoordinatorImplBE#010801Optional

If the build is canceled, a request to cancel the overall task group for the build is invoked.
BE#010801 Cancelled task group <Build ID>: <Status>BE#010801 Cancelled task group 27364933-0325-41c3-8bbe-e28c79f1924: DONEBE#193546 FAILED TO STOP EXECUTION: groupId=<Build ID>
Provides data about the number of items handled in the process, including base tables processed.{"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Metrics SchemaMetricType","Log_Level":"INFO","Log_Component":"translation","Log_DateTime":"2021-10-26T08:56:47Z","Log_Thread":"MetricsCollector","appTypeName":"translation","DatasourcesCount_avg":1.0,"DatasourcesCount_count":2,"DatasourcesCount_max":1.0,"DatasourcesCount_min":1.0,"RelationshipsCount_avg":0.0,"RelationshipsCount_count":2,"RelationshipsCount_max":4.9E-324,"RelationshipsCount_min":0.0,"SchemaSize_avg":8981.0,"SchemaSize_count":2,"SchemaSize_max":8981.0,"SchemaSize_min":8981.0,"TablesCount_avg":4.0,"TablesCount_count":2,"TablesCount_max":4.0,"TablesCount_min":4.0,"ViewsCount_avg":3.0,"ViewsCount_count":2,"ViewsCount_max":3.0,"ViewsCount_min":3.0} {"Log_Type":"Structured","Log_Version":"2.0","Log_Message":"Metrics SchemaMetricType","Log_Level":"INFO","Log_Component":"translation","Log_DateTime":"2021-10-26T08:56:47Z","Log_Thread":"MetricsCollector","appTypeName":"translation","DatasourcesCount_avg":1.0,"DatasourcesCount_count":2,"DatasourcesCount_max":1.0,"DatasourcesCount_min":1.0,"RelationshipsCount_avg":0.0,"RelationshipsCount_count":2,"RelationshipsCount_max":4.9E-324,"RelationshipsCount_min":0.0,"SchemaSize_avg":8981.0,"SchemaSize_count":2,"SchemaSize_max":8981.0,"SchemaSize_min":8981.0,"TablesCount_avg":4.0,"TablesCount_count":2,"TablesCount_max":4.0,"TablesCount_min":4.0,"ViewsCount_avg":3.0,"ViewsCount_count":2,"ViewsCount_max":3.0,"ViewsCount_min":3.0}

MANAGEMENT SERVICE

Class Message # Description Successful Log entry Example Possible Error Messages Exception Handling/Potential Root Cause Workaround/Solution
PHASE - INITIALIZATION
POD - MANAGEMENT
LOG: management.log

The request is received by the management pod to bring up a bld-pod to handle the build request.
Receive the request from Build to validate that all is ok to create the cube. (Note: These steps will occur when the Build service receives the request for a build, before it fully completes processing the request and requests the management pod to initiate brining up the pod.
RequestFilter BE#683109 This request is received from the build pod immediately upon the build receiving a build request. The build pod is not yet created at this point.

The request is routed via a REST API call.
BE#683109 Start handling request - url=http://management.sisense:8082/internal/, userId=<UserID>, userRole=<UserRole> BE#683109 Start handling request - url=http://management:8082/internal/, userId=6124c5dc2b42f5001ad4541a, userRole=super
ManagementService BE#007911 Register the build request, based on the build ID. However, the process is not starting yet. Prerequisite checks will be performed to ensure readiness for the build. If anything fails, the build will be canceled.

Check if this is a build of a new cube or if it is an existing query cube.

Also, checks are performed to ensure that the platform is ready to process a build including:
  1. The data group assigned to cube
  2. The platform is not halted from performing EC and build type activities
  3. Whether the disk has breached a utilization threshold
  4. Backup/restore activities are being performed
BE#007911 Start build id <BuildID> BE#007911 Start build id e51d6ac3-819c-48f7-b1c4-7003f8cdff7e BE#101507 cannot find data group and default data group for <CubeID>BE#918092 EC import/build temporally haltedBE#027859 Will not perform import/build due to insufficient disk resourcesBE#613453 Will not perform import/build due to back up activityBE#605131 Will not perform import/build due to restore activity The Cube ID is not found or is invalid.

Returned System EC import and build activity halted by management.

Disk breached its utilization limit.

Backup process is running and the build cannot take place.

A restore process is running and the build cannot take place.
Check to see whether the Cube ID has been deleted from MongoDB.

You might need to duplicate/clone the cube and perform a new fuil build to rectify the build issue. Also, make sure that the cube is assigned a Data Group.

Check the reason for the build failure and enable the system to resume operations.

Perform the disk utilization analysis outlined in the Disk Monitoring process in this document.

Wait for the backup activity (if it is taking place) to finish.

Wait for the restore activity (if it is taking place) to finish.
RequestFilter BE#587104 Request processing has completed. BE#587104 Finished handling request - <RequestURL> BE#587104 Finished handling request - http://management:8082/internal/
The Management Pod receives the request to create the build pod (EC-BLD)
RequestFilter BE#683109 Receiving a request from the build to initiate the build process, which will start the creation of the build cube. BE#683109 Start handling request - url=<request url>, userId=<UserID>, userRole=<user role> BE#683109 Start handling request - url=http://management.sisense:8082/internal/asyncRequest, userId=6124c5dc2b42f5001ad4541a, userRole=super
InternalRestController BE#984150 Send the request to create a BLD pod, then create the pod for the build of the cube. BE#984150 got async request - client=<client ip>, service=ManagementService, method=getCubeForBuild, requestId <request ID sent to management pod by build> BE#984150 got async request - client=10.233.101.198, service=ManagementService, method=getCubeForBuild, requestId 30ecc294-fd8e-4bc5-8825-6b14cb42dea0 E#750123 exception during method execution - service=ManagementService; method=getCubeForBuild An exception occurred while invoking this request. Check the Management Service status by describing the pod and see if any issues are noted.
RequestFilter BE#587104 Finished processing the request. BE#587104 Finished handling request - <RequestURL> BE#587104 Finished handling request - http://management.sisense:8082/internal/asyncRequest
ManagementService BE#934758 The parameters received contain the information required for build, including the schema details for the cube. This includes the schema, connection details to external data sources, etc.

The message indicates the type of build:

intentType = NEW (indicates this is a full build, live cube, or an acceleration)intentType = EXIT (indicates the build is accumulative or a schema change to the model)

The message also indicates whether the request originated from the GUI which is always a "synch" request or via SI which could be "synch" or "async" based on what was selected in the request.
BE#934758 getCubeForBuild, request:CreateCubeForBuildRequest(cubeName=<Cube Title>, ecmModel=EcmModel(oid=<Cube ID>, title=<Cube Title>, schemaId=<Schema ID>, server=LocalHost, lastBuildTime=<Last Time the build for cube ran>, lastBuildAttempt=null, lastUpdated=<Last time cube was updated>, datasets=[Dataset(name=<Data Set Name in MongoDB for each cube table set>, oid=<Data Set Doc ID in MongoDB for each cube table set>, type=extract, lastUpdated=<Last time dataset was updated>, schema=Schema(id=null,Table(oid=<MongoDB Doc ID for the given logical table>, id=<table id>, name=<table name>, type=<base or custom>, lastUpdated=2021-09-09T08:22:35.117Z, hidden=<hidden in the setup>, columns=[Column(id=<Column ID>, name=<Column Name>, type=<Column Data Type>, size=11, precision=11, scale=0, oid=<Column ID MongoDB>, hidden=<Set to be hidden in setup>, indexed=<Indexed Column>)], tupleTransformations=[], buildBehavior=BuildBehavior(type=<synch/accumulativeSync, accumulativeConfig=AccumulativeConfig(column=<Accumulate By Column ID in MongoDB>, type=<Data Type>)), expression=null, vTag=null, schemaName=<External Data Source Schema>, configOptions=<what options??, customCode=<Indicates if custom code is used)]),connection=Connection(provider=<provider>, schema=<Schema Name>, resultLimit=null, timeout=<Timeout setting>, parameters=Parameters (7)))], relations=[], modelingTransformations=null, buildDestination=elasticube), intentType=<New Build NEW or EXIT for existing>, buildID=BuildID(buildId=<Build ID>), requiredConnectors=[<Provider>], isCustomCodeRequired=<indicates if there is custom code in place or not>) BE#934758 getCubeForBuild, request:CreateCubeForBuildRequest(cubeName=TestMySQL, ecmModel=EcmModel(oid=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, title=TestMySQL, schemaId=0545564c-1341-3920-96eb-e19021de1b02, server=LocalHost, lastBuildTime=2021-10-20T14:21:42.912Z, lastBuildAttempt=null, lastUpdated=2021-09-09T08:22:35.119Z, datasets=[Dataset(name=0cfac4ad-e553-44e9-b536-c5e966252231, oid=4ad279b7-dfb7-49c9-9bd4-32ad273465af, type=extract, lastUpdated=2021-10-26T08:53:56.499Z, schema=Schema(id=null,schema=Schema(id=null,Table(oid=e1d22688-59f9-4733-8553-906d2393ab2b, id=employees, name=employees, type=base, lastUpdated=2021-09-09T08:22:35.117Z, hidden=false, columns=[Column(id=First Name, name=First Name, type=8, size=11, precision=11, scale=0, oid=81c057d5-9492-4788-8e45-cbd70cde11b0, hidden=false, indexed=true)], tupleTransformations=[], buildBehavior=BuildBehavior(type=accumulativeSync, accumulativeConfig=AccumulativeConfig(column=caad4cfb-6a03-4af9-97a0-699589cd1eb8, type=null)), expression=null, vTag=null, schemaName=SISENSE, configOptions=null, customCode=null)]),connection=Connection(provider=MySql, schema=SISENSE, resultLimit=null, timeout=300, parameters=Parameters (7)))], relations=[], modelingTransformations=null, buildDestination=elasticube), intentType=NEW, buildID=BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244), requiredConnectors=[MySql], isCustomCodeRequired=false) BE#534117 Query cube with name <Cube Title> not found"

BE#550327 Cube <Cube Title> is busy, new build request failed

BE#365717 missing index size, using default size: SHORT
This is a non critical message, and does not affect the continuation of the build.

Since the query pod is busy and cannot be brought down, the build will fail.

This error will appear if no index is defined in the data group that the cube is associated with (SHORT or LONG). In this case, the default of SHORT will be used. The build will continue.
ManagementService BE#969994 Optional - Only applicable for the building of an existing cube.

It indicates that the build is starting for a cube that already exists.
BE#969994 Starting a build of exiting cube: <Cube Name> BE#969994 Starting a build of exiting cube: TestMySQL BE#948590 Completing previous build as failed. cube info: <cube info> The build process denies placing more than one build request on the same cube simultaneously. However, the management pod protects this by "force" failing the cube build process, and deleting anything partially built, so we can start the more recent build request being processed here. In many of the cases outlined here, the solution would be to execute a full build that will recreate the storage from scratch and start a new build process.
ManagementService BE#762768 BE#762768: Indicates that the cube is getting created for the first time, NEW. BE#762768 building new cube <Cube Title> BE#762768 building new cube TestMySQL BE#425269 IntentType, Not implemented yet request: <requested build behavior>BE#590302 Single build pod should be on a single machine Internal Error. This is when, for some reason, the build is not that of a new or existing cube (not likely to happen).

The actual exception that is stated is not likely to happen since the same BLD cannot run at the same time on two different nodes. However, it points to a potential issue in which the BLD pod for the cube being built has failed to terminate and, therefore, the system denies bringing up a new BLD pod as long as the one (in terminating mode) exists.

Run the following command to see if there is a stuck pod for the cube still running (look for the instance name of the cube):

kubectl -n sisense get pods -l mode=build -w
n/a


You will need to terminate the BLD pod with the following command, which will force delete the pod. Then, you can rerun the build, which should now run:

kubectl -n <namespace> delete pod <the BLD pod that is stuck in terminating mode> --grace-period=0 --force
ManagementService BE#699155 BE#699155: Indicates that the cube already exists and has been previously built already. BE#699155 building Existing cube <Cube Name> BE#001903 Cannot find such cube in dbBE#349721 Failed to prepare files for build <requested details received from build service request to build>BE#367876 failed parsing or missing LastUpdatedTime: <returned value from cube information> The cube is not found in MonetDB, resulting in the build failing.

This is the case where the farm is not able to be prepared for the cube. It could happen when the build is set to already have a farm in place with the fold name being <Cube URI Location>_next" and it does not exist.

When there is no LastUpdateTime in the cube metadata record in MongoDB, the build will continue.
You will need to perform a full build to resolve this.

The solution is to either do a full build or switch off the "Do Full Copy Instead of Nbab" in "service Configuration >> Management". configuration. After the build is complete, return the configuration to on.
ManagementService BE#840887 This will show that when an existing cube is being built, and "Do Full Copy Instead of Nbab" is configured off, the URI storage location will be updated. BE#840887 Moving _next folder to new location BE#840887 Moving _next folder to new location n/a n/a n/a
ManagementService BE#956131 If no folder for the cube exists on the storage, the folder is duplicated from the original cube. BE#956131 Old instance does not hold such folder, duplicating from original: <uri storage farm location>BE#167341 <existing uri storage farm location> Duplicated to <new uri storage farm location> in: <copy duration> seconds BE#956131 Old instance does not hold such folder, duplicating from original: /opt/sisense/storage/farms/aMySQLTest_2022.01.19.14.17.44.425BE#167341 /opt/sisense/storage/farms/aMySQLTest_2022.01.17.12.20.53.331 Duplicated to /opt/sisense/storage/farms/aMySQLTest_2022.01.19.11.52.18.154 in: 0.594 seconds] BE#084547 The cp process from directory: <old farm folder - URI> to directory: <new farm folder - URI> didn't successBE#559694 Failed to copy <old farm folder - URI> to directory: <new farm folder - URI>
ManagementService BE#820743 Indicates that the data group that is associated with the cube is set to build in local storage. BE#820743 using local storage: <URI - location of folder in local storage> BE#820743 using local storage: opt/sisense/storage/farms/aECIAAaTest_2021.06.03.09.07.30.165
ManagementService BE#751435 Indicates that the data group associated with the cube is set to build on S3 with the URI (the location in the storage). BE#751435 Accumulative build on S3 with new uri <location of folder in S3> BE#751435 Accumulative build on S3 with new uri <location of folder in S3>
ManagementService BE#956131OR Optional - For Accumulative/Change Schema only:

BE#956131

This is for the case where the build behavior is of a type that relies on performing a comparison between the existing cube data and model to the newly defined data set (e.g., Change Only in Schema, Accumulative)

If a folder (a cube in a farm) is not found, the farm will be duplicated.

Note that if the mechanism for preparing the cube for the next build (e.g., the duplication of the farm was already done in the previous build), this step is not required.

BE#840887

There is already a farm that can be used for this new build, and that location will be used for this build (URI).
BE#956131 Old instance does not hold such folder, duplicating from original: <location of folder in farm>ORBE#840887 Moving _next folder to new location BE#956131 Old instance does not hold such folder, duplicating from original: opt/sisense/storage/farms/aECIAAaTest_2021.06.03.09.07.30.165
HostAgentHandlerImpl BE#884580 Optional - For Accumulative/Change Schema only BE#884580 copy directory: <location of existing farm> to : <new farm location> BE#884580 copy directory: opt/sisense/storage/farms/aECIAAaTest_2021.06.03.09.07.30.165 to : /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080] BE#845264 failed copy <original frame location> to <new location>BE#364023 cleaning up <new farm location> Note: The copy command, which is sent for execution on the storage that contains the farm, might take a long time and, therefore, could lead to the expiration of the process that is waiting for it to complete. For example, assuming it takes a very long time (due to a storage issue) for the copy to be performed, it is possible that the Management pod will hang/restart.

A failure here does not necessarily mean that an error will appear in the log and/or the Management pod might not time out this process and, therefore, will be late in responding to the Build Service that is waiting for it to finish the process.
If this process is taking a long time, this could potentially be a storage speed issue (taking a long time to copy the farm), a non-stable connection to shared storage, or any other latency that is affecting the completion of this operation in time.

If possible, assess the storage to further investigate. You can perform the copy manually within the storage and see if the same operation takes as long.

Also, is there another build or other operation keeping the storage busy around the same time?

Assess the speed on local storage and, if better, suggest a move to Local Storage solution for the build and/or query.
ManagementService BE#167341 Optional - For Accumulative/Change Schema only: BE#167341 : <location of existing farm> Duplicated to <new farm location> in <# of seconds> seconds BE#167341 opt/sisense/storage/farms/aECIAAaTest_2021.06.03.09.07.30.165 Duplicated to /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080] in: 10 seconds
HostAgentHandlerImpl BE#318099 Optional - For Full Build:

Create a new folder for the cube and subfolders, and indicate the location of the created folder for the build.

The new storage path will be created in the configured farm location, and the cube farm folder naming convention will be the "Cube Title" + DateTime (formatted with yyyy.MM.dd.HH.mm.ss.SSS).
BE#318099 uri:/<location of folder in farm for new build> BE#318099 uri:/opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080 BE#708572 Failed to create dirs <full path>/sql_logs Failed to create the subdirectory for SQL logs.
ManagementService BE#658227 Start building the cube, and note the cube title being built.

The cube is created with the environment resource utilization restrictions set in Data Group, including:
  1. Limit CPU Cores
  2. Core Usage Percentage
  3. Affinity setting if enabled for a specific node(s) for build.
Also, Kubernetes will be asked to monitor the pod that is being created, and report on status of pod initiating, based on the value set in the configuration. The configuration is in "Base Configuration" under the "KubernetesLatencyThresholdReport.value" parameter, whose value is in seconds.
BE#658227 startCubeForBuild. Cube title <Cube name> BE#658227 startCubeForBuild. Cube title TestMySQL BE#538215 startCubeForBuild failed - cube=<Cube Title>"BE#957647 no datagroupBE#714343 Pod with same name = <Cube Title> already exists. Delete it before creating a new onBE#649148 Pod with same name = {} already exists and can not be deleted. Pod creation for Build failed Some request responses returned to the Build service are:
  • Failed to get data group
  • Could not deploy cube for build <error description>
  • Failed to create configuration files
  • Unsupported index size - A value that is not compatible is set in the Data Group.
  • Failed to create service - not able to create a pod service
This will appear if the Build pod was already initiated, or for some reason still exists, and an error will be raised if it is not able to be shut down, which in turn will fail.
CubeProcessManagerImpl BE#113228 Indicates the location of the build if the build is set to run an interim build on local. Indicates whether the query is set to local or not. BE#113228 BUILD_ON_LOCAL_STORAGE is used for: <Cube title>, local path=<farm path on local storage>, with QUERY_ON_LOCAL set to <Indicates if query is set to run on local based on data group, true or false>". BE#113228 BUILD_ON_LOCAL_STORAGE is used for: TestMySQL, local path=/opt/sisense/local_storage, with QUERY_ON_LOCAL set to <true>".
CubeProcessManagerImpl BE#853877 Indicates the S3 location of the file used when a cumulative build is performed on S3. BE#853877 Accumulative build on S3 : <indicates location on S3>. BE#853877 Accumulative build on S3 :
CubeProcessManagerImpl BE#792649 Specify the name of the cube received from Kubernetes.

The cube name is constructed, based on the naming convention "ec-<Cube title>-bld-<instance id of cube>.
BE#792649 name: <Build ID> BE#792649 name: ec-testmysql-bld-376cc277-ccbf-5 info: bld-376cc277-ccbf-5
MonetDbConfigManagerImpl BE#297342 Get the parameters required to access MonetDB for the cube build. BE#297342 Monet command: --zk_system_name=<?>--zk_address=<zookeeper address> --external_server=<?> --instance_id=<instance id of cube> --dbname=<MonetDB name, Schema ID> --farmstate=Building --dbfarm=<folder location in farm> --set mapi_port=<mapi port> BE#297342 Monet command: --zk_system_name=S1 --zk_address=sisense-zookeeper.sisense:2181 --external_server=ec-testmysql-bld-376cc277-ccbf-5.sisense--instance_id=bld-33e277fd-7f90-9 --dbname=aTestMySQL--farmstate=Building --dbfarm=/opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080/dbfarm--set mapi_port=50000
CubeProcessManagerImpl BE#792649 Indicates the build ID and cube instance ID being processed.

This message will repeat itself (sort of pinging the process) until the cube is up.
BE#792649 name: <build id> info: <instance id of cube> BE#792649 name: ec-testmysql-bld-376cc277-ccbf-5 info: bld-376cc277-ccbf-5
CubeProcessManagerImpl BE#249583 Indicates that S3 is being used for the build, based on the configuration in "Service Configuration >> Management" under the parameters:"S3 Bucket" and " S3 Path" and the Data Group configuration is set with S3 enabled. BE#249583 using s3 storage with bucket : <S3 Bucket> and path : <S3 Path> BE#249583 using s3 storage with bucket : sisensetest and path : sisense/Test
CubeProcessManagerImpl BE#768721 Sets the affinity of the pod to an appropriate node - by default, a node that has a build label associated with it - unless the data group configuration has a node(s) specific affinity setting. BE#768721 Build affinity based data group label BE#768721 Build affinity based data group label
CubeProcessManagerImpl BE#840386 Get the memory values set in the Data Group associated with the cube. This includes the Build Node setting for: Reserved RAM (MB), Max RAM (MB).

Note: If values are set to none for any of the parameters (e.g., -1), no limits will be set and, therefore, will not be written to the log.

(At this point there are no additional parameters).
BE#840386 ResourceRequirements: ResourceRequirements(limits={memory=<MAX Memory>M, cpu=<Max Cores>}, requests={memory=<Reserved Memory>M, cpu=<Reserved Cores>}, additionalProperties={}) BE#840386 ResourceRequirements: ResourceRequirements(limits={memory=8192M, cpu=10.0}, requests={memory=512M, cpu=5.0}, additionalProperties={})]
CubeProcessManagerImpl BE#535421 Pulls the Docker secret required to provide access to Kubernetes.

The secret is optional, depending on how Kubernetes was deployed.
BE#716209 using secret: <docker secret configured to access kubernetes>ORBE#535421 secret was not provided in DOCKER_SECRET env, deploying without secret BE#716209 using secret: {}BE#535421 secret was not provided in DOCKER_SECRET env, deploying without secret
CubeProcessManagerImpl n/a The LivenessProbe is a mechanism used to instruct Kubernetes to monitor the deployed EC BLD pod and check that it is responsible throughout the life of the pod.

If "Elasticube Liveness" is true, Kubernetes will be set to monitor the cube using Kubernetes Liveness with the following parameters:
  • Liveness Initial Delay (seconds) - The time provided for the pod to start
  • Liveness Timeout (seconds - The time allowed for the pod to respond before the probing times out
  • Liveness Period (seconds) - The frequency at which liveness is probed.
  • Liveness Failure Threshold - The number of times Kubernetes will try to test for liveness before assuming a failure.
The ReadinessProbe is a mechanism used to instruct Kubernetes to monitor the deployed EC BLD Pod during the pod startup.

If "Elasticube Readiness" is set to true, Kubernetes will be set to monitor the cube using Kubernetes Readiness, using the following configuration parameter settings:
  • Readiness Initial Delay (seconds) - The number of seconds between container start and probe start to allow for services to initialize.
  • Readiness Timeout (seconds) - The timeout for probe responses.
  • Readiness Period (seconds) - The frequency of readiness tests.
  • Readiness Failure Threshold - The number of consecutive failed results needed to switch the probe status to “Failure”.
The configuration is set in the "Service Configuration >> Management" section of the configuration manager. These values can be adjusted when we want to troubleshoot build performance issues when the MonetDB EC BLD pod is deployed. For example, increasing the time to allow the pod to load.
Note the below configuration is written as part of the service load, but is not written to the log during the build. n/a
CubeProcessManagerImpl n/a If the build method is to MonetDB (BUILD_TO_MONET), deploy a MonetDB container with the following parameters: name, dataSourceTitle, buildId, deployment, resourceRequirements,readinessProbe, livenessProbe, lifecycle, args, envList, port).

Another build method is "BUILD_TO_DESTINATION", which builds to a destination such as SnowFlake directly.
n/a n/a BE#627094 missing variable MonetDbImage (MONET_DB_IMAGE) This will appear if access to MonetDB is not available or is configured incorrectly.
CubeProcessManagerImpl BE#041108 If "FsxInitEcContainers.value" is set in the management section of the configuration manager, a container will be initialized for the FSX storage. This container will only run initially, and check to see that there is a connection to the storage. BE#194125 adding init container for the storage : <storage> BE#194125 adding init container for the storage : <fsx>
CubeProcessManagerImpl BE#052726 If the platform is not single-node and a mount is defined, it will be initiated (e.g., with path /opt/sisense/storage). BE#052726 cluster - add storage mount BE#052726 cluster - add storage mount
CubeProcessManagerImpl No entry Create a connector container accordingly.

If the "old (legacy) framework" is set for one of the connectors, "internal" or "system" based on the value set in the data group for "Connector Running Mode". If internal, the container will run inside the Build pod.

If system, create a connector container.

A service is going to be created, based on the connector setting, to allow for connectivity to external data providers. The service will be initiated to support both the old framework and the new frameworks for connectors.

For legacy (old framework) connectors, ports 8097, 43795, and 9000-9019) will be opened.

For new (new framework) connectors, the port is defined by the connector manager.

The Max Java Heap Size in MB for connectors will be set as a memory protection. The default is 4096 MB.

n/a n/a BE#445252 Failed creating service for <name of service>BE#226448 missing variable buildConnectorImage (BUILD_CONNECTOR_IMAGE)");BE#779555 missing variable connectorImage (CONNECTOR_IMAGE)"
CubeProcessManagerImpl BE#693563 Create a Jupyter notebook container if custom code is assigned to build (INTERNAL_CUSTOM_CODE).

For custom code, as deployed via a Jupyter notebook, the configured port will be used (e.g., the default port 8888).
BE#693563 Custom code using internal mode. Add JupyterNotebook container BE#693563 Custom code using internal mode. Add JupyterNotebook container BE#450683 missing variable jupyterNotebookImage (CUSTOM_CODE_IMAGE)
CubeProcessManagerImpl No entry Add a container for the local build If build is set to local storage, that is, the "interim build on local storage" parameter is set to true in the data group setting. n/a - there is no log entry for this. n/a
CubeProcessManagerImpl BE#373245ORBE#344042 Indicated whether the build runs on a single node environment with a defined mount, or the build runs on a local storage. BE#373245 cluster - add storage mountORBE#344042 mini kube - use local storage mount BE#373245 cluster - add storage mountBE#344042 mini kube - use local storage mount
CubeProcessManagerImpl BE#521943 Hot swap mode is usually used only for troubleshooting issues with MonetDB. BE#521943 using monet hot swap directory: <monetHotSwapDirectory> BE#521943 using monet hot swap directory: /opt/sisense/monetdb/bin
CubeProcessManagerImpl BE#792649 Writes the name of the build pod that was created to the log. BE#792649 name: <pod name> info: <pod name without cube name> BE#792649 name: ec-ec-test-bld-33e277fd-7f90-9 info: bld-33e277fd-7f90-9
ProcessUtilsImpl BE#129108 Waiting for Kubernetes to finish creating the cube and for it to start.

Waiting for the Build pod to start. The number of retries to connect to the Build pod for the cube will be based on "Build Connection Retries", which is set in the Management section of the configuration manager. If the number of retries is breached, the overall build will fail.
BE#129108 trying to connect: <pod name><pod port> retrying <attempted>/<number set in configuration>BE#755182 Failed to connect, retrying <attempted>/<number set in configuration>.. BE#129108 trying to connect: ec-ec-test-bld-33e277fd-7f90-9 50000 retrying 90/90]BE#755182 Failed to connect, retrying 90/90..BE#129108 trying to connect: ec-ec-test-bld-33e277fd-7f90-9 9090 retrying 90/90]-8e298d0c2e29]BE#755182 Failed to connect, retrying 90/90... BE#755182 Failed to connect, retrying <attempt #>/<build connection retries - configured>...
CubeProcessManagerImpl BE#399188 Indicates the number of attempted retries to connect to the pod. BE#399188 Connected, try number:2] BE#399188 Connected, try number:<number of retries attempted>
CubeProcessManagerImpl BE#357324 Confirms that the Build pod has started for the cube. BE#357324 EC container started - name=<Cube name> BE#357324 EC container started - name=EC Test If Kubernetes returns status for pod that it failed due to insufficient memory:BE#718473 insufficient memoryIf process crushed by Kubernetes:BE#528476 CrashLoopBackOffBE#662111 failed to connect to new ec; failed to get pod for cube name: <Cube ID>, instanceId: <InstanceID>, <failure string>BE#967529 failed to connect to new ec - cube name: <Cube Title>, instanceId: <Instance ID>, pod: <Cube ID> There are various reasons for the process failure, specifically when attempting connection to the bld-pod:
Failed to connect to the MonetDB container (EC_NOT_DETECTED)
Failed to connect to Connector container (CONNECTOR_NOT_DETECTED)
Failed to connect to the Jupyter notebook container (FAILED_TO_START_JUPYTER_NOTEBOOK_insufficient memory CrashLoopBackOff)
CubeProcessManagerImpl BE#157338 If a connector pod is initiated, this confirms the pod is up. BE#157338 Connector container started BE#157338 Connector container started
CubeProcessManagerImpl BE#232226 If a custom code container is needed, this confirms it is up. BE#232226 JupyterNotebook container started BE#232226 JupyterNotebook container started
CubeProcessManagerImpl BE#957948 Indicates that the cube has been brought up. BE#957948 Cube started successfully - cube={} hostName={} BE#957948 Cube started successfully - cube={} hostName={} BE#418622 Cube process failed to start - cube=<Cube ID>BE#156351 startCubeForBuild failed for cube <Cube ID>
CubeProcessManagerImpl BE#792649 The name of the cube being processed. BE#792649 name: <Cube ID> BE#792649 name: ec-ec-test-bld-33e277fd-7f90-9 info: bld-33e277fd-7f90-9
ManagementService BE#194310 Provides the details about the BLD pod that was created.

buildConnectorRunningMode - This value is set in the Data Group to which the cube is assigned in the "Connector Running Mode" configuration. "Internal" means that the connector container will run inside the EC BLD pod. "External" has the connector running in a separate pod.

You can determine here which of the provider connectors used for this specific build were set to the old framework, and which were based on the new framework.
BE#194310 CubeForBuildResponse:CubeForBuildResponse(status=null>, uri=<uri>, uriForMalCmd=<uri for mal cmd>, instanceId=<Pod Instance ID>, indexSize=<index setting in data group>, hostIp=<server pod is running on>, serviceIp=<service ip>, hostName=<host name>, physicalName=<db name>, buildConnectorRunningMode=<running mode set for connectors in the data group>, newConnectors=<connector list using new framework>, legacyConnectors=<connector list using old framework>, newConnectorsPort=<connector port>) BE#194310 CubeForBuildResponse:CubeForBuildResponse(status=null, uri=/opt/sisense/storage/farms/aECIAAaTest_2021.06.03.09.07.30.165, uriForMalCmd=/opt/sisense/storage/farms/aECIAAaTest_2021.06.03.09.07.30.165, instanceId=bld-33e277fd-7f90-9, indexSize=SHORT, hostIp=10.50.65.129, serviceIp=10.233.100.191, hostName=ec-ec-test-bld-33e277fd-7f90-9-7798c7c8fc-jsrlq, physicalName=aECIAAaTest, buildConnectorRunningMode=INTERNAL_CONNECTORS, newConnectors=[MySql], legacyConnectors=[], newConnectorsPort=9090)
PHASE - INITIALIZATION - ENDPOD - MANAGEMENTLOG: management.log
PHASE - BASE TABLES - STARTPOD - MANAGEMENTLOG: management.log
RequestFilter BE#683109 Receive a request from the build pod (service). BE#683109 Start handling request - url=<request URL>, userId=<userid>, userRole=<user role> BE#683109 Start handling request - url=http://management.sisense:8082/internal/, userId=6124c5dc2b42f5001ad4541a, userRole=super
ManagementService BE#955774 Provide the Build pod the physical name for the cube. BE#955774 available datasources info request: <physical name of the cube> BE#955774 available datasources info request: aECIAAaTest There could be multiple issues here that would return a failure message back to the Build service including:
  1. Bad ElastiCube name
  2. User is unauthorized for this operation
This would fail if the cube name request is invalid or the user request does not have permission to the cube.
ManagementService BE#304344 Respond with the value of the physical cube. BE#304344 response: <physical name of the cube> BE#304344 response: aECIAAaTest
RequestFilter BE#587104 Finished processing the request. BE#587104 Finished handling request - <request URL> BE#587104 Finished handling request - http://management.sisense:8082/internal/
PHASE - BASE TABLES - ENDPOD - MANAGEMENTLOG: management.log
PHASE - CUSTOM ELEMENTS - STARTPOD - MANAGEMENTLOG: management.log
RequestFilter BE#683109 n/a BE#683109 Start handling request - url=<request URL>, userId=<userid>, userRole=<user role> BE#683109 Start handling request - url=http://management.sisense:8082/internal/, userId=6124c5dc2b42f5001ad4541a, userRole=super
Deployment Monitor Checks the deployment watcher during the stage where custom elements are processed by Build service and EC BLD pod (if applicable). Message is only in debug mode and indicates a modified event is sent to the deployment watcher.BE#015371 Event of type {} on node {} received BE#015371 Event of type MODIFIED on node node1 received
RequestFilter BE#587104 n/a BE#587104 Finished handling request - <request URL> BE#587104 Finished handling request - http://management.sisense:8082/internal/
RequestFilter BE#683109 n/a BE#683109 Start handling request - url=<request URL>, userId=<userid>, userRole=<user role> BE#683109 Start handling request - url=http://management.sisense:8082/internal/, userId=6124c5dc2b42f5001ad4541a, userRole=super
RequestFilter BE#587104 n/a BE#587104 Finished handling request - <request URL> BE#587104 Finished handling request - http://management.sisense:8082/internal/
PHASE - CUSTOM ELEMENTS - ENDPOD - MANAGEMENTLOG: management.log
PHASE - FINALIZATION - STARTPOD - MANAGEMENTLOG: management.log
RequestFilter BE#683109 Receive a request from the Build service to begin the finalization phase of the build. BE#683109 Start handling request - url=<request URL>, userId=<UserID>, userRole=<UserRole> BE#683109 Start handling request - url=http://management.sisense:8082/internal/, userId=6124c5dc2b42f5001ad4541a, userRole=super
InternalRestController BE#984150 Confirms that a request from the Build service was sent to start the finalization processes. BE#984150 got async request - client=10.233.101.198, service=ManagementService, method=stopCubeForBuild, requestId <Request ID sent from Build Service> BE#984150 got async request - client=10.233.101.198, service=ManagementService, method=stopCubeForBuild, requestId 6d10e195-0a43-4fea-b822-3c1cad9ac957 E#750123 exception during method execution - service=ManagementService; method=getCubeForBuild An exception occurred while invoking this request. Check the status of the Management service by describing the pod, and see if any issues are noted.
RequestFilter BE#587104 Confirms that a request is being handled. BE#587104 Finished handling request - http://management:8082/internal/asyncRequest BE#587104 Finished handling request - http://management:8082/internal/asyncRequest
ManagementService BE#963710 Request to stop the bld-pod of the cube since the build process has completed. BE#963710 stopCubeForBuild, dbfarm: <location in farm>, InstanceId: <instance ID for cube>, isSuccess: <status of the cube stop> BE#963710 stopCubeForBuild, dbfarm: /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080, InstanceId: bld-376cc277-ccbf-5, isSuccess: true BE#051797 Instance was not found: <instance ID for cube>Management service will return "ElastiCube was not found" to Build service. If the cube is not found, this will fail the build completely
CubeProcessManagemetnImpl BE#477446 Stop the cube within the time allowed.

The pod will wait for the cube to stop, based on the configured "buildStopPodWait" parameter value in the configuration manager under"Management".
BE#477446 Stopping cube - cube=<Cube Title>, instance=<instance ID for cube>, wait=<buildStopPodWait configured> BE#477446 Stopping cube - cube=TestMySQL, instance=bld-376cc277-ccbf-5, wait=3600 BE#569381 Failed removing service, cube: <instance ID for cube>BE#647376 Failed stopping, cube: <instance ID for cube>BE#213096 Failed to delete deployment but cube <instance ID for cube> doesn't existIf failure is not due to S3 upload, an error response of "Internal error" will be returned to the build service. The build Will fail if the pod does not stop in the allowed time.

This will occur if the Pod has already been stopped.

This will occur if the pod does not exist.
ManagementService n/a If it is a local build on S3, then upload the cube to that farm and ensure that it succeeds. n/a n/a BE#054013 Could not read existing mserverScriptError.log file"BE#845564 Error in complete build operation failed to upload to S3 farm.An error response will be returned to the Build service ""Failed to upload farm to S3". If the cube is stored on S3, and cube upload to S3 fails, this will fail the build. In the process, a log file is created. It can be used to analyze what went wrong in the process of uploading to S3. The file is generated in the cube farm location with the name "mserverScriptError.log". The build will need to be re-run after the root cause of the failed upload is investigated and connectivity to S3 is confirmed.
CubeProcessManagemetnImpl BE#792649 Indicates the cube that was stopped. BE#792649 name: <cCubeID> info: <instance id of cube> BE#792649 name: ec-testmysql-bld-376cc277-ccbf-5 info: bld-376cc277-ccbf-5 n/a n/a n/a
KubernetesManagerImpl BE#795423 Send a request to the Delete service. BE#795423 Deleting service: <service name> BE#795423 Deleting service: io.fabric8.kubernetes.client.dsl.internal.core.v1.ServiceOperationsImpl@18da615c BE#198261 Failed deleting service for: <service name>BE#097897 Failed to remove service : <service name> in namespace : <namespace>BE#723845 Kubernetes client request timeout detectedConditional:BE#440998 Should restart on timeout This indicates that the process of deleting the services responsible for the BLD pod has failed.

If this error message appears, there is a timeout from Kubernetes, based on the configuration setting in "Base Configuration" under "RestartOnKubernetesTimeout.value". The Management Pod will proactively be restarted.
In any of these cases, check to see if a BLD pod is still running. If so, delete the pod using the Kubernetes "Delete" pod command. Once confirmed that it is deleted, re-run the build.
CubeProcessManagerImpl BE#470672 Remove the bld-pod that was deployed for the build. BE#470672 Removing Pod - name=<BLD-POD ID> BE#470672 Removing Pod - name=ec-testmysql-bld-376cc277-ccbf-5 n/a n/a n/a
CubeProcessManagerImpl BE#822486 Confirmation the pod has been removed. BE#822486 Pod - name=<BLD-POD ID> removed successfully BE#822486 Pod - name=ec-testmysql-bld-376cc277-ccbf-5 removed successfully n/a n/a n/a
CubeProcessManagerImpl BE#792649 Indicates the Cube ID. BE#792649 name: <CubeID> info: <instance id of cube> BE#792649 name: ec-testmysql-bld-376cc277-ccbf-5 info: bld-376cc277-ccbf-5 n/a n/a n/a
CubeProcessManagerImpl BE#202008 Doublechecks if the bld-pod still exists, and waits for the configured time until it is shut down. There must be a valid (and enough) configured number of seconds for the build to continue successfully, meaning that the wait needs to be long enough to recognize that the EC BLD pod has been deleted. If not, the build fails completely.

The configuration is located under "Service Configuration >> Management"in the "Termination Grace Period Seconds For Build" parameter.
BE#202008 Removed pod <CubeID> is still exists. Wait up to <Termination Grace Period Seconds For Build>seconds for pod to go down BE#202008 Removed pod ec-testmysql-bld-376cc277-ccbf-5 is still exists. Wait up to 3600 seconds for pod to go down. BE#415088 failed to remove Pod - name=<PodID>BE#723845 Kubernetes client request timeout detectedConditional:BE#440998 Should restart on timeout The value of "Terminate Grace Period Seconds for Build" needs to be valid (greater than 0 seconds) and long enough to allow the pod removal process to complete. Otherwise, the build will fail.

Note: In this case, the default value for this configuration in the code is 0. Therefore, if the configuration value retrieval from Zookeeper fails, the number will not be sufficient (in this case showing -1 in log).

If this error message appears, there is a timeout from Kubernetes, based on the configuration setting in "Base Configuration" under "RestartOnKubernetesTimeout.value", the Management Pod will proactively be restarted.
Increase the value if potentially the pod is large and/or the storage is slow to delete the pod.
CubeProcessManagerImpl BE#792649 This log entry indicates retries for the duration configured in "Termination Grace Period Seconds For Build", until receiving confirmation from Kubernetes that the pod has been removed. Therefore, this log entry will keep appearing until the BLD pod is removed, or the retry duration has expired. BE#792649 name: <CubeID> info: <Instance ID of Cube> BE#792649 name: ec-testmysql-bld-376cc277-ccbf-5 info: bld-376cc277-ccbf-5 n/a n/a n/a
KubernetesManagerImpl BE#608046 Confirms that the cube has been removed. BE#608046 The pod: <CubeID> successfully deleted. BE#608046 The pod: ec-testmysql-bld-376cc277-ccbf-5 successfully deleted. n/a There could be a case where the deletion of the pod is successful, however it did not wait long enough for the pod to go down. In this case, the build will fail completely. Check the wait configuration above. n/a
KubernetesManagerImpl BE#411468 Wait for the service deployment to be deleted, according to the configured value of the "Termination Grace Period Seconds For Build" parameter in the configuration manager under "Service Configuration >> Management". BE#411468 Making sure deployment and pod are down, number of polling retries: <Termination Grace Periods Seconds For Build> BE#411468 Making sure deployment and pod are down, number of polling retries: 3600 BE#243122 Pod <Cube ID> did not terminate, deleting pod with grace time <buildStopPodWait configured>BE#183179 Failed stopping deployment <Cube ID>
KubernetesManagerImpl BE#798915 If the pod would not shut down, force delete the pod. BE#798915 Forcefully deleting pod if exist BE#798915 Forcefully deleting pod if exist BE#231555 Exception forcefully stopping deployment in namespace <namespace>, with label \app\, with name <cube id> This indicates that the pod cannot be stopped. In this case, manually delete the BLD pod for the cube, using the Kubernetes command for pod delete.
KubernetesManagerImpl BE#434431 Notes whether the BLD pod for the cube is still in terminating mode (pending), or has been brought down already. BE#434431 Pod <cube id> is in phase PendingORBE#672053 The pod went down <cube id> BE#434431 Pod ec-ec-test-bld-33e277fd-7f90-9 is in phase Pending n/a n/a n/a
KubernetesManagerImpl BE#786964 Confirms that the pod has been terminated and deleted. BE#786964 Deployment and pods were removed, name: <cube id> BE#786964 Deployment and pods were removed, name: ec-ec-test-bld-33e277fd-7f90-9 n/a n/a n/a
RequestFilter BE#683109 A request is received by the management pod from the build service. BE#683109 Start handling request - url=<request URL>, userId=<UserID>, userRole=<UserRole> BE#683109 Start handling request - url=http://management:8082/internal/asyncRequest, userId=6124c5dc2b42f5001ad4541a, userRole=super n/a n/a n/a
InternalRestController BE#984150 Confirmation from the Build service that finalization can be performed on the cube. The new cube will be associated (attached) with the Query pod. BE#984150 got async request - client=<Client IP requesting> service=ManagementService, method=attachNewlyBuiltCube, requestId 1a7cc8eb-f7ba-4944-8eef-068b3d6a5606 BE#984150 got async request - client=10.233.101.198, service=ManagementService, method=attachNewlyBuiltCube, requestId 1a7cc8eb-f7ba-4944-8eef-068b3d6a5606 BE#750123 exception during method execution - service=ManagementService; method=getCubeForBuild An exception occurred while invoking this request. Check the status of the Management service by describing the pod and see if any issues are noted.
RequestFilter BE#587104 Finished handling the request. BE#587104 Finished handling request - http://management:8082/internal/asyncRequest BE#587104 Finished handling request - http://management:8082/internal/asyncRequest n/a n/a n/a
ManagementService BE#444333 Associate the newly built pod with the given data model. The input is received from a file that contains all the cube's details. In addition, the Management service checks whether a query pod is up for this cube and, if so, updates the query that the build is completed (updates the state machine). BE#444333 Attaching newly build cube, instanceId: <instance id>, dbfarm: <storage location>, physicalName:<DB Physical Name>, buildId BuildID(buildId=<BuildID>)) BE#444333 Attaching newly build cube, instanceId: bld-376cc277-ccbf-5, dbfarm: /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080, physicalName: aTestMySQL, buildId BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244) BE#332959 Build <physical name> was failed, deleting :<farm path>The Management service will return a response error "Internal Error" to the build service. This warning will occur if the build fails and, as a result, a rollback is performed, leading to deletion of the new farm. Find the root cause of the failure and, once handled, rerun the build.

Potential issues could be:
  1. Disk space-related
  2. A lack of access to MongoDB
  3. A problem with Kubernetes that potentially requires a restart
  4. A problem with accessing the farm
CubeProcessManagerImpl BE#792649 Indicates the cube ID and instance ID. Check that it exists and that the Build pod is not still running. BE#792649 name: <cube id> info: <instance id> BE#792649 name: ec-testmysql-bld-376cc277-ccbf-5 info: bld-376cc277-ccbf-5 If cube is not found, an error response will be returned to the Build Service:
  • "ElastiCube was not found"
  • "ElastiCube failed to stop"
If the cube is not found in the records and/or a build pod is still running for the cube, the build will fail. Delete the BLD pod for the cube and rerun the build.
ManagementService BE#572084 The query pod for the newly built cube is initiated and started. If this was an existing cube build and it has a running query cube, the query cube deployment is updated. BE#572084 attachCube request for path: <Storage location>, instanceId: <instance id>, should startcube: true BE#572084 attachCube request for path: /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080 , instanceId: qry-6f63cf35-cda4-4, should startcube: true BE#712755 Failed to attach cube, status: FailBE#489396 Cube already attached, id: <cube id> The process of attaching failed, resulting in a failed build. If the cube was already attached this will be indicated. Check the status of the QRY pod for the cube, if it is running.
ManagementService BE#926364 This step determines the size of the cube that will need to run for the Query pod, and checks that against the size that is available in the farm path. The size of the cube is derived from the "cubeSize" file that is located at: <location of farm of cube -URI> /cubeSize.txt.

The text file is generated by the EC BLD pod when it runs the build for the cube. At this point the disk usage is checked to see that the threshold is not breached.
BE#926364 Calculating size of directory from file BE#926364 Calculating size of directory from file BE#277749 Failed to calculate the size if the farm : <location of farm of cube -URI>BE#858316 failed reading ecm model from json - ElastiCube missing model The cubeSize.txt file is not found in the cube farm. This failure will result in a build failure. Investigate the EC BLD pod log to see if there is a reason why the file was not generated.
ManagementService n/a Check if the system status permits the operation to continue. n/a n/a BE#918092 EC import/build temporally halted Returned System EC import and the build activity is halted by management. Check the reason for this and enable operations.
ManagementService n/a If, for some reason, the cube is already attached to the query pod, it will be force removed. n/a n/a BE#756844 Failed to remove existing cube [<Cube Title>]BE#345286 failed to delete cube <Cube Title>, <exception> If management is not able to detach the cube from the query pod, an "ElastiCube not valid" error response will be returned to the Build service and the build will fail. Check if there is a problem with the QRY pod by using the Kuberenetes describe command. Potentially, delete the QRY pod and restart the build, so a new QRY pod will be generated.
ManagementService n/a Find out the physical name for MonetDB. The file resides in the farm and is called "ElastiCubePhysicalDbSchema.json". n/a n/a BE#734204 Failed reading physical DB schema from jsonBE#490856 Failed reading physicalDbSchema json file The file does not exist. The management will return an "ElastiCube missing mode" error response. In this case, the build will need to be initiated again.
HostAgentHandlerImpl BE#775910 The file that contains the metadata for the cube is mainly used by the build service. Ignore any warnings/errors, as the file is not required. The information includes the Build ID and, for each table (indicated by a Table ID), the number of rows. BE#775910 File <farm location>/BuildMetaData.json not found BE#775910 File /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080/BuildMetaData.json not found BE#920291 Failed reading "buildmetadata.json" file Failed to read the JSON file from the farm. The file is not required and, therefore, any failure here will be ignored and the build will continue. n/a
ManagementService n/a The cube will be associated with the default Data Group, if it was not already assigned a Data Group. n/a n/a BE#670396 Failed to assign cube <Cube Title> to dataGroup Failed to attach the ElastiCube. n/a
ManagementService BE#106588 A dump of the cube information that indicates several things. The information is processed into MongoDB. A message will be sent to RabbitMQ that the cube attachment has completed. BE#106588 newCubeInfo:CubeInfoItem(instanceId=<Instance ID>, tenantId=null, uri=<New farm location>, shadowUri=/<Old Farm location>, nextBuildUri=<will indicate if already a build ID was assigned to next build>, uriDirectorySize=<Size of new farm>, shadowUriDirectorySize=<size of old farm>, nextBuildUriDirectorySize=null, hostName=<Query Pod attached>, ecubePID=null, newConnectorsPID=null, newConnectorsPort=null, shadowEcubePID=null, name=<Cube Title>, runtimeStatus=RUNNING, cubeOperationStatus=NONE, lastFailureMessage=null, cubeStatusErrorCode=null, buildId=BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244), lastBuildTime=Tue Oct 26 08:54:07 UTC 2021, lastBuildAttemptTime=null, lastUpdatedTime=Thu Sep 09 08:22:35 UTC 2021, physicalName=<Physical DB Name>, dataSourceId=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, port=50000, schemaId=0545564c-1341-3920-96eb-e19021de1b02, indexSize=<Long or Short Index>, concurrentQueries=8, queryTimeout=300, generation=1, isBuilding=false, isFinalizing=true, isLastBuildFailed=false, dataGroupOid=null, dataSourceType=<The build destination>) BE#106588 newCubeInfo:CubeInfoItem(instanceId=qry-6f63cf35-cda4-4, tenantId=null, uri=/opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080, shadowUri=/opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051, nextBuildUri=null, uriDirectorySize=807485, shadowUriDirectorySize=807057, nextBuildUriDirectorySize=null, hostName=ec-testmysql-qry-6f63cf35-cda4-4.sisense, ecubePID=null, newConnectorsPID=null, newConnectorsPort=null, shadowEcubePID=null, name=TestMySQL, runtimeStatus=RUNNING, cubeOperationStatus=NONE, lastFailureMessage=null, cubeStatusErrorCode=null, buildId=BuildID(buildId=27364933-0325-41c3-8bbe-e28c79f19244), lastBuildTime=Tue Oct 26 08:54:07 UTC 2021, lastBuildAttemptTime=null, lastUpdatedTime=Thu Sep 09 08:22:35 UTC 2021, physicalName=aTestMySQL, dataSourceId=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, port=50000, schemaId=0545564c-1341-3920-96eb-e19021de1b02, indexSize=SHORT, concurrentQueries=8, queryTimeout=300, generation=1, isBuilding=false, isFinalizing=true, isLastBuildFailed=false, dataGroupOid=null, dataSourceType=MonetDb) BE#901591 failed to insert cubeInfoItem It failed due to an inability to write the details into MongoDB, which will lead to a complete build failure. Check if MongoDB is up. In MongoDB, check if a cube exists.
ManagementService BE#809954 Attached the query pod that will be associated with the cube that was just built. BE#809954 creating countdown latch for synchronization purpose for instance <Query Pod ID associated with Cube>Or BE#489396 Cube already attached, id:<Query Pod ID> BE#809954 creating countdown latch for synchronization purpose for instance qry-6f63cf35-cda4-4Or BE#489396 Cube already attached, id:qry-6f63cf35-cda4-4
ManagementService BE#235989 Starting the query pod for the cube that was just built. This is a critical step for the build process and, if it fails, the build will be noted as failing.

The full name of the Kubernetes pod will be that of the name specified here + the additional characters assigned to the instance by Kubernetes.

The Sisense Pod ID is based on the cube title and assigned Instance ID which has the format:
Ec-<Cube Title>-qry-<instance ID>
For example:Ec-testmysql-qry-6f63cf35-cda4-4

When the QRY pod is brought up, Kubernetes will name the pod with the instance name + the additional characters that make the pod name unique:
Ec-<Cube Title>-qry-<instance ID>-<Kubernetes Assigned ID>
For example:Ec-testmysql-qry-6f63cf35-cda4-4-7d9cf4987d-cxmg9

Note, of course, that the <Kuberentes Assigned ID> will change after every build, as the QRY pod is re-instantiated.

This naming relationship can be derived when you describe the pod. That is, if you need to be sure of the association of the cube with the running Kubernetes cube.
Kubectl -n <namespace> describe pod Ec-testmysql-qry-6f63cf35-cda4-4-7d9cf4987d-cxmg9
The output will be:
Name: Ec-testmysql-qry-6f63cf35-cda4-4-7d9cf4987d-cxmg9InsanceID: Ec-testmysql-qry-6f63cf35-cda4-4
BE#235989 startCube: StartCubeRequest(instanceId=<query pod instance ID> useLocalStorage=<indicates if local storage is used for build>, isRestart=true) BE#235989 startCube: StartCubeRequest(instanceId=qry-6f63cf35-cda4-4, useLocalStorage=false, isRestart=true) BE#195623 Failed to start cube, status: <status>, id: <CubeID>BE#538250 no such cube, instanceId: <CubeID>BE#195623 Failed to start cube, status: TOO_MANY_CUBES_ALREADY_RUNNING, id: <qry pod instance ID> Unable to start the query pod, or no cube ID is found, which will lead to the build failing.

This failure occurs due to CubeID not existing in MongoDB, or not being able to retrieve the record from MongoDB.

The system is configured to run a maximum number of pods simultaneously. This is configured in the "Service Configuration >> Management" section in the "MaxNumberOfRunningCubes.value" parameter. If the number is breached, pod loading will fail and the build will fail altogether.
Start a new build. Potentially, you will need to clone the cube and start a new build.

This is a rare case. You might need to clone the cube and start a new build as MongoDB data appears to be corrupted.

Reduce the number of running pods or increase the number configured. However, of course this could lead to more system resource utilization that could lead to performance or other issues. Check the build performance and the monitor system after making changes.
CubeProcessManagerImpl BE#792649 This is used to continue to watch the query pod,while waiting for the pod to load successfully. This entry will repeat itself until the pod is up or the process fails. BE#792649 name: <CubeID> info: <Query Pod ID> BE#792649 name: ec-testmysql-qry-6f63cf35-cda4-4 info: qry-6f63cf35-cda4-4 n/a n/a n/a
MonetDbConfigManagerImpl BE#297342 Get the Monet database details. BE#297342 Monet command: --zk_system_name=S1 --zk_address=sisense-zookeeper.sisense:2181 --external_server=ec-testmysql-qry-6f63cf35-cda4-4.sisense --instance_id=<Instance ID>--dbname=<Physical DB name> --farmstate=Querying --dbfarm=/tmp/<Farm Path>/dbfarm --set mapi_port=50000 BE#297342 Monet command: --zk_system_name=S1 --zk_address=sisense-zookeeper.sisense:2181 --external_server=ec-testmysql-qry-6f63cf35-cda4-4.sisense --instance_id=qry-6f63cf35-cda4-4 --dbname=aTestMySQL --farmstate=Querying --dbfarm=/tmp/aTestMySQL_2021.10.26.08.53.59.080/dbfarm --set mapi_port=50000 n/a n/a n/a
CubeProcessManagerImpl BE#768721 Loads from the data group to check node affinity for the build. This will run the QRY in the nodes that are set. This is configured in the data group in "Query Nodes". BE#768721 Build affinity based data group label BE#768721 Build affinity based data group label BE#717375 no datagroup The cube is not associated with any data group. Assign the cube to a data group.
CubeProcessManagerImpl BE#938471ORBE#041652ORBE#434912 Indicates whether the query pod has started, is starting, or was brought up successfully. BE#938471 cube already starting"ORBE#041652 cube already runningORBE#434912 cube successfully started BE#938471 cube already starting"ORBE#041652 cube already runningORBE#434912 cube successfully started BE#882233 failed to start cube: <CubetTitle>BE#455816 start cube <Cube Title> error: cube failed to start.Some returned responses to request back to Build service could include:
  • Failed to get data group
  • Could not deploy cube for build <error description>
  • Failed to create configuration files
  • Unsupported index size - A value is set in the given Data Group that is not compatible.
  • Failed to create service - not able to create a pod service
Management Service will return "Cube failed to start" response error to the Build Service.
Failed to load the QRY pod. This could be the result of:
  1. A lack of system resources
  2. A Kuberentes issue
  3. An issue with MonetDB or the QRY pod
Investigate the QRY pod log to determine the issue.
CubeProcessManagerImpl n/a The QRY pod is set to run locally. BE#119493 Host local storage path defined - cube=<Cube Title>, local path=<Configured Local Storage Path> with QUERY_ON_LOCAL set to <True/false if to run Query on local> BE#119493 Host local storage path defined - cube=TestMySQL, local path=/opt/sisense/local_storage with QUERY_ON_LOCAL set to true n/a n/a n/a
CubeProcessManagerImpl BE#757750 Get the data group setting for the query pod, which will be used to determine how to launch the query pod. This includes: Reserved Cores, Max Cores, Reserved RAM, and Max RAM. BE#757750ResourceRequirements: ResourceRequirements(limits={memory=<Max RAM MB>, cpu=Max Cores}, requests={memory=<Reserved Ram MB>, cpu=<Reserved Cores>}, additionalProperties={})] BE#757750 ResourceRequirements: ResourceRequirements(limits={memory=8192M, cpu=4.0}, requests={memory=512M, cpu=3.0}, additionalProperties={})] BE#717375 no datagroup The cube is not associated with any data group. Assign the cube to a data group.
CubeProcessManagerImpl BE#719375 This indicates that the cube has started successfully. This step indicates that a pod service has been deployed and the QRY pod for the cube was brought up. BE#719375 start OK:<Cube Title> BE#719375 start OK:TestMySQL BE#486734 failed to get deployment for existing instance id : <qry pod instance ID with error : <returned kubernetes error>BE#203520 failed to get deployment for existing instance id : <qry pod instance ID with error : <if not kubernetes error>If kubernetes failure:
  • BE#870681 Failed to get deployment : <qry pod instance ID> in namespace : <namespace for Sisense>
  • BE#515641 Service already exists, name: <Service Name>
  • BE#547741 Exception creating service
  • BE#120529 Failed creating service for: <Service Name>
  • BE#723845 Kubernetes client request timeout detected
This could occur due to:
  1. Kubernetes issues with bringing up the pod due to resource limitations
  2. Deployment of a service and/or pod that, for some reason, already exists
An analysis of system resources will need to be performed to solve the problem.
ManagementService BE#997281 Notify the query pod to update its status to up. This updates the latch process that the build finished successfully. BE#997281 notifyInstanceGenerationUpdate: Not signalling. Cube generation: 1 preGeneration: 1 BE#997281 notifyInstanceGenerationUpdate: Not signalling. Cube generation: 1 preGeneration: 1 n/a n/a n/a
ManagementService BE#720309 Update the event that the cube is up and ready. BE#720309 deployment watch instanceId:<Existing Query instance ID> name:<Query Pod ID> generation:1 message: ReplicaSet "<New Kubernetes Query Pod ID>" has successfully progressed. BE#720309 deployment watch instanceId:qry-6f63cf35-cda4-4 name:ec-testmysql-qry-6f63cf35-cda4-4 generation:1 message: ReplicaSet "ec-testmysql-qry-6f63cf35-cda4-4-7c9b4b8b9b" has successfully progressed. n/a n/a n/a
ManagementService BE#285395 Update the event that the cube is up and ready.
This also indicates whether the QRY pod is running locally.
These statues are received from Kubernetes.
BE#285395 startCube success StartCubeRequest(instanceId=<Query pod ID>, useLocalStorage=<indicates if local storage used for query>, isRestart=true BE#285395 startCube success StartCubeRequest(instanceId=qry-6f63cf35-cda4-4, useLocalStorage=false, isRestart=true n/a n/a n/a
ManagementService BE#285395 Indicates the status for the QRY pod. BE#285395 startCubeResponse: ServiceResponse(status=OK, value=null, error=, statusCode=0 BE#815906 startCubeResponse: ServiceResponse(status=OK, value=null, error=, statusCode=0 n/a n/a n/a
ManagementService BE#775607 Display the response received from the QRY pod startup process. BE#775607 startStatus: OK BE#775607 startStatus: OK BE#195623 Failed to start cube, status: Fail id: <Cube ID> Indicates that the overall process to bring up the QRY pod has failed. Reasons are outlined above this log entry. n/a
ManagementService BE#386821 Finished the attach cube process that associated the Monet DB instance with the QRY pod. BE#386821 attachCube success for instance id: <Query Pod instance ID> BE#386821 attachCube success for instance id: qry-6f63cf35-cda4-4 n/a n/a n/a
ManagementService BE#786105 Start up the MonetDB instance.Waiting for MonetDB to come up. BE#786105 Wait for elasticube <DB Physical Name> to start running BE#786105 Wait for elasticube aTestMySQL to start running BE#414429 Latch object is null for physicalName: <DB Physical Name>, instanceId: <Instance ID>
ManagementService BE#076009 Waiting for the generation update complete for the pod. This updates the QRY pod (restarts the pod) with the MonetDB instance that is coming up.The wait time is based on the configuration that defines how long the management pod will wait to confirm that the post has been launched. The configuration is defined in "Service Configuration >> Management" under "Time to wait for pod to start". BE#076009 Starting generationUpdate Guaranteer with AttachCubeResponse(status=null, instanceId=<Query Pod Instance ID>, cubeTitle=<Cube Title>), instanceId <Query Pod Instance ID>, Url <new farm location> BE#076009 Starting generationUpdate Guaranteer with AttachCubeResponse(status=null, instanceId=qry-6f63cf35-cda4-4, cubeTitle=TestMySQL), instanceId qry-6f63cf35-cda4-4, Url /opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080 BE#291756 Failed complete build: <BuildID>, elasticube did not run before timeout, timeout is <configured Time to wait for pod to start> secondsManagement Service will return a response error of "Failed loading cube".BE#807818 Failed complete build: <DB Farm Location> the management thread was interrupted while waiting for the elasticube to start running" Check that the configuration is read properly, by confirming the time allowed that is specified in the log. This issue can occur if there is not enough time allowed for the operation to complete.

For some reason, there was an interruption in the Management Service waiting for the MonetDB script to run.
CubeSyncManager BE#936610 Generation update has completed, and the pod is ready to be restarted. BE#936610 generation changed for: <Cube Title> BE#936610 generation changed for: TestMySQL
CubeSyncManager BE#447282 Indicates whether the pod is in the stopping state. This will continuously appear until the pod has stopped. BE#447282 notifyInstanceGenerationUpdate - instance <Query Pod instance ID> is stopping/stopped BE#447282 notifyInstanceGenerationUpdate - instance qry-6f63cf35-cda4-4 is stopping/stopped
CubesSyncManager BE#397251 Indicates that the Qry pod status is moved into a starting mode. BE#397251 notifyInstanceGenerationUpdate - instance <Query Pod instance ID> is starting - moving to running BE#397251 notifyInstanceGenerationUpdate - instance qry-6f63cf35-cda4-4 is starting - moving to running
CubesSyncManager BE#332015 Indicates the completion of the waiting (latch) process for the pod to come up. BE#332015 Signalled completion for instance <query pod instance id> BE#332015 Signalled completion for instance qry-6f63cf35-cda4-4 n/a n/a n/a
ManagementService BE#389030ORBE#455235 For new cube builds (that have not existed yet), no NBAB will be performed, regardless of setting.

For existing cubes, NBAB will only be performed if it is enabled.

NBAB provides users the ability to build accumulative builds while executing queries. During the build, there are two cubes that are in place, one that is active and can be used for queries, and another that is being built on.

It also provides failure recovery when an accumulative build fails.

NBAB can also be set to be skipped, via the “Do Full Copy Instead of nbab” property in “Service Configuration >> Management”.

When NBAB is switched off, the farm for the cube is cloned into a directory with “_next” at the end of the farm folder name. This farm location will be used in the next build, to make it more efficient.
BE#389030 IntentType.NEW -> full build, skip NBABOR BE#455235 Start NBAB for cube <CubeTitle> BE#389030 IntentType.NEW -> full build, skip NBABOR BE#455235 Start NBAB for cube <CubeTitle> BE#220010 cube info missing for instanceId <Cube Instance ID>(warning) BE#751765 nextBuildUrl: <indicate the next build URL that was set already> already exists, skipping... If the next build URL was already set, the NBAB process is skipped.
ManagementService BE#720309 Indicates that the rolling update has finished on the cube. BE#720309 deployment watch instanceId:qry-6f63cf35-cda4-4 name:ec-testmysql-qry-6f63cf35-cda4-4 generation:2 message: ReplicaSet "ec-testmysql-qry-6f63cf35-cda4-4-67b4fd6d4" has successfully progressed. BE#720309 deployment watch instanceId:qry-6f63cf35-cda4-4 name:ec-testmysql-qry-6f63cf35-cda4-4 generation:2 message: ReplicaSet "ec-testmysql-qry-6f63cf35-cda4-4-67b4fd6d4" has successfully progressed.
CopyFarmTask BE#449293 This is the original URI of the cube before this build. Therefore, this is not needed anymore and its content is deleted. BE#449293 delete shadow Url: <The previous location of the farm> BE#449293 delete shadow Url: /opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051
HostAgentHandlerImpl BE#907101 This is the original folder of the cube before this build. Therefore, this is not needed anymore and is deleted. BE#907101 deleteUrl folder: <the previous location of the farm> BE#907101 deleteUrl folder: /opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051
HostAgentHandlerImpl BE#240867 The confirmation folder is deleted. BE#240867 folder /opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051 deleted BE#240867 folder /opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051 deleted n/a Note: The copy command that is sent to be performed on the storage that contains the farm might take a long time. Therefore, this could lead to an expiration of the process that is waiting for it to complete.

Also, a failure here does not necessarily mean that an error will appear in the log, and/or the management pod might not time out this process and therefore would be late in responding back to the Build Service that is waiting for it to finish the process.
n/a
CopyFarmTask BE#458375 If "Do Full Copy Instead of Nbab" is switched on, it copies the farm from the existing location into a new folder, in order to be able to perform the build. BE#458375 copy directory <old farm> to <new farm> startedBE#249700 copy directory <old farm> to <new farm> complete BE#458375 copy directory /opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051 to /opt/sisense/storage/farms/aTestMySQL_2021.10.20.14.55.10.051 startedBE#249700 copy directory /opt/sisense/storage/farms/aTestMySQL_2021.10.20.13.55.10.051 to /opt/sisense/storage/farms/aTestMySQL_2021.10.20.14.55.10.051 complete BE#062373 failed to copy <old farm> to <new farm> Note: The copy command that is sent to be performed on the storage that contains the farm might take a long time. Therefore, this could lead to an expiration of the process that is waiting for it to complete.

For example, assuming it takes a very long time (due to a storage issue) for the copy to be performed, it is possible that the Management pod will hang/restart.

Also, a failure here does not necessarily mean that an error will appear in the log, and/or the management pod might not time out this process and therefore would be late in responding back to the Build Service that is waiting for it to finish the process.
CopyFarmTask BE#193284 The time it took to create the copy of the cube. BE#193284 Copy took <duration> ms BE#193284 Copy took 177 ms
ElasticCubeBuildCommand BE#557337 The build has completed and the full details are written to the log. BE#557337 buildId: <Build ID> summary: BuildSummaryMessage(buildEndTime=2021-10-26T08:54:36.489851Z, buildDuration=PT0S, type=BUILD_END, message=Build ended, trackId=null, serverId=7c3f73dd-0b38-48dc-9347-c78811bd80c4, buildType=full, schemaType=schema-changes, serverName=localhost, dataSourceTitle=<Cube Title>, sessionId=8877aafd-5f4f-4a08-ad53-b7c49c23dc6d, source=NextECM, rowLimit=null, buildStatus=<Status>, buildProcessId=-431453317, executionFault=null, instanceId=bld-376cc277-ccbf-5, buildId=<Build ID>, dataSourceId=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, url=<Farm Location>) BE#557337 buildId: 27364933-0325-41c3-8bbe-e28c79f19244 summary: BuildSummaryMessage(buildEndTime=2021-10-26T08:54:36.489851Z, buildDuration=PT0S, type=BUILD_END, message=Build ended, trackId=null, serverId=7c3f73dd-0b38-48dc-9347-c78811bd80c4, buildType=full, schemaType=schema-changes, serverName=localhost, dataSourceTitle=TestMySQL, sessionId=8877aafd-5f4f-4a08-ad53-b7c49c23dc6d, source=NextECM, rowLimit=null, buildStatus=Success, buildProcessId=-431453317, executionFault=null, instanceId=bld-376cc277-ccbf-5, buildId=27364933-0325-41c3-8bbe-e28c79f19244, dataSourceId=e51d6ac3-819c-48f7-b1c4-7003f8cdff7e, url=/opt/sisense/storage/farms/aTestMySQL_2021.10.26.08.53.59.080) BE#893805 buildSummaryMessageNotification exception <Message>
ElasticCubeBuildCommand BE#039541 A RabbitMQ notification of the build status was sent if the build was received via the CLI. BE#039541 RabbitMq message: <Message>, for cube: <Cube Title> BE#039541 RabbitMq message: <Message>, for cube: <TestMySQL> BE#703803 build failure for cube: <Cube Title> build status: <Status>BE#893805 buildSummaryMessageNotification exception <Exception Message>