Dataset Marketplace

The Dataset Marketplace is the core platform for enterprise-level data management and governance, providing a unified data infrastructure for data analysis and visualization creation. Through multi-level data organization and refined permission management, the Dataset Marketplace can meet the data needs of different departments, users, and business scenarios.

Overview

The Data Marketplace serves as the foundational layer of the data analysis ecosystem, taking on the critical responsibilities of data integration, processing, governance, and distribution. Users first build and manage data assets in the Data Marketplace, and then perform visualization analysis and dashboard development in App Creation based on these data.

Data Package: Serves as a container for data, including datasets, data models, parameters, and other complete data analysis infrastructure.
Folder: Provides a hierarchical organizational structure, supporting the classification and management of data packages by business domain, department, or project.
Permission System: Ensures data security and compliant access through granular permission control.

The Data Marketplace supports the complete data lifecycle management, from data source integration, data processing, data modeling and association, to data exploration and analysis. Through unified data governance and permission control, it ensures both data security and consistency while enhancing data reusability and maintainability.

Data Package

Create a Data Package

Users with both System Management and Data Management roles can create a new data package in the Data Mart. If a user only has the Data Management role, they can create a data package in folders where they are authorized as a Manager or Editor.

There are two types of data packages:

Blank Data Package

The default name for the data package is "New Data Package," which can be modified to a name that better aligns with business logic.

Users can create datasets, process data, establish relationship models, build charts and tables, and fulfill specific analytical needs within the data package.

Import Template

Simply upload a previously exported .hstpl file to successfully import the template.

Operations that can be performed on a data package include:

Permission Management
Export Template
Import and Replace
Create Copy
Copy To
Replace Data Connection
Rename
Move
Delete

Permission Management

Users with the Manager permission for a data package have the authority to grant Manager, Editor, or Viewer permissions to users, user groups, or organizational structures.

When analysts enter the dataset marketplace and have the Manager permission for a data package, they can perform the following operations on the data package:

Permission Management
Export Template
Import and Replace
Create Copy
Copy To
Replace Data Connection
Rename
Move
Delete

When analysts enter the dataset marketplace and have the Editor permission for a data package, they can perform the following operations on the data package:

Export Template
Create Copy
Copy To
Replace Data Connection
Rename
Move
Delete

When analysts enter the dataset marketplace and only have the Viewer permission for a data package, they can perform the following operation:

Export Template

Export Template

Export Template refers to exporting the datasets, chart library, data models, and other contents from the current data package to generate a JSON file. Users can use the template to quickly create a new data package.

Import Template

Create a New Data Package Import Template

Click Create New Data Package -> Import Template, and open the data package template in the pop-up file dialog box to complete the template import process, creating a new data package.

The new data package includes the dataset and data model from the template data package, allowing users to perform data exploration based on it.

Name of the Imported Template Data Package: The name of the imported template data package is the same as the template name. If there is a data package with the same name in the directory, the naming convention for the imported template data package will be: "Template Name" + "(1)". Users can also customize the data package name.
Application Data of the Imported Template: The imported template data package requires users to re-confirm the data.
- If the datasets in the template are local files, these datasets will be empty, containing no data, and data can only be added by replacing the datasets.
- If the datasets are direct connection datasets or SQL query datasets, the corresponding data will be displayed based on the current user's permissions for the data connection associated with the datasets.
- The relationships established between datasets in the data model will be retained. For association models built using local files, the relationships are preserved, and data can be re-added for use. For association models built between direct connection datasets and SQL query datasets, both the relationships and data are retained.
Permission Configuration of the Imported Template: If the template includes permission configurations, the relevant permission configuration content can be imported. This is an optional operation and will only take effect if selected.
- When importing permissions, if the system encounters unmatched user objects, the authorization rules will be retained during the import process, but the authorization objects will be discarded.

Existing Data Package Import Template

Existing data packages also support importing templates. By selecting Import and Replace, you can use the template content to replace the datasets, chart libraries, and other contents in the current data package.

Create a Copy

In the data package menu, there is a Create a Copy button. Click Create a Copy to generate an application with the same data as this data package.

Creating a copy only generates a new data package in the current path, with the name appended by + (1) after the original data package name.

Copy To

In the Data Package menu, there is a Copy To button. Clicking Copy To will display a copy popup. In the popup, select the path where the data package will be copied, and after clicking Confirm, the data package will be copied to the corresponding path.

The Copy To function allows you to modify the data package name and select the copy path.

Replace Data Connection

In the dataset menu, there is a Replace Data Connection button. Click Replace Data Connection to select the data connection you want to replace.

Rename

The menu of the data package includes an operation button for renaming. If a data package with the same name already exists in the current path of the dataset marketplace, a prompt will appear: "The application name already exists, please choose another name."

Move

The menu of the dataset includes an operation button for moving, which allows you to move the dataset to a folder with editing permissions.

Delete

The menu of the data package contains an operation button for deletion. When deleting a data package, the datasets and data models within the data package will be deleted simultaneously.

Dataset

A dataset is a collection of data and serves as the core component of data analysis. Subsequent data exploration and data management are all based on the dataset.

During the process of using datasets for data analysis, topics such as Dataset Creation and Data Management are involved. You can click the links to access relevant usage instructions.

Users with Manager or Editor permissions for the data package can create new datasets and manage datasets within the data package.

Users with Viewer permissions for the data package can only view dataset information they are authorized to access.

Data Model

The data model, also known as the relational model, allows you to establish join or union relationships between datasets and incorporate these relationships when creating charts.

When building a data model, the source dataset and target dataset must originate from the same data connection. For detailed instructions on using the data model, please click link to learn more.

Users with Manager or Editor permissions for the data package can create new data models within the package; users with Viewer permissions for the data package can only view the established data models.

Parameters

Parameters are primarily designed to enable flexible control of data. Previously, achieving certain version functionalities required modifications in multiple places, but by creating parameters, all issues can now be resolved with a single adjustment.

For detailed instructions on how to use the parameter functionality, please click link to learn more.

Data Lineage

Data lineage is the lifecycle of data, showcasing the transformation process from the data source to the data target.

You can click link to learn more about the data lineage feature.

Settings

Data Permissions

Data permissions include three types: App Author, Dataset Author, and User, with the App Author option selected by default.

Mode Switching: During the switching of dataset permission modes, datasets imported into the engine need to be re-imported. Switching from App Author or Dataset Author mode to User mode will cause the datasets imported into the engine to become invalid.

For a detailed introduction to data permissions, see Data Permissions.

Data Mode

Data typically carries different security levels, making it suitable for different users. For example, high-security-level data can only be shared and viewed by specific personnel, while low-security-level data, such as internally public data, can be displayed to all members. The system introduces strict mode and non-strict mode to accommodate data of different levels.

Strict permission mode is suitable for high-security-level data, requiring all users to have permission for the data in order to view and use it.
Non-strict permission mode is suitable for low-security-level data, allowing applications or data packages using this data to display it to users without data permissions for viewing and usage.

Permission Control

For a detailed introduction to permission control, see Permission Control.

Inheritance Management

When an app needs to inherit data from a dataset for data analysis, the app will send an inheritance request to the dataset. The dataset manager decides whether to allow the app to inherit. Once the inheritance toggle is enabled, the app will display the inherited dataset, which can be used for data analysis.

When a dataset allows other apps to inherit, deleting the dataset will prompt a warning indicating that apps are inheriting from it. At this point, you can force delete the dataset. After deletion, all inherited datasets in the apps will disappear, and charts using the inherited datasets will encounter errors.

Start Exploring

Create Exploration

In the Data Mart, explorations can only be created within a dataset. Click the Start Exploration icon at the top right corner of the dataset to open the "Start Exploration" window. After selecting the chart type, you can begin the exploration. For chart-related operations, please refer to the Dashboard Controls user guide.

Save Chart

After creating a new exploration, you can choose to save the chart to a dashboard within My Creations or Team Space in App Creation. The dashboard can either be newly created or an existing one.

After successfully saving, you can choose to navigate to the chart page in "App Creation->App->Dashboard"; or you can choose to return to the dataset page.

Note

If you delete the dataset used for charting after creating a new exploration, a prompt will appear in App Creation stating "The dataset does not exist in the application of the dataset mart" when viewing the chart.

Folder

Only users with the System Administrator role can create new folders in the Data Mart. Within a folder, you can still create subfolders or new data packages.

When creating a new folder, if the folder is empty, a prompt will appear: "Please enter a folder name."

Actions that can be performed on folders include:

Permission Management
Move
Rename
Delete