Task Management

Task management involves the management of background task execution plans and the monitoring of execution records within the system, enabling users to understand task operation status and manage tasks uniformly.

Task management allows users to view task status, execution records, modify execution plans, etc. Regular users can view and manage all execution plans they own, while system administrators (users with the System Management Role) can view, modify, and delete execution plans for all resources, providing unified management of system operation and maintenance resources.

Click Settings -> Task Management to view the task management on the platform.

Task Types

Task management includes seven types of tasks: App Updates, Dataset Updates, Metric Alerts, Email Notifications, Data Integration, and Batch Synchronization. After setting the execution plan for the above tasks, whether the plan is enabled or not, their status and operation can be viewed in task management.

Users with different roles are allowed to manage different types of tasks.

1. Users with the System Administrator Role can manage all types of tasks;
2. Except for system administrators, users of other roles can only manage tasks related to resources owned by their roles;
3. The tenant system and the platform system support different functional modules. The tenant system only includes App Updates and Dataset Updates. Tenant system administrators can manage all tasks in the tenant system, while ordinary tenants can only manage execution tasks related to resources owned by their roles.

Execution Plan

Click Settings -> Task Management -> Execution Plan to enter the execution plan page. This page displays a list of scheduled execution plans for various types of tasks, categorized by task type. The list includes basic information about the tasks, such as type, project name, creator, path, engine, dataset size, etc., with different basic information for different types of tasks. If the plan is enabled, the list will also show task execution details, including execution frequency, next execution time, last execution time, execution duration, and execution status.

When viewing tasks in the execution plan, you can search for tasks by project name, filter tasks by task priority, running status, creator, and enable status, and sort tasks.

The execution plan supports operations such as enabling, disabling, immediate execution, modifying the plan, deleting the plan, and viewing the DAG for tasks, making it convenient for task owners or system administrators to monitor and manage tasks as necessary. Tasks of the dataset update type also support schema update operations.

The execution plan supports batch operations, including immediate update, enable, disable, and delete.

Tips

1. Indicator alert tasks in the execution plan only support enabling/disabling the plan and do not support other operations. To modify or delete indicator alert tasks, please go to the corresponding application path.
2. After an execution plan fails, you can set up email notifications to remind relevant responsible persons to pay attention to the failed tasks and handle them in a timely manner. This feature applies to all tasks within the system. Please check if the SMTP service is enabled before setting it up.

DAG Diagram of Execution Plan

Each task in the execution plan can view its DAG diagram, which displays the current status of the task, the dependency graph between tasks, and the operation status of upstream and downstream tasks in the task chain. The DAG diagram usually only shows the 2-level predecessor dependencies and 1-level downstream tasks of the current task. If the task chain is long, you can drag to view all tasks.

In the DAG diagram of the execution plan, you can view all instances of the task and modify the task plan. Viewing all instances of the task means viewing the Execution Records of the task.

Execution Records

Click Settings -> Task Management -> Execution Records to enter the execution records page. The execution records page displays a list of execution records for tasks over the past 30 days, categorized by task type. The list shows basic information and execution details of the tasks. Basic information includes type, project name, creator, path, engine, dataset size, which may vary depending on the task type. Execution details include execution code, submission time, execution start time, execution priority, execution duration, and execution status.

When viewing tasks on the execution records page, you can search for tasks by project name, filter tasks by task priority, task status, and user, and sort tasks.

In the execution records, you can perform operations such as rerunning tasks, increasing priority, decreasing priority, viewing DAG, and viewing logs.

Tip

1. For indicator warning tasks in the execution records, only viewing logs is supported; other operations are not supported.

Rerun

Click on the "Rerun" option in the task's three-dot menu to bring up the Rerun Task Settings interface. Here, you can set to rerun the current task and its downstream tasks. The tasks set to rerun will be executed immediately and enter the task execution queue.

Execution Log

Click to view the logs recorded during the current execution of the task, commonly used to analyze exceptions that occurred during the task execution process.

DAG Chart of Execution Records

The DAG chart can also be viewed in the execution records, displaying the same content as the DAG chart in the execution plan, including the current status of tasks, the dependency graph between tasks, and the running status of upstream and downstream tasks in the task chain. However, the supported operations are different from those in the DAG chart of the execution plan.

In the DAG chart of execution records, you can view all instances of a task, modify task priority, rerun with one click, and view logs.

• Viewing all instances of a task means viewing all execution records of the task within the last 30 days.
• Supports increasing and decreasing the priority of tasks.
• Supports one-click rerun settings, which means rerunning the task and its downstream tasks.
• Supports viewing the execution logs of tasks.

Single Task Execution Records

The complete execution records of a single task over the past 30 days can be viewed in the following two ways:

• Click on the task within the execution plan page to enter the execution record list page of that task, displaying the execution records of the task over the past 30 days. Click the back button to return to the execution plan page.
• In the DAG chart, click to view all instances of the task, which will display the execution records of the task over the past 30 days.