Skip to content

Commit 5b0347e

Browse files
aiwenmoaiwenmo
authored
[Task] Add Dinky task to better support the development and execution of FlinkSQL (#10640)
1 parent 182b9c9 commit 5b0347e

28 files changed

Lines changed: 863 additions & 4 deletions

File tree

‎docs/configs/docsdev.js‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,10 @@ export default {
169169
title: 'DVC',
170170
link: '/en-us/docs/dev/user_doc/guide/task/dvc.html',
171171
},
172+
{
173+
title: 'Dinky',
174+
link: '/en-us/docs/dev/user_doc/guide/task/dinky.html',
175+
},
172176
],
173177
},
174178
{
@@ -553,6 +557,10 @@ export default {
553557
title: 'DVC',
554558
link: '/zh-cn/docs/dev/user_doc/guide/task/dvc.html',
555559
},
560+
{
561+
title: 'Dinky',
562+
link: '/zh-cn/docs/dev/user_doc/guide/task/dinky.html',
563+
},
556564
],
557565
},
558566
{

‎docs/docs/en/guide/task/dinky.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Dinky
2+
3+
## Overview
4+
5+
Use `Dinky Task` to create a dinky-type task and support one-stop development, debugging, operation and maintenance of FlinkSql, Flink jar and SQL. When the worker executes `Dinky Task`,
6+
it will call `Dinky API` to trigger dinky task. Click [here](http://www.dlink.top/) for details about `Dinky`.
7+
8+
## Create Task
9+
10+
- Click Project Management-Project Name-Workflow Definition, and click the "Create Workflow" button to enter the DAG editing page.
11+
- Drag <img src="../../../../img/tasks/icons/dinky.png" width="15"/> from the toolbar to the canvas.
12+
13+
## Task Parameter
14+
15+
| **Parameter** | **Description** |
16+
| ------- | ---------- |
17+
| Node Name | Set the name of the task. Node names within a workflow definition are unique. |
18+
| Run flag | Indicates whether the node can be scheduled normally. If it is not necessary to execute, you can turn on the prohibiting execution switch. |
19+
| Description | Describes the function of this node. |
20+
| Task priority | When the number of worker threads is insufficient, they are executed in order from high to low according to the priority, and they are executed according to the first-in, first-out principle when the priority is the same. |
21+
| Worker group | The task is assigned to the machines in the worker group for execution. If Default is selected, a worker machine will be randomly selected for execution. |
22+
| Task group name | The group in Resources, if not configured, it will not be used. |
23+
| Environment Name | Configure the environment in which to run the script. |
24+
| Number of failed retries | The number of times the task is resubmitted after failure. It supports drop-down and manual filling. |
25+
| Failure Retry Interval | The time interval for resubmitting the task if the task fails. It supports drop-down and manual filling. |
26+
| Timeout alarm | Check Timeout Alarm and Timeout Failure. When the task exceeds the "timeout duration", an alarm email will be sent and the task execution will fail. |
27+
| Dinky Address | The url for a dinky server. |
28+
| Dinky Task ID | The unique task id for a dinky task. |
29+
| Online Task | Specify whether the current dinky job is online. If yes, the submitted job can only be submitted successfully when it is published and there is no corresponding Flink job instance running. |
30+
31+
## Task Example
32+
33+
### Dinky Task Example
34+
35+
This example illustrates how to create a dinky task node.
36+
37+
![demo-dinky](../../../../img/tasks/demo/dinky.png)
38+
39+
![demo-get-dinky-task-id](../../../../img/tasks/demo/dinky_task_id.png)
40+

‎docs/docs/zh/guide/task/dinky.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Dinky
2+
3+
## Overview
4+
5+
`Dinky`任务类型，用于创建并执行`Dinky`类型任务以支撑一站式的开发、调试、运维 FlinkSQL、Flink Jar、SQL。worker 执行该任务的时候，会通过`Dinky API`触发`Dinky 的作业`。
6+
点击[这里](http://www.dlink.top/) 获取更多关于`Dinky`的信息。
7+
8+
## Create Task
9+
10+
- 点击项目管理-项目名称-工作流定义，点击"创建工作流"按钮，进入DAG编辑页面。
11+
- 工具栏中拖动 <img src="../../../../img/tasks/icons/dinky.png" width="15"/> 到画板中，即可完成创建。
12+
13+
## Task Parameter
14+
15+
| **参数** | **描述** |
16+
|-------------|--------------------------------------------------------------------|
17+
| 任务名称 | 设置任务的名称。一个工作流定义中的节点名称是唯一的。 |
18+
| 运行标志 | 标识这个节点是否可以正常调度。如果不需要执行，可以打开禁止执行开关。 |
19+
| 描述 | 描述该节点的功能。 |
20+
| 任务优先级 | worker 线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。 |
21+
| Worker 分组 | 任务分配给 worker 组的机器机执行，选择 Default，会随机选择一台 worker 机执行。 |
22+
| 任务组名 | 资源中的组，如果未配置，将不会使用。 |
23+
| 环境名称 | 配置运行脚本的环境。 |
24+
| 失败重试次数 | 任务失败重新提交的次数，支持下拉和手填。 |
25+
| 失败重试间隔 | 任务失败重新提交任务的时间间隔，支持下拉和手填。 |
26+
| 超时告警 | 勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败. |
27+
| Dinky 地址 | Dinky 服务的 url。 |
28+
| Dinky 任务 ID | Dinky 作业对应的唯一ID。 |
29+
| 上线作业 | 指定当前 Dinky 作业是否上线，如果是，则该被提交的作业只能处于已发布且当前无对应的 Flink Job 实例在运行才可提交成功。 |
30+
31+
## Task Example
32+
33+
### Dinky Task Example
34+
35+
这个示例展示了如何创建 Dinky 任务节点：
36+
37+
![demo-dinky](../../../../img/tasks/demo/dinky.png)
38+
39+
![demo-get-dinky-task-id](../../../../img/tasks/demo/dinky_task_id.png)
40+

‎docs/img/tasks/demo/dinky.png‎

143 KB
Loading

‎docs/img/tasks/demo/dinky_task_id.png‎

167 KB
Loading

‎docs/img/tasks/icons/dinky.png‎

6.3 KB
Loading

‎dolphinscheduler-task-plugin/dolphinscheduler-task-all/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,12 @@
177177
<artifactId>dolphinscheduler-task-dvc</artifactId>
178178
<version>${project.version}</version>
179179
</dependency>
180+
181+
<dependency>
182+
<groupId>org.apache.dolphinscheduler</groupId>
183+
<artifactId>dolphinscheduler-task-dinky</artifactId>
184+
<version>${project.version}</version>
185+
</dependency>
180186
</dependencies>
181187

182188
</project>

‎dolphinscheduler-task-plugin/dolphinscheduler-task-dinky/pom.xml‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
~ Licensed to the Apache Software Foundation (ASF) under one or more
4+
~ contributor license agreements. See the NOTICE file distributed with
5+
~ this work for additional information regarding copyright ownership.
6+
~ The ASF licenses this file to You under the Apache License, Version 2.0
7+
~ (the "License"); you may not use this file except in compliance with
8+
~ the License. You may obtain a copy of the License at
9+
~
10+
~ http://www.apache.org/licenses/LICENSE-2.0
11+
~
12+
~ Unless required by applicable law or agreed to in writing, software
13+
~ distributed under the License is distributed on an "AS IS" BASIS,
14+
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
~ See the License for the specific language governing permissions and
16+
~ limitations under the License.
17+
-->
18+
<project xmlns="http://maven.apache.org/POM/4.0.0"
19+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
20+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
21+
<parent>
22+
<artifactId>dolphinscheduler-task-plugin</artifactId>
23+
<groupId>org.apache.dolphinscheduler</groupId>
24+
<version>dev-SNAPSHOT</version>
25+
</parent>
26+
<modelVersion>4.0.0</modelVersion>
27+
<packaging>jar</packaging>
28+
29+
<artifactId>dolphinscheduler-task-dinky</artifactId>
30+
<dependencies>
31+
<dependency>
32+
<groupId>org.apache.dolphinscheduler</groupId>
33+
<artifactId>dolphinscheduler-spi</artifactId>
34+
<scope>provided</scope>
35+
</dependency>
36+
<dependency>
37+
<groupId>org.apache.dolphinscheduler</groupId>
38+
<artifactId>dolphinscheduler-task-api</artifactId>
39+
<version>${project.version}</version>
40+
</dependency>
41+
<dependency>
42+
<groupId>org.apache.httpcomponents</groupId>
43+
<artifactId>httpclient</artifactId>
44+
</dependency>
45+
<dependency>
46+
<groupId>org.apache.httpcomponents</groupId>
47+
<artifactId>httpcore</artifactId>
48+
</dependency>
49+
</dependencies>
50+
</project>

‎dolphinscheduler-task-plugin/dolphinscheduler-task-dinky/src/main/java/org/apache/dolphinscheduler/plugin/task/dinky/DinkyParameters.java‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.dolphinscheduler.plugin.task.dinky;
19+
20+
import org.apache.dolphinscheduler.plugin.task.api.model.ResourceInfo;
21+
import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
22+
import org.apache.dolphinscheduler.spi.utils.StringUtils;
23+
24+
import java.util.Collections;
25+
import java.util.List;
26+
27+
public class DinkyParameters extends AbstractParameters {
28+
29+
/**
30+
* parameters for dinky Open API
31+
*
32+
* @see <a href="http://www.dlink.top/docs/administrator_guide/studio/openapi">Dinky_Open_API</a>
33+
*/
34+
private String address;
35+
private String taskId;
36+
private boolean online = false;
37+
38+
@Override
39+
public boolean checkParameters() {
40+
return StringUtils.isNotEmpty(this.address) && StringUtils.isNotEmpty(this.taskId);
41+
}
42+
43+
@Override
44+
public List<ResourceInfo> getResourceFilesList() {
45+
return Collections.emptyList();
46+
}
47+
48+
public String getAddress() {
49+
return address;
50+
}
51+
52+
public void setAddress(String address) {
53+
this.address = address;
54+
}
55+
56+
public String getTaskId() {
57+
return taskId;
58+
}
59+
60+
public void setTaskId(String taskId) {
61+
this.taskId = taskId;
62+
}
63+
64+
public boolean isOnline() {
65+
return online;
66+
}
67+
68+
public void setOnline(boolean online) {
69+
this.online = online;
70+
}
71+
72+
@Override
73+
public String toString() {
74+
return "DinkyParameters{" +
75+
"address='" + address + '\'' +
76+
", taskId='" + taskId + '\'' +
77+
", online='" + online + '\'' +
78+
'}';
79+
}
80+
}

0 commit comments

Comments
 (0)