You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Apache DolphinScheduler is the modern data workflow orchestration platform with powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available `out of the box`
19
18
20
19
The key features for DolphinScheduler are as follows:
20
+
21
21
- Easy to deploy, we provide 4 ways to deploy, such as Standalone deployment,Cluster deployment,Docker / Kubernetes deployment and Rainbond deployment
22
22
- Easy to use, there are four ways to create workflows:
23
+
23
24
- Visually, create tasks by dragging and dropping tasks
24
25
-[PyDolphinScheduler](https://dolphinscheduler.apache.org/python/main/index.html), Creating workflows via Python API, aka workflow-as-code
25
26
- Yaml definition, mapping yaml into workflow(have to install PyDolphinScheduler currently)
26
27
- Open API, Creating workflows
27
28
28
29
- Highly Reliable,
29
-
DolphinScheduler uses a decentralized multi-master and multi-worker architecture, which naturally supports horizontal scaling and high availability
30
+
DolphinScheduler uses a decentralized multi-master and multi-worker architecture, which naturally supports horizontal scaling and high availability
30
31
- High performance, its performance is N times faster than other orchestration platform and it can support tens of millions of tasks per day
Please refer the official website document: [QuickStart in Docker](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/start/docker.html)
57
+
Please refer the official website document: [QuickStart in Docker](https://dolphinscheduler.apache.org/#/en-us/docs/3.1.2/guide/start/docker)
57
58
58
59
## QuickStart in Kubernetes
59
60
60
-
Please refer to the official website document: [QuickStart in Kubernetes](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/installation/kubernetes.html)
61
+
Please refer to the official website document: [QuickStart in Kubernetes](https://dolphinscheduler.apache.org/#/en-us/docs/3.1.2/guide/installation/kubernetes)
You are very welcome to communicate with the developers and users of Dolphin Scheduler. There are two ways to find them:
85
+
84
86
1. Join the Slack channel [Slack](https://asf-dolphinscheduler.slack.com/)
85
87
2. Follow the [Twitter account of DolphinScheduler](https://twitter.com/dolphinschedule) and get the latest news on time
86
88
87
89
## How to Contribute
88
90
89
91
The community welcomes everyone to contribute, please refer to this page to find out more: [How to contribute](docs/docs/en/contribute/join/contribute.md).
90
92
91
-
92
93
## Thanks
93
94
94
95
DolphinScheduler is based on a lot of excellent open-source projects, such as Google guava, grpc, netty, quartz, and many open-source projects of Apache and so on.
Copy file name to clipboardExpand all lines: docs/docs/en/contribute/architecture-design.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,13 +15,13 @@ Before explaining the architecture of the schedule system, let us first understa
15
15
16
16
**Process definition**: Visualization **DAG** by dragging task nodes and establishing associations of task nodes
17
17
18
-
**Process instance**: A process instance is an instantiation of a process definition, which can be generated by manual startup or scheduling. The process definition runs once, a new process instance is generated
18
+
**Process instance**: A process instance is an instantiation of a process definition, which can be generated by manual startup or scheduling. The process definition runs once, a new process instance is generated
19
19
20
20
**Task instance**: A task instance is the instantiation of a specific task node when a process instance runs, which indicates the specific task execution status
21
21
22
22
**Task type**: Currently supports SHELL, SQL, SUB_PROCESS (sub-process), PROCEDURE, MR, SPARK, PYTHON, DEPENDENT (dependency), and plans to support dynamic plug-in extension, note: the sub-**SUB_PROCESS** is also A separate process definition that can be launched separately
23
23
24
-
**Schedule mode** : The system supports timing schedule and manual schedule based on cron expressions. Command type support: start workflow, start execution from current node, resume fault-tolerant workflow, resume pause process, start execution from failed node, complement, timer, rerun, pause, stop, resume waiting thread. Where **recovers the fault-tolerant workflow** and **restores the waiting thread** The two command types are used by the scheduling internal control and cannot be called externally
24
+
**Schedule mode** : The system supports timing schedule and manual schedule based on cron expressions. Command type support: start workflow, start execution from current node, resume fault-tolerant workflow, resume pause process, start execution from failed node, complement, timer, rerun, pause, stop, resume waiting thread. Where **recovers the fault-tolerant workflow** and **restores the waiting thread** The two command types are used by the scheduling internal control and cannot be called externally
25
25
26
26
**Timed schedule**: The system uses **quartz** distributed scheduler and supports the generation of cron expression visualization
27
27
@@ -48,7 +48,7 @@ Before explaining the architecture of the schedule system, let us first understa
48
48
49
49
#### 2.2 Architectural description
50
50
51
-
***MasterServer**
51
+
-**MasterServer**
52
52
53
53
MasterServer adopts the distributed non-central design concept. MasterServer is mainly responsible for DAG task split, task submission monitoring, and monitoring the health status of other MasterServer and WorkerServer.
54
54
When the MasterServer service starts, it registers a temporary node with Zookeeper, and listens to the Zookeeper temporary node state change for fault tolerance processing.
@@ -63,7 +63,7 @@ Before explaining the architecture of the schedule system, let us first understa
63
63
64
64
-**MasterTaskExecThread** is mainly responsible for task persistence
65
65
66
-
***WorkerServer**
66
+
-**WorkerServer**
67
67
68
68
- WorkerServer also adopts a distributed, non-central design concept. WorkerServer is mainly responsible for task execution and providing log services. When the WorkerServer service starts, it registers the temporary node with Zookeeper and maintains the heartbeat.
69
69
@@ -90,7 +90,7 @@ Before explaining the architecture of the schedule system, let us first understa
90
90
91
91
-**UI**
92
92
93
-
The front-end page of the system provides various visual operation interfaces of the system. For details, see the [quick start](https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/about/introduction.html) section.
93
+
The front-end page of the system provides various visual operation interfaces of the system. For details, see the [quick start](https://dolphinscheduler.apache.org/#/en-us/docs/3.1.2/about/introduction) section.
94
94
95
95
#### 2.3 Architectural Design Ideas
96
96
@@ -246,7 +246,7 @@ In the early scheduling design, if there is no priority design and fair scheduli
246
246
* task log appender
247
247
*/
248
248
Public classTaskLogAppenderextendsFileAppender<ILoggingEvent> {
0 commit comments