Apache DolphinScheduler 3.4.1 Released with Task Dispatch Timeout Detection
Source: Dev.to
# Apache DolphinScheduler 3.4.1 Released
The **3.4.1** version of **Apache DolphinScheduler** has been officially released by the community.
As a maintenance release in the **3.4.x** series, this update focuses on:
- Improving scheduling stability
- Enhancing task‑execution control
- Fixing various system issues
## New Features
- **Task dispatch timeout detection** – automatically identifies and handles tasks that exceed their dispatch window.
- **Maximum‑runtime control** – allows you to set upper limits for workflow and task instance execution times.
## Additional Improvements
- Refined scheduling logic and plugin handling
- Updated API behavior for better consistency
- Optimized system documentation, development processes, and project structure
## Resources
- **Release notes:** [Apache DolphinScheduler 3.4.1 Release Notes](https://github.com/apache/dolphinscheduler/releases/tag/3.4.1)
- **Source code download:** [Apache DolphinScheduler 3.4.1 Source Archive](https://github.com/apache/dolphinscheduler/archive/refs/tags/3.4.1.zip)
---
*Source: [Dev.to article](https://dev.to/seatunnel/apache-dolphinscheduler-341-released-with-task-dispatch-timeout-detection-3i5c)*Key Highlights
1. Task Dispatch Timeout Detection
- Added task‑dispatch timeout logic to the Master scheduling module.
- When a task is sent to a Worker:
- If the Worker group does not exist or no Worker nodes are available, the scheduler detects the exception within a configurable period.
- The dispatch failure is then handled automatically, preventing tasks from staying in a waiting state indefinitely.
- Improves fault tolerance for resource‑related anomalies.
- (Related issues: #17795, #17796)
2. Configurable Maximum Runtime for Workflows & Tasks
- Users can now set a maximum runtime for:
- Workflow instances
- Task instances
- If execution exceeds the configured limit, a timeout is triggered and the system performs the appropriate handling (e.g., fail, retry, alert).
- Prevents hanging processes and uncontrolled resource consumption, enhancing overall operational controllability.
- (Related issues: #17931, #17932)
Key Fixes and Improvements
Scheduling System Stability
- Fixed task‑timeout alerts not being triggered (#17820, #17818)
- Fixed workflow failure strategy not taking effect (#17834, #17851)
- Automatically mark a task as failed when task‑execution context initialization fails (#17758, #17821)
- Fixed incorrect parallelism calculation in backfill tasks under parallel‑execution mode (#17831, #17853)
Database & Compatibility
- Fixed SQL execution errors for dependent tasks in PostgreSQL environments (#17690, #17837)
- Fixed mismatched
INT/BIGINTcolumn types in database tables (#17979, #17988)
API & Permission
- Removed the
WAIT_TO_RUNstate and added a FAILOVER state when querying workflow instances (#17838, #17839) - Added tenant validation for the Workflow API (#17969, #17970)
- Fixed an issue where non‑admin users could not delete their own Access Tokens (#17995, #17997)
Plugin & Task Execution
- Fixed incorrect JVM‑parameter position in Java Task (#17848, #17850)
- Fixed Procedure Task parameters not being passed correctly (#17967, #17968)
- Fixed
ProcedureTasknot returning parameters or executing stored‑procedure queries (#17971, #17973) - Fixed HTTP plugin not sending nested JSON structures (#17912, #17911)
- Fixed inconsistent timeout units in the HTTP alert plugin (#17915, #17920)
UI & Documentation
- Removed the STOP state from task instances in the UI (#17864, #17865)
- Fixed locks not being released when workflow‑definition list loading failed (#17984, #17989)
- Fixed Keycloak login‑icon 404 issue (#18006, #18007)
- Corrected errors in the installation documentation (#17901, #17903)
- Fixed SeaTunnel documentation link 404 issue (#17904, #17905)
In‑Depth Feature Analysis
In modern data‑platform architectures, scheduling systems serve as key infrastructure connecting various computing engines (e.g., Apache Spark, Flink, Hive). Production environments often face challenges such as:
- Worker resource anomalies that prevent task scheduling.
- Uncontrollable task execution time leading to unpredictable runtimes.
- Unstable plugin execution behavior that can cause failures.
New Features
| Feature | Description | Issue IDs |
|---|---|---|
| Task dispatch timeout detection | The scheduler quickly identifies anomalies when workers are missing or resources are unavailable, preventing tasks from waiting indefinitely. | #17795, #17796 |
| Maximum runtime control | Allows setting a maximum runtime for workflows or individual tasks. If a task exceeds this limit, the system can intervene, avoiding long‑running or hung tasks that waste resources. | #17931, #17932 |
These enhancements further improve DolphinScheduler’s stability and controllability in production‑grade data‑platform environments.
Acknowledgements
The release of Apache DolphinScheduler 3.4.1 would not have been possible without the contributions of community developers. Special thanks to the release manager @ruanwenjun and the following contributors:
- SbloodyS
- njnu‑seafish
- Mrhs121
- ylq5126
- qiong‑zhou
- XpengCen
- iampratap7997‑dot
- yzeng1618
- Alexander1902
- maomao199691
- asadjan4611
- dill21yu
Final Thoughts
Apache DolphinScheduler 3.4.1 is a maintenance release focused on improving scheduling stability and enhancing task‑runtime control.
With the introduction of scheduling fault‑tolerance mechanisms, maximum task‑runtime control, and numerous bug fixes, this version further strengthens the system’s reliability in production environments.
As the community continues to grow, Apache DolphinScheduler is steadily improving its capabilities in the data‑workflow orchestration space, providing enterprises with a more stable and efficient infrastructure for building modern data platforms. We welcome more contributors to join the community and help drive the development of the project forward.