Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.1.0
-
None
Description
This is similar to SPARK-6614, but there a race condition where a task may fail (e.g. Executor heartbeat timeout) and still manage to go through the commit protocol successfully. After this any retries of the task will fail indefinitely because of TaskCommitDenied.
Attachments
Issue Links
- relates to
-
SPARK-19790 OutputCommitCoordinator should not allow another task to commit after an ExecutorFailure
- Resolved
- links to