Uploaded image for project: 'camunda BPM'
  1. camunda BPM
  2. CAM-9562

External tasks are not unlocked in case of orphaned async fetch and lock requests

    Details

      Description

      Steps to reproduce

      1. A task executor sends an async 'fetch and lock' request to the REST API with an asyncResponseTimeout
      2. The REST API queues the request as no external tasks are available to fetch and lock; The connection is kept alive
      3. The task executor does not wait for the async response and cancels the connection
      4. An external task instance is started

      Observed behavior
      The external task instance is fetched and locked although no client is actively waiting for it.

      Expected behavior
      The fetched and locked external task is unlocked if the response could not be sent back to the client as the client-side socket has been cleaned-up by the operating system.

      Hints
      The server can only unlock the fetched and locked external task if the client-side socket has been cleaned-up by the kernel. If the client-side socket is in state FIN_WAIT_2, the server completes the cancelled request successfully. This is an operating system specific and beyond control of Camunda BPM.

        Issue Links

          Activity

          Hide
          roman.smirnov Smirnov Roman added a comment -

          Possible solution to avoid this: Add a configuration flag that indicates orphaned requests can be cancelled when they have the same worker id.

          Show
          roman.smirnov Smirnov Roman added a comment - Possible solution to avoid this: Add a configuration flag that indicates orphaned requests can be cancelled when they have the same worker id.
          Hide
          thorben.lindhauer Thorben Lindhauer added a comment -

          Implemented solution: There is a configuration flag in web.xml called fetch-and-lock-unique-worker-request that if activated ensures that only at most one request per worker id is present at all times. That way, orphaned requests are cleared as soon as the worker re-connects. This can be combined with manually unlocking tasks when re-connecting to solve the overall problem that tasks are locked for a worker that never receives the task.

          Show
          thorben.lindhauer Thorben Lindhauer added a comment - Implemented solution: There is a configuration flag in web.xml called fetch-and-lock-unique-worker-request that if activated ensures that only at most one request per worker id is present at all times. That way, orphaned requests are cleared as soon as the worker re-connects. This can be combined with manually unlocking tasks when re-connecting to solve the overall problem that tasks are locked for a worker that never receives the task.

            People

            • Assignee:
              Unassigned
              Reporter:
              tassilo.weidner Tassilo Weidner
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development