Skip to content

Sessions: attempts on the explicit recovery VM, without the selector - #314

Open
slinkydeveloper wants to merge 1 commit into
mainfrom
explicit-recovery-vm
Open

slinkydeveloper wants to merge 1 commit into
mainfrom
explicit-recovery-vm

Conversation

@slinkydeveloper

Copy link
Copy Markdown
Contributor

Session objects now run their attempts on shared-core's new ExplicitRecoveryVM (restatedev/sdk-shared-core#118), instead of the SDK's journaled context with a selector patched in.

Changes

  • Binding: PyExplicitRecoveryVM (src/explicit_recovery.rs). Commands return handles, and results come back as notifications. The database's own handles are settled in Rust.
  • Driver (sessions/_driver.py):
    • one input pump, the only reader of the request body;
    • a tick after every call into the VM, which writes the output, settles the futures, and notices the end of the attempt;
    • cancellation through is_cancelled();
    • heartbeats while an annotation is open.
  • Store: a commit syncs with one acked write of the database's head, instead of an empty sync run.
  • Removed: the selector, and Sessions: a handler that watches for its cancellation is not suspended on it #312's working run. Nothing suspends on the cancel signal anymore.
  • Dependency: shared-core is patched to the PR's branch until it's released.

Testing

  • pytest tests/*: 691 passed. ruff, mypy, pyright and clippy are clean.
  • Locally against Restate 1.8.1: tutorial lessons 02, 05, 06 and 08, and basic/notes.py.
  • Restate Cloud through the tunnel:
    • writes, views, retries, calls and SQL awaited together, transaction sends, traces;
    • shutdown and resume on a long call;
    • heartbeats;
    • implicit cancellation, including the child call.
  • Not run: the OpenAI examples, because the API key in restate.env has expired.

🤖 Generated with Claude Code

A session object's attempts no longer run on the SDK's journaled context with a selector
patched in to fake one await point over do_progress: they run on shared-core's
ExplicitRecoveryVM (restatedev/sdk-shared-core#118), which never replays and has no await
point at all.

- src/explicit_recovery.rs: PyExplicitRecoveryVM. Commands return handles, results come
  back as notifications; those of the database's own handles go straight to the Rust store.
- sessions/_driver.py: one attempt. One input pump, the only reader of the body; a tick
  after every call into the VM writes the output, settles the futures, and notices the end
  of the attempt (a shutdown when Restate closes the input, a failure, or the end).
  Cancellation: the VM's is_cancelled(), checked at every tick. An open annotation sends
  heartbeats, so a long model request is not shut down for inactivity.
- The store over the VM syncs a commit with one acked write of the database's head
  (KvStore::put_and_sync), instead of an empty run.
- Handler.attempt_runner: the server hands a session object's request to it.
- The selector and its working run (#312) are gone: nothing suspends on the cancel signal.
- shared-core is patched to the PR's branch until it is released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Implicit cancellation (explicit_cancellation=False, the default for session handlers) can now be silently dropped. If no future is pending when the cancellation arrives, nothing fails, and the handler runs to a normal answer. Details inline.

Reviewed changes

The whole PR at c5e1f7d: session object attempts move from the SDK's journaled context and selector onto shared-core's ExplicitRecoveryVM.

  • New VM binding — src/explicit_recovery.rs adds PyExplicitRecoveryVM. Actions return handles, and notifications are routed either to Rust waiters (wait_in_rust) or to Python.
  • Attempt driver — sessions/_driver.py adds AttemptDriver (input pump, the tick, cancellation, heartbeats) and VmHost. server.py hands off to Handler.attempt_runner.
  • Store sync by ack — VmStore reads, writes and lists keys directly on the VM, and syncs with an acked write of the head (or of v22/sync). SQL_WAIT becomes SQL_COMMIT, which runs the outbox callbacks between the write and the wait for its ack.
  • Selector removed — _selector.py and its tests are deleted, along with #312's working run.
  • Nested annotations — annotate(..., parent=) adds trace parents.

ℹ️ shared-core is pinned to a git branch

Cargo.toml patches restate-sdk-shared-core to the explicit-recovery-vm branch of restatedev/sdk-shared-core#118. A release built from this would depend on a moving branch. Switch to a published shared-core version before merging.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

self.act(operator.methodcaller("cancel_invocation", call.known_invocation_id()))
self._cancelling = _quiet(asyncio.ensure_future(self.cancel_calls()))
cancelled = TerminalError("cancelled", 409)
for handle, future in list(self._futures.items()):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With implicit cancellation, _cancelled runs once: on the tick that first sees vm.is_cancelled(). It fails only the futures pending at that moment. If nothing is pending then, the cancellation is lost:

  • Local work. The cancellation arrives while the handler awaits a model request. The later ctx.call(...) completes normally, and its child is never cancelled.
  • Earlier attempt. The cancellation arrived in an earlier attempt. start() schedules the tick before run() creates the handler's task, so _cancelled() sees no futures and no calls. The handler then reruns to a normal answer.

The old path delivered the cancellation at the handler's next await: the regular VM's do_progress returns CancelSignalReceived and only then flips it. I reproduced the drop with a fake VM: cancellation.done() is True, yet a call made afterwards returns its result.

Technical details
# Implicit cancellation dropped when nothing is pending at its arrival

## Affected sites
- python/restate/sessions/_driver.py:303 — `_tick` calls `_cancelled()` once, when `vm.is_cancelled()` first turns true ("acted on once")
- python/restate/sessions/_driver.py:369-387 — `_cancelled` fails only `self._futures` present now, and cancels only `self.calls` present now
- python/restate/sessions/_driver.py:146-167 — `start()` schedules the first tick before `run()` creates the handler task, so a cancellation from an earlier attempt is handled before the handler has any futures

## Required outcome
- With `explicit_cancellation=False`, a cancellation that finds nothing pending is still delivered: the handler's next Restate await fails with `TerminalError("cancelled", 409)`, and calls made from then on are cancelled. This matches the old selector and the regular VM.
- This holds both for a cancellation arriving during local work and for one received by an earlier attempt.
- Add an in-process test of each case, e.g. a fake VM whose `is_cancelled()` is true before the handler's first action. The new Docker test only covers a cancellation while a call is pending.

## Suggested approach (optional)
- Keep a "cancellation not yet delivered" flag. Clear it once `_cancelled` actually fails at least one future. While it is set, `future()` (or `act()`) fails the new future with 409 and cancels calls as they are created.


T = typing.TypeVar("T")

HEARTBEAT_INTERVAL = timedelta(seconds=20)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The heartbeat interval is fixed at 20s. SessionObject and its handlers accept inactivity_timeout, though. Any object configured below 20s gets its annotated work shut down and restarted. Your own daydreams test shows those restarts don't count against the retry policy, so a model request longer than the timeout restarts forever. Consider deriving the interval from the handler's or service's inactivity_timeout when one is set, e.g. a third of it, capped at 20s.

Comment on lines +125 to +128
self.calls: list[_CallFuture] = []
# The cancellation of the calls whose invocation id a cancellation waits for.
self._cancelling: asyncio.Future[Any] | None = None
"""The attempt's calls, for ``cancel_calls``."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This attribute docstring is attached to _cancelling, but it describes calls.

Suggested change
self.calls: list[_CallFuture] = []
# The cancellation of the calls whose invocation id a cancellation waits for.
self._cancelling: asyncio.Future[Any] | None = None
"""The attempt's calls, for ``cancel_calls``."""
self.calls: list[_CallFuture] = []
"""The attempt's calls, for ``cancel_calls``."""
# The cancellation of the calls whose invocation id a cancellation waits for.
self._cancelling: asyncio.Future[Any] | None = None

@github-actions

Copy link
Copy Markdown

Test Results

  8 files  ±0    8 suites  ±0   3m 58s ⏱️ -17s
 71 tests ±0   71 ✅ ±0  0 💤 ±0  0 ❌ ±0 
322 runs  ±0  322 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit c5e1f7d. ± Comparison against base commit 377d634.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant