Skip to content

Data streams: receive via FFI readers; add sendText / sendBytes / sendFile - #321

Draft
1egoman wants to merge 1 commit into
mainfrom
data-streams-ffi-receive-and-send
Draft

1egoman wants to merge 1 commit into
mainfrom
data-streams-ffi-receive-and-send

Conversation

@1egoman

@1egoman 1egoman commented Nov 20, 2026

Copy link
Copy Markdown

First of two PRs that move C++ data streams onto the livekit-ffi data streams API, the same port Python did in python-sdks#769. This one fixes receiving. The writer port is stacked on top of it in the follow-up PR.

Why

Since #318 (livekit-ffi 0.15.0), the FFI always advertises data streams v2 for every client. JS, Swift, Android, Python, Node, Rust and Unity senders therefore send C++ participants single-packet ("inline") and deflate-compressed streams. C++ rebuilt streams from the low-level stream_header/chunk/trailer_received events, which can't represent v2 traffic:

  • Inline streams never end. readAll() hangs forever.
  • Compressed streams arrive as raw deflate bytes.
  • Every incoming stream leaked. The FFI hands over an owned reader handle per stream in text/byte_stream_opened. C++ ignored it, so the FFI kept each stream's full content buffered for the life of the process.

C++-to-C++ still worked only because C++ sends v1 framing.

What changes

Receive

  • Room consumes text/byte_stream_opened:
    • If no handler is registered for the topic, it drops the reader handle immediately.
    • If a handler is registered, it starts *_read_incremental and routes the resulting *_stream_reader_events (keyed by reader handle) to the reader.
    • The low-level stream_*_received events are now ignored.
  • TextStreamReader / ByteStreamReader:
    • Abnormal ends (sender close reason, size limit, disconnect) throw DataStreamError, after any buffered chunks have been returned.
    • New close() stops reading early and wakes blocked reads.
    • New ByteStreamReader::readAll().
    • Room disconnect wakes blocked readers with "Disconnected while receiving".
  • RoomOptions::data_stream.max_payload_byte_length sets the receive size limit.

Send

  • LocalParticipant::sendText / sendBytes / sendFile (one-shot sends, compressed and single-packet when recipients support it). They block like performRpc and throw DataStreamError on failure.
  • StreamTextOptions / StreamByteOptions, with compress defaulting to true.
  • Sends throw once the room has disconnected instead of reaching the FFI. Rust unwrap()s the session there, and C++ turns an FFI panic into SIGTERM.

API compatibility: source compatible. Everything is additive except the behaviour change below. TextStreamOpenedEvent / ByteStreamOpenedEvent::reader_handle stays 0 and is now documented @deprecated. Existing writers are untouched here; FFI readers accept their v1 framing.

Behaviour change: readers now throw when a stream ends abnormally. Before, they silently returned truncated data.

Notes for reviewers

  • Reader events are routed by Room, not per-reader FfiClient listeners. The plan suggested one listener per reader. Routing through the room's existing listener avoids N listeners (each FFI event fans out to every listener) and any remove-listener-from-own-callback issues. It also keeps reader lifetime simple: the room holds the reader until EOS or shutdown.
  • info() data race. info() returned a reference that the FFI thread mutated at end of stream. End-of-stream attributes are now merged on the reading thread, when readNext() / readAll() reaches the end. The const& signature is unchanged, and info() is documented as not safe to call concurrently with readNext().
  • A reader's readNext() logs a warning if it would block on the FFI callback thread. That thread also delivers chunks, so blocking there is a real deadlock risk; this is now documented on the handler APIs.
  • Dropping handles after FfiClient::shutdown() is safe in C++ (pitfall 6 / python-sdks#857). The FFI library stays loaded, and drop_handle after dispose is a map removal.

Testing

All runs were local on macOS arm64, against livekit-server --dev with tokens from scripts/set-test-tokens.sh.

  • Unit tests: 381/381 pass. New coverage:
    • the reader state machine (chunks, end of stream, sticky errors, close() waking a blocked read, attribute merge)
    • option and info proto conversion
    • RoomOptions.data_stream → proto
    • sends refusing an uninitialized FFI
  • Integration, DataStreamE2ETest: 17/17 pass (3 existing, 14 new):
    • sendText small, 50 KB compressed, uncompressed, and multi-byte Unicode across chunk boundaries
    • sendBytes
    • sendFile, plus a missing file
    • send after disconnect
    • receive size limit
    • trailer attributes merged into the info
    • abnormal close
    • receiver disconnecting mid-stream
    • reader close() waking a pending read
    • handle release
  • Handle release check. StreamHandlesAreReleased sends 20 streams on an unhandled topic plus one handled stream, then checks that every reader handle is gone from the FFI. With the unhandled-topic drop removed, the test fails ("reader handle N was leaked").
  • Full integration suite: 82 pass, 1 skipped (token-server endpoint test; no LIVEKIT_CREATE_TOKEN_URL locally). 5 fail, and all 5 also fail on main on this machine:
    • DataTrackE2ETest.{DefineAndGetSchema, DefineAndGetCustomSchemaEncoding, DefineDuplicateSchemaFails}
    • RoomTest.ServerDeletedRoomDisconnectsAndTearsDownLocally
    • PlatformAudioIntegrationTest.PlatformAudioFramesReachRemote (local audio device)
  • Cross-SDK check (manual). A Python participant (PyPI livekit 1.3.0) called send_text with a short message and a 50 KB message to a C++ receiver:
    • On main, the C++ side received 0 of 2 streams; both hung.
    • On this branch, both arrived intact, in 5 of 6 runs. The first run received nothing; I couldn't reproduce that in the following runs and believe it was local environment noise.
  • Tooling:
    • ./scripts/generate-docs.sh passes with no warnings.
    • check_no_private_symbols.py passes on a release build.
    • clang-format is clean. I ran clang-format 23 directly; the cpp-tools/clang-format.sh wrapper fails to parse under macOS's bash 3.2.
    • clang-tidy (23 locally; CI uses 19) reports nothing on lines this PR adds.
  • Not run: Windows and Linux. CI will cover them; watch the uint64_t / size_t conversions and the sendFile basename handling on Windows.

Not in this PR

  • The cpp-example-collection/simple_data_stream update is deferred and will be a follow-up.
  • Docs site C++ tabs (sendBytes, compression, limits) and changelog notes should follow once a release ships.

🤖 Generated with Claude Code

…sendFile

Incoming data streams were rebuilt from the low-level stream_header/chunk/
trailer events. Since livekit-ffi 0.15.0 always advertises data streams v2,
modern senders send C++ participants single-packet and deflate-compressed
streams, which that path can't read: inline streams hang readAll() forever
and compressed chunks arrive as raw deflate bytes. The FFI also handed C++ an
owned reader handle for every incoming stream, which C++ ignored, so every
stream's content stayed buffered in the FFI for the life of the process.

Receive:
- Room consumes text/byte_stream_opened: unhandled topics drop the reader
  handle; handled topics start read_incremental and route the resulting
  *_stream_reader_events (keyed by reader handle) to the reader. The
  low-level stream_*_received events are now ignored.
- Readers surface abnormal ends (sender close reason, size limit, disconnect)
  as DataStreamError after draining buffered chunks, merge end-of-stream
  attributes into info() on the reading thread, gain close(), and
  ByteStreamReader gains readAll(). Room shutdown wakes blocked readers.
- RoomOptions::data_stream.max_payload_byte_length sets the receive limit.

Send:
- LocalParticipant::sendText / sendBytes / sendFile use the FFI one-shot
  sends (compression and single-packet delivery when recipients support
  them). They throw DataStreamError on failure and refuse to reach the FFI
  once the room has disconnected, since the FFI panics there.

TextStreamOpenedEvent / ByteStreamOpenedEvent::reader_handle stays 0 and is
documented as deprecated. Existing writers are unchanged (v1 framing is still
accepted by FFI readers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
///
/// Derives from std::runtime_error, so existing
/// `catch (const std::runtime_error&)` handlers keep working.
class LIVEKIT_API DataStreamError : public std::runtime_error {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open question: the new data stream calls throw DataStreamError. Readers now also throw when a stream ends with an error (see readNext below), where they used to return truncated data silently. That keeps the existing data stream API's throw style, but AGENTS.md prefers Result<T, E> for typed errors, and the reader change is a behaviour change for existing callers. Is throwing the right call, or should these return Result / expose an error() accessor instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant