Skip to content

fix(browsers): Clean up BrowserPool and its plugins after a failed entry or exit - #2446

Open
Mantisus wants to merge 1 commit into
apify:v2from
Mantisus:browser-pool-lifecycle
Open

Mantisus wants to merge 1 commit into
apify:v2from
Mantisus:browser-pool-lifecycle

Conversation

@Mantisus

@Mantisus Mantisus commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Description

  • A failed or cancelled entry or exit of BrowserPool, PlaywrightBrowserPlugin or StagehandBrowserPlugin leaves it inactive, so it can be entered again. A pool whose entry fails exits the plugins it already entered, and a plugin whose entry fails stops the Playwright driver it started.
  • Exiting a pool closes every browser and plugin even when one of them fails, raises the first failure and logs the rest.

Issues

Testing

  • New unit tests for failed and cancelled entries and exits of the pool and both plugins.

✍️ Drafted by Claude Code

@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 14, 2026
@Mantisus Mantisus self-assigned this Nov 14, 2026
@Mantisus
Mantisus marked this pull request as draft November 14, 2026 22:16
@Mantisus
Mantisus force-pushed the browser-pool-lifecycle branch 3 times, most recently from 92e630a to 05d91fa Compare November 15, 2026 22:34
@Mantisus
Mantisus force-pushed the browser-pool-lifecycle branch from 05d91fa to 8626312 Compare November 18, 2026 14:08
@Mantisus
Mantisus marked this pull request as ready for review November 18, 2026 14:08
@Mantisus
Mantisus requested a review from vdusek November 18, 2026 14:08
half_started_context_manager = self._playwright_context_manager
self._playwright_context_manager = async_playwright()
try:
await half_started_context_manager.__aexit__(None, None, None)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: This stop has no time limit. The pool runs __aenter__ under asyncio.wait_for(..., operation_timeout), and wait_for can't raise until this rollback finishes. With a driver that doesn't exit when stdin closes, the pool entry now hangs. On v2 it logs "timed out, will be skipped" after operation_timeout (reproduced with a fake driver and a 1 s timeout). Could the stop get its own timeout, here and in StagehandBrowserPlugin.__aenter__?

✍️ Drafted by Claude Code

self._playwright_context_manager = async_playwright()
try:
await half_started_context_manager.__aexit__(None, None, None)
except Exception:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: When the driver never spawned (a missing executable, or a cancel right after the start), __aexit__ fails with AttributeError: 'PipeTransport' object has no attribute '_output'. Users then get this warning and a second traceback next to the real error. In the early-cancel case the driver still starts later and keeps running. Maybe log this at DEBUG, and narrow the comment above, since the driver isn't always stopped.

✍️ Drafted by Claude Code


try:
self._playwright = await self._playwright_context_manager.__aenter__()
except BaseException:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: __aexit__ now always resets the state, so the rollback could be try: await self.__aexit__(None, None, None) except Exception: logger.warning(...). That removes the near copy in the Stagehand plugin, where the variable name already differs, and the timeout from the comment below would live in one place.

✍️ Drafted by Claude Code

failures: list[BaseException] = []
try:
try:
await self._stop_recurring_tasks()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: If _stop_recurring_tasks() is cancelled, no browser is closed, but the finally still clears both lists and marks the pool inactive. The browsers are then lost, with their persistent temp directories. Could the browser closes run in a finally after the task stop?

✍️ Drafted by Claude Code

for plugin in self._plugins:
await plugin.__aexit__(exc_type, exc_value, exc_traceback)
try:
# Plugins left inactive by a timed-out entry have nothing to exit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This also skips the plugins after a timed-out one, which were never entered, since the except TimeoutError ends the loop. Keeping entered_plugins on self and exiting exactly those would make this explicit.

✍️ Drafted by Claude Code

],
)
async def test_failed_enter_stops_playwright(monkeypatch: pytest.MonkeyPatch, error: BaseException) -> None:
"""A plugin whose Stagehand client fails to start stops Playwright and can be entered again."""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The test covers the cancelled case too.

Suggested change
"""A plugin whose Stagehand client fails to start stops Playwright and can be entered again."""
"""A plugin whose Stagehand client fails to start or is cancelled stops Playwright and can be entered again."""

✍️ Drafted by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Let crawlers manage only the collaborators they create and extract per-run state

2 participants