Skip to content

fix(janitor): skip cleanup gracefully when get_data_object fails after delete error - #6223

Open
boomesh11 wants to merge 1 commit into
SQLMesh:mainfrom
boomesh11:fix/cleanup-snapshot-resilient-get-data-object
Open

boomesh11 wants to merge 1 commit into
SQLMesh:mainfrom
boomesh11:fix/cleanup-snapshot-resilient-get-data-object

Conversation

@boomesh11

@boomesh11 boomesh11 commented Nov 16, 2026 •

Copy link
Copy Markdown

Fixes #5775.

Problem

In SnapshotEvaluator._cleanup_snapshot() (sqlmesh/core/snapshot/evaluator.py), if evaluation_strategy.delete() raises an exception, the except Exception: block calls �dapter.get_data_object(table_name) unguarded. On engines such as Microsoft Fabric where catalog switching or connection teardown fails on a missing/stale catalog, get_data_object() also raises, aborting the janitor batch before delete_expired_snapshots() can delete the expired snapshot records.

Fix

  • Wrapped �dapter.get_data_object(table_name) inside _cleanup_snapshot()'s except Exception handler in a ry/except, logging a warning and continuing if existence verification itself fails, while still re-raising the original delete error when get_data_object() confirms the table still exists.

Verification

  • Added ests/core/test_cleanup_snapshot.py covering both get_data_object() failure (skip + on_complete) and confirmed-existing table (re-raises).
  • Ran pytest -o addopts="" --noconftest tests/core/test_cleanup_snapshot.py: 2 passed.

Summary by CodeRabbit

  • Bug Fixes
    • Snapshot cleanup now continues when checking whether a table exists also fails. If the table is confirmed to remain after a deletion error, the error is still raised.

@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Snapshot cleanup now handles errors from the data-object lookup after a table deletion fails. Tests cover lookup errors and the case where the table still exists.

Changes: Snapshot cleanup

Layer / File(s) Summary
Handle cleanup verification failures
sqlmesh/core/snapshot/evaluator.py, tests/core/test_cleanup_snapshot.py
If the data-object lookup raises, cleanup logs a warning and continues. If the table exists, cleanup re-raises the original deletion error. Tests cover both cases.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: izeigerman

Merge Risk: Minimal · up to 67860

Snapshot cleanup no longer aborts when the post-failure existence check itself errors; it logs a warning and moves on. Confirmed-existing tables still surface the original error. The risk of merging is low and the change is contained.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: the janitor cleanup now continues when get_data_object fails after a delete error. It matches the raw summary, which describes the same handling in _cleanup_snapshot. …
Description check Passed The description follows the template. It has a Problem section, a Fix section, and a Verification section, which covers the Test Plan content. It names the file, the change, the tests added, and the p…
Linked Issues check Passed Issue #5775 is the only active direct target. Its suggested fix has two options: (a) catch exceptions from get_data_object in _cleanup_snapshot and treat the table as unknown or skip, or (b) make the …
Out of Scope Changes check Passed The whole-PR diff against the merge base changes only sqlmesh/core/snapshot/evaluator.py (the _cleanup_snapshot exception handling) and adds tests/core/test_cleanup_snapshot.py. The evaluator change i…
Full details: Title check

Explanation

The title names the main change: the janitor cleanup now continues when get_data_object fails after a delete error. It matches the raw summary, which describes the same handling in _cleanup_snapshot. The title follows the conventional commit style and is 83 characters long, which is acceptable.

Full details: Description check

Explanation

The description follows the template. It has a Problem section, a Fix section, and a Verification section, which covers the Test Plan content. It names the file, the change, the tests added, and the pytest result. The Checklist items from the template are missing: style run, DCO sign-off, and full test-suite status are not stated. The description also has no section headings that match the template exactly.

Resolution

Optionally rename the Verification section to Test Plan to match the template. Add the checklist items from the template, and state whether make style was run, whether commits are signed off, and whether make fast-test passes.

Full details: Linked Issues check

Explanation

Issue #5775 is the only active direct target. Its suggested fix has two options: (a) catch exceptions from get_data_object in _cleanup_snapshot and treat the table as unknown or skip, or (b) make the Fabric adapter handle catalog context before the query. The PR implements option (a). In sqlmesh/core/snapshot/evaluator.py, the get_data_object call is now wrapped in its own try/except. On failure, it logs a warning and returns without raising, so the janitor continues to delete_expired_snapshots. When get_data_object returns a non-None object, the original delete error is still re-raised. tests/core/test_cleanup_snapshot.py covers both paths. Limitation: Mode B in the issue (drop fails and the table still exists) still re-raises. In that case, snapshot records are still retained. The issue does not require this case to be fixed by the suggested approach, and the PR description does not claim it. The issue remains open for that case.

Full details: Out of Scope Changes check

Explanation

The whole-PR diff against the merge base changes only sqlmesh/core/snapshot/evaluator.py (the _cleanup_snapshot exception handling) and adds tests/core/test_cleanup_snapshot.py. The evaluator change is the targeted fix for #5775. The test file verifies that fix. No unrelated changes were found in the inspected diff.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bad670a7-f412-4e2f-8f3a-81fc360404f4
Commits

Reviewing files that changed from the base of the PR and between 3df452c and 67860f9.

Files selected for processing: 2

  • sqlmesh/core/snapshot/evaluator.py
  • tests/core/test_cleanup_snapshot.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Fabric/TSQL] Removed models persist in state snapshot when using virtual_environment_mode = "dev_only"

1 participant