You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(janitor): skip cleanup gracefully when get_data_object fails after delete error - #6223
In SnapshotEvaluator._cleanup_snapshot() (sqlmesh/core/snapshot/evaluator.py), if evaluation_strategy.delete() raises an exception, the except Exception: block calls �dapter.get_data_object(table_name) unguarded. On engines such as Microsoft Fabric where catalog switching or connection teardown fails on a missing/stale catalog, get_data_object() also raises, aborting the janitor batch before delete_expired_snapshots() can delete the expired snapshot records.
Fix
Wrapped �dapter.get_data_object(table_name) inside _cleanup_snapshot()'s except Exception handler in a ry/except, logging a warning and continuing if existence verification itself fails, while still re-raising the original delete error when get_data_object() confirms the table still exists.
Verification
Added ests/core/test_cleanup_snapshot.py covering both get_data_object() failure (skip + on_complete) and confirmed-existing table (re-raises).
Ran pytest -o addopts="" --noconftest tests/core/test_cleanup_snapshot.py: 2 passed.
Summary by CodeRabbit
Bug Fixes
Snapshot cleanup now continues when checking whether a table exists also fails. If the table is confirmed to remain after a deletion error, the error is still raised.
No actionable comments were generated in the recent review.
Walkthrough
Snapshot cleanup now handles errors from the data-object lookup after a table deletion fails. Tests cover lookup errors and the case where the table still exists.
If the data-object lookup raises, cleanup logs a warning and continues. If the table exists, cleanup re-raises the original deletion error. Tests cover both cases.
Snapshot cleanup no longer aborts when the post-failure existence check itself errors; it logs a warning and moves on. Confirmed-existing tables still surface the original error. The risk of merging is low and the change is contained.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: the janitor cleanup now continues when get_data_object fails after a delete error. It matches the raw summary, which describes the same handling in _cleanup_snapshot. …
Description check
The description follows the template. It has a Problem section, a Fix section, and a Verification section, which covers the Test Plan content. It names the file, the change, the tests added, and the p…
Linked Issues check
Issue #5775 is the only active direct target. Its suggested fix has two options: (a) catch exceptions from get_data_object in _cleanup_snapshot and treat the table as unknown or skip, or (b) make the …
Out of Scope Changes check
The whole-PR diff against the merge base changes only sqlmesh/core/snapshot/evaluator.py (the _cleanup_snapshot exception handling) and adds tests/core/test_cleanup_snapshot.py. The evaluator change i…
Full details: Title check
Explanation
The title names the main change: the janitor cleanup now continues when get_data_object fails after a delete error. It matches the raw summary, which describes the same handling in _cleanup_snapshot. The title follows the conventional commit style and is 83 characters long, which is acceptable.
Full details: Description check
Explanation
The description follows the template. It has a Problem section, a Fix section, and a Verification section, which covers the Test Plan content. It names the file, the change, the tests added, and the pytest result. The Checklist items from the template are missing: style run, DCO sign-off, and full test-suite status are not stated. The description also has no section headings that match the template exactly.
Resolution
Optionally rename the Verification section to Test Plan to match the template. Add the checklist items from the template, and state whether make style was run, whether commits are signed off, and whether make fast-test passes.
Full details: Linked Issues check
Explanation
Issue #5775 is the only active direct target. Its suggested fix has two options: (a) catch exceptions from get_data_object in _cleanup_snapshot and treat the table as unknown or skip, or (b) make the Fabric adapter handle catalog context before the query. The PR implements option (a). In sqlmesh/core/snapshot/evaluator.py, the get_data_object call is now wrapped in its own try/except. On failure, it logs a warning and returns without raising, so the janitor continues to delete_expired_snapshots. When get_data_object returns a non-None object, the original delete error is still re-raised. tests/core/test_cleanup_snapshot.py covers both paths. Limitation: Mode B in the issue (drop fails and the table still exists) still re-raises. In that case, snapshot records are still retained. The issue does not require this case to be fixed by the suggested approach, and the PR description does not claim it. The issue remains open for that case.
Full details: Out of Scope Changes check
Explanation
The whole-PR diff against the merge base changes only sqlmesh/core/snapshot/evaluator.py (the _cleanup_snapshot exception handling) and adds tests/core/test_cleanup_snapshot.py. The evaluator change is the targeted fix for #5775. The test file verifies that fix. No unrelated changes were found in the inspected diff.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: bad670a7-f412-4e2f-8f3a-81fc360404f4
Commits
Reviewing files that changed from the base of the PR and between 3df452c and 67860f9.
Files selected for processing: 2
sqlmesh/core/snapshot/evaluator.py
tests/core/test_cleanup_snapshot.py
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5775.
Problem
In SnapshotEvaluator._cleanup_snapshot() (sqlmesh/core/snapshot/evaluator.py), if evaluation_strategy.delete() raises an exception, the except Exception: block calls �dapter.get_data_object(table_name) unguarded. On engines such as Microsoft Fabric where catalog switching or connection teardown fails on a missing/stale catalog, get_data_object() also raises, aborting the janitor batch before delete_expired_snapshots() can delete the expired snapshot records.
Fix
Verification
Summary by CodeRabbit