Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to fail the gh-ost process on SQL warnings #1498

Open
grodowski opened this issue Feb 13, 2025 · 1 comment
Open

Add option to fail the gh-ost process on SQL warnings #1498

grodowski opened this issue Feb 13, 2025 · 1 comment

Comments

@grodowski
Copy link

My team is investigating using Gh-ost for online schema migrations and we'd like to propose a feature we've been relying on in our current approach with LHM.

A new CLI option (raise_on_warnings?) would query for SHOW WARNINGS after each insert batch and fail the migration if warnings are found. This would optionally prevent data loss when e.g.:

  • adding a unique index with duplicate rows
  • changing table's collation causing new duplicates
  • adding a not null constraint with null values
  • resizing columns

An equivalent in LHM is raise_on_warnings: https://github.com/Shopify/lhm/blob/46204927a2933cfe239c77ac84ab58b5c7deb3a0/lib/lhm/chunker.rb#L92-L100

I have a prototype that shows the direction of how this option could be implemented in Applier.ApplyIterationInsertQuery: Shopify#1

Some open questions:

  • Is it desired to always process warnings (with the CLI option enabled) or only if a row count discrepancy has been detected. The latter is the current, less strict behaviour of Shopify/lhm

I'd love to hear your suggestions before we propose a PR contribution, thanks!

@meiji163
Copy link
Contributor

meiji163 commented Feb 24, 2025

We think this is a good feature to have in gh-ost. I believe the lhm approach would be best: check warnings if there is a discrepancy in rows affected, then raise any warnings that aren't "Duplicate entry for <shared unique key>".

We welcome a PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants