Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spirit fails on metadata lock when table name is really long #347

Closed
pamelayin opened this issue Sep 19, 2024 · 4 comments · Fixed by #350
Closed

Spirit fails on metadata lock when table name is really long #347

pamelayin opened this issue Sep 19, 2024 · 4 comments · Fixed by #350
Assignees

Comments

@pamelayin
Copy link
Collaborator

Square has some apps with long table names. It fails to start due to lock name over the length limit specified here.

@morgo
Copy link
Collaborator

morgo commented Sep 20, 2024

Thanks for the bug report. I'll look into if this can be fixed.

64 chars is the limit for metadata locks, but what we can do is try and take a shorter lock name.

@morgo morgo self-assigned this Sep 20, 2024
@morgo
Copy link
Collaborator

morgo commented Sep 27, 2024

I looked at this. This is a regression of #342

The maximum tablename supported by default is 54 characters (versus in MySQL: 64). This is because we need to support a table of the name:

_%s_sentinel

However, when the option SkipDropAfterCutover is enabled, the tableName must be 43 characters or less. These are well tested in https://github.com/cashapp/spirit/blob/main/pkg/check/tablename.go .

The regression is because the MDL lock is using the full quoted name of the table and the schema (which our schema names tend to be large too, so it could eat into the maximum size quickly). We'll have to change it back to just being the table name. I will ask @prudhvi to work on it, since it affects some of his other code.

@pamelayin
Copy link
Collaborator Author

We implemented similar lock feature to shift in general and ran into a table name that was 47 characters long. On shift, we created lock name shift_schema_tablename which became 81 characters long altogether. On our end, we're thinking of hashing the table name as potential solution.

@morgo
Copy link
Collaborator

morgo commented Sep 27, 2024

Our fix here is just to use table name and not schema. Thus, you will no longer be able to run concurrent modifications across schemas with tables of the same name. I think this is an acceptable limitation for now.

We can not reduce the 54/43 char limit much further, since these are unhashed table objects. We could document it better though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants