-
-
Notifications
You must be signed in to change notification settings - Fork 866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
s3: Fix newline handling for text-mode files #1352
Conversation
7eae49e
to
9b1b76f
Compare
Thanks @skim618 for finishing this off |
There were two possible ways of doing this:
The changes in this PR implement (2).
|
@@ -123,7 +124,6 @@ def __init__(self, name, mode, storage, buffer_size=None): | |||
self._storage = storage | |||
self.name = name[len(self._storage.location) :].lstrip("/") | |||
self._mode = mode | |||
self._force_mode = (lambda b: b) if "b" in mode else (lambda b: b.decode()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are the changes to _force_mode
necessary? It's interesting that there are no failing tests when those are moved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the TextIOWrapper takes care of decoding the file as it's read, so this is no longer necessary
This change looks good but tests are failing. Is this due to 3.7 not being dropped yet? Once I release the next version the intent is to drop that. |
yes, looks like moto dropped 3.7 support: getmoto/moto@2fd5e80 I've fixed the tests by undoing my previous moto upgrade and instead pinning to <5 so the tests will run |
are those gcloud failures expected? I also get them locally. Seems unrelated to the changes in this PR though |
Seems like a breaking change related to a google cloud dependency. I'm moving your moto fix and that to a separate PR so I can unblock everything else. Thanks for flagging. |
This actually also exposed a bug in Google Cloud Storage handling of gzip files. I was looking into who wrote the tests in such a confusing way and of course it was...me. |
Alright we are back to green, go ahead and rebase. And thanks again. |
Moved this to #1381 |
This fixes the newline issue at #1351. Files open in text mode (
r
) now get newline handling consistent with other python file-like objects.run these specific tests with