Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit memory usage when loading files from S3 #140

Merged
merged 1 commit into from
Mar 5, 2024
Merged

Conversation

RealOrangeOne
Copy link
Member

@RealOrangeOne RealOrangeOne commented Mar 4, 2024

Description of Changes Made

By default, files are always loaded into memory, regardless of their size. Now, files larger than 2MB are saves to disk instead.

I've also made a change upstream to change the default: jschneier/django-storages#1359

How to Test

Screenshots

MR Checklist

  • Add a description of your pull request and instructions for the reviewer to verify your work.
  • If your pull request is for a specific ticket, link to it in the description.
  • Stay on point and keep it small so the merge request can be easily reviewed.
  • Tests and linting passes.

Unit tests

  • Added
  • Not required

Documentation

Browser testing

  • I have tested in the following browsers and environments (edit the list as required)
    • Latest version of Chrome on mac
    • Latest version of Firefox on mac
    • Latest version of Safari on mac
    • Safari on last two versions of iOS
    • Chrome on last two versions of Android
  • Not required

Data protection

  • Not relevant
  • This adds new sources of PII and documents it and modifies Birdbath processors accordingly

Accessibility

  • Automated WCAG 2.1 tests pass
  • Manual WCAG 2.1 tests completed
  • I have tested in a screen reader
  • I have tested in high-contrast mode
  • Any animations removed for prefers-reduced-motion
  • Not required

Sustainability

  • Images are optimised and lazy-loading used where appropriate
  • SVGs have been optimised
  • Perfomance and transfer of data considered
  • If JavaScript is needed alternatives have been considered
  • Not required

Pattern library

  • The pattern library component for this template displays correctly, and does not break parent templates
  • The styleguide is updated if relevant
  • Changes are not relevant the pattern library

By default, files are always loaded into memory, regardless of their size. Now, files larger than 2MB are saves to disk instead.
@helenb helenb requested a review from engineervix March 4, 2024 11:01
Copy link
Member

@engineervix engineervix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks @RealOrangeOne

@helenb
Copy link
Member

helenb commented Mar 4, 2024

@engineervix are we merging this one straight to main - it would be good to get the benefit straight away, but we'll need to update the integration branch if so.

@engineervix
Copy link
Member

@engineervix are we merging this one straight to main - it would be good to get the benefit straight away, but we'll need to update the integration branch if so.

yes we can merge it straight to main, then update the integration branch

@helenb helenb merged commit 5fcc101 into main Mar 5, 2024
4 checks passed
@helenb helenb deleted the chore/spool-s3-files branch March 5, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants