Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundled package size increases by ~10mb between versions 5.25 & 5.26 #5018

Open
mikeholloway-kaluza opened this issue Dec 16, 2024 · 7 comments
Assignees

Comments

@mikeholloway-kaluza
Copy link

mikeholloway-kaluza commented Dec 16, 2024

Hey 👋

I've noticed a ~10mb increase in my bundled package size (lambda) when using 5.26, that isn't happening in 5.25.

I've skimmed the changelog but I'm not totally sure what may have caused it, however, it feels like a significant increase. I wondered if this is known & expected?

Cheers.

@tlhunter
Copy link
Member

I can confirm that the size increases from 71MB to 99MB by running the following command in two separate directories:

npm init -y && npm install [email protected] && du -h
npm init -y && npm install [email protected] && du -h

Diffing the output of the two du commands gives me this:

177a178,179
> 36K	./node_modules/@isaacs/ttlcache
> 36K	./node_modules/@isaacs
236c238
< 2.5M	./node_modules/@datadog/native-appsec/prebuilds/linuxglibc-x64
---
> 2.6M	./node_modules/@datadog/native-appsec/prebuilds/linuxglibc-x64
255c257,265
< 45M	./node_modules/@datadog
---
> 4.8M	./node_modules/@datadog/libdatadog/prebuilds/darwin-x64
> 5.5M	./node_modules/@datadog/libdatadog/prebuilds/linuxglibc-x64
> 4.4M	./node_modules/@datadog/libdatadog/prebuilds/linuxmusl-arm64
> 3.6M	./node_modules/@datadog/libdatadog/prebuilds/darwin-arm64
> 4.4M	./node_modules/@datadog/libdatadog/prebuilds/linuxglibc-arm64
> 5.5M	./node_modules/@datadog/libdatadog/prebuilds/linuxmusl-x64
> 28M	./node_modules/@datadog/libdatadog/prebuilds
> 28M	./node_modules/@datadog/libdatadog
> 74M	./node_modules/@datadog
395,396c405,406
< 28K	./node_modules/dd-trace/packages/datadog-core/src
< 32K	./node_modules/dd-trace/packages/datadog-core
---
> 32K	./node_modules/dd-trace/packages/datadog-core/src
> 36K	./node_modules/dd-trace/packages/datadog-core
404,405c414,415
< 44K	./node_modules/dd-trace/packages/datadog-plugin-cypress/src
< 44K	./node_modules/dd-trace/packages/datadog-plugin-cypress
---
> 48K	./node_modules/dd-trace/packages/datadog-plugin-cypress/src
> 48K	./node_modules/dd-trace/packages/datadog-plugin-cypress
471,472c481,482
< 700K	./node_modules/dd-trace/packages/datadog-instrumentations/src
< 704K	./node_modules/dd-trace/packages/datadog-instrumentations
---
> 708K	./node_modules/dd-trace/packages/datadog-instrumentations/src
> 712K	./node_modules/dd-trace/packages/datadog-instrumentations
480,482c490,492
< 36K	./node_modules/dd-trace/packages/dd-trace/src/opentelemetry
< 100K	./node_modules/dd-trace/packages/dd-trace/src/plugins/util
< 180K	./node_modules/dd-trace/packages/dd-trace/src/plugins
---
> 40K	./node_modules/dd-trace/packages/dd-trace/src/opentelemetry
> 104K	./node_modules/dd-trace/packages/dd-trace/src/plugins/util
> 184K	./node_modules/dd-trace/packages/dd-trace/src/plugins
490a501
> 12K	./node_modules/dd-trace/packages/dd-trace/src/crashtracking
500c511
< 176K	./node_modules/dd-trace/packages/dd-trace/src/appsec/iast/analyzers
---
> 180K	./node_modules/dd-trace/packages/dd-trace/src/appsec/iast/analyzers
508c519
< 372K	./node_modules/dd-trace/packages/dd-trace/src/appsec/iast
---
> 376K	./node_modules/dd-trace/packages/dd-trace/src/appsec/iast
513c524
< 848K	./node_modules/dd-trace/packages/dd-trace/src/appsec
---
> 856K	./node_modules/dd-trace/packages/dd-trace/src/appsec
524c535
< 12K	./node_modules/dd-trace/packages/dd-trace/src/log
---
> 16K	./node_modules/dd-trace/packages/dd-trace/src/log
550,552c561,563
< 3.8M	./node_modules/dd-trace
< 71M	./node_modules
< 71M	.
---
> 3.9M	./node_modules/dd-trace
> 99M	./node_modules
> 99M	.

So it appears that libdatadog adds 28MB of size to the package. I thought we had CI jobs to catch this...

@mikeholloway-kaluza
Copy link
Author

Thanks for confirming @tlhunter.

The impact this is having for me is that the lambda limit is now being hit for my build, which is preventing me from upgrading to the latest version of the lib without causing a bit of disruption (i.e using S3 to sidestep the 50MB cap).

Is this something Datadog could look to reduce again?

@rochdev
Copy link
Member

rochdev commented Jan 6, 2025

@mikeholloway-kaluza Are you using dd-trace directly or the Lambda extension? In the Lambda extension we have a script to remove that dependency before it's published, so the size increase should not be impacting Lambda as long as the version of dd-trace provided by the extension is used.

@mikeholloway-kaluza
Copy link
Author

@rochdev I want to say directly.

I'm doing something similar to this:

import tracer from 'dd-trace'

// ...business logic

const results = await tracer.wrap(handlerTag, handler as ActionHandler)(
    actions,
    context,
    prevMetadata
  )

@rochdev
Copy link
Member

rochdev commented Jan 7, 2025

What I meant is that you shouldn't have to install dd-trace manually as long as you use the Lambda extension since it comes with dd-trace already, and it removes some of the fat that is not needed in Serverless environments like libdatadog.

@jianzuguan
Copy link

Hi @rochdev, Is there a plan to reduce the package size?
This will help me decide whether to investigate Lambda extension or roll my fat removal script.

@rochdev
Copy link
Member

rochdev commented Jan 14, 2025

Is there a plan to reduce the package size?

Not in the short term.

This will help me decide whether to investigate Lambda extension or roll my fat removal script.

I would definitely recommend using the Lambda extension as that is the official way to use dd-trace in Lambda. You can also find the script that the extension itself uses here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants