-
Notifications
You must be signed in to change notification settings - Fork 9
Heroku Build Configuration and Package.json
Background: 2022Jul07 - I (@ddfridley) have been having a lot of trouble building on heroku over the past week or 2. I've been doing a lot to debug it, and make it work. While I have it working, I have not been able to prove that I found the root cause, though I have done several things to make it better. I am capturing what I have done, and why for better understanding. This project, undebate-ssp depends on undebate, civil-server and civil-client. The undebate projects (which depends on civil-server) builds separately on heroku okay, and the civil-server, which depends on civil-client also builds separately on heroku.
There is a 15 minute limit to builds on Heroku
The problem I've been having is that builds never finish. Often they hang for days. But the key is that they don't complete in 15 minutes. It seems like 15 minutes should be a problem, but with this layers repo architecture where undebate-ssp includes undebate and civil-client and civil-server the build time accumulates.
Move storybook, jest, eslint, and other 'test' dependencies to optionalDependencies in package.json.
"optionalDependencies": {
"@jest/globals": "^27.4.6",
"@shelf/jest-mongodb": "^2.2.0",
"@storybook/addon-actions": "^6.4.9",
"@storybook/addon-essentials": "^6.4.9",
"@storybook/addon-interactions": "^6.4.19",
"@storybook/addon-links": "^6.4.9",
"@storybook/builder-webpack5": "^6.4.9",
"@storybook/jest": "^0.0.10",
"@storybook/manager-webpack5": "^6.4.9",
"@storybook/react": "^6.4.9",
"@storybook/testing-library": "^0.0.9",
"concurrently": "^6.4.0",
"enzyme": "^3.11.0",
"enzyme-adapter-react-16": "^1.15.6",
"eslint": "^8.4.1",
"eslint-config-airbnb": "^19.0.2",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-import": "^2.25.3",
"eslint-plugin-jsx-a11y": "^6.5.1",
"eslint-plugin-prettier": "^4.0.0",
"eslint-plugin-react": "^7.27.1",
"eslint-plugin-react-hooks": "^4.3.0",
"husky": "^4.3.8",
"jest": "^27.4.7",
"jest-enzyme": "^7.1.2",
"nodemon": "^2.0.15",
"prettier": "^2.5.1",
"pretty-quick": "^3.1.2",
"socket.io": "^4.4.1",
"socket.io-client": "^4.4.1",
"webpack-dev-server": "^4.6.0"
},
Set the NPM_CONFIG_OMIT env variable to omit the optional dependencies.
heroku config:set NPM_CONFIG_OMIT=optional
Npm on heroku will not waste time and memory fetching and building them. Slug size went from 130M to 81M.
In all the repos I moved the postinstall.sh code into build.sh and ran that under the prepare script rather than postinstall.
For production, you may set NODE_ENV=production which will cause heroku to remove the devDependencies after the build. That is it removes them after the prepare script, and before the postinstall script. So they won't be there when needed.
The build.sh scripts now check for an NPM_PROJECT environment variable and they don't run webpack if the script isn't for the project being built.
if test \"$NPM_PROJECT\" = \"\" || test \"$NPM_PROJECT\" == \"undebate-ssp\" ; then {
npm run packbuild || {
echo Could not webpack;
exit 1
}
}; fi
The undebate repo is a dependency and being built when loaded into undebate-ssp project, undebate's main.js does not need to be generated. Webpack consumes a lot of memory when it runs and can lead to out of memory errors. So, instead just don't build it if it's not necessary.
bcrypt is a package that wants to compile and install C code on node in order to provide optimal password encryption. Sometimes new versions of node don't work with bcrypt.
Some of the builds that failed were on [email protected] and then things that were working were on [email protected] just a few days apart. But when I set the node engine back to 16.15.1 after all of the other changed described here, the build succeeded.
"engines": {
"npm": "8.13.2",
"node": "16.16.0"
}
The node and npm you run locally won't change very often - unless you change it. But the node and npm that get used on heroku will change depending on what's the latest available, and what Heroku stack you are using (eg heroku-22). Also see why for bcrypt.
Heroku has started using npm ci instead of npm install when you push your repo. Npm ci will erase your node_modules directory and start anew every time you push a commit. This will take longer, and if you are running into problems with your build taking to long, then if you use npm install you can do it incrementally. The first push may fail, but if you push it again, then it may work.
webpack-prod.config.js
module.exports = {
mode: 'production',
context: path.resolve(__dirname, 'dist'), // dist because app failed when building this as a node_module in another component
entry: {
main: './client/main-app.js',
},
devtool: 'inline-cheap-source-map', // cheap so it won't run out of memory on heroku
...
In the production build for webpack, use this devtool to reduce out of memory errors.
To prevent out of memory errors
Because undebate-ssp depends on undebate, which depends on civil-server and civil-client I put them on in peerDependencies
"peerDependencies": {
"civil-client": "github:EnCiv/civil-client#heroku-oom",
"civil-server": "github:EnCiv/civil-server#heroku-oom",
"undebate": "github:EnCiv/undebate#heroku-oom"
}
If undebate-ssp depends on one version of civil-server, while undebate depends on a different one, then civil-server may get fetched and built twice. This is especially a concern since these things are changing frequently. So they are in peerDependencies in all repos in order to minimize extra fetches and builds.