Sweep: Find a better templating solution

[aqidd]

The current component-generator is using bash and .template folder to generate test & component. Find another method, preferably in the same language ( javascript ) to make maintenance easier.

[sweep-ai]

Here's the PR! #24.

⚡ Sweep Free Trial: I used GPT-4 to create this ticket. You have 3 GPT-4 tickets left. For more GPT-4 tickets, visit our payment portal.To get Sweep to recreate this ticket, leave a comment prefixed with "sweep:" or edit the issue.

• Install Sweep Configs: Pull Request

---

Step 1: 🔍 Code Search

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I looked at (click to expand). If some file is missing from here, you can mention the path in the ticket description.

express-component-starter/README.md

Lines 1 to 77 in f8c3656

	# express-component-starter
	an express starter project with component based grouping.
	**Goals** :
	Creating a scalable architecture that can be translated into micro-services easily ( with the help of `component-generator` runnable script )
	---
	## Components vs Services
	**Services** are ideal for highly resilient systems whereby parts of your infrastructure can crash but the rest keeps going gracefully.
	**Components** are designed to fit together to deliver functionality. This doesn't mean that they aren't reusable as a component can be an API that gets used in a wide range of systems and applications.
	Services and components aren't mutually exclusive architectures as a service can be made from components. Components may also call services.
	## Independent business logic
	Business logic inside components should not depends on external libraries. Any external dependencies should be wrapped / implemented in *libraries* folder. Components should access files from *libraries* and should not access any node_modules dependencies directly, without valid reasons / exceptions. We don't want to end up with problems from abandoned external libraries.
	some suggestions & rules :
	- **Controller** should be a pure javascript class / functions without direct `node_modules` dependencies ( with some exceptions e.g. `lodash` | other `node_modules` dependencies should be wrapped and placed inside *libraries*
	)
	- **Model** & **Repository** files will be tightly coupled with ORM / DB library used
	- **Route** files will be tightly coupled with Express / Backend Framework
	- **Validator** files might be coupled with external validator libraries
	- Communication between components should use events / message queues / wrapped in a bridge files inside *libraries*
	## Style Guide & Code Formatting
	For code formatting, we use `standard` - Read more here https://github.com/standard/standard
	Please run `standard --fix` before committing. Or use IDE plugin to auto format on save ( `npm run dev` or `yarn dev` automatically run `standard --fix` - make sure you have `standard` installed as global module )
	## Folder/Architecture Descriptions
	- `app/config` : containing configuration files to run your app. Including but not limited to : database configuration, server configuration, provider used
	- `app/providers` : containing specific module configuration that your backend framework will use (in this case, ExpressJS). Please note that you need to update `app/config/service-provider.js` after adding file inside this folder.
	- `app/*.js` : manager for configurations and server files.
	- `components` : the most important part. it contains several layer for you to create your API
	- **Model** : represented by `module-name.js` - feel free to add more model if necessary
	- **Repository** : represented by `module-name-repository.js` - it contains data processing functions for your model & database
	- **Controller** : represented by `module-name-controller.js` - it contains business logic for your components / application
	- **Event** : represented by `module-name-event.js` - it contains a repeatable job / event based action for your application. We're using `agendajs` for this repository.
	- **Transformer** : represented by `module-name-transformer.js` - it transform data from API and making sure the schema is always valid & consistent
	- **Validator** : represented by `module-name-validator.js` - it validates requests & responses and making sure all data input & output will conform to client's requirement
	- **Router** : represented by `module-name-api.js` - it contains your route list and logics.
	- `libraries` : containing module configuration that your components will use ( e.g for communicating with each other )
	- `tests` : your test files. we use `supertest` as the testing library.
	## Component Generator
	To make things easier, we've made a simple bash script to generate components for you. Call it like this `sh component-generator ComponentName` - and it will generate a folder with files inside `components` folder - it will also generate a folder with test suite template inside `tests/api` folder. Please use `UpperCamelCase` for the parameter.
	After creating the component, you can access it from `<HOST>:<PORT>/<API_PREFIX>/<module-name>` - you can also check the test suite template by running `npm run test` / `yarn test`. Currently available component is `user` component using `mongodb` as sample.
	## Database Options
	You can choose between NoSQL ( using `mongodb` database driver ) or SQL ( using `sequelize` database driver ). In this repository, you're given example configuration on using mongo / postgres. Please see `app/providers/database` for details
	## Docker & Docker Compose
	Running `docker-compose up` will give you several connected containers ready for tinkering : Application container using node 8.11, MongoDB 3.6 with persistence volume, and Postgres 9.5 with persistence volume. ( will create `data/mongo` and `data/postgres` folder if not exists )
	## Notes & References
	The main principle of this component based architecture is loose-coupled relation between components. But depending on your business domain, there might be some components that are tighly coupled and that's also quite alright.
	For the loosely coupled component, you need to interact by sending messages/by REST API, just like how mocroservices interact. A component can call API / use message broker to call another components related functions.
	For the tightly coupled components you just call the method directly, will be better if they're inside some kind of parent component so the context is clearer.
	---
	Follow lots of guides from https://github.com/i0natan/nodebestpractices , http://www.appcontinuum.io/
	Watch repository pattern here https://www.youtube.com/watch?v=rtXpYpZdOzM&feature=youtu.be

express-component-starter/tests/api/user/index.js

Lines 1 to 73 in f8c3656

	/**
	* Template file for API testing
	* Work In Progress
	* TODO : complete the files, remove this comment
	*/
	const chalk = require('chalk')
	describe(chalk.cyanBright.bold('test suite template for user'), function () {
	before(async function (done) {
	this.timeout(1000)
	done()
	})
	after(async function (done) {
	this.timeout(1000)
	done()
	})
	describe('define success case for user ', function () {
	before(function (done) {
	this.timeout(1000)
	done()
	})
	after(async function (done) {
	this.timeout(1000)
	done()
	})
	it('should return 200 status code')
	it('should return correct object')
	})
	describe('define error case for user ', function () {
	before(function (done) {
	this.timeout(1000)
	done()
	})
	after(async function (done) {
	this.timeout(1000)
	done()
	})
	it('should return 400 status code')
	it('should return correct error object')
	})
	describe('define empty case for user ', function () {
	before(function (done) {
	this.timeout(1000)
	done()
	})
	after(async function (done) {
	this.timeout(1000)
	done()
	})
	it('should return 404 status code')
	it('should return correct error object')
	})
	describe('define corner cases for user ', function () {
	before(function (done) {
	this.timeout(1000)
	done()
	})
	after(async function (done) {
	this.timeout(1000)
	done()
	})
	it('should return 400 status code')
	it('should return correct error object')
	})
	})

express-component-starter/app/component-manager.js

Lines 1 to 43 in f8c3656

	const fs = require('fs')
	const path = require('path')
	class ComponentManager {
	constructor (express) {
	this.express = express
	this.path = path.join(process.env.PWD, 'components')
	this.disabledComponents = []
	}
	registerComponents () {
	fs.readdirSync(this.path).forEach(dir => {
	this.registerComponent(dir)
	this.registerModels(path.join(this.path, dir), dir)
	})
	}
	registerComponent (name) {
	if (this.isValidComponent(name)) {
	this.express.use(`/${process.env.API_PREFIX}/` + name, require('components/' + name + '/' + name + '-api'))
	} else {
	console.error(path.join(this.path, name) + ' is not valid or may be disabled')
	}
	}
	registerModels (dir, name) {
	fs.readdirSync(dir).forEach(file => {
	let classFile = path.join(name, path.basename(file))
	require('components/' + classFile)
	})
	}
	isValidComponent (name) {
	let files = fs.readdirSync(path.join(this.path, name))
	let hasFile = files.includes(name + '-api.js')
	let notDisabled = !this.disabledComponents.includes(name)
	return hasFile && notDisabled
	}
	}
	module.exports = ComponentManager

express-component-starter/component-generator

Lines 1 to 48 in f8c3656

	#!/bin/bash
	uppercase=$1
	kebabcase="$(echo ${uppercase} | sed -e 's/\([A-Z]\)/\-\1/g' -e 's/\-//' | tr '[:upper:]' '[:lower:]')"
	echo "creating folder"
	cp -R .template/api components/$kebabcase
	cp -R .template/test tests/api/$kebabcase
	echo "modifying files for api"
	cd ./components/$kebabcase
	for file in $(find . -maxdepth 1 -name "*(qq)*" -type f)
	do
	#change variable inside file
	sed -i 's/(QQ)/'$uppercase'/g' "$file"
	sed -i 's/(qq)/'$kebabcase'/g' "$file"
	echo ' change' "$file" 'to' `echo "${file//(qq)/$kebabcase}"`
	mv "$file" `echo "${file//(qq)/$kebabcase}"`
	done
	# for D in $(find . -depth -name "*(qq)*" -type d) '.'
	# do
	# pushd $D >> /dev/null
	# echo 'directory: ' "$D"
	# for file in $(find . -maxdepth 1 -name "*(qq)*" -type f)
	# do
	# #change variable inside file
	# sed -i 's/(QQ)/'$uppercase'/g' "$file"
	# sed -i 's/(qq)/'$kebabcase'/g' "$file"
	# echo ' change' "$file" 'to' `echo "${file//(qq)/$kebabcase}"`
	# mv "$file" `echo "${file//(qq)/$kebabcase}"`
	# done
	# popd >> /dev/null
	# echo 'change' "$D" 'to' `echo "$D"`
	# #change only the last occurence of (qq)
	# mv "$D" `echo "$D"`
	# done
	ls -al .
	echo "modifying files for test"
	cd ../../tests/api/$kebabcase
	sed -i 's/(qq)/'$kebabcase'/g' "index.js"

express-component-starter/components/user/user-validator.js

Lines 1 to 21 in f8c3656

	/**
	* Template file for Validate functions
	* Work In Progress
	* TODO : complete the files, remove this comment
	*/
	module.exports = {
	request: {
	create (req, res, next) {
	console.warn('implement your request validation here')
	next()
	}
	},
	response: {
	create (req, res, next) {
	console.warn('implement your response validation here')
	next()
	}
	}
	}

---

Step 2: 🧐 Snippet Analysis

From looking through the relevant snippets, I decided to make the following modifications:

File Path	Proposed Changes
component-generator	Replace the bash script with a JavaScript script. Use the 'fs' module to read the template files, replace placeholders with the actual component name, and write the new component and test files.
package.json	Add a new npm script that runs the JavaScript component-generator. The script could be something like "generate-component": "node component-generator.js".

---

Step 3: 📝 Planning

I have created a plan for writing the pull request. I am now working my plan and coding the required changes to address this issue. Here is the planned pull request:

Replace bash script with JavaScript component-generator
sweep/replace-bash-script

Description

This PR replaces the existing bash script with a JavaScript component-generator for easier maintenance and better integration with the project. The bash script was using a .template folder to generate components and tests, which made it harder to debug and maintain. The new JavaScript script uses the Node.js 'fs' module to read and write files, and string replacement functions to replace placeholders in the template files.

Summary of Changes

• Replaced the bash script with a JavaScript component-generator.
• Used the Node.js 'fs' module to read and write files.
• Implemented string replacement functions to replace placeholders in the template files.
• Modified the 'package.json' file to add a new npm script that runs the JavaScript component-generator.

Please review and merge this PR to improve the maintainability and usability of the component-generator.

---

Step 4: ⌨️ Coding

File	Instructions	Progress
component-generator	Replace the bash script with a JavaScript script. Use the 'fs' module to read the template files, replace placeholders with the actual component name, and write the new component and test files.	✅ Done with commit 2627df6
package.json	Add a new npm script that runs the JavaScript component-generator. The script could be something like "generate-component": "node component-generator.js".	✅ Done with commit 4995d63

---

Step 5: 🔁 Code Review

Here are the my self-reviews of my changes at sweep/replace-bash-script.

Here is the 1st review

Thanks for your contribution. There is a small issue that needs to be addressed:

In the component-generator file, the fs.copyFileSync function is used to copy directories, but this function can only copy files, not directories. This will likely cause a runtime error. You might want to use a package like fs-extra which can copy directories, or write a function to copy directories manually.

Here is the line that needs to be changed:

• Change required in component-generator on lines 38-39

Once this change is made, everything else looks good to go. Keep up the good work!

I finished incorporating these changes.

---

To recreate the pull request, leave a comment prefixed with "sweep:" or edit the issue.
^{Join Our Discord}