Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(dev_env_setup): use relative packages everywhere #8910

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix(dev_env_setup): use relative packages everywhere #8910

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
182 changes: 56 additions & 126 deletions dev_env_setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,145 +12,75 @@ cd "$SCRIPT_DIR"
echo "Running 'yarn install' in the root directory..."
yarn install

# Step 2: Run yarn build in the root directory
echo "Running 'yarn build' in the root directory..."
yarn build
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

originally yarn build fires the rollup to build all the UMDs/ESMs in packages... You switched to trying running yarn build inside each pkg. But does this produce the same output?
Taking package packages/cubejs-client-core as an example:
There is no build script in package.json and there is no ts files. So new version won't do anything, while original rollup will prepare UMD for it.


# Step 3: Run yarn build in packages/cubejs-playground
echo "Running 'yarn build' in packages/cubejs-playground..."
cd packages/cubejs-playground
yarn build
cd ../..

# Step 4: Run yarn tsc for the first time
echo "Running 'yarn tsc --build'..."
yarn tsc

# Step 5: List available drivers and ask user to select
echo "Listing available drivers..."
available_drivers=$(ls packages | grep "driver")

PS3='Please select the drivers you want to use (enter number, then press Enter): '
# Step 2: Build all packages in packages directory
echo "Building all packages..."
for package in packages/*; do
if [ -d "$package" ]; then
echo "Building $package..."
cd "$package"
if ! yarn build; then
echo "yarn build failed for $package, trying yarn tsc..."
yarn tsc || true # Continue even if tsc fails
fi
cd "$SCRIPT_DIR"
fi
done

# Display drivers without the prefix "cubejs-"
select selected_driver in $(echo "$available_drivers" | sed 's/cubejs-//') "Finish selection"
do
if [[ "$selected_driver" == "Finish selection" ]]; then
break
# Step 3: Link all packages
echo "Linking all packages..."
for package in packages/*; do
if [ -d "$package" ]; then
echo "Linking $package..."
cd "$package"
yarn link
cd "$SCRIPT_DIR"
fi
selected_drivers+=("$selected_driver")
echo "Selected drivers: ${selected_drivers[*]}"
done

# Step 6-7: Run yarn link and yarn install in packages/cubejs-<pkg>
for driver in "${selected_drivers[@]}"
do
echo "Linking and installing dependencies for $driver..."
cd "packages/cubejs-$driver"
yarn link
yarn install
cd ../..
# Step 4: Ask for application name and database type
read -p "Enter the application name: " APP_NAME

# Get available database types from packages directory
db_types=()
for package in packages/cubejs-*-driver; do
if [ -d "$package" ]; then
db_name=$(basename "$package" | sed 's/cubejs-\(.*\)-driver/\1/')
db_types+=("$db_name")
fi
done

# Step 8: Run yarn link @cubejs-backend/<pkg> in packages/cubejs-server-core
cd packages/cubejs-server-core
for driver in "${selected_drivers[@]}"
echo "Available database types:"
PS3='Please select the database type: '
select DB_TYPE in "${db_types[@]}"
do
echo "Linking @cubejs-backend/$driver in packages/cubejs-server-core..."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure linking drivers in server-core is no longer needed for proper working?

yarn link @cubejs-backend/"$driver"
if [[ -n "$DB_TYPE" ]]; then
break
else
echo "Invalid selection. Please try again."
fi
done
cd ../..

# Step 9: Run yarn link in packages/cubejs-server-core
echo "Running 'yarn link' in packages/cubejs-server-core..."
cd packages/cubejs-server-core
yarn link
cd ../..

# Change back to the original directory
cd "$CURRENT_DIR"

# Step 10: Ask user if they want to create a new test project
read -p "Do you want to create a new test project? (yes/no, default: yes): " CREATE_PROJECT
CREATE_PROJECT=${CREATE_PROJECT:-yes}

if [[ "$CREATE_PROJECT" == "yes" || "$CREATE_PROJECT" == "y" ]]; then
read -p "Enter the application name: " APP_NAME

# List of available database types (hardcoded for now as of https://cube.dev/docs/reference/cli)
db_types=("postgres" "mysql" "athena" "mongodb" "bigquery" "redshift" "mssql" "clickhouse" "snowflake" "presto" "druid")

echo "Listing available database types..."
PS3='Please select the database type: '
select DB_TYPE in "${db_types[@]}"
do
if [[ -n "$DB_TYPE" ]]; then
break
else
echo "Invalid selection. Please try again."
fi
done

# Create new project using cubejs-cli
echo "Creating new project with name $APP_NAME and database type $DB_TYPE..."
npx cubejs-cli create "$APP_NAME" -d "$DB_TYPE"

# Step 11: Run yarn link @cubejs-backend/server-core in your project directory
echo "Linking @cubejs-backend/server-core in the project directory..."
cd "$APP_NAME"
yarn link @cubejs-backend/server-core
cd ../
else
echo "Ok. No problem!"
echo "You need to run 'yarn link @cubejs-backend/server-core' in your project directory manually"
fi

# Step 11: Ask user if they plan to make changes to Rust code
read -p "Do you plan to make changes to Rust code? (yes/no, default: no): " RUST_CHANGES
RUST_CHANGES=${RUST_CHANGES:-no}
# Create new project using cubejs-cli
echo "Creating new project with name $APP_NAME and database type $DB_TYPE..."
node "$SCRIPT_DIR/packages/cubejs-cli/dist/src/index.js" create "$APP_NAME" -d "$DB_TYPE"

if [[ "$RUST_CHANGES" == "yes" || "$RUST_CHANGES" == "y" ]]; then
# Run yarn link:dev in the script directory
cd "$SCRIPT_DIR"
echo "Running 'yarn link:dev' in the root directory..."
yarn link:dev
# Step 5: Link all packages in the new project
echo "Linking packages in the new project..."
cd "$APP_NAME"

if [[ "$CREATE_PROJECT" == "yes" || "$CREATE_PROJECT" == "y" ]]; then
dev_pkgs=("@cubejs-backend/shared"
"@cubejs-backend/cloud"
"@cubejs-backend/native"
"@cubejs-backend/server"
"@cubejs-backend/server-core"
"@cubejs-backend/api-gateway"
"@cubejs-backend/schema-compiler"
"@cubejs-backend/query-orchestrator"
"@cubejs-backend/athena-driver"
"@cubejs-backend/duckdb-driver"
"@cubejs-backend/bigquery-driver"
"@cubejs-backend/postgres-driver"
"@cubejs-backend/databricks-jdbc-driver"
"@cubejs-backend/mssql-driver"
"@cubejs-backend/clickhouse-driver"
"@cubejs-backend/snowflake-driver"
"@cubejs-backend/cubestore-driver"
"@cubejs-backend/templates"
"@cubejs-client/core"
"@cubejs-client/ws-transport"
"@cubejs-client/playground"
)

cd "$CURRENT_DIR/$APP_NAME"
echo "Linking dev packages in $APP_NAME project..."

for pkg in "${dev_pkgs[@]}"
do
echo "Linking $pkg..."
yarn link "$pkg"
done
else
echo "Don't forget to link packages that you plan to modify inside your project!"
for package in "$SCRIPT_DIR"/packages/*; do
if [ -d "$package" ]; then
package_name=$(node -p "require('$package/package.json').name")
echo "Linking $package_name..."
yarn link "$package_name"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably good to cleanup the scripts in root package.json to remove potential confusion for those who will find it and try to use but it is outdated and not actual anymore. Like "link:dev"

fi
fi
done

echo "All steps completed successfully!"
echo "Run 'yarn dev' to start your testing project and verify changes."
echo "Project setup completed!"
echo "You can now run 'yarn dev' in the $APP_NAME directory to start your project."

# Change back to the original directory
cd "$CURRENT_DIR"
Loading