Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative path is not working within artifacts #1927

Closed
jiropardo opened this issue Nov 21, 2024 · 2 comments · Fixed by #2128
Closed

Relative path is not working within artifacts #1927

jiropardo opened this issue Nov 21, 2024 · 2 comments · Fixed by #2128
Assignees
Labels
Bug Something isn't working DABs DABs related issues No Autoclose

Comments

@jiropardo
Copy link

jiropardo commented Nov 21, 2024

Describe the issue

Trying to deploy an artifcat referencing a location from an upstream folder when building an artifact using Relative path fails with a path resoultion error

Configuration

Please provide a minimal reproducible configuration for the issue

Steps to reproduce the behavior

Please list the steps required to reproduce the issue, for example:

1 Init a bundle , select python template , include python packae

  1. Create an upstream folder with build files upstream to the databricks.yml file
    ├── lib
    │ ├── setup.py
    │ └── src
    ├── cus_test
    │ │
    │ └── test_x
    │ │
    │ ├── setup.py
    │ └── src
    │ │
    │ ├── databricks.yml

databricks.yml file

bundle:
  name: test_x

targets:
  dev:
    mode: production
    default: true
    workspace:
      host: https://xxxxXXXXXX.cloud.databricks.com
      artifact_path: /Shared/.artifacts/${bundle.name}
  
artifacts:
  default:
    type: whl
    build: python3 setup.py bdist_wheel
    path: .

  lib:
    type: whl
    build: python3 setup.py bdist_wheel
    path:  ../../lib

sync:
 paths:
  - ../../lib
  - .

If I use absolute path in the artifacts path, it works with no issues

artifacts:
  default:
    type: whl
    build: python3 setup.py bdist_wheel
    path: /Users/admin/Documents/tests/bundle/path_resolution/cus_test/test_x

  lib:
    type: whl
    build: python3 setup.py bdist_wheel
    path: /Users/admin/Documents/tests/bundle/lib

Expected Behavior

Path error should have not come up and they deployment should end up successfully

Actual Behavior

Clear and concise description of what actually happened
It give a path resolution error

OS and CLI version

Please provide the version of the CLI (eg: v0.1.2) and the operating system (eg: windows). You can run databricks --version to get the version of your Databricks CLI

Databricks CLI v0.234.0

Is this a regression?

Did this work in a previous version of the CLI? If so, which versions did you try? Never worked, not even in pre release CLI

Debug Logs

Output logs if you run the command with debug logs enabled. Example: databricks bundle deploy --log-level=debug. Redact if needed

Building default...
09:59:10 ERROR Error: build failed default, error: chdir /Users/admin/Documents/tests/bundle/path_resolution/cus_test/test_x/cus_test/test_x: no such file or directory, output: pid=59588 mutator=seq mutator=build mutator=seq mutator=artifacts.BuildAll mutator=seq mutator=artifacts.Build(default) mutator=seq mutator=artifacts.whl.Build(default)

@jiropardo jiropardo added the DABs DABs related issues label Nov 21, 2024
@pietern pietern self-assigned this Nov 25, 2024
@pietern pietern added the Bug Something isn't working label Nov 25, 2024
@pietern
Copy link
Contributor

pietern commented Nov 25, 2024

Thanks for reporting. Working on a repro.

Copy link

github-actions bot commented Jan 2, 2025

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

@github-actions github-actions bot added the Stale label Jan 2, 2025
@andrewnester andrewnester removed the Stale label Jan 2, 2025
github-merge-queue bot pushed a commit that referenced this issue Jan 14, 2025
## Changes

We perform a check during path translation that the path being
referenced is contained in the bundle's sync root. If it isn't, it's not
a valid remote reference. However, this doesn't apply to paths that are
_always_ local, such as the artifact path. An artifact's build command
is executed in its path. Files created by the artifact build (e.g.
wheels or JARs) don't need to be in the sync root because they have a
dedicated and different upload path into `${workspace.artifact_path}`.

Therefore, this check that a path is contained in the bundle's sync root
doesn't apply to artifact paths. This change modifies the structure of
path translation to allow opting out of this check.

Fixes #1927.

## Tests

* Existing and new tests pass.
* Manually confirmed that building and using a wheel built outside the
sync root path works as expected.
* No acceptance tests because we don't run build as part of validate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working DABs DABs related issues No Autoclose
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants