Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove mlflow, and explicit mlflow-skinny version #3666

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

remove mlflow, and explicit mlflow-skinny version #3666

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
380d29f
try upgrade mlflow
runhli Dec 12, 2024
837b9b3
update
runhli Dec 12, 2024
70ed41b
remove promptflow
runhli Dec 12, 2024
2896cdc
Merge remote-tracking branch 'origin/main' into runhli/try_upgrade_ml…
runhli Dec 12, 2024
4a11152
bump version
runhli Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .../large_language_models/components_pipelines/data_ingestion_dataset_to_acs_basic/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: pipeline
tags:
Preview: ""

version: 0.0.85
version: 0.0.86
name: llm_ingest_dataset_to_acs_basic
display_name: LLM - Dataset to ACS Pipeline
is_deterministic: false
Expand Down Expand Up @@ -110,7 +110,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_validate_deployments:0.0.80'
component: 'azureml:llm_rag_validate_deployments:0.0.81'
identity:
type: user_identity
inputs:
Expand All @@ -136,7 +136,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_crack_and_chunk:0.0.77'
component: 'azureml:llm_rag_crack_and_chunk:0.0.78'
inputs:
input_data: ${{parent.inputs.input_data}}
input_glob: ${{parent.inputs.input_glob}}
Expand All @@ -159,7 +159,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_generate_embeddings:0.0.70'
component: 'azureml:llm_rag_generate_embeddings:0.0.71'
inputs:
chunks_source:
type: uri_folder
Expand Down Expand Up @@ -209,7 +209,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_update_acs_index:0.0.74'
component: 'azureml:llm_rag_update_acs_index:0.0.75'
inputs:
embeddings:
type: uri_folder
Expand All @@ -229,7 +229,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_register_mlindex_asset:0.0.74'
component: 'azureml:llm_rag_register_mlindex_asset:0.0.75'
inputs:
storage_uri: ${{parent.jobs.create_acs_index_job.outputs.index}}
asset_name: ${{parent.inputs.embeddings_dataset_name}}
Expand Down
12 changes: 6 additions & 6 deletions ...arge_language_models/components_pipelines/data_ingestion_dataset_to_acs_user_id/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: pipeline
tags:
Preview: ""

version: 0.0.83
version: 0.0.84
name: llm_ingest_dataset_to_acs_user_id
display_name: LLM - Dataset to ACS Pipeline
is_deterministic: false
Expand Down Expand Up @@ -110,7 +110,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_validate_deployments:0.0.80'
component: 'azureml:llm_rag_validate_deployments:0.0.81'
identity:
type: user_identity
inputs:
Expand All @@ -136,7 +136,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_crack_and_chunk:0.0.77'
component: 'azureml:llm_rag_crack_and_chunk:0.0.78'
identity:
type: user_identity
inputs:
Expand All @@ -161,7 +161,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_generate_embeddings:0.0.70'
component: 'azureml:llm_rag_generate_embeddings:0.0.71'
identity:
type: user_identity
inputs:
Expand Down Expand Up @@ -215,7 +215,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_update_acs_index:0.0.74'
component: 'azureml:llm_rag_update_acs_index:0.0.75'
identity:
type: user_identity
inputs:
Expand All @@ -237,7 +237,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_register_mlindex_asset:0.0.74'
component: 'azureml:llm_rag_register_mlindex_asset:0.0.75'
identity:
type: user_identity
inputs:
Expand Down
12 changes: 6 additions & 6 deletions ...arge_language_models/components_pipelines/data_ingestion_dataset_to_faiss_basic/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: pipeline
tags:
Preview: ""

version: 0.0.84
version: 0.0.85
name: llm_ingest_dataset_to_faiss_basic
display_name: LLM - Dataset to FAISS Pipeline
is_deterministic: false
Expand Down Expand Up @@ -102,7 +102,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_validate_deployments:0.0.80'
component: 'azureml:llm_rag_validate_deployments:0.0.81'
identity:
type: user_identity
inputs:
Expand All @@ -125,7 +125,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_crack_and_chunk:0.0.77'
component: 'azureml:llm_rag_crack_and_chunk:0.0.78'
inputs:
input_data: ${{parent.inputs.input_data}}
input_glob: ${{parent.inputs.input_glob}}
Expand All @@ -148,7 +148,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_generate_embeddings:0.0.70'
component: 'azureml:llm_rag_generate_embeddings:0.0.71'
inputs:
chunks_source:
type: uri_folder
Expand Down Expand Up @@ -198,7 +198,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_create_faiss_index:0.0.75'
component: 'azureml:llm_rag_create_faiss_index:0.0.76'
inputs:
embeddings:
type: uri_folder
Expand All @@ -216,7 +216,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_register_mlindex_asset:0.0.74'
component: 'azureml:llm_rag_register_mlindex_asset:0.0.75'
inputs:
storage_uri: ${{parent.jobs.create_faiss_index_job.outputs.index}}
asset_name: ${{parent.inputs.embeddings_dataset_name}}
Expand Down
12 changes: 6 additions & 6 deletions ...ge_language_models/components_pipelines/data_ingestion_dataset_to_faiss_user_id/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: pipeline
tags:
Preview: ""

version: 0.0.83
version: 0.0.84
name: llm_ingest_dataset_to_faiss_user_id
display_name: LLM - Dataset to FAISS Pipeline
is_deterministic: false
Expand Down Expand Up @@ -102,7 +102,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_validate_deployments:0.0.80'
component: 'azureml:llm_rag_validate_deployments:0.0.81'
identity:
type: user_identity
inputs:
Expand All @@ -125,7 +125,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_crack_and_chunk:0.0.77'
component: 'azureml:llm_rag_crack_and_chunk:0.0.78'
identity:
type: user_identity
inputs:
Expand All @@ -150,7 +150,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_generate_embeddings:0.0.70'
component: 'azureml:llm_rag_generate_embeddings:0.0.71'
identity:
type: user_identity
inputs:
Expand Down Expand Up @@ -204,7 +204,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_create_faiss_index:0.0.75'
component: 'azureml:llm_rag_create_faiss_index:0.0.76'
identity:
type: user_identity
inputs:
Expand All @@ -224,7 +224,7 @@ jobs:
properties:
compute_specification:
automatic: true
component: 'azureml:llm_rag_register_mlindex_asset:0.0.74'
component: 'azureml:llm_rag_register_mlindex_asset:0.0.75'
identity:
type: user_identity
inputs:
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/crack_and_chunk/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.77
version: 0.0.78
name: llm_rag_crack_and_chunk
display_name: LLM - Crack and Chunk Data
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/crack_and_chunk_and_embed/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.48
version: 0.0.49
name: llm_rag_crack_and_chunk_and_embed
display_name: LLM - Crack, Chunk and Embed Data
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/crack_chunk_embed_index_and_register/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: llm_rag_crack_chunk_embed_index_and_register
version: 0.0.36
version: 0.0.37
tags:
Preview: ""

Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/crawl_url/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.35
version: 0.0.36
name: llm_rag_crawl_url
display_name: LLM - Crawl URL to Retrieve Data
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/create_faiss_index/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.75
version: 0.0.76
name: llm_rag_create_faiss_index
display_name: LLM - Create FAISS Index
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/create_promptflow/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.86
version: 0.0.87
name: llm_rag_create_promptflow
display_name: LLM - Create Prompt Flow
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/data_import_acs/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.71
version: 0.0.72
name: llm_rag_data_import_acs
display_name: LLM - Import Data from ACS
is_deterministic: false
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/generate_embeddings/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.70
version: 0.0.71
name: llm_rag_generate_embeddings
display_name: LLM - Generate Embeddings
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/generate_embeddings_parallel/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: parallel
tags:
Preview: ""

version: 0.0.76
version: 0.0.77
name: llm_rag_generate_embeddings_parallel
display_name: LLM - Generate Embeddings Parallel
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/git_clone/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.74
version: 0.0.75
name: llm_rag_git_clone
display_name: LLM - Clone Git Repo
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/image_embed_index/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.28
version: 0.0.29
name: llm_rag_image_embed_index
display_name: LLM - Embedding images with Florence
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/qa_data_generation/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.74
version: 0.0.75
name: llm_rag_qa_data_generation
display_name: LLM - Generate QnA Test Data
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/register_mlindex_asset/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.74
version: 0.0.75
name: llm_rag_register_mlindex_asset
display_name: LLM - Register Vector Index Asset
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/register_qa_data_asset/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.67
version: 0.0.68
name: llm_rag_register_qa_data_asset
display_name: LLM - Register QA Generation Data Asset
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/update_acs_index/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.74
version: 0.0.75
name: llm_rag_update_acs_index
display_name: LLM - Update ACS Index
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/update_azure_cosmos_mongo_vcore_index/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.30
version: 0.0.31
name: llm_rag_update_cosmos_mongo_vcore_index
display_name: LLM - Update Azure Cosmos Mongo vCore Index
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/update_milvus_index/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.30
version: 0.0.31
name: llm_rag_update_milvus_index
display_name: LLM - Update Milvus Index
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/update_pinecone_index/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.42
version: 0.0.43
name: llm_rag_update_pinecone_index
display_name: LLM - Update Pinecone Index
is_deterministic: true
Expand Down
2 changes: 1 addition & 1 deletion assets/large_language_models/rag/components/validate_deployments/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: command
tags:
Preview: ""

version: 0.0.80
version: 0.0.81
name: llm_rag_validate_deployments
display_name: LLM - Validate Deployments
is_deterministic: false
Expand Down
2 changes: 0 additions & 2 deletions assets/large_language_models/rag/environments/rag/context/conda_dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ dependencies:
- inference-schema>=1.8.0
- lxml
- markdown
- mlflow-skinny==2.3.2
- mlflow>=2.6.0
- msal~=1.31.0
- nltk==3.9.1
- openai>=0.27.4
Expand Down
Loading
Loading