README.md

Tutorial: Using External Models or Provisioned Throughput

1. Create an External Model or Provisioned Throughput endpoint

   • See 1_openai_as_external_model for a click-to-run Notebook to create External Models for OpenAI and Azure OpenAI endpoints

2. Once your endpoint is created, update your RAG chain.py to reference the model using ChatDatabricks (for chat models) or Databricks (for completions models).

from langchain_community.chat_models import ChatDatabricks

model = ChatDatabricks(
    endpoint="name-of-your-external-model-or-pt-endpoint",
    # extra_params={}, # optional e.g., temperature, etc
)

chain = (
    {
        "question": itemgetter("messages") | RunnableLambda(extract_user_query_string),
    }
    | prompt
    | model
    | StrOutputParser()
)

3. When logging your chain, RAG Studio will automaticaly detect the endpoint and provision the appropiate credentials for your deployed model to use it.