[GEN AI] Improving Langfuse's traces #1793
Conversation
assouktim
force-pushed
the
featur/dercbot-langfuse-xxx
branch
from
a4222d9 to
1878fe2
Compare
assouktim
changed the title
| history: list[ChatMessage] = Field( | ||
| description="Conversation history, used to reformulate the user's question." | ||
| ) | ||
| dialog: DialogDetails = Field(description='The user dialog details.') |
There was a problem hiding this comment.
Make it optional for non "conversational" use cases of the orchestrator, I agree it a bit hypothetical
| dialog_id: Optional[str] = Field(description="The dialog ID.", default=None, examples=["uuid-0123"]) | ||
| user_id: Optional[str] = Field(description="The user ID.", default=None, examples=["[email protected]"]) | ||
| history: list[ChatMessage] = Field(description="Conversation history, used to reformulate the user's question.") | ||
| tags: list[str] = Field(description='List of tags', examples=[["my-Tag"]]) |
There was a problem hiding this comment.
| tags: list[str] = Field(description='List of tags', examples=[["my-Tag"]]) | |
| tags: list[str] = Field(description='List of tags, attached to the observability trace, if the observability provider support it .', examples=[["my-Tag"]]) |
| class DialogDetails(BaseModel): | ||
| """The dialog details model""" | ||
|
|
||
| dialog_id: Optional[str] = Field(description="The dialog ID.", default=None, examples=["uuid-0123"]) |
There was a problem hiding this comment.
| dialog_id: Optional[str] = Field(description="The dialog ID.", default=None, examples=["uuid-0123"]) | |
| dialog_id: Optional[str] = Field(description="The dialog/session ID, attached to the observability traces if the observability provider support it.", default=None, examples=["uuid-0123"]) |
| """The dialog details model""" | ||
|
|
||
| dialog_id: Optional[str] = Field(description="The dialog ID.", default=None, examples=["uuid-0123"]) | ||
| user_id: Optional[str] = Field(description="The user ID.", default=None, examples=["[email protected]"]) |
There was a problem hiding this comment.
| user_id: Optional[str] = Field(description="The user ID.", default=None, examples=["[email protected]"]) | |
| user_id: Optional[str] = Field(description="The user ID, attached to the observability traces if the observability provider support it.", default=None, examples=["[email protected]"]) |
| setting=observability_setting | ||
| ).get_callback_handler(trace_name=trace_name.value) | ||
| setting=observability_setting, | ||
| ).get_callback_handler(**kwargs) |
There was a problem hiding this comment.
Those are directly mapped to the langfuse attributes ... I'm thinking that this is ok untill we just have langfuse but what about langsmith or Arize Phoenix so I made some research in case we would someday implement them :
- Arize Phoenix uses auto-instrumentation (compatible with OpenTelemetry standard and OpenInference), meaning it's not done though an explicit langchain callback
- Langsmith SDK is now called LangChainTracer, it seems that they aren't using callbacks any more : context managers are use in the document, tags are passed by wrapping the LCEL chain ...
It seems that langfuse also recommends using decorators : https://langfuse.com/docs/sdk/python/decorators#decorator-context
I think that it would be great to investigate a bit more and maybe instead of using a callback, wrap chains calls inside python's context managers ... where the manager is build by a factory.
Also we should test LCEL chain tags with_configs (to pass tags) in a more generic way, maybe we would have to pass them to the observability provider.
There was a problem hiding this comment.
yes, it's a good thing, we need to move towards this new way of tracing. As seen together on a separate ticket.
assouktim
force-pushed
the
featur/dercbot-langfuse-xxx
branch
2 times, most recently
from
5042a54 to
3a19167
Compare
assouktim
force-pushed
the
featur/dercbot-langfuse-xxx
branch
from
c49bc63 to
6072ada
Compare
Add dialogs details to Langfuse trace & Return Langfuse trace (id, name, url) in RAG Response