[BUG]Not able to use long term memory with Azure Open AI #1577

talrejanikhil · 2024-11-11T10:14:35Z

Description

When using memory=True for a crew that uses Azure Open AI, there is an error creating long term memory.

Steps to Reproduce

import os

from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
from crewai import Agent, Crew, Process, Task, LLM
from crewai.memory import ShortTermMemory, EntityMemory
from crewai.memory.storage.rag_storage import RAGStorage
from crewai.project import CrewBase, agent, crew, task
from dotenv import load_dotenv

load_dotenv()
model = "GPT4_MINI_"


@CrewBase
class LatestAiDevelopmentCrew():
    """LatestAiDevelopment crew"""

    def __init__(self):
        self.llm = LLM(
            api_key=os.getenv(model + "AZURE_OPENAI_API_KEY"),
            base_url=os.getenv(model + "AZURE_OPENAI_ENDPOINT"),
            api_version=os.getenv(model + "OPENAI_API_VERSION"),
            model="azure/" + os.getenv(model + "OPENAI_DEPLOYMENT"),
            azure=True,
        )

    @agent
    def researcher(self) -> Agent:
        return Agent(
            config=self.agents_config['researcher'],
            # tools=[MyCustomTool()], # Example of custom tool, loaded on the beginning of file
            verbose=True,
            llm=self.llm
        )

    @agent
    def reporting_analyst(self) -> Agent:
        return Agent(
            config=self.agents_config['reporting_analyst'],
            verbose=True,
            llm=self.llm
        )

    @task
    def research_task(self) -> Task:
        return Task(
            config=self.tasks_config['research_task'],
        )

    @task
    def reporting_task(self) -> Task:
        return Task(
            config=self.tasks_config['reporting_task'],
            output_file='report.md'
        )

    @crew
    def crew(self) -> Crew:
        """Creates the LatestAiDevelopment crew"""
        return Crew(
            agents=self.agents,  # Automatically created by the @agent decorator
            tasks=self.tasks,  # Automatically created by the @task decorator
            process=Process.sequential,
            verbose=True,
            memory=True,
            embedder=OpenAIEmbeddingFunction(
                api_key=os.getenv(model + "AZURE_OPENAI_API_KEY"),
                api_base=os.getenv(model + "AZURE_OPENAI_ENDPOINT"),
                api_type="azure",
                api_version="2023-05-15",
                model_name="embedding-ada-002"
            ),
            short_term_memory=ShortTermMemory(
                storage=RAGStorage(
                    type="short_term",
                    embedder_config=
                    {
                        "provider": "azure",
                        "config": {
                            "model": "text-embedding-ada-002",
                            "api_key": os.getenv(model + "AZURE_OPENAI_API_KEY"),
                            "api_base": os.getenv(model + "AZURE_OPENAI_ENDPOINT"),
                            "api_version": "2023-05-15"
                        },
                    }
                )
            ),
            entity_memory=EntityMemory(
                storage=RAGStorage(
                    type="entities",
                    embedder_config=
                    {
                        "provider": "azure",
                        "config": {
                            "model": "text-embedding-ada-002",
                            "api_key": os.getenv(model + "AZURE_OPENAI_API_KEY"),
                            "api_base": os.getenv(model + "AZURE_OPENAI_ENDPOINT"),
                            "api_version": "2023-05-15"
                        },
                    }
                )
            ),
        )

Expected behavior

Long term and short term memory should be created

Screenshots/Code snippets

Agent: AI LLMs Senior Data Researcher

Task: Conduct a thorough research about AI LLMs Make sure you find any interesting and relevant information given the current year is 2024.

Agent: AI LLMs Senior Data Researcher

Final Answer:

Multimodal Capabilities: As of 2024, AI LLMs have evolved to include multimodal abilities, allowing them to process and generate not just text but also images, audio, and video, thereby increasing their application potential across different fields such as entertainment, education, and healthcare.
Robustness and Safety Improvements: Ongoing research has led to improved safety mechanisms in LLMs to minimize harmful outputs. Companies are implementing reinforcement learning from human feedback (RLHF) to better align AI outputs with human values and ethical norms.
Smaller, More Efficient Models: New architectures and training techniques have facilitated the development of smaller, more efficient LLMs that maintain performance comparable to larger ones, reducing computational costs and making deployment more feasible on edge devices.
Augmented Human Intelligence: LLMs are increasingly being integrated into tools that assist professionals (like writers, programmers, and designers) in their creative processes, enabling enhanced productivity and creative exploration through features like code suggestions, content generation, and brainstorming support.
Pretraining with Diverse Data: The 2024 landscape shows a shift toward using more diverse datasets for pretraining LLMs, aiming for broader generalizability and better performance in multilingual and cross-domain tasks, catering to a more global audience.
Legal and Regulatory Frameworks: As AI technology continues to integrate deeper into society, countries and organizations are establishing legal and regulatory frameworks to govern the development and deployment of LLMs, addressing issues such as data privacy, bias, and accountability.
Fine-tuning and Customization: Organizations are finding value in fine-tuning existing LLMs for specific industry applications, resulting in more tailored models that provide superior performance in specialized areas like law, medicine, and customer service compared to general-purpose models.
Community-driven Model Development: Open-source contributions and community-driven research are accelerating LLM advancements, with collaborative projects enabling rapid iterations, best practices sharing, and reduced barriers to entry for smaller entities in AI development.
AI Ethics and Fairness Initiatives: 2024 has seen a pronounced emphasis on AI ethics, with many institutions creating frameworks and best practices for ensuring fairness, transparency, and accountability in LLMs, leading to the formation of dedicated groups focused on ethical AI research.
Real-time Interactivity: The latest advancements include LLMs that can engage in real-time conversations with users, learning from interactions to provide more personalized and contextually aware responses, making them increasingly viable for applications such as virtual customer support and interactive educational environments.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Agent: AI LLMs Reporting Analyst

Task: Review the context you got and expand each topic into a full section for a report. Make sure the report is detailed and contains any and all relevant information.

Agent: AI LLMs Reporting Analyst

Final Answer:

Comprehensive Report on the Evolution of AI LLMs in 2024

1. Multimodal Capabilities

As of 2024, AI large language models (LLMs) have made significant strides in developing multimodal capabilities. This evolution allows these models to process and generate various forms of media, including text, images, audio, and video. The implications of this multimodal functionality are profound, expanding the potential application of LLMs across diverse fields.

In the entertainment industry, for instance, LLMs can analyze scripts, suggest visual elements, and even create preliminary video edits, facilitating a more streamlined creative process. In education, they offer interactive learning experiences, combining text-based explanations with visual aids and audio prompts to cater to different learning styles. Healthcare applications include processing medical imaging data alongside patient records, enabling more comprehensive diagnostics and treatment plans. This versatility positions multimodal LLMs as valuable assets in driving innovation and enhancing user experiences across numerous sectors.

2. Robustness and Safety Improvements

The ongoing research into AI LLMs has prioritized robustness and safety to mitigate the risks associated with harmful outputs. Throughout 2024, developers and organizations are increasingly implementing reinforcement learning from human feedback (RLHF) to better align AI outputs with human values and ethical standards.

These processes involve training models on human evaluations of responses, helping the LLM learn from its mistakes and refine its future outputs. Organizations now conduct rigorous testing phases to assess the models' ability to handle sensitive topics and respond appropriately, minimizing the risk of misinformation or offensive content. This commitment to safety not only enhances the reliability of LLMs but also builds trust among users and stakeholders, fostering a more responsible deployment of AI technologies.

3. Smaller, More Efficient Models

Innovations in architecture and training techniques have led to the creation of smaller, more efficient LLMs that maintain performance levels comparable to their larger counterparts. The significance of this development cannot be overstated, as reduced computational requirements open pathways for deploying these models on edge devices such as smartphones, tablets, and other IoT devices.

By optimizing model architectures, organizations are capable of delivering valuable AI functionalities without the excessive computational costs traditionally associated with large models. This translates to broader accessibility and usability, ensuring that even smaller firms and individual developers can integrate AI capabilities into their applications. Consequently, this reduces barriers to entry in AI application development and promotes a more competitive ecosystem.

4. Augmented Human Intelligence

The advancements in AI LLMs in 2024 are significantly enhancing human intelligence across various professional domains. By integrating LLMs into creative tools, professionals such as writers, programmers, and designers are experiencing increased productivity and effectiveness in their work processes.

Features like code suggestions, automated content generation, and brainstorming support allow users to leverage the analytical prowess of LLMs to augment their creativity rather than replace it. This synergistic relationship fosters an environment of enhanced exploration where professionals can focus on higher-level creative tasks while leaning on LLMs to handle routine and labor-intensive aspects of their work. The outcome is a notable advancement in professional output quality and efficiency.

5. Pretraining with Diverse Data

In the evolving landscape of AI LLMs, there is a marked shift towards pretraining these models using more diverse datasets. This approach is instrumental in enhancing the models’ generalizability and enabling them to perform better in multilingual and cross-domain tasks.

By broadening the dataset horizon, developers ensure that models are equipped to cater to a more global audience, addressing the needs of users from various cultural and linguistic backgrounds. The diverse training material leads to a richer understanding of context and nuances in language, ultimately resulting in more effective and inclusive AI interactions. This emphasis on diversity in data not only improves performance but also contributes to a fairer representation of different demographics in AI outputs.

6. Legal and Regulatory Frameworks

With the increasing integration of AI technology into societal frameworks, the necessity for legal and regulatory structures has become paramount. In 2024, various countries and organizations are actively establishing guidelines governing the development and deployment of LLMs.

These regulations focus on critical issues such as data privacy, bias, and accountability, ensuring that AI technologies are used responsibly and ethically. Policymakers are engaging with stakeholders from multiple sectors to create comprehensive legislation that fosters innovation while safeguarding individuals' rights. The establishment of these legal frameworks is crucial for balancing the benefits of LLMs with societal concerns, ensuring that AI technologies contribute positively to overall progress.

7. Fine-tuning and Customization

Organizations are increasingly realizing the value of fine-tuning existing LLMs to address specific industry needs. This customization process results in more tailored models that can provide superior performance in specialized areas such as law, medicine, and customer service.

By adapting general-purpose LLMs to meet the unique requirements of particular sectors, organizations can achieve higher accuracy and relevance in model outputs. For instance, law firms can utilize fine-tuned models to analyze legal documents quickly, enhancing efficiency in case preparation. In healthcare, customized LLMs can assist professionals with patient data analysis, diagnostic suggestions, and streamlined communication. This specialized approach ensures that LLMs deliver maximum value to users by providing solutions that are closely aligned with industry practices.

8. Community-driven Model Development

The collaborative nature of open-source contributions has significantly accelerated advancements in LLM technology. In 2024, community-driven research efforts have become more prevalent, fostering environments where collective input leads to rapid iterations and improvements in AI models.

Collaborative projects enable smaller entities to partake in AI development, reducing barriers to entry and encouraging innovation across the board. This model of development promotes the sharing of best practices and resources, cultivating a rich ecosystem of knowledge and expertise. The result is a more vibrant AI landscape where diverse perspectives drive advancements and enhance the capabilities of LLMs.

9. AI Ethics and Fairness Initiatives

In response to growing concerns regarding the ethical implications of AI, there has been a pronounced emphasis on establishing AI ethics and fairness initiatives throughout 2024. Many institutions are now creating frameworks and best practices aimed at ensuring transparency, accountability, and fairness in LLM operations.

Dedicated groups focused on ethical AI research are examining bias mitigation, the impact of AI on diverse populations, and the importance of ethical decision-making processes in AI development. These initiatives aim to foster a culture of ethical responsibility among developers and organizations, promoting the idea that AI systems should not only be effective but also just and equitable. The ongoing discourse surrounding AI ethics is crucial to maintaining public trust and acceptance of LLM technologies.

10. Real-time Interactivity

The latest advancements in AI LLMs include enhanced real-time interactivity capabilities. As of 2024, these models can engage in dynamic conversations with users, learning and adapting from interactions to provide more personalized and contextually aware responses.

This ability makes LLMs increasingly viable for applications such as virtual customer support and interactive educational environments. By continuously learning from users, LLMs can tailor their responses based on individual preferences and historical interactions, thus improving user satisfaction and engagement. The shift towards real-time interactivity represents a significant progression in user experience, enabling AI to function as not just a tool but a responsive and engaging partner in communication.

In summary, the evolution of AI LLMs in 2024 highlights significant advancements in multimodal capabilities, safety improvements, efficiency, customization, and ethical considerations, positioning them to play a crucial role in various industries globally.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Process finished with exit code 0

Operating System

macOS Sonoma

Python Version

3.10

crewAI Version

0.76.9

crewAI Tools Version

0.13.4

Virtual Environment

Conda

Evidence

Traceback (most recent call last):
  File "/opt/anaconda3/envs/engine/lib/python3.10/site-packages/litellm/llms/AzureOpenAI/azure.py", line 380, in completion
    if "gateway.ai.cloudflare.com" in api_base:
TypeError: argument of type 'NoneType' is not iterable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/anaconda3/envs/engine/lib/python3.10/site-packages/litellm/main.py", line 1160, in completion
    response = azure_chat_completions.completion(
  File "/opt/anaconda3/envs/engine/lib/python3.10/site-packages/litellm/llms/AzureOpenAI/azure.py", line 545, in completion
    raise AzureOpenAIError(
litellm.llms.AzureOpenAI.azure.AzureOpenAIError: argument of type 'NoneType' is not iterable

Possible Solution

class Converter(OutputConverter):
    """Class that converts text into either pydantic or json."""

    def to_pydantic(self, current_attempt=1):
        """Convert text to pydantic."""
        try:
            if self.llm.supports_function_calling():
                return self._create_instructor().to_pydantic() # The error seems to original somewhere here:
            else:
                return self.llm.call(
                    [
                        {"role": "system", "content": self.instructions},
                        {"role": "user", "content": self.text},
                    ]
                )
        except Exception as e:
            if current_attempt < self.max_attempts:
                return self.to_pydantic(current_attempt + 1)
            return ConverterError(
                f"Failed to convert text into a pydantic model due to the following error: {e}"
            )

This method below does not use the right LLM to convert the response to pydantic:

    def to_pydantic(self):
        messages = [{"role": "user", "content": self.content}]
        if self.instructions:
            messages.append({"role": "system", "content": self.instructions})
        model = self._client.chat.completions.create(
            model=self.llm.model, response_model=self.model, messages=messages #Only LLM model name is used and not all the paramters
        )
        return model

Additional context

The to_pydantic method in internal_instructor should use the correct LLM parameters

The text was updated successfully, but these errors were encountered:

talrejanikhil · 2024-11-11T13:34:52Z

I noticed that if I comment out line 74 from CrewAgentExecutorMixin it fixes this bug

There was an exception here:
evaluation = ast.literal_eval(evaluation)

talrejanikhil added the bug Something isn't working label Nov 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG]Not able to use long term memory with Azure Open AI #1577

[BUG]Not able to use long term memory with Azure Open AI #1577

talrejanikhil commented Nov 11, 2024 •

edited

Loading

talrejanikhil commented Nov 11, 2024

[BUG]Not able to use long term memory with Azure Open AI #1577

[BUG]Not able to use long term memory with Azure Open AI #1577

Comments

talrejanikhil commented Nov 11, 2024 • edited Loading

Description

Steps to Reproduce

Expected behavior

Screenshots/Code snippets

Agent: AI LLMs Senior Data Researcher

Task: Conduct a thorough research about AI LLMs Make sure you find any interesting and relevant information given the current year is 2024.

Agent: AI LLMs Senior Data Researcher

Final Answer:

Agent: AI LLMs Reporting Analyst

Task: Review the context you got and expand each topic into a full section for a report. Make sure the report is detailed and contains any and all relevant information.

Agent: AI LLMs Reporting Analyst

Final Answer:

Comprehensive Report on the Evolution of AI LLMs in 2024

1. Multimodal Capabilities

2. Robustness and Safety Improvements

3. Smaller, More Efficient Models

4. Augmented Human Intelligence

5. Pretraining with Diverse Data

6. Legal and Regulatory Frameworks

7. Fine-tuning and Customization

8. Community-driven Model Development

9. AI Ethics and Fairness Initiatives

10. Real-time Interactivity

Operating System

Python Version

crewAI Version

crewAI Tools Version

Virtual Environment

Evidence

Possible Solution

Additional context

talrejanikhil commented Nov 11, 2024

talrejanikhil commented Nov 11, 2024 •

edited

Loading