learn/synthetic_data/Llama_3_1_Synthetic_Data.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "d2a32163",
      "metadata": {},
      "source": [
        "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1arL7bWuF2P3soS3p19MWJeUDtW0Eu5tk?usp=sharing)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "71a43144",
      "metadata": {
        "id": "71a43144"
      },
      "source": [
        "# Get Started with Llama 3.1 Models\n",
        "\n",
        "\n",
        "Llama 3.1 release comes with three sizes of models 7B, 70B and 405B\n",
        "\n",
        "In this notebook, we will look at :\n",
        "\n",
        "*  How to access the Llama 3.1 models over a API?\n",
        "*  Generate Structured Synthetic Instruction Dataset with Llama 3.1 405B\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "TWZqnPr0pcv8",
      "metadata": {
        "id": "TWZqnPr0pcv8"
      },
      "source": [
        "## Setup\n",
        "\n",
        "Install all the dependencies and import the required python modules."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "YYRdRzHuHkjz",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YYRdRzHuHkjz",
        "outputId": "c497ab98-5b58-44cc-e1ef-3a67c65a4e7e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: fireworks-ai in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (0.15.8)\n",
            "Requirement already satisfied: httpx in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from fireworks-ai) (0.27.2)\n",
            "Requirement already satisfied: httpx-ws in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from fireworks-ai) (0.6.2)\n",
            "Requirement already satisfied: httpx-sse in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from fireworks-ai) (0.4.0)\n",
            "Requirement already satisfied: pydantic in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from fireworks-ai) (2.9.2)\n",
            "Requirement already satisfied: Pillow in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from fireworks-ai) (10.4.0)\n",
            "Requirement already satisfied: anyio in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpx->fireworks-ai) (4.6.0)\n",
            "Requirement already satisfied: certifi in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpx->fireworks-ai) (2024.8.30)\n",
            "Requirement already satisfied: httpcore==1.* in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpx->fireworks-ai) (1.0.5)\n",
            "Requirement already satisfied: idna in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpx->fireworks-ai) (3.10)\n",
            "Requirement already satisfied: sniffio in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpx->fireworks-ai) (1.3.1)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpcore==1.*->httpx->fireworks-ai) (0.14.0)\n",
            "Requirement already satisfied: wsproto in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from httpx-ws->fireworks-ai) (1.2.0)\n",
            "Requirement already satisfied: annotated-types>=0.6.0 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pydantic->fireworks-ai) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.23.4 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pydantic->fireworks-ai) (2.23.4)\n",
            "Requirement already satisfied: typing-extensions>=4.6.1 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pydantic->fireworks-ai) (4.12.2)\n",
            "\n",
            "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
            "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip3 install --upgrade pip\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!pip3 install --upgrade fireworks-ai"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "sV4u2hezqV4X",
      "metadata": {
        "id": "sV4u2hezqV4X"
      },
      "source": [
        "## Setup your API Key\n",
        "\n",
        "In order to use the Llama 3.1, you must first obtain Fireworks API Keys. If you don't already have one, you can one by following the instructions [here](https://docs.fireworks.ai/getting-started/quickstart)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "M9EtJxjJrlyD",
      "metadata": {
        "id": "M9EtJxjJrlyD"
      },
      "outputs": [],
      "source": [
        "from fireworks.client import Fireworks\n",
        "\n",
        "#replace the FIREWORKS_API_KEY with the key copied in the above step.\n",
        "client = Fireworks(api_key=\"FIREWORKS_API_KEY\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "FrFtrGQ6r-Mk",
      "metadata": {
        "id": "FrFtrGQ6r-Mk"
      },
      "source": [
        "## Accessing Llama 3.1 Models using API\n",
        "\n",
        "We are sending a request to Llama 3.1 405B model, alternatively you can change the model string to access the otherm models.\n",
        "\n",
        "* accounts/fireworks/models/llama-v3p1-70b-instruct\n",
        "* accounts/fireworks/models/llama-v3p1-8B-instruct"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "TVGsp9wvtO2Q",
      "metadata": {
        "id": "TVGsp9wvtO2Q"
      },
      "source": [
        "### Chat Completions API"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "DHAgBTk8Hpg_",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DHAgBTk8Hpg_",
        "outputId": "a2060088-0fac-4c29-c388-60c3760e8c0f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "I'm an artificial intelligence model known as Llama. Llama stands for \"Large Language Model Meta AI.\"\n"
          ]
        }
      ],
      "source": [
        "model_name = \"accounts/fireworks/models/llama-v3p1-405b-instruct\"\n",
        "\n",
        "response = client.chat.completions.create(\n",
        "\tmodel=model_name,\n",
        "\tmessages=[{\n",
        "\t\t\"role\": \"user\",\n",
        "\t\t\"content\": \"Who are you?\",\n",
        "\t}],\n",
        ")\n",
        "print(response.choices[0].message.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "O5i7rqkhJQJN",
      "metadata": {
        "id": "O5i7rqkhJQJN"
      },
      "source": [
        "## Generate Synthetic Data\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "yxVgS4xuBhFF",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yxVgS4xuBhFF",
        "outputId": "1ce8ebe2-c924-4065-d2c7-87e2a8d5e019"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: pydantic in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (2.9.2)\n",
            "Requirement already satisfied: annotated-types>=0.6.0 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pydantic) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.23.4 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pydantic) (2.23.4)\n",
            "Requirement already satisfied: typing-extensions>=4.6.1 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pydantic) (4.12.2)\n",
            "\n",
            "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
            "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip3 install --upgrade pip\u001b[0m\n",
            "Note: you may need to restart the kernel to use updated packages.\n"
          ]
        }
      ],
      "source": [
        "pip install pydantic"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "wcOA_lp1B9lD",
      "metadata": {
        "id": "wcOA_lp1B9lD"
      },
      "outputs": [],
      "source": [
        "from pydantic import BaseModel, Field"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "D9xgnLuGCAQi",
      "metadata": {
        "id": "D9xgnLuGCAQi"
      },
      "outputs": [],
      "source": [
        "from pydantic import BaseModel, Field\n",
        "from typing import List, Optional\n",
        "from enum import Enum\n",
        "\n",
        "\n",
        "class Category(str, Enum):\n",
        "    COUNTRIES = \"Countries\"\n",
        "    CAPITALS = \"Capitals\"\n",
        "    RIVERS = \"Rivers\"\n",
        "    MOUNTAINS = \"Mountains\"\n",
        "    LANDMARKS = \"Landmarks\"\n",
        "    CLIMATE = \"Climate\"\n",
        "    CULTURE = \"Culture\"\n",
        "\n",
        "class Difficulty(str, Enum):\n",
        "    EASY = \"Easy\"\n",
        "    MEDIUM = \"Medium\"\n",
        "    HARD = \"Hard\"\n",
        "    EXPERT = \"Expert\"\n",
        "\n",
        "class QuestionType(str, Enum):\n",
        "    MULTIPLE_CHOICE = \"Multiple Choice\"\n",
        "    TRUE_FALSE = \"True/False\"\n",
        "    FILL_IN_THE_BLANK = \"Fill in the Blank\"\n",
        "    SHORT_ANSWER = \"Short Answer\"\n",
        "\n",
        "class Question(BaseModel):\n",
        "    instruction: str\n",
        "    context: str\n",
        "    response: str\n",
        "    question_type: QuestionType\n",
        "    category: Category\n",
        "    difficulty: Difficulty\n",
        "\n",
        "class GeographyQuizDataset(BaseModel):\n",
        "    title: str = \"World Geography Challenge Dataset\"\n",
        "    description: str = \"Dataset for geography quiz questions and answers\"\n",
        "    questions: List[Question]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "5fdLrBLx9vwL",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5fdLrBLx9vwL",
        "outputId": "f4e84aff-af6c-4075-e48f-7674d764ab06"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'instruction': 'Which river is the longest in South America and flows through Brazil, Peru, and Colombia before emptying into the Pacific Ocean?', 'context': 'Rivers of the World', 'response': 'Amazon River', 'question_type': 'Multiple Choice', 'category': 'Rivers', 'difficulty': 'Medium'}\n",
            "Generated question 1/10: Which river is the longest in South America and flows through Brazil, Peru, and Colombia before emptying into the Pacific Ocean?\n",
            "{'instruction': \"What is the world's largest desert, covering about 9,200,000 square kilometers (3,600,000 sq mi), and spanning across several countries in North Africa?\", 'context': 'Deserts are large areas of land with very little rainfall and limited vegetation. They can be hot or cold and are found on every continent. The largest hot desert in the world is a significant geographical feature that affects climate, culture, and ecosystems across North Africa.', 'response': 'Sahara', 'question_type': 'Short Answer', 'category': 'Landmarks', 'difficulty': 'Medium'}\n",
            "Generated question 2/10: What is the world's largest desert, covering about 9,200,000 square kilometers (3,600,000 sq mi), and spanning across several countries in North Africa?\n",
            "{'instruction': 'Which river, approximately 6,400 kilometers long, flows through Brazil, Peru, and Colombia before emptying into the Pacific Ocean?', 'context': 'This question tests knowledge of major rivers in South America.', 'response': 'Amazon River', 'question_type': 'Multiple Choice', 'category': 'Rivers', 'difficulty': 'Medium'}\n",
            "Generated question 3/10: Which river, approximately 6,400 kilometers long, flows through Brazil, Peru, and Colombia before emptying into the Pacific Ocean?\n",
            "{'instruction': 'Which river is the longest in South America?', 'context': 'South America is home to many significant rivers, including the Orinoco, São Francisco, and Magdalena. However, one river stands out for its exceptional length.', 'response': 'Amazon River', 'question_type': 'Multiple Choice', 'category': 'Rivers', 'difficulty': 'Easy'}\n",
            "Generated question 4/10: Which river is the longest in South America?\n",
            "{'instruction': 'What is the name of the largest island in the Mediterranean Sea?', 'context': 'The Mediterranean Sea is a semi-enclosed sea connected to the Atlantic Ocean, surrounded by the Mediterranean region and almost completely enclosed by land: on the north by Southern Europe and Anatolia, on the south by North Africa, and on the east by the Levant.', 'response': 'Sicily', 'question_type': 'Multiple Choice', 'category': 'Landmarks', 'difficulty': 'Easy'}\n",
            "Generated question 5/10: What is the name of the largest island in the Mediterranean Sea?\n",
            "{'instruction': 'What is the name of the strait that separates the continents of Asia and Africa?', 'context': 'This strait is a significant shipping route and connects the Red Sea to the Gulf of Aden.', 'response': 'Bab-el-Mandeb', 'question_type': 'Short Answer', 'category': 'Landmarks', 'difficulty': 'Medium'}\n",
            "Generated question 6/10: What is the name of the strait that separates the continents of Asia and Africa?\n",
            "{'instruction': \"What is the world's largest desert, covering over 9,000,000 square kilometers (3,500,000 sq mi), and spanning across several countries in North Africa?\", 'context': 'Deserts of the world', 'response': 'Sahara', 'question_type': 'Short Answer', 'category': 'Landmarks', 'difficulty': 'Medium'}\n",
            "Generated question 7/10: What is the world's largest desert, covering over 9,000,000 square kilometers (3,500,000 sq mi), and spanning across several countries in North Africa?\n",
            "{'instruction': \"What is the world's largest desert, covering about 9,200,000 square kilometers (3,600,000 sq mi), and spanning across several countries in North Africa?\", 'context': 'Deserts are known for their extreme heat and arid conditions. This particular desert covers a significant portion of the African continent.', 'response': 'Sahara', 'question_type': 'Short Answer', 'category': 'Landmarks', 'difficulty': 'Medium'}\n",
            "Generated question 8/10: What is the world's largest desert, covering about 9,200,000 square kilometers (3,600,000 sq mi), and spanning across several countries in North Africa?\n",
            "{'instruction': \"What is the world's largest desert, covering about 9,200,000 square kilometers (3,600,000 sq mi), and spanning across several countries in North Africa?\", 'context': 'Deserts are vast expanses of arid land, often characterized by extreme heat and limited precipitation.', 'response': 'Sahara', 'question_type': 'Short Answer', 'category': 'Landmarks', 'difficulty': 'Medium'}\n",
            "Generated question 9/10: What is the world's largest desert, covering about 9,200,000 square kilometers (3,600,000 sq mi), and spanning across several countries in North Africa?\n",
            "{'instruction': 'Which river is the longest in South America and flows through Brazil, Peru, and Colombia before emptying into the Pacific Ocean?', 'context': 'Rivers of South America', 'response': 'Amazon River', 'question_type': 'Multiple Choice', 'category': 'Rivers', 'difficulty': 'Medium'}\n",
            "Generated question 10/10: Which river is the longest in South America and flows through Brazil, Peru, and Colombia before emptying into the Pacific Ocean?\n",
            "Generated and saved 10 questions to geography_quiz_dataset.jsonl\n"
          ]
        }
      ],
      "source": [
        "import json\n",
        "def generate_question():\n",
        "    prompt = \"\"\"Generate a geography quiz question. Format your response as a JSON object with the following structure:\n",
        "    {\n",
        "        \"instruction\": \"The full question text\",\n",
        "        \"context\": \"Provide context about the question\",\n",
        "        \"response\": \"The correct answer\",\n",
        "        \"question_type\": \"The type of question (e.g., 'Multiple Choice')\",\n",
        "        \"category\": \"The category should be marked as one of these: Countries, Capitals, Rivers, Mountains, Landmarks, Climate, Culture\",\n",
        "        \"difficulty\": \"The difficulty level of the question (e.g., 'Easy')\"\n",
        "    }\"\"\"\n",
        "\n",
        "    response = client.chat.completions.create(\n",
        "        model=\"accounts/fireworks/models/llama-v3p1-405b-instruct\",\n",
        "        response_format={\"type\": \"json_object\"},\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": \"You are a geography expert creating quiz questions.\"},\n",
        "            {\"role\": \"user\", \"content\": prompt}\n",
        "        ]\n",
        "    )\n",
        "\n",
        "    question_data = json.loads(response.choices[0].message.content)\n",
        "    print(question_data)\n",
        "    return Question(**question_data)\n",
        "\n",
        "def main(num_questions=10):\n",
        "    with open(\"geography_quiz_dataset.jsonl\", \"w\") as f:\n",
        "        for i in range(num_questions):\n",
        "            question = generate_question()\n",
        "            json.dump(question.dict(), f)\n",
        "            f.write(\"\\n\")\n",
        "            print(f\"Generated question {i+1}/{num_questions}: {question.instruction}\")\n",
        "\n",
        "    print(f\"Generated and saved {num_questions} questions to geography_quiz_dataset.jsonl\")\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "sDoxKlYwDORC",
      "metadata": {
        "id": "sDoxKlYwDORC"
      },
      "source": [
        "## Conclusion\n",
        "\n",
        "We’re excited to see how the community leverages Llama 3.1 API to create interesting applications.\n",
        "\n",
        "\n",
        "For more information and to get started with Llama 3.1, visit [docs.fireworks.ai](https://docs.fireworks.ai) or join our [discord community](https://discord.gg/fireworks-ai)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}