Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(agents-api): init nlp pipeline text-search #1045

Open
wants to merge 19 commits into
base: dev
Choose a base branch
from

Conversation

Vedantsahai18
Copy link
Member

@Vedantsahai18 Vedantsahai18 commented Jan 12, 2025

PR Type

Enhancement, Tests


Description

  • Enhanced NLP pipeline for text search with improved keyword extraction.

  • Refactored query-building logic for efficiency and clarity.

  • Added tests for embedding generation and text-to-query conversion.

  • Introduced utility for generating vectors with specific cosine similarity.


Changes walkthrough 📝

Relevant files
Enhancement
nlp.py
Enhanced NLP pipeline and query-building logic                     

agents-api/agents_api/common/nlp.py

  • Improved stopword filtering for noun chunks and spans.
  • Refactored query-building logic for efficiency and clarity.
  • Renamed and optimized functions for text-to-query conversion.
  • Increased caching capacity for query-related functions.
  • +61/-64 
    search_docs_by_text.py
    Preprocess raw text queries for search                                     

    agents-api/agents_api/queries/docs/search_docs_by_text.py

    • Added preprocessing step for raw text queries.
    +2/-0     
    utils.py
    Added vector generation utility for testing                           

    agents-api/tests/utils.py

  • Introduced utility to generate vectors with specific cosine
    similarity.
  • Added mathematical handling for edge cases in vector generation.
  • +47/-0   
    Tests
    fixtures.py
    Added test embeddings with confidence levels                         

    agents-api/tests/fixtures.py

  • Added embeddings with varying confidence levels for testing.
  • Inserted test data into the database for embeddings.
  • +53/-0   
    test_docs_queries.py
    Added tests for document search and text-to-query utility

    agents-api/tests/test_docs_queries.py

  • Added tests for searching documents with technical terms.
  • Verified document retrieval based on query relevance.
  • Added tests for text-to-query utility function.
  • +171/-0 

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information


    Important

    Enhances NLP pipeline for text search, refactors query logic, and adds tests for embedding and text-to-query conversion.

    • Enhancements:
      • Improved NLP pipeline in nlp.py with better stopword filtering and optimized query-building logic.
      • Added preprocessing step for raw text queries in search_docs_by_text.py.
      • Introduced utility in utils.py for generating vectors with specific cosine similarity.
    • Tests:
      • Added tests in test_docs_queries.py for document search and text-to-query conversion.
      • Added embeddings with varying confidence levels in fixtures.py for testing.
    • Refactoring:
      • Refactored query-building logic in nlp.py for efficiency and clarity.
      • Renamed functions for text-to-query conversion in nlp.py.

    This description was created by Ellipsis for 27ed1f4. It will automatically update as commits are pushed.

    @Vedantsahai18 Vedantsahai18 linked an issue Jan 12, 2025 that may be closed by this pull request
    1 task
    Copy link
    Contributor

    qodo-merge-pro-for-open-source bot commented Jan 12, 2025

    CI Failure Feedback 🧐

    (Checks updated until commit 6a07a54)

    Action: Test

    Failed stage: Run tests [❌]

    Failed test name: test_agent_routes

    Failure summary:

    The action failed due to a database connection pool configuration error. Specifically, the min_size
    (10) was set larger than the max_size (4) in the asyncpg connection pool settings, which is invalid.
    This caused the database connection initialization to fail during test startup, preventing the test
    fixtures from being properly resolved.

    Key error details:

  • ValueError: min_size is greater than max_size
  • min_size = 10
  • max_size = 4
  • This affected multiple test cases in test_agent_routes.py

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    1327:  PASS  test_agent_queries:28 query: create agent sql                          1%
    1328:  PASS  test_agent_queries:44 query: create or update agent sql                2%
    1329:  PASS  test_agent_queries:63 query: update agent sql                          2%
    1330:  PASS  test_agent_queries:85 query: get agent not exists sql                  3%
    1331:  PASS  test_agent_queries:96 query: get agent exists sql                      3%
    1332:  PASS  test_agent_queries:111 query: list agents sql                          4%
    1333:  PASS  test_agent_queries:122 query: patch agent sql                          4%
    1334:  PASS  test_agent_queries:143 query: delete agent sql                         5%
    1335:  FAIL  test_agent_routes:9 route: unauthorized should fail                    6%
    1336:  FAIL  test_agent_routes:26 route: create agent                               6%
    1337:  FAIL  test_agent_routes:43 route: create agent with instructions             7%
    1338:  ─────────────────────── route: unauthorized should fail ────────────────────────
    1339:  Failed at tests/test_agent_routes.py                                          
    ...
    
    1460:  │ │              name = '_dsn'                                           │ │  
    1461:  │ │              self = TestArgumentResolver(                            │ │  
    1462:  │ │                     │   test=Test(                                   │ │  
    1463:  │ │                     │   │   fn=<function _ at 0x7f93751a6ca0>,       │ │  
    1464:  │ │                     │   │   module_name='test_agent_routes',         │ │  
    1465:  │ │                     │   │   id='a459cf0eb64d4e74b9323b8604a16dec',   │ │  
    1466:  │ │                     │   │   marker=None,                             │ │  
    1467:  │ │                     │   │   description='route: unauthorized should  │ │  
    1468:  │ │                     fail',                                           │ │  
    ...
    
    1587:  │ │ self = <anyio._backends._asyncio.BlockingPortal object at            │ │  
    1588:  │ │        0x7f9373f74650>                                               │ │  
    1589:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    1590:  │                                                                          │  
    1591:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    1592:  │ python3.12/concurrent/futures/_base.py:456 in result                     │  
    1593:  │                                                                          │  
    1594:  │   453 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFI │  
    1595:  │   454 │   │   │   │   │   raise CancelledError()                         │  
    1596:  │   455 │   │   │   │   elif self._state == FINISHED:                      │  
    1597:  │ ❱ 456 │   │   │   │   │   return self.__get_result()                     │  
    1598:  │   457 │   │   │   │   else:                                              │  
    1599:  │   458 │   │   │   │   │   raise TimeoutError()                           │  
    ...
    
    1631:  │   221 │   │   except self._cancelled_exc_class:                          │  
    1632:  │                                                                          │  
    1633:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    1634:  │ │                args = ()                                             │ │  
    1635:  │ │                func = <bound method TestClient.wait_startup of       │ │  
    1636:  │ │                       <starlette.testclient.TestClient object at     │ │  
    1637:  │ │                       0x7f9373f75700>>                               │ │  
    1638:  │ │              future = <Future at 0x7f9373f56990 state=finished       │ │  
    1639:  │ │                       raised ValueError>                             │ │  
    ...
    
    1643:  │ │               scope = None                                           │ │  
    1644:  │ │                self = <anyio._backends._asyncio.BlockingPortal       │ │  
    1645:  │ │                       object at 0x7f9373f74650>                      │ │  
    1646:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    1647:  │                                                                          │  
    1648:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    1649:  │ ges/starlette/testclient.py:774 in wait_startup                          │  
    1650:  │                                                                          │  
    1651:  │   771 │   │   │   "lifespan.startup.failed",                             │  
    1652:  │   772 │   │   )                                                          │  
    1653:  │   773 │   │   if message["type"] == "lifespan.startup.failed":           │  
    1654:  │ ❱ 774 │   │   │   await receive()                                        │  
    1655:  │   775 │                                                                  │  
    1656:  │   776 │   async def wait_shutdown(self) -> None:                         │  
    1657:  │   777 │   │   async def receive() -> typing.Any:                         │  
    1658:  │                                                                          │  
    1659:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    1660:  │ │ message = {                                                          │ │  
    1661:  │ │           │   'type': 'lifespan.startup.failed',                     │ │  
    ...
    
    1680:  │ │ message = None                                                       │ │  
    1681:  │ │    self = <starlette.testclient.TestClient object at 0x7f9373f75700> │ │  
    1682:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    1683:  │                                                                          │  
    1684:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    1685:  │ python3.12/concurrent/futures/_base.py:449 in result                     │  
    1686:  │                                                                          │  
    1687:  │   446 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFI │  
    1688:  │   447 │   │   │   │   │   raise CancelledError()                         │  
    ...
    
    1724:  │   221 │   │   except self._cancelled_exc_class:                          │  
    1725:  │                                                                          │  
    1726:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    1727:  │ │                args = ()                                             │ │  
    1728:  │ │                func = <bound method TestClient.lifespan of           │ │  
    1729:  │ │                       <starlette.testclient.TestClient object at     │ │  
    1730:  │ │                       0x7f9373f75700>>                               │ │  
    1731:  │ │              future = <Future at 0x7f9373f57380 state=finished       │ │  
    1732:  │ │                       raised ValueError>                             │ │  
    ...
    
    1839:  │ │           waiting_senders=OrderedDict()), _closed=False),            │ │  
    1840:  │ │           receive_stream=MemoryObjectReceiveStream(_state=MemoryObj… │ │  
    1841:  │ │           buffer=deque([]), open_send_channels=1,                    │ │  
    1842:  │ │           open_receive_channels=1, waiting_receivers=OrderedDict(),  │ │  
    1843:  │ │           waiting_senders=OrderedDict()), _closed=False))>           │ │  
    1844:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    1845:  │                                                                          │  
    1846:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    1847:  │ ges/starlette/middleware/errors.py:152 in __call__                       │  
    ...
    
    1867:  │ │   scope = {                                                          │ │  
    1868:  │ │           │   'type': 'lifespan',                                    │ │  
    1869:  │ │           │   'state': {},                                           │ │  
    1870:  │ │           │   'app': <fastapi.applications.FastAPI object at         │ │  
    1871:  │ │           0x7f93904e5280>,                                           │ │  
    1872:  │ │           │   'router': <fastapi.routing.APIRouter object at         │ │  
    1873:  │ │           0x7f937852c620>                                            │ │  
    1874:  │ │           }                                                          │ │  
    1875:  │ │    self = <starlette.middleware.errors.ServerErrorMiddleware object  │ │  
    ...
    
    2097:  │ ges/starlette/routing.py:693 in lifespan                                 │  
    2098:  │                                                                          │  
    2099:  │   690 │   │   app: typing.Any = scope.get("app")                         │  
    2100:  │   691 │   │   await receive()                                            │  
    2101:  │   692 │   │   try:                                                       │  
    2102:  │ ❱ 693 │   │   │   async with self.lifespan_context(app) as maybe_state:  │  
    2103:  │   694 │   │   │   │   if maybe_state is not None:                        │  
    2104:  │   695 │   │   │   │   │   if "state" not in scope:                       │  
    2105:  │   696 │   │   │   │   │   │   raise RuntimeError('The server does not su │  
    ...
    
    2141:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2142:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2143:  │                                                                          │  
    2144:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2145:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2146:  │   209 │   │   try:                                                       │  
    2147:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2148:  │   211 │   │   except StopAsyncIteration:                                 │  
    2149:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2175:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2176:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2177:  │                                                                          │  
    2178:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2179:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2180:  │   209 │   │   try:                                                       │  
    2181:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2182:  │   211 │   │   except StopAsyncIteration:                                 │  
    2183:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2209:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2210:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2211:  │                                                                          │  
    2212:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2213:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2214:  │   209 │   │   try:                                                       │  
    2215:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2216:  │   211 │   │   except StopAsyncIteration:                                 │  
    2217:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2243:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2244:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2245:  │                                                                          │  
    2246:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2247:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2248:  │   209 │   │   try:                                                       │  
    2249:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2250:  │   211 │   │   except StopAsyncIteration:                                 │  
    2251:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2277:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2278:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2279:  │                                                                          │  
    2280:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2281:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2282:  │   209 │   │   try:                                                       │  
    2283:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2284:  │   211 │   │   except StopAsyncIteration:                                 │  
    2285:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2311:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2312:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2313:  │                                                                          │  
    2314:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2315:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2316:  │   209 │   │   try:                                                       │  
    2317:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2318:  │   211 │   │   except StopAsyncIteration:                                 │  
    2319:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2345:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2346:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2347:  │                                                                          │  
    2348:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2349:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2350:  │   209 │   │   try:                                                       │  
    2351:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2352:  │   211 │   │   except StopAsyncIteration:                                 │  
    2353:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2379:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2380:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2381:  │                                                                          │  
    2382:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2383:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2384:  │   209 │   │   try:                                                       │  
    2385:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2386:  │   211 │   │   except StopAsyncIteration:                                 │  
    2387:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2413:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2414:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2415:  │                                                                          │  
    2416:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2417:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2418:  │   209 │   │   try:                                                       │  
    2419:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2420:  │   211 │   │   except StopAsyncIteration:                                 │  
    2421:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2447:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2448:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2449:  │                                                                          │  
    2450:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2451:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2452:  │   209 │   │   try:                                                       │  
    2453:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2454:  │   211 │   │   except StopAsyncIteration:                                 │  
    2455:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2481:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2482:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    2483:  │                                                                          │  
    2484:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    2485:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    2486:  │   209 │   │   try:                                                       │  
    2487:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    2488:  │   211 │   │   except StopAsyncIteration:                                 │  
    2489:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    2552:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2553:  │                                                                          │  
    2554:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    2555:  │ ges/asyncpg/pool.py:361 in __init__                                      │  
    2556:  │                                                                          │  
    2557:  │    358 │   │   │   │   'min_size is expected to be greater or equal to z │  
    2558:  │    359 │   │                                                             │  
    2559:  │    360 │   │   if min_size > max_size:                                   │  
    2560:  │ ❱  361 │   │   │   raise ValueError('min_size is greater than max_size') │  
    2561:  │    362 │   │                                                             │  
    2562:  │    363 │   │   if max_queries <= 0:                                      │  
    2563:  │    364 │   │   │   raise ValueError('max_queries is expected to be great │  
    ...
    
    2577:  │ │                        max_size = 4                                  │ │  
    2578:  │ │                        min_size = 10                                 │ │  
    2579:  │ │                           reset = None                               │ │  
    2580:  │ │                            self = <asyncpg.pool.Pool object at       │ │  
    2581:  │ │                                   0x7f93751722c0>                    │ │  
    2582:  │ │                           setup = None                               │ │  
    2583:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2584:  ╰──────────────────────────────────────────────────────────────────────────╯  
    2585:  ValueError: min_size is greater than max_size                                 
    ...
    
    2662:  │ │         │   }                                                        │ │  
    2663:  │ │         )                                                            │ │  
    2664:  │ │  self = TestArgumentResolver(                                        │ │  
    2665:  │ │         │   test=Test(                                               │ │  
    2666:  │ │         │   │   fn=<function _ at 0x7f93751a6ca0>,                   │ │  
    2667:  │ │         │   │   module_name='test_agent_routes',                     │ │  
    2668:  │ │         │   │   id='a459cf0eb64d4e74b9323b8604a16dec',               │ │  
    2669:  │ │         │   │   marker=None,                                         │ │  
    2670:  │ │         │   │   description='route: unauthorized should fail',       │ │  
    ...
    
    2789:  │ │      resolved_args = {}                                              │ │  
    2790:  │ │               self = TestArgumentResolver(                           │ │  
    2791:  │ │                      │   test=Test(                                  │ │  
    2792:  │ │                      │   │   fn=<function _ at 0x7f93751a6ca0>,      │ │  
    2793:  │ │                      │   │   module_name='test_agent_routes',        │ │  
    2794:  │ │                      │   │   id='a459cf0eb64d4e74b9323b8604a16dec',  │ │  
    2795:  │ │                      │   │   marker=None,                            │ │  
    2796:  │ │                      │   │   description='route: unauthorized should │ │  
    2797:  │ │                      fail',                                          │ │  
    ...
    
    2822:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2823:  │                                                                          │  
    2824:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    2825:  │ ges/ward/testing.py:637 in _resolve_single_arg                           │  
    2826:  │                                                                          │  
    2827:  │   634 │   │   │   else:                                                  │  
    2828:  │   635 │   │   │   │   fixture.resolved_val = arg(**args_to_inject)       │  
    2829:  │   636 │   │   except (Exception, SystemExit) as e:                       │  
    2830:  │ ❱ 637 │   │   │   raise FixtureError(f"Unable to resolve fixture '{fixtu │  
    ...
    
    2943:  │ │              name = '_dsn'                                           │ │  
    2944:  │ │              self = TestArgumentResolver(                            │ │  
    2945:  │ │                     │   test=Test(                                   │ │  
    2946:  │ │                     │   │   fn=<function _ at 0x7f93751a6ca0>,       │ │  
    2947:  │ │                     │   │   module_name='test_agent_routes',         │ │  
    2948:  │ │                     │   │   id='a459cf0eb64d4e74b9323b8604a16dec',   │ │  
    2949:  │ │                     │   │   marker=None,                             │ │  
    2950:  │ │                     │   │   description='route: unauthorized should  │ │  
    2951:  │ │                     fail',                                           │ │  
    ...
    
    2970:  │ │                     │   │   timer=<ward._testing._Timer object at    │ │  
    2971:  │ │                     0x7f9373fa02f0>,                                 │ │  
    2972:  │ │                     │   │   tags=[]                                  │ │  
    2973:  │ │                     │   ),                                           │ │  
    2974:  │ │                     │   iteration=0                                  │ │  
    2975:  │ │                     )                                                │ │  
    2976:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2977:  ╰──────────────────────────────────────────────────────────────────────────╯  
    2978:  FixtureError: Unable to resolve fixture 'client'                              
    2979:  ───────────────────────────── route: create agent ──────────────────────────────
    2980:  Failed at tests/test_agent_routes.py                                          
    ...
    
    3227:  │ │ self = <anyio._backends._asyncio.BlockingPortal object at            │ │  
    3228:  │ │        0x7f9373f54380>                                               │ │  
    3229:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    3230:  │                                                                          │  
    3231:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3232:  │ python3.12/concurrent/futures/_base.py:456 in result                     │  
    3233:  │                                                                          │  
    3234:  │   453 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFI │  
    3235:  │   454 │   │   │   │   │   raise CancelledError()                         │  
    3236:  │   455 │   │   │   │   elif self._state == FINISHED:                      │  
    3237:  │ ❱ 456 │   │   │   │   │   return self.__get_result()                     │  
    3238:  │   457 │   │   │   │   else:                                              │  
    3239:  │   458 │   │   │   │   │   raise TimeoutError()                           │  
    ...
    
    3271:  │   221 │   │   except self._cancelled_exc_class:                          │  
    3272:  │                                                                          │  
    3273:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    3274:  │ │                args = ()                                             │ │  
    3275:  │ │                func = <bound method TestClient.wait_startup of       │ │  
    3276:  │ │                       <starlette.testclient.TestClient object at     │ │  
    3277:  │ │                       0x7f9373f572f0>>                               │ │  
    3278:  │ │              future = <Future at 0x7f9373f54e60 state=finished       │ │  
    3279:  │ │                       raised ValueError>                             │ │  
    ...
    
    3283:  │ │               scope = None                                           │ │  
    3284:  │ │                self = <anyio._backends._asyncio.BlockingPortal       │ │  
    3285:  │ │                       object at 0x7f9373f54380>                      │ │  
    3286:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    3287:  │                                                                          │  
    3288:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    3289:  │ ges/starlette/testclient.py:774 in wait_startup                          │  
    3290:  │                                                                          │  
    3291:  │   771 │   │   │   "lifespan.startup.failed",                             │  
    3292:  │   772 │   │   )                                                          │  
    3293:  │   773 │   │   if message["type"] == "lifespan.startup.failed":           │  
    3294:  │ ❱ 774 │   │   │   await receive()                                        │  
    3295:  │   775 │                                                                  │  
    3296:  │   776 │   async def wait_shutdown(self) -> None:                         │  
    3297:  │   777 │   │   async def receive() -> typing.Any:                         │  
    3298:  │                                                                          │  
    3299:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    3300:  │ │ message = {                                                          │ │  
    3301:  │ │           │   'type': 'lifespan.startup.failed',                     │ │  
    ...
    
    3320:  │ │ message = None                                                       │ │  
    3321:  │ │    self = <starlette.testclient.TestClient object at 0x7f9373f572f0> │ │  
    3322:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    3323:  │                                                                          │  
    3324:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3325:  │ python3.12/concurrent/futures/_base.py:449 in result                     │  
    3326:  │                                                                          │  
    3327:  │   446 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFI │  
    3328:  │   447 │   │   │   │   │   raise CancelledError()                         │  
    ...
    
    3364:  │   221 │   │   except self._cancelled_exc_class:                          │  
    3365:  │                                                                          │  
    3366:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    3367:  │ │                args = ()                                             │ │  
    3368:  │ │                func = <bound method TestClient.lifespan of           │ │  
    3369:  │ │                       <starlette.testclient.TestClient object at     │ │  
    3370:  │ │                       0x7f9373f572f0>>                               │ │  
    3371:  │ │              future = <Future at 0x7f9373f54f20 state=finished       │ │  
    3372:  │ │                       raised ValueError>                             │ │  
    ...
    
    3479:  │ │           waiting_senders=OrderedDict()), _closed=False),            │ │  
    3480:  │ │           receive_stream=MemoryObjectReceiveStream(_state=MemoryObj… │ │  
    3481:  │ │           buffer=deque([]), open_send_channels=1,                    │ │  
    3482:  │ │           open_receive_channels=1, waiting_receivers=OrderedDict(),  │ │  
    3483:  │ │           waiting_senders=OrderedDict()), _closed=False))>           │ │  
    3484:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    3485:  │                                                                          │  
    3486:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    3487:  │ ges/starlette/middleware/errors.py:152 in __call__                       │  
    ...
    
    3507:  │ │   scope = {                                                          │ │  
    3508:  │ │           │   'type': 'lifespan',                                    │ │  
    3509:  │ │           │   'state': {},                                           │ │  
    3510:  │ │           │   'app': <fastapi.applications.FastAPI object at         │ │  
    3511:  │ │           0x7f93904e5280>,                                           │ │  
    3512:  │ │           │   'router': <fastapi.routing.APIRouter object at         │ │  
    3513:  │ │           0x7f937852c620>                                            │ │  
    3514:  │ │           }                                                          │ │  
    3515:  │ │    self = <starlette.middleware.errors.ServerErrorMiddleware object  │ │  
    ...
    
    3737:  │ ges/starlette/routing.py:693 in lifespan                                 │  
    3738:  │                                                                          │  
    3739:  │   690 │   │   app: typing.Any = scope.get("app")                         │  
    3740:  │   691 │   │   await receive()                                            │  
    3741:  │   692 │   │   try:                                                       │  
    3742:  │ ❱ 693 │   │   │   async with self.lifespan_context(app) as maybe_state:  │  
    3743:  │   694 │   │   │   │   if maybe_state is not None:                        │  
    3744:  │   695 │   │   │   │   │   if "state" not in scope:                       │  
    3745:  │   696 │   │   │   │   │   │   raise RuntimeError('The server does not su │  
    ...
    
    3781:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3782:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3783:  │                                                                          │  
    3784:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3785:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3786:  │   209 │   │   try:                                                       │  
    3787:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3788:  │   211 │   │   except StopAsyncIteration:                                 │  
    3789:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    3815:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3816:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3817:  │                                                                          │  
    3818:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3819:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3820:  │   209 │   │   try:                                                       │  
    3821:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3822:  │   211 │   │   except StopAsyncIteration:                                 │  
    3823:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    3849:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3850:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3851:  │                                                                          │  
    3852:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3853:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3854:  │   209 │   │   try:                                                       │  
    3855:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3856:  │   211 │   │   except StopAsyncIteration:                                 │  
    3857:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    3883:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3884:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3885:  │                                                                          │  
    3886:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3887:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3888:  │   209 │   │   try:                                                       │  
    3889:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3890:  │   211 │   │   except StopAsyncIteration:                                 │  
    3891:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    3917:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3918:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3919:  │                                                                          │  
    3920:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3921:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3922:  │   209 │   │   try:                                                       │  
    3923:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3924:  │   211 │   │   except StopAsyncIteration:                                 │  
    3925:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    3951:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3952:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3953:  │                                                                          │  
    3954:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3955:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3956:  │   209 │   │   try:                                                       │  
    3957:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3958:  │   211 │   │   except StopAsyncIteration:                                 │  
    3959:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    3985:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    3986:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    3987:  │                                                                          │  
    3988:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    3989:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    3990:  │   209 │   │   try:                                                       │  
    3991:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    3992:  │   211 │   │   except StopAsyncIteration:                                 │  
    3993:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    4019:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    4020:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    4021:  │                                                                          │  
    4022:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    4023:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    4024:  │   209 │   │   try:                                                       │  
    4025:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    4026:  │   211 │   │   except StopAsyncIteration:                                 │  
    4027:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    4053:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    4054:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    4055:  │                                                                          │  
    4056:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    4057:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    4058:  │   209 │   │   try:                                                       │  
    4059:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    4060:  │   211 │   │   except StopAsyncIteration:                                 │  
    4061:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    4087:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    4088:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    4089:  │                                                                          │  
    4090:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    4091:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    4092:  │   209 │   │   try:                                                       │  
    4093:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    4094:  │   211 │   │   except StopAsyncIteration:                                 │  
    4095:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    4121:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    4122:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    4123:  │                                                                          │  
    4124:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    4125:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    4126:  │   209 │   │   try:                                                       │  
    4127:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    4128:  │   211 │   │   except StopAsyncIteration:                                 │  
    4129:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    4192:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    4193:  │                                                                          │  
    4194:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    4195:  │ ges/asyncpg/pool.py:361 in __init__                                      │  
    4196:  │                                                                          │  
    4197:  │    358 │   │   │   │   'min_size is expected to be greater or equal to z │  
    4198:  │    359 │   │                                                             │  
    4199:  │    360 │   │   if min_size > max_size:                                   │  
    4200:  │ ❱  361 │   │   │   raise ValueError('min_size is greater than max_size') │  
    4201:  │    362 │   │                                                             │  
    4202:  │    363 │   │   if max_queries <= 0:                                      │  
    4203:  │    364 │   │   │   raise ValueError('max_queries is expected to be great │  
    ...
    
    4217:  │ │                        max_size = 4                                  │ │  
    4218:  │ │                        min_size = 10                                 │ │  
    4219:  │ │                           reset = None                               │ │  
    4220:  │ │                            self = <asyncpg.pool.Pool object at       │ │  
    4221:  │ │                                   0x7f9373384580>                    │ │  
    4222:  │ │                           setup = None                               │ │  
    4223:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    4224:  ╰──────────────────────────────────────────────────────────────────────────╯  
    4225:  ValueError: min_size is greater than max_size                                 
    ...
    
    4603:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    4604:  │                                                                          │  
    4605:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    4606:  │ ges/ward/testing.py:637 in _resolve_single_arg                           │  
    4607:  │                                                                          │  
    4608:  │   634 │   │   │   else:                                                  │  
    4609:  │   635 │   │   │   │   fixture.resolved_val = arg(**args_to_inject)       │  
    4610:  │   636 │   │   except (Exception, SystemExit) as e:                       │  
    4611:  │ ❱ 637 │   │   │   raise FixtureError(f"Unable to resolve fixture '{fixtu │  
    ...
    
    4750:  │ │                     │   │   timer=<ward._testing._Timer object at    │ │  
    4751:  │ │                     0x7f9373f74350>,                                 │ │  
    4752:  │ │                     │   │   tags=[]                                  │ │  
    4753:  │ │                     │   ),                                           │ │  
    4754:  │ │                     │   iteration=0                                  │ │  
    4755:  │ │                     )                                                │ │  
    4756:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    4757:  ╰──────────────────────────────────────────────────────────────────────────╯  
    4758:  FixtureError: Unable to resolve fixture 'client'                              
    4759:  ──────────────────── route: create agent with instructions ─────────────────────
    4760:  Failed at tests/test_agent_routes.py                                          
    ...
    
    5008:  │ │ self = <anyio._backends._asyncio.BlockingPortal object at            │ │  
    5009:  │ │        0x7f937518b6e0>                                               │ │  
    5010:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    5011:  │                                                                          │  
    5012:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5013:  │ python3.12/concurrent/futures/_base.py:456 in result                     │  
    5014:  │                                                                          │  
    5015:  │   453 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFI │  
    5016:  │   454 │   │   │   │   │   raise CancelledError()                         │  
    5017:  │   455 │   │   │   │   elif self._state == FINISHED:                      │  
    5018:  │ ❱ 456 │   │   │   │   │   return self.__get_result()                     │  
    5019:  │   457 │   │   │   │   else:                                              │  
    5020:  │   458 │   │   │   │   │   raise TimeoutError()                           │  
    ...
    
    5052:  │   221 │   │   except self._cancelled_exc_class:                          │  
    5053:  │                                                                          │  
    5054:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    5055:  │ │                args = ()                                             │ │  
    5056:  │ │                func = <bound method TestClient.wait_startup of       │ │  
    5057:  │ │                       <starlette.testclient.TestClient object at     │ │  
    5058:  │ │                       0x7f939192dfd0>>                               │ │  
    5059:  │ │              future = <Future at 0x7f937518b020 state=finished       │ │  
    5060:  │ │                       raised ValueError>                             │ │  
    ...
    
    5064:  │ │               scope = None                                           │ │  
    5065:  │ │                self = <anyio._backends._asyncio.BlockingPortal       │ │  
    5066:  │ │                       object at 0x7f937518b6e0>                      │ │  
    5067:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    5068:  │                                                                          │  
    5069:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    5070:  │ ges/starlette/testclient.py:774 in wait_startup                          │  
    5071:  │                                                                          │  
    5072:  │   771 │   │   │   "lifespan.startup.failed",                             │  
    5073:  │   772 │   │   )                                                          │  
    5074:  │   773 │   │   if message["type"] == "lifespan.startup.failed":           │  
    5075:  │ ❱ 774 │   │   │   await receive()                                        │  
    5076:  │   775 │                                                                  │  
    5077:  │   776 │   async def wait_shutdown(self) -> None:                         │  
    5078:  │   777 │   │   async def receive() -> typing.Any:                         │  
    5079:  │                                                                          │  
    5080:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    5081:  │ │ message = {                                                          │ │  
    5082:  │ │           │   'type': 'lifespan.startup.failed',                     │ │  
    ...
    
    5101:  │ │ message = None                                                       │ │  
    5102:  │ │    self = <starlette.testclient.TestClient object at 0x7f939192dfd0> │ │  
    5103:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    5104:  │                                                                          │  
    5105:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5106:  │ python3.12/concurrent/futures/_base.py:449 in result                     │  
    5107:  │                                                                          │  
    5108:  │   446 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFI │  
    5109:  │   447 │   │   │   │   │   raise CancelledError()                         │  
    ...
    
    5145:  │   221 │   │   except self._cancelled_exc_class:                          │  
    5146:  │                                                                          │  
    5147:  │ ╭─────────────────────────────── locals ───────────────────────────────╮ │  
    5148:  │ │                args = ()                                             │ │  
    5149:  │ │                func = <bound method TestClient.lifespan of           │ │  
    5150:  │ │                       <starlette.testclient.TestClient object at     │ │  
    5151:  │ │                       0x7f939192dfd0>>                               │ │  
    5152:  │ │              future = <Future at 0x7f937518bcb0 state=finished       │ │  
    5153:  │ │                       raised ValueError>                             │ │  
    ...
    
    5260:  │ │           waiting_senders=OrderedDict()), _closed=False),            │ │  
    5261:  │ │           receive_stream=MemoryObjectReceiveStream(_state=MemoryObj… │ │  
    5262:  │ │           buffer=deque([]), open_send_channels=1,                    │ │  
    5263:  │ │           open_receive_channels=1, waiting_receivers=OrderedDict(),  │ │  
    5264:  │ │           waiting_senders=OrderedDict()), _closed=False))>           │ │  
    5265:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    5266:  │                                                                          │  
    5267:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    5268:  │ ges/starlette/middleware/errors.py:152 in __call__                       │  
    ...
    
    5288:  │ │   scope = {                                                          │ │  
    5289:  │ │           │   'type': 'lifespan',                                    │ │  
    5290:  │ │           │   'state': {},                                           │ │  
    5291:  │ │           │   'app': <fastapi.applications.FastAPI object at         │ │  
    5292:  │ │           0x7f93904e5280>,                                           │ │  
    5293:  │ │           │   'router': <fastapi.routing.APIRouter object at         │ │  
    5294:  │ │           0x7f937852c620>                                            │ │  
    5295:  │ │           }                                                          │ │  
    5296:  │ │    self = <starlette.middleware.errors.ServerErrorMiddleware object  │ │  
    ...
    
    5518:  │ ges/starlette/routing.py:693 in lifespan                                 │  
    5519:  │                                                                          │  
    5520:  │   690 │   │   app: typing.Any = scope.get("app")                         │  
    5521:  │   691 │   │   await receive()                                            │  
    5522:  │   692 │   │   try:                                                       │  
    5523:  │ ❱ 693 │   │   │   async with self.lifespan_context(app) as maybe_state:  │  
    5524:  │   694 │   │   │   │   if maybe_state is not None:                        │  
    5525:  │   695 │   │   │   │   │   if "state" not in scope:                       │  
    5526:  │   696 │   │   │   │   │   │   raise RuntimeError('The server does not su │  
    ...
    
    5562:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5563:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5564:  │                                                                          │  
    5565:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5566:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5567:  │   209 │   │   try:                                                       │  
    5568:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5569:  │   211 │   │   except StopAsyncIteration:                                 │  
    5570:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5596:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5597:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5598:  │                                                                          │  
    5599:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5600:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5601:  │   209 │   │   try:                                                       │  
    5602:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5603:  │   211 │   │   except StopAsyncIteration:                                 │  
    5604:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5630:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5631:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5632:  │                                                                          │  
    5633:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5634:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5635:  │   209 │   │   try:                                                       │  
    5636:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5637:  │   211 │   │   except StopAsyncIteration:                                 │  
    5638:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5664:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5665:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5666:  │                                                                          │  
    5667:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5668:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5669:  │   209 │   │   try:                                                       │  
    5670:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5671:  │   211 │   │   except StopAsyncIteration:                                 │  
    5672:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5698:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5699:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5700:  │                                                                          │  
    5701:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5702:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5703:  │   209 │   │   try:                                                       │  
    5704:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5705:  │   211 │   │   except StopAsyncIteration:                                 │  
    5706:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5732:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5733:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5734:  │                                                                          │  
    5735:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5736:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5737:  │   209 │   │   try:                                                       │  
    5738:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5739:  │   211 │   │   except StopAsyncIteration:                                 │  
    5740:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5766:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5767:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5768:  │                                                                          │  
    5769:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5770:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5771:  │   209 │   │   try:                                                       │  
    5772:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5773:  │   211 │   │   except StopAsyncIteration:                                 │  
    5774:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5800:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5801:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5802:  │                                                                          │  
    5803:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5804:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5805:  │   209 │   │   try:                                                       │  
    5806:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5807:  │   211 │   │   except StopAsyncIteration:                                 │  
    5808:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5834:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5835:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5836:  │                                                                          │  
    5837:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5838:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5839:  │   209 │   │   try:                                                       │  
    5840:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5841:  │   211 │   │   except StopAsyncIteration:                                 │  
    5842:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5868:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5869:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5870:  │                                                                          │  
    5871:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5872:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5873:  │   209 │   │   try:                                                       │  
    5874:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5875:  │   211 │   │   except StopAsyncIteration:                                 │  
    5876:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5902:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    5903:  │ python3.12/contextlib.py:210 in __aenter__                               │  
    5904:  │                                                                          │  
    5905:  │   207 │   │   # they are only needed for recreation, which is not possib │  
    5906:  │   208 │   │   del self.args, self.kwds, self.func                        │  
    5907:  │   209 │   │   try:                                                       │  
    5908:  │ ❱ 210 │   │   │   return await anext(self.gen)                           │  
    5909:  │   211 │   │   except StopAsyncIteration:                                 │  
    5910:  │   212 │   │   │   raise RuntimeError("generator didn't yield") from None │  
    ...
    
    5973:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    5974:  │                                                                          │  
    5975:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    5976:  │ ges/asyncpg/pool.py:361 in __init__                                      │  
    5977:  │                                                                          │  
    5978:  │    358 │   │   │   │   'min_size is expected to be greater or equal to z │  
    5979:  │    359 │   │                                                             │  
    5980:  │    360 │   │   if min_size > max_size:                                   │  
    5981:  │ ❱  361 │   │   │   raise ValueError('min_size is greater than max_size') │  
    5982:  │    362 │   │                                                             │  
    5983:  │    363 │   │   if max_queries <= 0:                                      │  
    5984:  │    364 │   │   │   raise ValueError('max_queries is expected to be great │  
    ...
    
    5998:  │ │                        max_size = 4                                  │ │  
    5999:  │ │                        min_size = 10                                 │ │  
    6000:  │ │                           reset = None                               │ │  
    6001:  │ │                            self = <asyncpg.pool.Pool object at       │ │  
    6002:  │ │                                   0x7f9373385600>                    │ │  
    6003:  │ │                           setup = None                               │ │  
    6004:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    6005:  ╰──────────────────────────────────────────────────────────────────────────╯  
    6006:  ValueError: min_size is greater than max_size                                 
    ...
    
    6386:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    6387:  │                                                                          │  
    6388:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    6389:  │ ges/ward/testing.py:637 in _resolve_single_arg                           │  
    6390:  │                                                                          │  
    6391:  │   634 │   │   │   else:                                                  │  
    6392:  │   635 │   │   │   │   fixture.resolved_val = arg(**args_to_inject)       │  
    6393:  │   636 │   │   except (Exception, SystemExit) as e:                       │  
    6394:  │ ❱ 637 │   │   │   raise FixtureError(f"Unable to resolve fixture '{fixtu │  
    ...
    
    6534:  │ │                     │   │   timer=<ward._testing._Timer object at    │ │  
    6535:  │ │                     0x7f9373f8f0e0>,                                 │ │  
    6536:  │ │                     │   │   tags=[]                                  │ │  
    6537:  │ │                     │   ),                                           │ │  
    6538:  │ │                     │   iteration=0                                  │ │  
    6539:  │ │                     )                                                │ │  
    6540:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    6541:  ╰──────────────────────────────────────────────────────────────────────────╯  
    6542:  FixtureError: Unable to resolve fixture 'client'                              
    6543:  ────────────────────────────────────────────────────────────────────────────────
    6544:  ╭──────────── Results ─────────────╮
    6545:  │  12  Tests Encountered           │
    6546:  │   9  Passes             (75.0%)  │
    6547:  │   3  Failures           (25.0%)  │
    6548:  ╰──────────────────────────────────╯
    6549:  ─────────────────────────── FAILED in 53.72 seconds ────────────────────────────
    6550:  ##[error]Process completed with exit code 1.
    

    ✨ CI feedback usage guide:

    The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
    The tool analyzes the failed checks and provides several feedbacks:

    • Failed stage
    • Failed test name
    • Failure summary
    • Relevant error logs

    In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

    /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"
    

    where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed...

    @Vedantsahai18 Vedantsahai18 changed the title fix(agetns-api): init nlp pipeline text-search fix(agents-api): init nlp pipeline text-search Jan 14, 2025
    @Vedantsahai18 Vedantsahai18 marked this pull request as ready for review January 14, 2025 04:00
    Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Performance

    The LRU cache size of 1000 for text_to_tsvector_query may need tuning based on actual usage patterns and memory constraints

    @lru_cache(maxsize=1000)
    def text_to_tsvector_query(
    Code Duplication

    Similar code blocks for processing groups and building queries are duplicated between text_to_tsvector_query and batch_text_to_tsvector_queries functions

    for group in groups:
        if len(group) > 1:
            # Sort by length descending to prioritize longer phrases
            sorted_group = sorted(group, key=len, reverse=True)
            # For truly proximate multi-word groups, group words
            queries.add(" OR ".join(sorted_group))
        else:
            # For non-proximate words or single words, add them separately
            queries.update(group)
    Commented Code

    Large block of commented test code should either be implemented or removed to maintain code clarity

    # @test("query: search docs by embedding with different confidence levels")
    # async def _(
    #     dsn=pg_dsn, agent=test_agent, developer=test_developer, doc=test_doc_with_embedding
    # ):
    #     pool = await create_db_pool(dsn=dsn)
    
    #     # Get query embedding (using original doc's embedding)
    #     query_embedding = make_vector_with_similarity(EMBEDDING_SIZE, 0.7)
    
    #     # Test with different confidence levels
    #     confidence_tests = [
    #         (0.99, 0),  # Very high similarity threshold - should find no results
    #         (0.7, 1),   # High similarity - should find 1 result (the embedding with all 1.0s)
    #         (0.3, 2),   # Medium similarity - should find 2 results (including 0.3-0.7 embedding)
    #         (-0.8, 3),  # Low similarity - should find 3 results (including -0.8 to 0.8 embedding)
    #         (-1.0, 4)   # Lowest similarity - should find all 4 results (including alternating -1/1)
    #     ]
    
    #     for confidence, expected_min_results in confidence_tests:
    #         results = await search_docs_by_embedding(
    #             developer_id=developer.id,
    #             owners=[("agent", agent.id)],
    #             embedding=query_embedding,
    #             k=3,
    #             confidence=confidence,
    #             metadata_filter={"test": "test"},
    #             connection_pool=pool,
    #         )
    
    #         print(f"\nSearch results with confidence {confidence}:")
    #         for r in results:
    #             print(f"- Doc ID: {r.id}, Distance: {r.distance}")
    
    #         assert len(results) >= expected_min_results, (
    #             f"Expected at least {expected_min_results} results with confidence {confidence}, got {len(results)}"
    #         )
    
    #         if results:
    #             # Verify that all returned results meet the confidence threshold
    #             for result in results:
    #                 assert result.distance >= confidence, (
    #                     f"Result distance {result.distance} is below confidence threshold {confidence}"
    #                 )

    Copy link
    Contributor

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Security
    Add input sanitization to prevent SQL injection and handle special characters

    The text_to_tsvector_query function is not handling special characters and SQL
    injection risks in the query string. Add proper escaping and sanitization of the
    query terms before joining them with OR operators.

    agents-api/agents_api/common/nlp.py [240]

    -queries.add(" OR ".join(sorted_group))
    +queries.add(" OR ".join(re.sub(r'[^\w\s-]', '', term).strip() for term in sorted_group if term.strip()))
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    Why: Critical security enhancement that prevents potential SQL injection vulnerabilities by properly sanitizing query terms before database operations.

    9
    Possible issue
    Enable query preprocessing to ensure proper text search functionality

    The commented out query preprocessing line should be uncommented to ensure proper
    text query handling. Without this preprocessing, raw queries may not be properly
    formatted for the search.

    agents-api/agents_api/queries/docs/search_docs_by_text.py [63-64]

    -#  Pre-process rawtext query
    -# query = text_to_tsvector_query(query)
    +# Pre-process rawtext query
    +query = text_to_tsvector_query(query)
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: Important fix that enables essential query preprocessing functionality, which is currently disabled but necessary for proper text search operation.

    8
    General
    Improve stopword filtering to retain meaningful multi-token phrases

    The current implementation may miss important keywords by filtering out spans where
    all tokens are stopwords. Some multi-token spans like "in front of" or "up to" are
    meaningful despite being composed of stopwords. Consider keeping spans that match
    specific patterns or are part of recognized phrases.

    agents-api/agents_api/common/nlp.py [110-112]

    -# Skip if all tokens in span are stopwords
    -if all(token.is_stop for token in span):
    +# Skip spans of only stopwords, except for recognized phrases
    +if all(token.is_stop for token in span) and not span.root.dep_ in {'pobj', 'pcomp', 'prep'}:
         continue
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: The suggestion addresses a significant limitation in keyword extraction by preserving meaningful multi-token phrases that would otherwise be filtered out, improving search accuracy and relevance.

    7

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍 Looks good to me! Reviewed everything up to cb86135 in 1 minute and 22 seconds

    More details
    • Looked at 543 lines of code in 5 files
    • Skipped 0 files when reviewing.
    • Skipped posting 1 drafted comments based on config settings.
    1. agents-api/agents_api/queries/docs/search_docs_by_text.py:64
    • Draft comment:
      The text_to_tsvector_query function is commented out. If this is intentional, consider removing it. Otherwise, uncomment it to ensure the text query is processed correctly.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The text_to_tsvector_query function is commented out in the search_docs_by_text function, which might be an oversight since the PR is about initializing the NLP pipeline for text search.

    Workflow ID: wflow_TVBvQPCSjdXPF22A


    You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    ❌ Changes requested. Incremental review on 27ed1f4 in 35 seconds

    More details
    • Looked at 42 lines of code in 1 files
    • Skipped 0 files when reviewing.
    • Skipped posting 0 drafted comments based on config settings.

    Workflow ID: wflow_DnFTLZ3o4XbKJR8j


    Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    agents-api/agents_api/common/nlp.py Outdated Show resolved Hide resolved
    Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    [Bug]: Fix text and vector searches
    3 participants