Failing Search Performance Test (pinecone)

[jimin0]

To fix this error, please pass a unique `key` argument to the `plotly_chart` element.
2024-10-25 09:58:35,230 | INFO: generated uuid for the tasks: 7b1ea8ec49064ea58973e7f3d9a2deb1 (interface.py:66) (14586)
2024-10-25 09:58:35,232 | INFO | DB             | CaseType     Dataset               Filter | task_label (task_runner.py:338)
2024-10-25 09:58:35,232 | INFO | -----------    | ------------ -------------------- ------- | -------    (task_runner.py:338)
2024-10-25 09:58:35,233 | INFO | Pinecone       | Performance  OpenAI-SMALL-50K        None | 2024102509 (task_runner.py:338)
2024-10-25 09:58:35,233 | INFO: task submitted: id=7b1ea8ec49064ea58973e7f3d9a2deb1, 2024102509, case number: 1 (interface.py:231) (14586)
2024-10-25 09:58:35.671 WARNING streamlit.runtime.scriptrunner_utils.script_run_context: Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode.
2024-10-25 09:58:36,111 | INFO: [1/1] start case: {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'OpenAI', 'size': 50000, 'dim': 1536, 'metric_type': <MetricType.COSINE: 'COSINE'>}}, 'db': 'Pinecone'}, drop_old=True (interface.py:164) (14706)
2024-10-25 09:58:36,111 | INFO: Starting run (task_runner.py:100) (14706)
2024-10-25 09:58:37,103 | WARNING: pre run case error: 'Pinecone' object has no attribute 'index' (task_runner.py:96) (14706)
2024-10-25 09:58:37,104 | WARNING: [1/1] case {'label': <CaseLabel.Performance: 2>, 'dataset': {'data': {'name': 'OpenAI', 'size': 50000, 'dim': 1536, 'metric_type': <MetricType.COSINE: 'COSINE'>}}, 'db': 'Pinecone'} failed to run, reason='Pinecone' object has no attribute 'index' (interface.py:184) (14706)
Traceback (most recent call last):
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/interface.py", line 165, in _async_task_v2
    case_res.metrics = runner.run(drop_old)
                       ^^^^^^^^^^^^^^^^^^^^
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/backend/task_runner.py", line 102, in run
    self._pre_run(drop_old)
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/backend/task_runner.py", line 97, in _pre_run
    raise e from None
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/backend/task_runner.py", line 88, in _pre_run
    self.init_db(drop_old)
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/backend/task_runner.py", line 78, in init_db
    self.db = db_cls(
              ^^^^^^^
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/backend/clients/pinecone/pinecone.py", line 179, in __init__
    index = self.pc.index(self.index_name)
            ^^^^^^^^^^^^^
AttributeError: 'Pinecone' object has no attribute 'index'
2024-10-25 09:58:37,112 | INFO |Task summary: run_id=7b1ea, task_label=2024102509 (models.py:349)
2024-10-25 09:58:37,113 | INFO |DB       | db_label case                label      | load_dur    qps        latency(p99)    recall        max_load_count | label (models.py:349)
2024-10-25 09:58:37,113 | INFO |-------- | -------- ------------------- ---------- | ----------- ---------- --------------- ------------- -------------- | ----- (models.py:349)
2024-10-25 09:58:37,114 | INFO |Pinecone |          Performance1536D50K 2024102509 | 0.0         0.0        0.0             0.0           0              | x     (models.py:349)
Traceback (most recent call last):
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/interface.py", line 199, in _async_task_v2
    test_result.flush()
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/models.py", line 170, in flush
    timestamp = date.today().timestamp()
                ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'datetime.date' object has no attribute 'timestamp'
2024-10-25 09:58:37,116 | WARNING: An error occurs when running task=2024102509, run_id=7b1ea8ec49064ea58973e7f3d9a2deb1, err='datetime.date' object has no attribute 'timestamp' (interface.py:208) (14706)
2024-10-25 09:58:42,780 | INFO: will force stop running task: 7b1ea8ec49064ea58973e7f3d9a2deb1 (interface.py:218) (14586)
2024-10-25 09:58:42,783 | WARNING: sending SIGTERM to child process: psutil.Process(pid=14705, name='python3.11', status='sleeping', started='09:58:34') (interface.py:247) (14586)
2024-10-25 09:58:47.782 Uncaught app exception
Traceback (most recent call last):
  File "/home/jimin/vectorDBTEST/.vectorDBBench3108/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
  File "/home/jimin/vectorDBTEST/.vectorDBBench3108/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 579, in code_to_exec
    exec(code, module.__dict__)
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/frontend/pages/run_test.py", line 69, in <module>
    main()
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/frontend/pages/run_test.py", line 59, in main
    submitTask(submitContainer, tasks, isAllValid)
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/frontend/components/run_test/submitTask.py", line 25, in submitTask
    controlPanel(controlPanelContainer, tasks, taskLabel, isAllValid)
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/frontend/components/run_test/submitTask.py", line 70, in controlPanel
    isRunning = benchMarkRunner.has_running()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/interface.py", line 108, in has_running
    self._try_get_signal()
  File "/home/jimin/vectorDBTEST/VectorDBBench/vectordb_bench/interface.py", line 90, in _try_get_signal
    sig, received = self.receive_conn.recv()
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/multiprocessing/connection.py", line 249, in recv
    buf = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/multiprocessing/connection.py", line 413, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/multiprocessing/connection.py", line 382, in _recv
    raise EOFError
EOFError
2024-10-25 09:58:47,789 | WARNING: force killing child process: psutil.Process(pid=14705, name='python3.11', status='sleeping', started='09:58:34') (interface.py:255) (14586)

How can i solve this error?

step:
init_bench

• streamlit -
  run_test
  

Thanks

[alwayslove2013]

@jimin0 fixed in #380. The results page will work if updated.

AttributeError: 'datetime.date' object has no attribute 'timestamp'

We've noticed the issue, it will only report a WARNING, and it will not affect other functions.
streamlit was recently updated about the id limit for charts and I'm working on fixing that.

To fix this error, please pass a unique key argument to the plotly_chart element.

It looks like Pinecone updated its API. I'll tweak VectorDBBench's Pinecone client to adapt as soon as possible.

AttributeError: 'Pinecone' object has no attribute 'index'

[alwayslove2013]

@jimin0 Thank you very much for identifying the issue. It has now been fully resolved. You can try updating to the latest version.

If you have any further questions or issues, please feel free to reach out to us.