Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom dataset increased test dataset testing vectorscale #16

Open
wants to merge 96 commits into
base: pvs-testing
Choose a base branch
from
Open

Custom dataset increased test dataset testing vectorscale #16

wants to merge 96 commits into from

Conversation

Sheharyar570
Copy link
Collaborator

No description provided.

wahajali and others added 30 commits August 28, 2024 12:52
@wahajali
Add pgdiskann client
3da8a9d
@wahajali
Add testing configuration, capture table/index sizes
b85d7d2
@wahajali
Increase timeout
e0e6309
@wahajali
update run count
4ad827a
@wahajali
Update parallel workers
0e74142
@wahajali
Update num-concurrency
4a8c1c9
@wahajali
Fix spelling
6380d82
@Sheharyar570
Saving benchmark results in DB
778fba8
@Sheharyar570
Fix spelling mistake
2900e13
@wahajali
Reduce concurrency and add K
7652731
@wahajali
Merge branch 'pgvector-testing' of github.com:EmumbaOrg/VectorDBBench…
b7c7f68 
… into pgvector-testing
@wahajali @Sheharyar570
Add pgdiskann client
e931034
@Sheharyar570
Add CLI support in pgdiskann
50ff354
@wahajali
Merge branch 'diskann-support' of github.com:EmumbaOrg/VectorDBBench …
0e36aeb 
…into diskann-support
@wahajali
Merge pull request #5 from EmumbaOrg/add-pgdiskann-cli-support
d081997 
Add CLI support in pgdiskann
@wahajali
Merge branch 'diskann-support' of github.com:EmumbaOrg/VectorDBBench …
021cbde 
…into diskann-support
@wahajali
Remove data from git
f984400
@wahajali
Merge branch 'pgvector-testing' into diskann-testing
b7f982b
@wahajali
Add get_size_info method for pgdiskann
c1cd526
@wahajali
Add automation for diskann
4b773c4
@wahajali
Add sample config files
9927505
@wahajali
Set storage to plain
a54a4f4
@wahajali
Update timeout
3555837
@wahajali
Update run.py
8468260
@Sheharyar570
added xxl instance config file
6d1a701
@wahajali
Merge pull request #7 from EmumbaOrg/add-xxl-config-file
421b000 
Added XXL instance config file
@wahajali
Add filter search sample config
4f187e2
@Sheharyar570
Add script that creates subsets of dataset.
9ada5ee
@Sheharyar570
Add script to compute ground truth for dataset subset
3ea17ed
@Sheharyar570
Fixed dataset file name issues
174dbf2
Sheharyar570 and others added 30 commits November 4, 2024 21:34
@Sheharyar570
fixed dataset args directory path
f50990d
@Sheharyar570
large test dataset related changes
b128c55
@Sheharyar570
10k vectors test dataset added
c61c616
@Sheharyar570
Configs updated to run parallel on 2 instances
8d2469b
@Sheharyar570
updated get_info_size in pgvector and pgdiskann
2e128c7
@Sheharyar570
set parallel index build params in init as well.
00b3571
@Sheharyar570
Diskann custom dataset run script added
f4a669b
@Sheharyar570
Updated config files to include diskann related params
fd27212
@Sheharyar570
set enable_seqscan off in all configs
8ee569f
@Sheharyar570
log level set to DEBUG
d3b0e7d
@Sheharyar570
fixed provider and instance_type info in one of the config files.
10c8cfa
@Sheharyar570
Merge branch 'main' of https://github.com/EmumbaOrg/VectorDBBench int…
fe7561b 
…o custom-dataset-increased-test-dataset-testing-diskann
@Sheharyar570
diskann cli custom_case support added
b738358
@Sheharyar570
increase optimize timeout for custom case to 2 days.
2e27bf0
@Sheharyar570
increase optimize timeout for custom case to 7 days.
1b06ecc
@Sheharyar570
Merge branch 'pvs-testing' of https://github.com/EmumbaOrg/VectorDBBench
8ed91c4 
 into custom-dataset-increased-test-dataset-testing-vectorscale
@Sheharyar570
Added custom dataset pgvectorscale benchmark run wrapper script
d52b0ee
@Sheharyar570
added get_stats and prewarm in the run script for diskann custom data…
6cd44de 
…set.
@Sheharyar570
Custom configs updated for pgvectorscale.
4b3e1ea
@Sheharyar570
Updated the configs.
c1c6112
@Sheharyar570
updated db-label in configs
f1fc197
@Sheharyar570
fix key error
512b4be
@Sheharyar570
fixed pgvectorscale command does not exist.
02be4b2
@Sheharyar570
removed parallel index build related queries from init
8dcbbf5
@Sheharyar570
serial search enabled for all run configs.
c4b54aa
@Sheharyar570
fixed issue causing serial and conc search to run even when set to fa…
a0263fb 
…lse.
@wahajali
Update test.parquet file
5d83c49
@Sheharyar570
update configs and scripts to run pgvectorscale diskann plain algorit…
47a13d0 
…hm on custom dataset
@Sheharyar570
maintenance work mem and parallel workers updated for 4xl instance
89a0be9
@Sheharyar570
Updated DB names
9177be1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Sheharyar570 @wahajali