Skip to content

Commit

Permalink
deploy: 106d337
Browse files Browse the repository at this point in the history
  • Loading branch information
zhjwpku committed Mar 30, 2024
1 parent ed376b4 commit cc6ea4e
Show file tree
Hide file tree
Showing 6 changed files with 16 additions and 6 deletions.
Binary file added assets/pdfs/FreshDiskANN_2021.pdf
Binary file not shown.
Binary file added assets/pdfs/OOD-DiskANN-2022.pdf
Binary file not shown.
5 changes: 5 additions & 0 deletions databases/vectordb/diskann.html
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,11 @@ <h4 id="vamana-graph-construction-algorithm"><a class="header" href="#vamana-gra
<p>从下图可以看出,第一行使用 α = 1 消除了很多不必要的边,第二行使用 α &gt; 1 将一些所谓的 long-range edges 加回到图中:</p>
<p><img src="/assets/images/vamana_graph_generation.png" alt="Progression of the graph generated by the Vamana" /></p>
<p>DiskANN 通过 BeamSearch(设置 beamwidth 一次读多个数据块) 和缓存最常访问的节点(eg. by caching all vertices that are C = 3 or 4 hops from the starting point s)来加速查询。 另外,DiskANN 将邻居节点的向量保存在磁盘索引文件中,来提高搜索的精度(Implicit Re-Ranking Using Full-Precision Vectors)。</p>
<h3 id="further-readings"><a class="header" href="#further-readings">Further readings</a></h3>
<ul>
<li><a href="/assets/pdfs/FreshDiskANN_2021.pdf">FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search</a>, 2021</li>
<li><a href="/assets/pdfs/OOD-DiskANN-2022.pdf">OOD-DiskANN: Efficient and Scalable Graph ANNS for Out-of-Distribution Queries</a>, 2022</li>
</ul>
<h4 id="references"><a class="header" href="#references">References:</a></h4>
<ul>
<li><a href="https://milvus.io/blog/2021-09-24-diskann.md">DiskANN, A Disk-based ANNS Solution with High Recall and High QPS on Billion-scale Dataset</a></li>
Expand Down
13 changes: 9 additions & 4 deletions print.html
Original file line number Diff line number Diff line change
Expand Up @@ -1248,6 +1248,11 @@ <h4 id="vamana-graph-construction-algorithm"><a class="header" href="#vamana-gra
<p>从下图可以看出,第一行使用 α = 1 消除了很多不必要的边,第二行使用 α &gt; 1 将一些所谓的 long-range edges 加回到图中:</p>
<p><img src="databases/vectordb//assets/images/vamana_graph_generation.png" alt="Progression of the graph generated by the Vamana" /></p>
<p>DiskANN 通过 BeamSearch(设置 beamwidth 一次读多个数据块) 和缓存最常访问的节点(eg. by caching all vertices that are C = 3 or 4 hops from the starting point s)来加速查询。 另外,DiskANN 将邻居节点的向量保存在磁盘索引文件中,来提高搜索的精度(Implicit Re-Ranking Using Full-Precision Vectors)。</p>
<h3 id="further-readings-2"><a class="header" href="#further-readings-2">Further readings</a></h3>
<ul>
<li><a href="databases/vectordb//assets/pdfs/FreshDiskANN_2021.pdf">FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search</a>, 2021</li>
<li><a href="databases/vectordb//assets/pdfs/OOD-DiskANN-2022.pdf">OOD-DiskANN: Efficient and Scalable Graph ANNS for Out-of-Distribution Queries</a>, 2022</li>
</ul>
<h4 id="references-4"><a class="header" href="#references-4">References:</a></h4>
<ul>
<li><a href="https://milvus.io/blog/2021-09-24-diskann.md">DiskANN, A Disk-based ANNS Solution with High Recall and High QPS on Billion-scale Dataset</a></li>
Expand Down Expand Up @@ -1612,7 +1617,7 @@ <h3 id="more-readings"><a class="header" href="#more-readings">More readings</a>
<li><strong><a href="datalayout/orc.html">Major Technical Advancements in Apache Hive</a></strong></li>
<li><strong><a href="datalayout/table-placement-methods.html">Table Placement Methods</a></strong></li>
</ul>
<h4 id="further-readings-2"><a class="header" href="#further-readings-2">Further readings</a></h4>
<h4 id="further-readings-3"><a class="header" href="#further-readings-3">Further readings</a></h4>
<p>[1] <a href="http://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html">Apache Arrow vs. Parquet and ORC: Do we really need a third Apache project for columnar data representation?</a> by Daniel Abadi, 2017<br></p>
<div style="break-before: page; page-break-before: always;"></div><h3 id="a-hrefdatalayoutassetspdfscstore-vldb05pdfc-store-a-column-oriented-dbmsa"><a class="header" href="#a-hrefdatalayoutassetspdfscstore-vldb05pdfc-store-a-column-oriented-dbmsa"><a href="datalayout/../assets/pdfs/cstore-vldb05.pdf">C-Store: A Column-oriented DBMS</a></a></h3>
<blockquote>
Expand Down Expand Up @@ -2493,7 +2498,7 @@ <h4 id="references-9"><a class="header" href="#references-9">References:</a></h4
<li><strong><a href="distributedsystem/consensus/raft.html">In Search of an Understandable Consensus Algorithm</a></strong></li>
<li><strong><a href="distributedsystem/consensus/paxos-vs-raft.html">Paxos vs Raft: have we reached consensus on distributed consensus?</a></strong></li>
</ul>
<h4 id="further-readings-3"><a class="header" href="#further-readings-3">Further readings</a></h4>
<h4 id="further-readings-4"><a class="header" href="#further-readings-4">Further readings</a></h4>
<p>[1] <a href="distributedsystem/consensus/../../assets/pdfs/disk-paxos.pdf">Disk Paxos</a> by Eli Gafni &amp; Leslie Lamport, 2002<br>
[2] <a href="distributedsystem/consensus/../../assets/pdfs/paxos-abcd.pdf">The ABCD’s of Paxos</a> by Butler W. Lampson, PODC 2001<br>
[3] Paxos for System Builders: <a href="distributedsystem/consensus/../../assets/pdfs/paxos-for-system-builders-an-overview.pdf">An Overview</a> and <a href="distributedsystem/consensus/../../assets/pdfs/paxos_for_system_builders.pdf">The Complete Specification</a> by Yair Amir and Jonathan Kirsch, 2008<br>
Expand Down Expand Up @@ -3175,7 +3180,7 @@ <h4 id="evaluating-global-predicates"><a class="header" href="#evaluating-global
<p>由于相对较难理解,还没深入看 :()</p>
<h4 id="detecting-behavioral-patterns"><a class="header" href="#detecting-behavioral-patterns">Detecting Behavioral Patterns</a></h4>
<p>由于相对较难理解,还没深入看 :()</p>
<h4 id="further-readings-4"><a class="header" href="#further-readings-4">Further readings</a></h4>
<h4 id="further-readings-5"><a class="header" href="#further-readings-5">Further readings</a></h4>
<p>[1] <a href="https://fileadmin.cs.lth.se/cs/Personal/Amr_Ergawy/dist-algos-papers/4.pdf">Timestamps in Message-Passing Systems That Preserve the Partial Ordering</a></p>
<div style="break-before: page; page-break-before: always;"></div><h3 id="a-hrefdistributedsystemassetspdfschandypdfdistributed-snapshots-determining-global-states-of-distributed-systemsa"><a class="header" href="#a-hrefdistributedsystemassetspdfschandypdfdistributed-snapshots-determining-global-states-of-distributed-systemsa"><a href="distributedsystem/../assets/pdfs/chandy.pdf">Distributed Snapshots: Determining Global States of Distributed Systems</a></a></h3>
<blockquote>
Expand Down Expand Up @@ -3341,7 +3346,7 @@ <h4 id="implementation"><a class="header" href="#implementation">Implementation<
<li><strong><a href="fs/gfs.html">The Google File System</a></strong></li>
<li><strong><a href="fs/polarfs.html">PolarFS: An Ultralow Latency and Failure Resilient Distributed File System for Shared Storage Cloud Database</a></strong></li>
</ul>
<h3 id="further-readings-5"><a class="header" href="#further-readings-5">Further readings</a></h3>
<h3 id="further-readings-6"><a class="header" href="#further-readings-6">Further readings</a></h3>
<ul>
<li><a href="https://www.usenix.org/system/files/conference/fast17/fast17-vangoor.pdf">To FUSE or Not to FUSE: Performance of User-Space File Systems</a>, FAST ’17</li>
</ul>
Expand Down
2 changes: 1 addition & 1 deletion searchindex.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion searchindex.json

Large diffs are not rendered by default.

0 comments on commit cc6ea4e

Please sign in to comment.