Skip to content

Commit

Permalink
Fix CR comments
Browse files Browse the repository at this point in the history
  • Loading branch information
MariusArhaug committed Jun 19, 2024
1 parent db50c8e commit 1a7e424
Show file tree
Hide file tree
Showing 7 changed files with 147 additions and 341 deletions.
6 changes: 2 additions & 4 deletions _data/sidebar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,8 @@ docs:
url: /en/stateless-model-evaluation.html
- page: Ranking With BM25
url: /en/reference/bm25.html
- page: Ranking With Significance Model
url: /en/reference/significance.html
- page: Using Significance Model
url: /en/significance.html
- page: Ranking With nativeRank
url: /en/nativerank.html
- page: Accelerated OR search using the WAND algorithm
Expand Down Expand Up @@ -412,8 +412,6 @@ docs:
url: /en/reference/stateless-model-reference.html
- page: Embedding Model Reference
url: /en/reference/embedding-reference.html
- page: Significance Model Reference
url: /en/reference/significance-reference.html

- title: Queries and results reference
documents:
Expand Down
25 changes: 15 additions & 10 deletions en/operations-selfhosted/vespa-cmdline-tools.html
Original file line number Diff line number Diff line change
Expand Up @@ -1910,8 +1910,8 @@ <h2 id="vespa-set-node-state">vespa-set-node-state</h2>

<!--h2 id="vespa-slobrok-cmd">vespa-slobrok-cmd</h2-->

<h2 id="vepsa-signficance">vespa-significance</h2>
<p><code>vepsa-signficance</code> is a tool that generates a significance model file based on <a href="">this</a> file format. Its input is a <code>vespa-feed</code> file.
<h2 id="vespa-significance">vespa-significance</h2>
<p><code>vespa-signficance</code> is a tool that generates a significance model <a href="../reference/significance-reference.html#significance-file-format">file</a>. Its input is a <a href="../reference/document-json-format.html"><em>vespa-feed</em></a> file.
</p>
<p>Synopsis: <code>vespa-significance [options]</code></p>
<p>Example</p>
Expand All @@ -1928,14 +1928,12 @@ <h2 id="vepsa-signficance">vespa-significance</h2>
<tbody>
<tr>
<th>-h, --help</th>
<td>
Help text
</td>
<td>Help text</td>
</tr>
<tr>
<th>-i, --input &lt;input file&gt;</th>
<th>-i, --in &lt;input file&gt;</th>
<td>
Vespa dump file to be used for generating the significance model
<a href="../reference/document-json-format.html">Vespa-feed</a> file to be used for generating the significance model
</td>
</tr><tr>
<th>-o, --out &lt;output file&gt;</th>
Expand All @@ -1945,17 +1943,24 @@ <h2 id="vepsa-signficance">vespa-significance</h2>
</tr><tr>
<th> -f, --field &lt;field&gt;</th>
<td>
Name of the text field to be used for tokenization
Name of the text field to be used for significance model
</td>
</tr><tr>
<th> -l, --language &lt;language&gt;</th>
<td>
Language of the text field, must be a valid language code from the <a href="https://www.rfc-editor.org/rfc/rfc5646">RFC5646</a> standard.
<p>
Language of the text field, must be a valid language code from the <a href="https://www.rfc-editor.org/rfc/rfc5646">RFC5646</a> standard.
<br >
It is used with
OpenNLP's tokenizer to tokenize the text field based on that language's rules.
</p>
</td>
</tr><tr>
<th> -d, --doc-type &lt;doc-id&gt;</th>
<td>
Document type identifier for the dump file
<p>Document type identifier for the vespa dump file. <br>
It becomes a part of the id for <a href="../reference/document-json-format.html#put">put</a> operations in the vespa-feed file. <code>&#123; "put": "id::&lt;doc-id&gt;::1" &#125; </code>
</p>
</td>
</tr>
</tbody>
Expand Down
2 changes: 1 addition & 1 deletion en/reference/schema-reference.html
Original file line number Diff line number Diff line change
Expand Up @@ -2488,7 +2488,7 @@ <h2 id="onnx-model">onnx-model</h2>

<h2 id="significance">significance</h2>
<p>
Constrained in <a href="#rank-profile">rank-profile</a>. True or false. By default this is false. When enabled Vespa will use the significance calculation based on provided significance models in the service.xml for the rank-profile it is defined in.
Contained in <a href="#rank-profile">rank-profile</a>. True or false. By default this is false. When enabled Vespa will use the significance calculation based on provided significance models in the service.xml for the rank-profile it is defined in.
<pre>
significance {
use-model: true
Expand Down
28 changes: 26 additions & 2 deletions en/reference/services-search.html
Original file line number Diff line number Diff line change
Expand Up @@ -331,10 +331,10 @@ <h2 id="renderer">renderer</h2>

<h2 id="significance">significance</h2>
<p>
The significance tag can include multiple models. Their order determines the model precedence for a given language, with the last element having the highest. The models' document frequency is used to set a token's significance value based on the inverse document frequency (IDF). To enable the use of these models, the schema needs to have a rank-profile field with the <em>significance</em> element and the <em>use-model</em> flag set to <em>true</em>.
The significance element can include multiple models. Their order determines the model precedence for a given language, with the last element having the highest. The models' document frequency is used to set a token's significance. To enable the use of these models, the schema needs to have a rank-profile with the <em>significance</em> element and the <em>use-model</em> set to <em>true</em>.
</p>

<p>Example of significance model with multiple models. These models are either provided by <em>Vespa</em> or can be generated with the <a href="vespa-cmdline-tools.html#vespa-significance">vepsa-signficance</a> cli. </p>
<p>Example with multiple <a href="config-files.html#model">model</a> files. These models are either provided by <em>Vespa</em> or can be generated with the <a href="vespa-cmdline-tools.html#vespa-significance">vespa-signficance</a> cli. </p>
<pre data-test="file" data-path="my-app/src/main/application/services.xml">
&lt;significance&gt;
&lt;model model-id="wikimedia"/&gt;
Expand All @@ -345,6 +345,30 @@ <h2 id="significance">significance</h2>
</p>


<h3 id="significance-reference-config">significance reference config</h3>
<table class="table">
<thead>
<tr>
<th>Name</th>
<th>Occurrence</th>
<th>Description</th>
<th>Type</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td>model</td>
<td>One To Many</td>
<td>Use to point to the significance model file</td>
<td><a href="#model-config-reference">model-type</a></td>
<td>N/A</td>
</tr>

</tbody>
</table>


<h2 id="chain">chain</h2>
<p>
Specifies how a search chain should be instantiated, and how the contained searchers should be ordered.
Expand Down
107 changes: 0 additions & 107 deletions en/reference/significance-reference.html

This file was deleted.

Loading

0 comments on commit 1a7e424

Please sign in to comment.