Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/sa compression #21

Merged
merged 27 commits into from
Jun 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
aa8f2b7
bitarray compression
tibvdm May 16, 2024
84edc18
add coverage for bitarray and small README
tibvdm May 16, 2024
a618019
document library
tibvdm May 16, 2024
bbe0bc7
cargo formatting + better panic message in testing
tibvdm May 16, 2024
3a426fe
add is_empty method for clippy
tibvdm May 16, 2024
768d128
cargo formatting
tibvdm May 16, 2024
7dd5c4b
function to compress data without having to store everything in memory
tibvdm May 17, 2024
df85796
compressed integration + some major refactoring
tibvdm May 17, 2024
3d9c4f8
dynamic resuired bits per value
tibvdm May 17, 2024
3f66e2a
update tests and documentation
tibvdm May 17, 2024
58772f1
update coverage config
tibvdm May 17, 2024
9f3d6f4
fmt + clippy
tibvdm May 17, 2024
c57d44d
format --all
tibvdm May 17, 2024
f26cf34
improve testing
tibvdm May 18, 2024
679043f
formatting
tibvdm May 18, 2024
a5638bb
fix bitarray tests
tibvdm May 18, 2024
41d6c45
some additional smaller tests
tibvdm May 22, 2024
6bf49d0
merge SA tests
tibvdm May 22, 2024
7c2b6dc
update codecov config
tibvdm May 22, 2024
263d697
ignore main files when covering code
tibvdm May 22, 2024
1091ff2
test failing reader and writer in test module
tibvdm May 22, 2024
c15e720
small cleanup
tibvdm May 22, 2024
010f80b
small cleanup
tibvdm May 22, 2024
64a0aba
some debug information
tibvdm May 27, 2024
7096a0d
allow preliminary enzyme numbers (x.x.x.nx)
tibvdm May 28, 2024
285f246
encode functional annotations on load
tibvdm May 28, 2024
859a66b
more debug info + some movement of SA code
tibvdm May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 104 additions & 1 deletion .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,33 @@ jobs:
toolchain: nightly
override: true



- name: Run cargo test (bitarray)
uses: actions-rs/cargo@v1
with:
command: test
args: --all-features --no-fail-fast -p bitarray
env:
CARGO_INCREMENTAL: 0
RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'

- name: Gather coverage information (bitarray)
id: coverage-bitarray
uses: actions-rs/[email protected]

- name: Upload coverage reports to Codecov (bitarray)
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ${{ steps.coverage-bitarray.outputs.report }}
flags: bitarray
verbose: true
fail_ci_if_error: true



- name: Run cargo test (fa-compression)
uses: actions-rs/cargo@v1
with:
Expand All @@ -42,6 +69,82 @@ jobs:
verbose: true
fail_ci_if_error: true



- name: Run cargo test (sa-builder)
uses: actions-rs/cargo@v1
with:
command: test
args: --all-features --no-fail-fast -p sa-builder
env:
CARGO_INCREMENTAL: 0
RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'

- name: Gather coverage information (sa-builder)
id: coverage-sa-builder
uses: actions-rs/[email protected]

- name: Upload coverage reports to Codecov (sa-builder)
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ${{ steps.coverage-sa-builder.outputs.report }}
flags: sa-builder
verbose: true
fail_ci_if_error: true



- name: Run cargo test (sa-compression)
uses: actions-rs/cargo@v1
with:
command: test
args: --all-features --no-fail-fast -p sa-compression
env:
CARGO_INCREMENTAL: 0
RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'

- name: Gather coverage information (sa-compression)
id: coverage-sa-compression
uses: actions-rs/[email protected]

- name: Upload coverage reports to Codecov (sa-compression)
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ${{ steps.coverage-sa-compression.outputs.report }}
flags: sa-compression
verbose: true
fail_ci_if_error: true



- name: Run cargo test (sa-index)
uses: actions-rs/cargo@v1
with:
command: test
args: --all-features --no-fail-fast -p sa-index
env:
CARGO_INCREMENTAL: 0
RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'

- name: Gather coverage information (sa-index)
id: coverage-sa-index
uses: actions-rs/[email protected]

- name: Upload coverage reports to Codecov (sa-index)
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ${{ steps.coverage-sa-index.outputs.report }}
flags: sa-index
verbose: true
fail_ci_if_error: true


- name: Run cargo test (sa-mappings)
uses: actions-rs/cargo@v1
with:
Expand All @@ -63,4 +166,4 @@ jobs:
file: ${{ steps.coverage-sa-mappings.outputs.report }}
flags: sa-mappings
verbose: true
fail_ci_if_error: true
fail_ci_if_error: true
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
target/
data/

.DS_Store
18 changes: 17 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
[workspace]
resolver = "2"

members = [
members = [ "bitarray",
"fa-compression",
"libsais64-rs",
"sa-builder",
"sa-builder", "sa-compression",
"sa-index",
"sa-mappings",
"sa-server"
Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,20 @@

![Codecov](https://img.shields.io/codecov/c/github/unipept/unipept-index?token=IZ75A2FY98&logo=Codecov)

The unipept index written entirely in `Rust`. This repository consists of multiple different Rust projects that depend on
each other. More information about each project can be found in their respective `README.md` file.
The unipept index written entirely in `Rust`. This repository consists of multiple different Rust projects that depend on each other. More information about each project can be found in their respective `README.md` file.

## Installation

Clone this repository with the following command:
> [!NOTE]
> To build and use the Unipept Index, you need to have Rust installed. If you don't have Rust installed, you can get it from [rust-lang.org](https://www.rust-lang.org/).

Clone this repository by executing the following command:

```bash
git clone https://github.com/unipept/unipept-index.git
```

And build the projects using:
Next, build everything by executing the following command in the root of the repository.

```bash
cargo build --release
Expand Down
8 changes: 8 additions & 0 deletions bitarray/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[package]
name = "bitarray"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
21 changes: 21 additions & 0 deletions bitarray/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Bitarray

![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/unipept/unipept-index/test.yml?logo=github)
![Codecov](https://img.shields.io/codecov/c/github/unipept/unipept-index?token=IZ75A2FY98&flag=bitarray&logo=codecov)
![Static Badge](https://img.shields.io/badge/doc-rustdoc-blue)

The `bitarray` offers a special array where each item is represented by a specified amount of bits (smaller than 64). The bitarray uses a pre-alocated vector and allows you to `set` or `get` a value from the array.

## Example

```rust
use bitarray;

fn main() {
let bitarray = BitArray::<40>::with_capacity(4);

bitarray.set(0, 0b0001110011111010110001000111111100110010);

assert_eq!(bitarray.get(0), 0b0001110011111010110001000111111100110010);
}
```
Loading
Loading