Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enforce no_std for all? protocol/v2 crates #1230

Merged
merged 9 commits into from
Nov 13, 2024
Merged

enforce no_std for all? protocol/v2 crates #1230

merged 9 commits into from
Nov 13, 2024
+70 −36

Conversation

Georges760
Copy link

@Georges760 Georges760 commented Oct 19, 2024 •
edited
Loading

Most of protocol/v2 crates are almost no_std, as listed here.

Having a lib crate that can be no_std or std by the selection of a feature :

#![cfg_attr(not(feature = "std"), no_std)]

or the current implementation :

#![cfg_attr(feature = "no_std", no_std)]

is only usefull if :

  • the crate could use some std related dependency (for instance std::vec::Vec)
  • but we want the crate's user to be able to not have a dependancy on std so we disable the functionalities using std

In some case we can replace the std dep by its alloc equivalent if exists (alloc::vec::Vec) and we can keep the functionality in no_std environement.

Finally, a std binary can use some no_std dependencies, but an no_std binary cannot use any std deps. So making the low level crates all no_std will make them copamtible for every kind of binary (app).

We should use the std feature only if some functionality requires std and not have any no_std equivalent.

As far as I tried all protocol/v2/ crates (libs) can be 100% no_std a,d does not need the std (or no_std) feature to optionally add std dep.

This PR is targeting to get rid of std dep without loosing any functionality or efficiency.

Note: I will make 1 commit per crate to explain what and how I am doing it.
Note2: being no_alloc is another level of complexity for deep embedded system, which is not at all this PR concern.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 19, 2024 •
edited
Loading

🐰 Bencher Report

Branch1230/merge
Testbedsv2

⚠️ WARNING: The following Measures do not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkEstimated Cyclesestimated cyclesInstructionsinstructionsL1 AccessesaccessesL2 AccessesaccessesRAM Accessesaccesses
client_sv2_handle_message_common📈 view plot
⚠️ NO THRESHOLD		2,073.00📈 view plot
⚠️ NO THRESHOLD		473.00📈 view plot
⚠️ NO THRESHOLD		733.00📈 view plot
⚠️ NO THRESHOLD		9.00📈 view plot
⚠️ NO THRESHOLD		37.00
client_sv2_handle_message_mining📈 view plot
⚠️ NO THRESHOLD		8,178.00📈 view plot
⚠️ NO THRESHOLD		2,137.00📈 view plot
⚠️ NO THRESHOLD		3,158.00📈 view plot
⚠️ NO THRESHOLD		38.00📈 view plot
⚠️ NO THRESHOLD		138.00
client_sv2_mining_message_submit_standard📈 view plot
⚠️ NO THRESHOLD		6,263.00📈 view plot
⚠️ NO THRESHOLD		1,750.00📈 view plot
⚠️ NO THRESHOLD		2,548.00📈 view plot
⚠️ NO THRESHOLD		22.00📈 view plot
⚠️ NO THRESHOLD		103.00
client_sv2_mining_message_submit_standard_serialize📈 view plot
⚠️ NO THRESHOLD		14,694.00📈 view plot
⚠️ NO THRESHOLD		4,694.00📈 view plot
⚠️ NO THRESHOLD		6,754.00📈 view plot
⚠️ NO THRESHOLD		48.00📈 view plot
⚠️ NO THRESHOLD		220.00
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD		27,746.00📈 view plot
⚠️ NO THRESHOLD		10,645.00📈 view plot
⚠️ NO THRESHOLD		15,506.00📈 view plot
⚠️ NO THRESHOLD		89.00📈 view plot
⚠️ NO THRESHOLD		337.00
client_sv2_open_channel📈 view plot
⚠️ NO THRESHOLD		4,413.00📈 view plot
⚠️ NO THRESHOLD		1,461.00📈 view plot
⚠️ NO THRESHOLD		2,158.00📈 view plot
⚠️ NO THRESHOLD		10.00📈 view plot
⚠️ NO THRESHOLD		63.00
client_sv2_open_channel_serialize📈 view plot
⚠️ NO THRESHOLD		14,024.00📈 view plot
⚠️ NO THRESHOLD		5,064.00📈 view plot
⚠️ NO THRESHOLD		7,324.00📈 view plot
⚠️ NO THRESHOLD		38.00📈 view plot
⚠️ NO THRESHOLD		186.00
client_sv2_open_channel_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD		22,832.00📈 view plot
⚠️ NO THRESHOLD		8,040.00📈 view plot
⚠️ NO THRESHOLD		11,687.00📈 view plot
⚠️ NO THRESHOLD		87.00📈 view plot
⚠️ NO THRESHOLD		306.00
client_sv2_setup_connection📈 view plot
⚠️ NO THRESHOLD		4,659.00📈 view plot
⚠️ NO THRESHOLD		1,502.00📈 view plot
⚠️ NO THRESHOLD		2,274.00📈 view plot
⚠️ NO THRESHOLD		15.00📈 view plot
⚠️ NO THRESHOLD		66.00
client_sv2_setup_connection_serialize📈 view plot
⚠️ NO THRESHOLD		16,134.00📈 view plot
⚠️ NO THRESHOLD		5,963.00📈 view plot
⚠️ NO THRESHOLD		8,659.00📈 view plot
⚠️ NO THRESHOLD		46.00📈 view plot
⚠️ NO THRESHOLD		207.00
client_sv2_setup_connection_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD		35,670.00📈 view plot
⚠️ NO THRESHOLD		14,888.00📈 view plot
⚠️ NO THRESHOLD		21,880.00📈 view plot
⚠️ NO THRESHOLD		98.00📈 view plot
⚠️ NO THRESHOLD		380.00
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 19, 2024 •
edited
Loading

🐰 Bencher Report

Branch1230/merge
Testbedsv2

⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
client_sv2_handle_message_common📈 view plot
⚠️ NO THRESHOLD		44.06
client_sv2_handle_message_mining📈 view plot
⚠️ NO THRESHOLD		75.36
client_sv2_mining_message_submit_standard📈 view plot
⚠️ NO THRESHOLD		14.67
client_sv2_mining_message_submit_standard_serialize📈 view plot
⚠️ NO THRESHOLD		256.91
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD		612.34
client_sv2_open_channel📈 view plot
⚠️ NO THRESHOLD		148.15
client_sv2_open_channel_serialize📈 view plot
⚠️ NO THRESHOLD		277.25
client_sv2_open_channel_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD		380.26
client_sv2_setup_connection📈 view plot
⚠️ NO THRESHOLD		161.96
client_sv2_setup_connection_serialize📈 view plot
⚠️ NO THRESHOLD		455.68
client_sv2_setup_connection_serialize_deserialize📈 view plot
⚠️ NO THRESHOLD		957.41
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 19, 2024 •
edited
Loading

🐰 Bencher Report

Branch1230/merge
Testbedsv1

⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
client-submit-serialize📈 view plot
⚠️ NO THRESHOLD		6,748.30
client-submit-serialize-deserialize📈 view plot
⚠️ NO THRESHOLD		7,707.10
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
⚠️ NO THRESHOLD		8,377.50
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
⚠️ NO THRESHOLD		943.45
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
⚠️ NO THRESHOLD		675.37
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
⚠️ NO THRESHOLD		262.34
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
⚠️ NO THRESHOLD		156.57
client-sv1-get-submit📈 view plot
⚠️ NO THRESHOLD		6,451.40
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
⚠️ NO THRESHOLD		289.67
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
⚠️ NO THRESHOLD		812.52
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
⚠️ NO THRESHOLD		607.60
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
⚠️ NO THRESHOLD		222.88
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 19, 2024 •
edited
Loading

🐰 Bencher Report

Branch1230/merge
Testbedsv1

⚠️ WARNING: The following Measures do not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkEstimated Cyclesestimated cyclesInstructionsinstructionsL1 AccessesaccessesL2 AccessesaccessesRAM Accessesaccesses
get_authorize📈 view plot
⚠️ NO THRESHOLD		8,304.00📈 view plot
⚠️ NO THRESHOLD		3,663.00📈 view plot
⚠️ NO THRESHOLD		5,114.00📈 view plot
⚠️ NO THRESHOLD		8.00📈 view plot
⚠️ NO THRESHOLD		90.00
get_submit📈 view plot
⚠️ NO THRESHOLD		95,226.00📈 view plot
⚠️ NO THRESHOLD		59,263.00📈 view plot
⚠️ NO THRESHOLD		85,076.00📈 view plot
⚠️ NO THRESHOLD		49.00📈 view plot
⚠️ NO THRESHOLD		283.00
get_subscribe📈 view plot
⚠️ NO THRESHOLD		7,871.00📈 view plot
⚠️ NO THRESHOLD		2,758.00📈 view plot
⚠️ NO THRESHOLD		3,831.00📈 view plot
⚠️ NO THRESHOLD		17.00📈 view plot
⚠️ NO THRESHOLD		113.00
serialize_authorize📈 view plot
⚠️ NO THRESHOLD		12,041.00📈 view plot
⚠️ NO THRESHOLD		5,240.00📈 view plot
⚠️ NO THRESHOLD		7,281.00📈 view plot
⚠️ NO THRESHOLD		14.00📈 view plot
⚠️ NO THRESHOLD		134.00
serialize_deserialize_authorize📈 view plot
⚠️ NO THRESHOLD		24,452.00📈 view plot
⚠️ NO THRESHOLD		9,786.00📈 view plot
⚠️ NO THRESHOLD		13,787.00📈 view plot
⚠️ NO THRESHOLD		40.00📈 view plot
⚠️ NO THRESHOLD		299.00
serialize_deserialize_handle_authorize📈 view plot
⚠️ NO THRESHOLD		30,171.00📈 view plot
⚠️ NO THRESHOLD		11,989.00📈 view plot
⚠️ NO THRESHOLD		16,951.00📈 view plot
⚠️ NO THRESHOLD		61.00📈 view plot
⚠️ NO THRESHOLD		369.00
serialize_deserialize_handle_submit📈 view plot
⚠️ NO THRESHOLD		126,220.00📈 view plot
⚠️ NO THRESHOLD		73,117.00📈 view plot
⚠️ NO THRESHOLD		104,760.00📈 view plot
⚠️ NO THRESHOLD		120.00📈 view plot
⚠️ NO THRESHOLD		596.00
serialize_deserialize_handle_subscribe📈 view plot
⚠️ NO THRESHOLD		27,761.00📈 view plot
⚠️ NO THRESHOLD		9,577.00📈 view plot
⚠️ NO THRESHOLD		13,521.00📈 view plot
⚠️ NO THRESHOLD		69.00📈 view plot
⚠️ NO THRESHOLD		397.00
serialize_deserialize_submit📈 view plot
⚠️ NO THRESHOLD		114,825.00📈 view plot
⚠️ NO THRESHOLD		67,894.00📈 view plot
⚠️ NO THRESHOLD		97,370.00📈 view plot
⚠️ NO THRESHOLD		68.00📈 view plot
⚠️ NO THRESHOLD		489.00
serialize_deserialize_subscribe📈 view plot
⚠️ NO THRESHOLD		23,052.00📈 view plot
⚠️ NO THRESHOLD		8,129.00📈 view plot
⚠️ NO THRESHOLD		11,427.00📈 view plot
⚠️ NO THRESHOLD		43.00📈 view plot
⚠️ NO THRESHOLD		326.00
serialize_submit📈 view plot
⚠️ NO THRESHOLD		99,505.00📈 view plot
⚠️ NO THRESHOLD		61,325.00📈 view plot
⚠️ NO THRESHOLD		87,940.00📈 view plot
⚠️ NO THRESHOLD		52.00📈 view plot
⚠️ NO THRESHOLD		323.00
serialize_subscribe📈 view plot
⚠️ NO THRESHOLD		11,176.00📈 view plot
⚠️ NO THRESHOLD		4,111.00📈 view plot
⚠️ NO THRESHOLD		5,696.00📈 view plot
⚠️ NO THRESHOLD		18.00📈 view plot
⚠️ NO THRESHOLD		154.00
🐰 View full continuous benchmarking report in Bencher

@codecov Codecov
Copy link

codecov bot commented Oct 19, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 25.13%. Comparing base (dee3cae) to head (8cd634b).
Report is 10 commits behind head on main.

Files with missing lines Patch % Lines
utils/buffer/src/buffer_pool/mod.rs 50.00% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1230      +/-   ##
==========================================
+ Coverage   19.30%   25.13%   +5.83%     
==========================================
  Files         164       20     -144     
  Lines       10849     1134    -9715     
==========================================
- Hits         2094      285    -1809     
+ Misses       8755      849    -7906
Flag Coverage Δ
binary_codec_sv2-coverage ?
binary_serde_sv2-coverage ?
binary_sv2-coverage ?
bip32_derivation-coverage ?
buffer_sv2-coverage ?
codec_sv2-coverage ?
common_messages_sv2-coverage ?
const_sv2-coverage ?
error_handling-coverage ?
framing_sv2-coverage ?
jd_client-coverage ?
jd_server-coverage ?
job_declaration_sv2-coverage ?
key-utils-coverage ?
mining-coverage ?
mining_device-coverage ?
mining_proxy_sv2-coverage ?
noise_sv2-coverage ?
pool_sv2-coverage ?
protocols ?
roles ?
roles_logic_sv2-coverage ?
sv1-mining-device-coverage ?
sv2_ffi-coverage ?
template_distribution_sv2-coverage ?
translator_sv2-coverage ?
utils 25.13% <50.00%> (ø)
v1-coverage ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Georges760 Georges760 force-pushed the std_logic branch 3 times, most recently from 03a2dae to fc640a7 Compare October 20, 2024 07:44
@plebhash plebhash mentioned this pull request Oct 25, 2024
Open
plebhash
plebhash reviewed Oct 25, 2024
View reviewed changes
protocols/DEPENDANCIES.md Outdated Show resolved Hide resolved
@Georges760 Georges760 mentioned this pull request Oct 26, 2024
Merged
@Georges760 Georges760 changed the title std logic enforce no_std for all protocol/v2 crates Oct 28, 2024
jbesraa
jbesraa reviewed Oct 28, 2024
View reviewed changes
Copy link
Contributor

@jbesraa jbesraa left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I think the general idea is very welcome, I think addressing #1200 first would make more sense as it can potentially change how we approach this. I think in #1200 we should aim to provide a logger for std and no std env and then impl Debug manually, which would allow us to remove the debug feature IIUC whats going here

@Georges760
Copy link
Author

Current binary_codec_sv2 :

  • don't compile without no_std feature (this doesn't mean it is no_std as there is no enforcement => #![cfg_attr(feature = "no_std", no_std)])
  • has no_std feature by default (default = ["no_std"])
  • nowhere (not even in the CI), its default features are disable (--no-default-features)

So it's pseudo std support os broken and unused. We can :

  • remove/clean it, and keep a real no_std crate only
  • debug it, and have a no_std crate by default, with a std feature with the debugged std functionalities

Maybe this can be discussed in #1132 ?

@Georges760
Copy link
Author

While I think the general idea is very welcome, I think addressing #1200 first would make more sense as it can potentially change how we approach this. I think in #1200 we should aim to provide a logger for std and no std env and then impl Debug manually, which would allow us to remove the debug feature IIUC whats going here

tracing has a std feature enabled by default, this means it actually add std dep to the crate that use it without disabling default features : sv1_api, binary_sv2, codec_sv2 and roles_logic_sv2 (the 4 crates listed in #1200). 2 of this 4 crates are targeted by this PR (binary_sv2 and codec_sv2) and in order for them to become no_std, I will have to no-default-features tracing, and enable std in tracing using a local feature too. This can be done before or after #1200 without problem.

If I do it before #1200, when #1200 will remove tracing with the custom logger, the std feature will be available (and can be removed if not used after all).
If #1200 should be done before, then we can limit this PR to crates not depending on binary_sv2 and codec_sv2, and wait to see what the new logger will propose for no_std support.

@Georges760 Georges760 changed the title enforce no_std for all protocol/v2 crates enforce no_std for all? protocol/v2 crates Oct 28, 2024
@jbesraa
Copy link
Contributor

jbesraa commented Oct 28, 2024

While I think the general idea is very welcome, I think addressing #1200 first would make more sense as it can potentially change how we approach this. I think in #1200 we should aim to provide a logger for std and no std env and then impl Debug manually, which would allow us to remove the debug feature IIUC whats going here

tracing has a std feature enabled by default, this means it actually add std dep to the crate that use it without disabling default features : sv1_api, binary_sv2, codec_sv2 and roles_logic_sv2 (the 4 crates listed in #1200). 2 of this 4 crates are targeted by this PR (binary_sv2 and codec_sv2) and in order for them to become no_std, I will have to no-default-features tracing, and enable std in tracing using a local feature too. This can be done before or after #1200 without problem.

If I do it before #1200, when #1200 will remove tracing with the custom logger, the std feature will be available (and can be removed if not used after all). If #1200 should be done before, then we can limit this PR to crates not depending on binary_sv2 and codec_sv2, and wait to see what the new logger will propose for no_std support.

Yea I think #1200 shouldn't necessarily be about removing tracing crate, but generalizing how we use the logger
and mainly enabling a logger in a non-std env. Making everything non-std by default would make it a bit tricky to debug the crates generally.
With that said, If #1200 will require too much time and would block this for too long, I am happy for this to get in anyway.

@Georges760 Georges760 force-pushed the std_logic branch 3 times, most recently from dfb23fe to 8ef9d5e Compare October 28, 2024 14:10
@Georges760
Copy link
Author

Actual binary_sv2 without default feature (core) and with with_serde feature, test is broken.

This PR is not fixing this.

CI does not see it because does not test it.

@Georges760
Copy link
Author

Actual framing_sv2 with with_serde feature, test is broken.

This PR is not fixing this.

CI does not see it because does not test it.

@plebhash
Copy link
Collaborator

@Georges760 please note that we're aiming to deprecate with_serde feature soon.

I cannot give you a precise timeline but Q1 2025 would be my rough guess.

I wonder if that would make this process simpler?

@Georges760
Copy link
Author

@Georges760 please note that we're aiming to deprecate with_serde feature soon.

I cannot give you a precise timeline but Q1 2025 would be my rough guess.

I wonder if that would make this process simpler?

It make sense, as it is already broken in binary_sv2 and framing_sv2 as stated above.

So I understand the official/only codec way will be the "core" option ? Good

@Georges760
Copy link
Author

noise_sv2 has its own PR #1238.
codec_sv2 is dependant on both noise_sv2 and all the crates of this PR.

So I stop here for this PR and open for reviews.
I think it is in good shape to fullfil the OP goal.

@plebhash
Copy link
Collaborator

plebhash commented Nov 5, 2024

completely removing std from protocols crates will imply making adaptations to the fuzz tests

@rrybarczyk rrybarczyk added the protocols Lowest level protocol logic label Nov 11, 2024
@plebhash
Copy link
Collaborator

plebhash commented Nov 11, 2024 •
edited
Loading

As far as I tried all protocol/v2/ crates (libs) can be 100% no_std a,d does not need the std (or no_std) feature to optionally add std dep.

if I understand correctly, we should expect this PR to leave all protocols crates (with exception of noise_sv2 + codec_sv2, which will be covered in other PRs) without ANY std/no_std feature flag, since they will be no_std by default.

but I'm confused:

why are we still leaving some crates with the no_std feature flag? I inspected this branch and found the no_std feature flag on the following crates:

  • v2/codec-sv2/Cargo.toml (expected, since this PR doesn't aim to touch this crate IIUC)
  • v2/subprotocols/job-declaration/Cargo.toml
  • v2/subprotocols/mining/Cargo.toml
  • v2/binary-sv2/no-serde-sv2/codec/Cargo.toml
  • v2/subprotocols/template-distribution/Cargo.toml
  • v2/subprotocols/common-messages/Cargo.toml

shouldn't no_std feature flag be completely removed from these crates?
shouldn't they KISS and only have a #![no_std] on the top of the module, instead of #![cfg_attr(feature = "no_std", no_std)]? e.g.: 422aee3

@plebhash
Copy link
Collaborator

plebhash commented Nov 12, 2024 •
edited
Loading

following this criteria, we should bump the PATCH version of the crates that are being changed on this PR

@Georges760
Copy link
Author

As far as I tried all protocol/v2/ crates (libs) can be 100% no_std a,d does not need the std (or no_std) feature to optionally add std dep.

if I understand correctly, we should expect this PR to leave all protocols crates (with exception of noise_sv2 + codec_sv2, which will be covered in other PRs) without ANY std/no_std feature flag, since they will be no_std by default.

That is the end goal yes

but I'm confused:

why are we still leaving some crates with the no_std feature flag? I inspected this branch and found the no_std feature flag on the following crates:

  • v2/codec-sv2/Cargo.toml (expected, since this PR doesn't aim to touch this crate IIUC)
  • v2/subprotocols/job-declaration/Cargo.toml
  • v2/subprotocols/mining/Cargo.toml

I didn't consider subprotocols/* crates either in this PR

  • v2/binary-sv2/no-serde-sv2/codec/Cargo.toml

details here above
TL;DR: kept no_std feature for an existing yet broken implementation (don't want to remove any piece of code, even if doesn't compile, but yes I think we should also clean what is under the #[cfg(not(feature = "no_std"))] in this crate, maybe part of #1132 )

  • v2/subprotocols/template-distribution/Cargo.toml
  • v2/subprotocols/common-messages/Cargo.toml

shouldn't no_std feature flag be completely removed from these crates? shouldn't they KISS and only have a #![no_std] on the top of the module, instead of #![cfg_attr(feature = "no_std", no_std)]? e.g.: 422aee3

YES totally, but this PR being multi crate already, try to limit its action to not refactor any code, just remove std dependencies where possible without functional impact.

@Georges760
Copy link
Author

following this criteria, we should bump the PATCH version of the crates that are being changed on this PR

exact, but usually the version bump is done in a separate PR/commit to merge multiple improvement in a single version increment. Should I do it in this PR anyway ?

Shourya742
Shourya742 approved these changes Nov 12, 2024
View reviewed changes
Copy link
Contributor

@Shourya742 Shourya742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@plebhash
Copy link
Collaborator

following this criteria, we should bump the PATCH version of the crates that are being changed on this PR

exact, but usually the version bump is done in a separate PR/commit to merge multiple improvement in a single version increment. Should I do it in this PR anyway ?

you're right, we can leave it for a new PR

@rrybarczyk rrybarczyk self-requested a review November 12, 2024 17:30
@plebhash
Copy link
Collaborator

@Georges760 could you please add a comment to the crates README.md mentioning that they are no_std by default?

some crates are missing a README.md (WIP under #845)

for the crates that are missing a README.md, feel free to create the file and only add this sentence

later, we're going to finish the README.md file with the rest of the info

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2024 •
edited
Loading

🐰 Bencher Report

Branchstd_logic
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,709.10
(-0.84%)		7,968.68
(84.19%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,678.60
(-0.08%)		9,000.13
(85.32%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,234.00
(-0.17%)		9,512.06
(86.56%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		877.55
(-0.63%)		1,017.70
(86.23%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		676.55
(-0.58%)		829.29
(81.58%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		245.74
(-0.72%)		265.23
(92.65%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		157.73
(+0.13%)		164.83
(95.69%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,554.80
(+0.12%)		7,887.68
(83.10%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		283.56
(+1.36%)		316.58
(89.57%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		736.17
(-1.40%)		888.03
(82.90%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		607.58
(+0.09%)		710.49
(85.52%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		207.85
(+0.57%)		226.99
(91.57%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2024 •
edited
Loading

🐰 Bencher Report

Branchstd_logic
Testbedsv2
🚨 6 ALERTS: Threshold Boundary Limits exceeded!
BenchmarkMeasure
Units		ViewBenchmark Result
(Result Δ%)		Lower Boundary
(Limit %)		Upper Boundary
(Limit %)
client_sv2_mining_message_submit_standard_serializeL2 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		54.00
(+23.92%)		53.90
(100.19%)
client_sv2_mining_message_submit_standard_serialize_deserializeEstimated Cycles
estimated cycles		📈 plot
🚨 alert
🚷 threshold		27,727.00
(+0.73%)		27,715.27
(100.04%)
client_sv2_mining_message_submit_standard_serialize_deserializeInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		10,648.00
(+0.66%)		10,637.94
(100.09%)
client_sv2_mining_message_submit_standard_serialize_deserializeL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		15,507.00
(+0.75%)		15,484.20
(100.15%)
client_sv2_mining_message_submit_standard_serialize_deserializeL2 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		92.00
(+13.15%)		89.92
(102.31%)
client_sv2_open_channel_serialize_deserializeL2 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		89.00
(+19.83%)		85.29
(104.35%)
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2,068.00
(-1.13%)		2,202.99
(93.87%)		📈 view plot
🚷 view threshold		476.00
(+0.59%)		488.78
(97.39%)		📈 view plot
🚷 view threshold		738.00
(+0.31%)		757.87
(97.38%)		📈 view plot
🚷 view threshold		7.00
(+26.91%)		14.29
(48.98%)		📈 view plot
🚷 view threshold		37.00
(-2.51%)		41.08
(90.07%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8,212.00
(+0.07%)		8,290.52
(99.05%)		📈 view plot
🚷 view threshold		2,137.00
(-0.03%)		2,143.08
(99.72%)		📈 view plot
🚷 view threshold		3,157.00
(-0.12%)		3,170.57
(99.57%)		📈 view plot
🚷 view threshold		38.00
(+6.64%)		42.35
(89.73%)		📈 view plot
🚷 view threshold		139.00
(-0.05%)		140.97
(98.60%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6,262.00
(-0.25%)		6,386.75
(98.05%)		📈 view plot
🚷 view threshold		1,753.00
(+0.13%)		1,767.16
(99.20%)		📈 view plot
🚷 view threshold		2,552.00
(-0.07%)		2,576.20
(99.06%)		📈 view plot
🚷 view threshold		21.00
(+23.42%)		24.81
(84.66%)		📈 view plot
🚷 view threshold		103.00
(-0.94%)		106.97
(96.29%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14,721.00
(-0.08%)		14,971.96
(98.32%)		📈 view plot
🚷 view threshold		4,697.00
(+0.05%)		4,711.16
(99.70%)		📈 view plot
🚷 view threshold		6,751.00
(-0.11%)		6,782.84
(99.53%)		📈 view plot
🚨 view alert
🚷 view threshold		54.00
(+23.92%)		53.90
(100.19%)		📈 view plot
🚷 view threshold		220.00
(-0.72%)		227.92
(96.53%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚨 view alert
🚷 view threshold		27,727.00
(+0.73%)		27,715.27
(100.04%)		📈 view plot
🚨 view alert
🚷 view threshold		10,648.00
(+0.66%)		10,637.94
(100.09%)		📈 view plot
🚨 view alert
🚷 view threshold		15,507.00
(+0.75%)		15,484.20
(100.15%)		📈 view plot
🚨 view alert
🚷 view threshold		92.00
(+13.15%)		89.92
(102.31%)		📈 view plot
🚷 view threshold		336.00
(+0.28%)		341.37
(98.43%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4,382.00
(-0.38%)		4,620.27
(94.84%)		📈 view plot
🚷 view threshold		1,464.00
(+0.19%)		1,476.78
(99.13%)		📈 view plot
🚷 view threshold		2,162.00
(+0.12%)		2,185.51
(98.92%)		📈 view plot
🚷 view threshold		10.00
(+10.73%)		17.86
(55.99%)		📈 view plot
🚷 view threshold		62.00
(-1.10%)		68.61
(90.36%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		13,975.00
(-0.64%)		14,468.13
(96.59%)		📈 view plot
🚷 view threshold		5,067.00
(+0.05%)		5,079.78
(99.75%)		📈 view plot
🚷 view threshold		7,325.00
(-0.01%)		7,353.26
(99.62%)		📈 view plot
🚷 view threshold		42.00
(+19.44%)		43.51
(96.53%)		📈 view plot
🚷 view threshold		184.00
(-1.88%)		199.17
(92.38%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22,809.00
(+0.65%)		22,896.79
(99.62%)		📈 view plot
🚷 view threshold		8,043.00
(+0.30%)		8,075.04
(99.60%)		📈 view plot
🚷 view threshold		11,689.00
(+0.21%)		11,755.91
(99.43%)		📈 view plot
🚨 view alert
🚷 view threshold		89.00
(+19.83%)		85.29
(104.35%)		📈 view plot
🚷 view threshold		305.00
(+0.46%)		312.01
(97.75%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4,654.00
(-0.64%)		4,772.18
(97.52%)		📈 view plot
🚷 view threshold		1,505.00
(+0.19%)		1,517.78
(99.16%)		📈 view plot
🚷 view threshold		2,279.00
(+0.02%)		2,299.53
(99.11%)		📈 view plot
🚷 view threshold		13.00
(+40.42%)		15.60
(83.35%)		📈 view plot
🚷 view threshold		66.00
(-2.07%)		69.87
(94.46%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16,111.00
(-0.40%)		16,476.39
(97.78%)		📈 view plot
🚷 view threshold		5,966.00
(+0.05%)		5,978.78
(99.79%)		📈 view plot
🚷 view threshold		8,661.00
(-0.03%)		8,692.85
(99.63%)		📈 view plot
🚷 view threshold		48.00
(+19.77%)		53.89
(89.07%)		📈 view plot
🚷 view threshold		206.00
(-1.39%)		216.79
(95.02%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35,651.00
(+0.29%)		35,735.48
(99.76%)		📈 view plot
🚷 view threshold		14,891.00
(+0.30%)		14,905.14
(99.91%)		📈 view plot
🚷 view threshold		21,881.00
(+0.33%)		21,915.42
(99.84%)		📈 view plot
🚷 view threshold		101.00
(+8.56%)		114.80
(87.98%)		📈 view plot
🚷 view threshold		379.00
(-0.07%)		384.68
(98.52%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2024 •
edited
Loading

🐰 Bencher Report

Branchstd_logic
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.07
(-1.75%)		47.65
(92.49%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		74.77
(-0.72%)		87.54
(85.41%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.65
(-0.04%)		14.76
(99.22%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		311.45
(+13.27%)		386.52
(80.58%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		610.78
(+3.49%)		698.56
(87.43%)
client_sv2_open_channel📈 view plot
🚷 view threshold		146.22
(-0.38%)		151.34
(96.61%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		284.23
(-0.80%)		317.21
(89.60%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		379.33
(+0.63%)		422.18
(89.85%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		159.36
(-1.41%)		184.86
(86.21%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		498.51
(+5.14%)		616.61
(80.85%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		1,007.30
(+3.67%)		1,130.47
(89.10%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2024 •
edited
Loading

🐰 Bencher Report

Branchstd_logic
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8,391.00
(-0.77%)		8,829.55
(95.03%)		📈 view plot
🚷 view threshold		3,689.00
(-1.57%)		3,890.95
(94.81%)		📈 view plot
🚷 view threshold		5,156.00
(-1.91%)		5,458.66
(94.46%)		📈 view plot
🚷 view threshold		10.00
(+40.35%)		12.50
(79.97%)		📈 view plot
🚷 view threshold		91.00
(+0.65%)		96.19
(94.60%)
get_submit📈 view plot
🚷 view threshold		95,453.00
(+0.01%)		96,154.14
(99.27%)		📈 view plot
🚷 view threshold		59,346.00
(-0.21%)		59,815.75
(99.21%)		📈 view plot
🚷 view threshold		85,218.00
(-0.24%)		85,914.28
(99.19%)		📈 view plot
🚷 view threshold		45.00
(-0.98%)		65.54
(68.66%)		📈 view plot
🚷 view threshold		286.00
(+2.18%)		289.55
(98.77%)
get_subscribe📈 view plot
🚷 view threshold		7,872.00
(-1.78%)		8,385.35
(93.88%)		📈 view plot
🚷 view threshold		2,765.00
(-2.44%)		2,970.86
(93.07%)		📈 view plot
🚷 view threshold		3,847.00
(-2.90%)		4,151.92
(92.66%)		📈 view plot
🚷 view threshold		14.00
(+16.59%)		20.75
(67.46%)		📈 view plot
🚷 view threshold		113.00
(-0.95%)		120.15
(94.05%)
serialize_authorize📈 view plot
🚷 view threshold		12,214.00
(-0.49%)		12,650.29
(96.55%)		📈 view plot
🚷 view threshold		5,266.00
(-1.00%)		5,461.05
(96.43%)		📈 view plot
🚷 view threshold		7,324.00
(-1.28%)		7,620.55
(96.11%)		📈 view plot
🚷 view threshold		12.00
(+33.22%)		15.58
(77.03%)		📈 view plot
🚷 view threshold		138.00
(+0.41%)		143.62
(96.09%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24,344.00
(-1.36%)		25,290.85
(96.26%)		📈 view plot
🚷 view threshold		9,838.00
(-0.54%)		10,054.66
(97.85%)		📈 view plot
🚷 view threshold		13,889.00
(-0.54%)		14,208.50
(97.75%)		📈 view plot
🚷 view threshold		33.00
(-5.01%)		44.09
(74.85%)		📈 view plot
🚷 view threshold		294.00
(-2.39%)		316.04
(93.03%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		30,088.00
(-0.75%)		30,801.44
(97.68%)		📈 view plot
🚷 view threshold		12,015.00
(-0.56%)		12,225.38
(98.28%)		📈 view plot
🚷 view threshold		16,998.00
(-0.65%)		17,317.53
(98.15%)		📈 view plot
🚷 view threshold		63.00
(+13.02%)		68.38
(92.13%)		📈 view plot
🚷 view threshold		365.00
(-1.17%)		382.16
(95.51%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126,471.00
(-0.00%)		127,200.85
(99.43%)		📈 view plot
🚷 view threshold		73,200.00
(-0.12%)		73,699.14
(99.32%)		📈 view plot
🚷 view threshold		104,911.00
(-0.15%)		105,668.01
(99.28%)		📈 view plot
🚷 view threshold		105.00
(-2.25%)		134.54
(78.04%)		📈 view plot
🚷 view threshold		601.00
(+0.81%)		605.60
(99.24%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27,476.00
(-1.24%)		28,656.12
(95.88%)		📈 view plot
🚷 view threshold		9,584.00
(-0.66%)		9,790.17
(97.89%)		📈 view plot
🚷 view threshold		13,541.00
(-0.77%)		13,850.12
(97.77%)		📈 view plot
🚷 view threshold		71.00
(+11.76%)		76.20
(93.18%)		📈 view plot
🚷 view threshold		388.00
(-2.00%)		417.43
(92.95%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115,163.00
(-0.07%)		115,990.40
(99.29%)		📈 view plot
🚷 view threshold		68,060.00
(-0.07%)		68,513.05
(99.34%)		📈 view plot
🚷 view threshold		97,653.00
(-0.09%)		98,354.05
(99.29%)		📈 view plot
🚷 view threshold		65.00
(+3.23%)		78.98
(82.30%)		📈 view plot
🚷 view threshold		491.00
(+0.00%)		499.67
(98.27%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		22,850.00
(-1.68%)		24,084.36
(94.87%)		📈 view plot
🚷 view threshold		8,143.00
(-0.73%)		8,346.75
(97.56%)		📈 view plot
🚷 view threshold		11,465.00
(-0.78%)		11,757.48
(97.51%)		📈 view plot
🚷 view threshold		37.00
(-0.94%)		44.62
(82.92%)		📈 view plot
🚷 view threshold		320.00
(-2.60%)		349.37
(91.59%)
serialize_submit📈 view plot
🚷 view threshold		99,854.00
(-0.01%)		100,524.67
(99.33%)		📈 view plot
🚷 view threshold		61,408.00
(-0.18%)		61,861.67
(99.27%)		📈 view plot
🚷 view threshold		88,074.00
(-0.22%)		88,762.92
(99.22%)		📈 view plot
🚷 view threshold		53.00
(+12.26%)		65.99
(80.32%)		📈 view plot
🚷 view threshold		329.00
(+1.37%)		334.04
(98.49%)
serialize_subscribe📈 view plot
🚷 view threshold		11,317.00
(-0.70%)		11,761.88
(96.22%)		📈 view plot
🚷 view threshold		4,118.00
(-1.52%)		4,316.83
(95.39%)		📈 view plot
🚷 view threshold		5,707.00
(-1.92%)		6,007.89
(94.99%)		📈 view plot
🚷 view threshold		16.00
(+24.42%)		20.66
(77.46%)		📈 view plot
🚷 view threshold		158.00
(+0.30%)		164.08
(96.30%)
🐰 View full continuous benchmarking report in Bencher

Georges Palauqui added 9 commits November 13, 2024 17:02
const_sv2: is a no_std crate
58c53c0
buffer_sv2: is a no_stdcrate by default
133f73e 
- only feature `debug` will add `std` dep which is not used by any other crates
- just had to replace `std::hint` by its `core` equivalent
- in `debug` removed a `mut` to clear a warning
binary_codec_sv2: is a no_std crate by default
433d9cb 
- std::vec::Vec -> alloc::vec::Vec
- std::mem -> core::mem
- std::slice -> core::slice
- std::convert::{TryFrom, TryInto} -> core::convert::{TryFrom, TryInto}
derive_codec_sv2: is a no_std crate
76670d9 
- std::format -> alloc::format
- std::vec::Vec -> alloc::vec::Vec
- std::string::{String, ToString} -> alloc::string::{String, ToString}
serde_sv2: is a no_std crate
83d07c0 
no need any feature for `std` stuff, there is none
binary_sv2: is a no_std crate
1a9012a 
- std::vec::Vec -> alloc::vec::Vec
framing_sv2: is a no_std crate
537a6ac
binary_sv2: factorize use alloc::vec::Vec; in tests
b4da137
Add no_std in README
8cd634b
@plebhash plebhash merged commit b900d0a into stratum-mining:main Nov 13, 2024
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbesraa jbesraa jbesraa left review comments

@plebhash plebhash plebhash approved these changes

@Shourya742 Shourya742 Shourya742 approved these changes

@rrybarczyk rrybarczyk rrybarczyk approved these changes

Assignees
No one assigned
Labels
protocols Lowest level protocol logic
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Georges760 @jbesraa @plebhash @rrybarczyk @Shourya742