stdlib prototype #10639

axic · 2020-12-16T19:43:54Z

Part of #10282.

TODO:

provide a way to export the standard library as a directory hierarchy from the commandline interface (or standard-json - then it would be a json map of filename -> string)
only include standard library files in the compiler stack input files if they were imported (use the import file resolution mechanism for that)
if a file is included whose name clashes with a file in the standard library (or maybe even the prefix std/), check if its contents match the built-in contents and don't do anything. Otherwise warn.
Changelog entry

If the three points are combined, you can export the standard library to a folder and importing from that folder would do the same as importing the built-in standard library.

The contents of the standard library are of course version-dependent.

chriseth · 2020-12-21T14:52:40Z

stdlib/assembly.sol

+assembly {
+  // allocates a memory block
+  // TODO: use memoryguard
+  function alloc(size) -> ptr {


Suggested change

function alloc(size) -> ptr {

function allocate(size) -> ptr {

I think actually these memory functions may be better as being part of the inline assembly dialect (as discussed on #10399).

Yes, but I think it would be good to also have the translation between the two dialects available.

chriseth · 2020-12-21T14:53:12Z

stdlib/assembly.sol

@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: Apache-2.0


Why apache?

Could be MIT too. The goal is to avoid license baggage for users. They may want to keep their deployed codebase with less restrictions than GPL-3.0.

One thing I really don't like about Apache license is this bit:

You must cause any modified files to carry prominent notices stating that You changed the files

I.e. it requires the source to be annotated with extensive info about who changed it. I think that such metadata belongs in the version control, not in the file. It would be pretty tedious to have to keep it in the source.

I think it just means stating it was changed and not what has been. Though in practice this does not take place in the original source repo, just in forks.

Also note Rust uses a dual MIT/Apache2 license and a large part of the Ethereum ecosystem uses Apache2.

chriseth · 2020-12-21T14:54:52Z

stdlib/globals.sol

+  assembly {
+    // Prepare input
+    // We are using the free memory pointer.
+    let input := mload(0x40)


Move this into a function?

So far these files were independently working, and the assembly file of course does not work yet. But yes, eventually it should use the assembly helpers.

chriseth · 2020-12-21T14:55:46Z

stdlib/globals.sol

+    mstore(output, 0)
+
+    // Call the precompile
+    let ret := staticcall(gas(), 1, input, 128, output, 32)


Doesn't ecrecover need more error checks?

This is currently a 1-on-1 mapping to ecrecover in the language. If we do semantic changes, then we can consider an empty output as failure and revert.

axic · 2020-12-21T15:15:40Z

stdlib/globals.sol

+    mstore(input, hash)
+    mstore(add(input, 32), v)
+    mstore(add(input, 64), r)
+    mstore(add(input, 96), s)


Could also use bytes manipulation (as per discussions in #9829):

bytes input = new bytes(96); input[0..32] = bytes32(bytes8(v)); input[32..64] = r; input[64..96] = s;

Or

bytes input = abi.encodePacked(bytes32(bytes8(v)), r, s);

If we manage to improve memory handling 😉

axic · 2021-04-23T11:40:18Z

I wonder if we should just start committing stdlib under semanticTests/externalSources and iterating on it? And once it is in a useful state, git move those files to /stdlib and get it exposed in the compiler.

cameel · 2021-04-24T01:00:49Z

stdlib/blocktx.sol

+function blockhash(uint blockNumber) returns (bytes32 ret) {
+  assembly {
+    ret := blockhash(blockNumber)
+  }
+}


Just a heads up before we drop the source in the repo - I think that the piece of code as prominent as the standard library should really follow our own coding style :)

Suggested change

function blockhash(uint blockNumber) returns (bytes32 ret) {

assembly {

ret := blockhash(blockNumber)

}

}

function blockhash(uint blockNumber) returns (bytes32 ret) {

assembly {

ret := blockhash(blockNumber)

}

}

EDIT: turns out the _ prefix for parameters is not in our Solidity style guidelines. So only indents are wrong.

Makes sense. Do we have any good guide for variable names (i.e. for ret)?

cameel · 2021-04-24T01:05:16Z

stdlib/utils.sol

+function min<T>(T a, T b) pure internal returns (T ret) {
+  ret = (a < b) ? a : b;
+}
+
+function max<T>(T a, T b) pure internal returns (T ret) {
+  ret = (a > b) ? a : b;
+}
+
+function abs<T: T.isSigned>(T a) pure internal returns (T ret) {
+  ret = (a < 0) ? -a : a;
+}


Looks like the future :)

chriseth · 2021-08-18T13:09:06Z

stdlib/globals.sol

+//import sha256 from std;
+//import chainid from std;
+
+contract C{


Suggested change

contract C{

Yup, this was before free functions :)

axic · 2021-08-18T13:12:51Z

stdlib/blocktx.sol

+  }
+}
+
+library Block {


Instead of libraries, we can split these into different files and import them based on file names.

Yes, I think we should split this in many small files.

(mainly to test out if the way we plan to do namespacing in the future works out well or not)

axic · 2021-08-18T13:13:27Z

stdlib/errors.sol

+uint constant PanicResourceError = 0x41;
+uint constant PanicInvalidInternalFunction = 0x51;
+
+error Panic(uint code);


Should we natspec the stdlib?

leonardoalt · 2022-04-04T09:02:58Z

@axic wen stdlib

axic · 2022-06-27T07:54:51Z

Pushed here a new approach, where a compiler flag enables the stdlib. This way perhaps we can merge these 3 precompiles as a start.

CMakeLists.txt

axic · 2022-06-27T07:56:13Z

solstdlib/solstdlib.h.in

+{
+
+static std::map<std::string, std::string> sources = {
+	{ "std/precompiles.sol", precompiles },


Didn't find a good cmake way to generate this file 👁️

axic · 2022-06-27T07:57:45Z

solstdlib/src/precompiles.sol

+function sha256(bytes memory input) returns (bytes32 ret) {
+  assembly {
+    let success := staticcall(gas(), 2, add(input, 32), mload(input), 0, 32)
+    if iszero(success) { revert(0, 0) }


These precompiles currently differ in two ways from the IR:

only support newer EVM versions where passing all gas is possible (would need Introduce solidity.evmVersion() to query the target EVM #10283 to support conditional options)

it does not pass through the revert data (this should be fixed before merging)

github-actions · 2023-03-15T12:05:07Z

This pull request is stale because it has been open for 14 days with no activity.
It will be closed in 7 days unless the stale label is removed.

ekpyron · 2023-03-15T12:15:40Z

Removing the stale label here - but @nikola-matic, you wanted to check and report the state of this, s.t. we can decide all remaining open questions, if any (I guess it's mainly stuff like the name used in importing and such?), and actually move this forward, right?

[For the record: this only occurs in the roadmap as a "future" roadmap item - but that is supposed to mean to be finished with this - this PR is only the very first small step towards moving everything that can be moved and the entire thing will both take a lot of time and require generics - but that doesn't mean that we shouldn't still start and do what we can do "now" or at least soonish - no immediate hurry, but we also shouldn't just keep this open and go to waste forever]

github-actions · 2023-03-30T12:04:35Z

This pull request is stale because it has been open for 14 days with no activity.
It will be closed in 7 days unless the stale label is removed.

nikola-matic · 2023-03-30T12:12:20Z

Removing the stale label here - but @nikola-matic, you wanted to check and report the state of this, s.t. we can decide all remaining open questions, if any (I guess it's mainly stuff like the name used in importing and such?), and actually move this forward, right?

[For the record: this only occurs in the roadmap as a "future" roadmap item - but that is supposed to mean to be finished with this - this PR is only the very first small step towards moving everything that can be moved and the entire thing will both take a lot of time and require generics - but that doesn't mean that we shouldn't still start and do what we can do "now" or at least soonish - no immediate hurry, but we also shouldn't just keep this open and go to waste forever]

We can add the roadmap label to this PR, so it doesn't get the bot's attention, right @r0qs?

r0qs · 2023-03-30T13:58:46Z

~~Yes, or even use must have or must have eventually with prevent the bot to mark it as stale.~~
Just a small correction since must have is only used for issues and not PRs. So just use roadmap as you said.

cameel · 2023-04-19T13:35:37Z

Decision from the call: we'd prefer a special syntax for the stdlib imports:

import std.cryprography;

nikola-matic · 2023-04-19T13:54:36Z

Decision from the call: we'd prefer a special syntax for the stdlib imports:
import std.cryprography;

Changes to Parser::parseImportDirective() and the grammar then. Would be nice to crosspost this to the stdlib issue. Might be a bit annoying (haven't looked into it yet) to protect against allowing this syntax without the pragma being enabled.

cameel · 2023-04-19T14:10:21Z

I did cross-post there already. Small changes in the parser will of course be needed (the stdlib import will use an identifier path) but this will all be behind an experimental pragma so non-breaking.

And I don't think we'll be keeping the ANTLR grammar up to date with the experimental changes. @ekpyron already thinks that even trying to keep it up to date as is is too much of a distraction.

nikola-matic · 2023-04-19T14:20:54Z

I did cross-post there already. Small changes in the parser will of course be needed (the stdlib import will use an identifier path) but this will all be behind an experimental pragma so non-breaking.

And I don't think we'll be keeping the ANTLR grammar up to date with the experimental changes. @ekpyron already thinks that even trying to keep it up to date as is is too much of a distraction.

Ah, so we're changin the pragma directive as well. Should this be implemented separately as a prerequisite for stdlib, and then this PR updated respectively?

cameel · 2023-04-19T14:24:20Z

I think separately, because we want it done and merged in the very near future. It should not be held back by discussions of other changes in this PR.

cameel · 2023-04-19T14:25:52Z

@ekpyron said on the call that anyone not busy with other tasks could start working on that already. We'll discuss further steps next week.

ekpyron · 2023-04-19T14:26:36Z

Yeah, we won't maintain an antlr grammar for the experimental version of the language - we can do that once we have stabilized it :-).

And yes, this PR will be rebased on top of a generic "pragma experimental solidity-next;" PR, which we'll have soon (after that we can drop the pragma stdlib here) - and that experimental pragma will work in such a way that it can enable an experimental parser mode that can then be permissive in the import directive and allow this new style.

@ekpyron said on the call that anyone not busy with other tasks could start working on that already. We'll discuss further steps next week.

"That" being the pragma experimental solidity-next; PR first - and once that's done rebasing this PR on top of that, if there's even more time to spare.

ekpyron · 2023-05-30T11:08:55Z

Closing in favour of #14249

chriseth reviewed Dec 21, 2020

View reviewed changes

axic commented Dec 21, 2020

View reviewed changes

ethereum deleted a comment from stackenbotten Apr 22, 2021

cameel reviewed Apr 24, 2021

View reviewed changes

axic force-pushed the stdlib branch from db6f47a to 169f4f7 Compare August 18, 2021 12:57

chriseth reviewed Aug 18, 2021

View reviewed changes

axic commented Aug 18, 2021

View reviewed changes

chriseth mentioned this pull request May 11, 2022

Native solidity support for EIP-196 EIP-197: ECADD and ECMOD for alt_bn128 #13003

Closed

axic dismissed a stale review via 2efd602 June 27, 2022 07:54

axic force-pushed the stdlib branch from 169f4f7 to 2efd602 Compare June 27, 2022 07:54

axic requested review from chriseth, cameel and leonardoalt June 27, 2022 07:54

axic commented Jun 27, 2022

View reviewed changes

CMakeLists.txt Show resolved Hide resolved

axic commented Jun 27, 2022

View reviewed changes

ethereum deleted a comment from stackenbotten Jun 27, 2022

axic force-pushed the stdlib branch from faeccbf to 4349fc9 Compare June 27, 2022 08:20

ethereum deleted a comment from stackenbotten Jun 27, 2022

nikola-matic added 5 commits February 2, 2023 11:17

Resolve stdlib imports on demand

aceb9dc

More general minimum EVM version check

5f6e606

Rename precompile.sol and add more tests

1bde61c

Force LF line endings in cmake generated files

ad5d528

Check for import without pragma and add cmdline test

2a07761

nikola-matic force-pushed the stdlib branch from d143837 to 2a07761 Compare February 2, 2023 10:17

cameel mentioned this pull request Feb 13, 2023

Moving fixed language features to Stdlib #13726

Open

github-actions bot added the stale The issue/PR was marked as stale because it has been open for too long. label Mar 15, 2023

ekpyron removed the stale The issue/PR was marked as stale because it has been open for too long. label Mar 15, 2023

github-actions bot added the stale The issue/PR was marked as stale because it has been open for too long. label Mar 30, 2023

nikola-matic removed the stale The issue/PR was marked as stale because it has been open for too long. label Mar 30, 2023

r0qs added the roadmap label Mar 30, 2023

nikola-matic mentioned this pull request Apr 28, 2023

Introduce pragma experimental solidity #14168

Merged

4 tasks

ekpyron closed this in #14168 May 15, 2023

nikola-matic reopened this May 15, 2023

nikola-matic mentioned this pull request May 19, 2023

Experimental standard library #14249

Merged

ekpyron closed this May 30, 2023

	function alloc(size) -> ptr {
	function allocate(size) -> ptr {

stdlib prototype #10639

stdlib prototype #10639

Conversation

axic commented Dec 16, 2020 • edited by nikola-matic Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

axic Dec 21, 2020 • edited Loading

Choose a reason for hiding this comment

axic Dec 21, 2020 • edited Loading

Choose a reason for hiding this comment

axic commented Apr 23, 2021

cameel Apr 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leonardoalt commented Apr 4, 2022

axic commented Jun 27, 2022

Choose a reason for hiding this comment

axic Jun 27, 2022 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Mar 15, 2023

ekpyron commented Mar 15, 2023 • edited Loading

github-actions bot commented Mar 30, 2023

nikola-matic commented Mar 30, 2023

r0qs commented Mar 30, 2023 • edited Loading

cameel commented Apr 19, 2023

nikola-matic commented Apr 19, 2023 • edited Loading

cameel commented Apr 19, 2023

nikola-matic commented Apr 19, 2023

cameel commented Apr 19, 2023

cameel commented Apr 19, 2023 • edited Loading

ekpyron commented Apr 19, 2023 • edited Loading

ekpyron commented May 30, 2023

axic commented Dec 16, 2020 •

edited by nikola-matic

Loading

axic Dec 21, 2020 •

edited

Loading

axic Dec 21, 2020 •

edited

Loading

cameel Apr 24, 2021 •

edited

Loading

axic Jun 27, 2022 •

edited

Loading

ekpyron commented Mar 15, 2023 •

edited

Loading

r0qs commented Mar 30, 2023 •

edited

Loading

nikola-matic commented Apr 19, 2023 •

edited

Loading

cameel commented Apr 19, 2023 •

edited

Loading

ekpyron commented Apr 19, 2023 •

edited

Loading