Call For Action: standard library for Solidity

[axic]

Call For Action: "Standard Solidity Library"

Solidity is quickly improving thanks to the work of everyone involved. The compiler is getting in a better shape every day and providing the low level features nicely, however support for higher level features still needs to improve. Many have created helper functions, but all these efforts are scattered around in different places.

I have created a tree called density as a start to this effort. So far it contains a very limited set of features:

• the two missing bitwise operators (shl, shr)
• the basic design patterns (owned, mortal and interface)
• interfaces for the standard contract APIs (so far, token only)

I've did the above a week ago, but only pushed it now. I have also some string utilities, such as conversion of address to string, which I haven't committed yet.

I wasn't able to find everything via Google before, but now more and more such snippets have surfaced. Either I'm getting bad at using Google or it is harder to get meaningful results out of it. The following "libraries" already exists out there:

• dapp-bin/library has a few utilities already (strings, linked list, and recently two pull requests for maths)
• the oraclize.it API has a few string related functions
• the ether.camp IDE has a snippet called std.sol

I think the standard library should contain:

• the basic design patterns (owner only, mortal, etc.)
• interfaces for the Standardized Contract APIs
• useful string helpers
• support for bitwise operators until EVM and Solidity supports it natively
• have a basic math library (which also supports ‘floating points’ to at least 2 decimal places) (also, FIXME: define "basic")
• has other basic helpers in place for conversion between types (those which aren’t natively supported)

I don’t think it should go into dapp-bin, but a separate tree for the library only. And that tree probably should belong to an org on github as opposed to a person.

The questions are:

• what structure should it have (a lot of questions here)?
• what should be included?
• what should be the goal for the code (readability/understandability, effectiveness i.e. gas cost, ..)?
• what license should it be under (safe to assume MIT, as every piece of code involved is already under it)?
• what should it be named?
• where should such a library be placed?
• anything else?

Based on git history, the following users have contributed to the libraries: @chriseth, @fivedogit, @obscuren, @bertani, @pipermerriam. Calling all you guys for input.

Appendix A. - List of useful helpers

• https://github.com/ethereum/dapp-bin/tree/master/library
• http://dev.oraclize.it/api.sol (string and number parsing)
• https://github.com/pipermerriam/ethereum-string-utils
• https://github.com/pipermerriam/ethereum-stack-depth-lib
• https://github.com/pipermerriam/ethereum-datetime
• https://github.com/axic/density (bitwise, address to string)
• https://github.com/pipermerriam/ethereum-murmur-hash

[chriseth]

In my opinion, the library should be extremely well documented using natspec and if possible, it should be formally verified.
It should also be semantically versioned, potentially providing a "staging area" with proposed updates which can be discussed and then accepted or rejected/amended.

Are there any opinions against putting it inside the solidity repository?

[pipermerriam]

the library should be extremely well documented using natspec and if possible

+1. I'm in favor of requiring inclusion of proper natspec as well as documentation.

it should be formally verified.

Future requirement? This isn't currently doable correct?

It should also be semantically versioned

+1 but we should be sure to declare what semantic versioning means.

Are there any opinions against putting it inside the solidity repository?

I would like to be able to have the documentation live alongside the code. Readthedocs is an easy to configure resource for this sort of thing, so if it goes in the solidity library, I'd like for the docs to be able to live in the same branch as the code (as opposed to the ghpages branch that the current docs have to live in. If they aren't side-by-side, it's hard to enforce documentation with code modifications and additions.

I'm advocating a hard requirement of documentation because a culture of good documentation will pay us back 10 fold in making it easier for new devs to get up to speed.

[simondlr]

My gut feel is that although it would be nice to have standard APIs in the standard libraries, it's a bit too early to start pushing them in there. We don't quite know where these standards will go. Even now, we are changing it (https://gist.github.com/frozeman/090ae32041bcfe120824).

[anthony-cros]

are functions a first-class citizen in solidity? if so, it would be great to have the map(), flatMap(), filter() and reduce() constructs at the very least!

and if they're not, then maybe using an "functional interface" equivalent as java 8 does...

[chriseth]

@anthony-cros hopefully, soon: https://www.pivotaltracker.com/n/projects/1189488/stories/101758856

[androlo]

Not sure if it's been fixed, but last time I documented with natspec it was not possible to document multiple return-values (like you do multiple params). It works, but it isn't formatted properly in devdoc. Natspec in general feels a bit under-developed, Perhaps start in that end?

I would be happy to contribute to this in any way I can, code, documentation, tests. Especially tests. I can probably put in as much as 20-25% time.

I think it is essential to have some way to run code directly through the EVM for testing. If the standard library is put in the Solidity rep like suggested, it would be simple to write either a separate suite or maybe include it in the regular library tests, depending on what's best.

[redsquirrel]

Any updates on this project/issue?

[androlo]

I'm doing some contract that might make it in at some point (in fact I do them on chriseth's request), including a secp256k1 implementation, some string stuff etc. Gonna have RLP encoding/decoding and other Ethereum related functionality as well.

There is still the issue of how tests will be carried out and such, which is why i work in a separate repo. There's a new issue on how to do it from RPC (#516), and when that's been settled maybe they will be accepted into official Solidity stl. In that case I will port the tests to that system.

Link: https://github.com/androlo/solidity-contracts

[androlo]

Standard library!

(Made by a group of people that checks each-others code etc.)

[redsquirrel]

See also @vbuterin's post: https://blog.ethereum.org/2016/06/19/thinking-smart-contract-security/

take common patterns and hardcode them ... create as many standardized mid-level components as possible

[chriseth]

This is an ongoing process and will never be "closed", although we need a kind of "registry" of library contracts. I hope that tools like dapphub and live-libs can fill this gap soon.

[axic]

support for bitwise operators until EVM and Solidity supports it natively

This from the original discussion is not a concern anymore after #527 is done.