The state of the project.

[husio]

I was wondering what is the state of this repository. I think crawshaw/sqlite is an amazing take on the SQLite driver for Go. It seems though that the original authors are not able to spend time on it right now.
There are many pull requests and issues open. Most of them are addressed initially, but the author is waiting for response by the repository maintainer. I understand this is because of lack of time rather than anything else.

I believe the repository is not dead, it's just resting :) Is there a way to help with the crawshaw/sqlite maintanance? Is there anything that can be done in order to help with the pending pull request list?

[AdamSLevy]

Hey all,

Many apologies for not staying current with this repo. I too really love this package but haven't had much occasion to use it as of late, and between life and work I have lapsed as a maintainer.

I will try to circle through the open issues and PRs and perform some triage and updates this weekend.

[anacrolix]

@AdamSLevy consider opening up maintainership to several contributors. I'm sure with some good communication they can manage the project appropriately. There are quite a few PRs that are very suitable to be merged, and others with some guidance.

[AdamSLevy]

Certainly.

I'm not sure I have the permissions to grant push access to others but I will come up with a short list based on past contributions and try to reach out to @crawshaw about adding maintainers.

If anyone is interested feel free to comment here for consideration.

[anacrolix]

Consider me for your short-list: I'm currently using this library extensively across several projects in production, in open-source and for my employer, in several different arrangements including the Pool and Blob implementations, different journal modes, and a mix of single and multi-process use cases. I have a fork or two, including using more recent versions of sqlite due to some needed features there, so quite a bit of test and use case coverage.

[AdamSLevy]

Brief update: I have not reached out to @crawshaw about this yet, but I have taken the time to go through as many PRs and issues as possible this weekend and have cleaned up and merged several as well as updating the package to the latest SQLite 3.36.0 release.

If we are going to add another maintainer I'd like to ensure we're on the same page with the direction of this repo and project. Up to this point it's just been my opinions, and my interpretations of David Crawshaw's intentions based on open issues and the code itself. But with more people getting involved its worth shoring that up. It would also make contributing and accepting contributions easier.

So here are some thoughts about that.

• One of the strengths of the sqlite package is that it is a relatively thin wrapper around the C APIs. In general if you understand the sqlite C APIs, you can understand the Go package APIs. At the same time the Go APIs are designed to hide all of the CGo complexity and protect you from major design pitfalls or implementation errors. For example the caching of prepared statements and the lifecycle management of those objects with the lifecycle management of a Conn. (Note this still isn't perfect. For example I still see people calling Stmt.Finalize when using Conn.Prepare.)
• This ties in with the last strength, but by staying close to the C API design, we offer the power of SQLite native features such as their nested transaction support and provide what is IMO a very nice Golang API for this. I would like to continue to expand support of SQLite3 features into the Go API.
• An apparent design goal of the sqlite package is to be TinyGo compatible. I haven't actually tried to use TinyGo with it, but this is the reason the sqlite package does not import context or perform reflection. sqlitex provides these nicer APIs that are more abstracted from the complexity of preparing statements. I would like to maintain this design goal and even expand CI testing to ensure sqlite can be built with the TinyGo toolchain.
• Another strength of this package is its test coverage and general code quality and documentation. Keeping this high is of top priority. Part of that is ensuring that the API design encourages proper use or is not easy to misuse or misunderstand. For example consider my comment on this rejected PR: added pool.Size() and pool.ExecScript() #108 (comment)
• It should go without saying but correct C memory management is of top importance. Haven't given it much thought but adding testing coverage to detect memory leaks would be valuable if possible.

In any case let me know what you think about these design goals and how we might formalize some of them.

[anacrolix]

I think this project is significantly lagging other implementations in this space. zombiezen.com/go/sqlite provides a much more complete experience and a much tidier API. This implementation only retains relevance vs. zombiezen.com/go/sqlite in that it uses C underneath, and retains compatibility with other C code intended for use with sqlite (such as extensions).

I do not think updates are being provided in a sufficiently timely manner. I also think that while test coverage is good, it should not be at the expense of relevance and a reasonable feature set. Upstream sqlite updates are slow to arrive. There has been no tagged release in several years.

It's my view that the current maintainer is not meeting the needs of the project. They had ample opportunity to seek support earlier in this thread and failed to do so.

I think open source authors don't owe anyone anything, but in this case the maintainer is not even the original author. Access to low-level sqlite3 functionality is of elevated interest in the community right now, and the handling of this repo is becoming a missed opportunity.

This wouldn't be such an issue if long-lived forks in Go weren't so difficult to manage as dependencies across non-trivial projects, but unfortunately they are and here we are.

[boutros]

Since the original creator is busy with other stuff (and also seems to be maintaining another sqlite lib over at github.com/tailscale/sqlite), I was thinking maybe some of us interested in seeing this lib moving forward could create a github organization and maintain a fork of this driver?

I don't mean to disrespect the effort which the creator and maintainers have put into this repo, and I am very greatfull for the value it allready has provided, but am simply interested in keeping upstream sqlite in sync and also try to look into the issues and bugs.

Just a thought!

[anacrolix]

I have started a fork here: https://github.com/go-llsqlite/llsqlite. Numerous fixes and PRs from this project have been merged into it already (it represents the crawshaw-based sqlite version I use in production on multiple projects. Feel free to message me about it, request maintainer privileges and submit PRs.