Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add Nix Flake support #867

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

WIP: Add Nix Flake support #867

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
238613b
Initial flake version
demusdev Sep 30, 2024
f40e9a5
Use Rust version from toolchain file
demusdev Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions flake.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 38 additions & 0 deletions flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
description = "Next-generation decentralized data lakehouse and a multi-party stream processing network";

inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
rust-overlay.url = "github:oxalica/rust-overlay";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a beginner NixOS user myself I'm pretty excited to see this ... but at the same time worried about pushing OS/package-manager-specific stuff into the repo root that will not be used (and therefore cannot be maintained) by the whole team.

On one hand, the ease of setup this offers for Nix users is pretty great - you can get all dev dependencies in seconds. It also guarantees that same exact versions of e.g. flatbuffers and protobuf are used, which is a real concern in our current setup.

On the other hand, I question whether it's a good idea to have e.g. kubo in here as it may result in user having two versions of kubo (system-wide, and one in the flake). In my NixOS I have installed rust via rustup and the rest of dependencies as my system packages and that seems to work OK.

Will need to think on this a bit more. Perhaps having this flake just as a "best-effort support" feature would be valuable enough.

Thanks a lot for contributing this though!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments!

I don't think it would be a big harm to have flake in repo despite only a few people will using it. I definitely don't propose it as the only or the main way to setup dev env. My idea was to have a kind of automation of setup described in DEVELOPER.md, so new contributors could have (isolated) environment immediately. For instance, I like how I can fetch Helix code, enter nix develop shell and build it, and also have rust-analyzer available in my editor.

But I understand that you as a maintainer want to be very careful with adding new things to the repo.

Anyway, I created this PR more as a starting point of discussion than something to be merged immediately.
I plan to push flake file to parent directory and continue extending it, so there will be something for start when you'll consider having flake in the repo.

Should I close this PR or keep it as a reminder?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep this open for sure. I want to find time to test it and perhaps contribute too.

Would be cool to eventually use this to automate things like all the cargo plugins that we currently suggest to install manually.

Hopefully we'll get to a point where flakes can also be used in Github CI actions :)

But lots of things to figure out still. For example I'd like to avoid duplicating this flake into kamu-node repo and risking flake.lock files going out of sync and bloating the host system. Perhaps the right direction would be to have a "kamu-wide" flake that bring in Rust, Node JS for our Web UI etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very new to nix and don't have an idea atm how to organise it better, but eager to experiment.

Hopefully we'll get to a point where flakes can also be used in Github CI actions :)

That would be perfect nix usage I believe.

flake-utils.url = "github:numtide/flake-utils";
};

outputs = { self, nixpkgs, rust-overlay, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
overlays = [ (import rust-overlay) ];
pkgs = import nixpkgs {
inherit system overlays;
};
darwinPkgs = [
pkgs.darwin.apple_sdk.frameworks.SystemConfiguration
pkgs.darwin.apple_sdk.frameworks.CoreServices
];
in
{
devShells.default = with pkgs; mkShell {
buildInputs = [
(rust-bin.fromRustupToolchainFile ./rust-toolchain)
rust-analyzer
jq
kubo
flatbuffers
protobuf
] ++ lib.optionals pkgs.stdenv.isDarwin darwinPkgs;

shellHook = ''
'';
};
}
);
}