feat: native guest macro for kernel functions

[TlatoaniHJ]

Revision of #1121. Introduces the openvm_native_guest_macro::kernel! macro. Its current behavior is as follows:

openvm_native_guest_macro::kernel! {
    fn function_name(arg1: RustType1 | EdslType1<F>, ..)-> RustReturnType | EdslReturnType<F> {
        compiler_output.txt
    }
}

Inside the macro, you provide a function declaration. Each type (argument types and return types) should be specified in the form A | B, where A is a Rust type and B is an eDSL type (which will generally have a generic F and possibly EF). Currently, only usize | Felt<F> is supported.

Inside the function body, you should provide a filename. The corresponding file should contain a series of u32s in plaintext format, which should be obtained from compiling an eDSL program. For each argument of the function, there should be one u32 equal to its fp in the eDSL program. Then, there should be a single u32 equal to the same for the return value. The remaining u32s should be the serialization of the compiled program.

The intention is that the eDSL program's inputs and outputs should correspond to the eDSL types specified in the macro invocation. However for now, although only Felt<F> is allowed by the macro, the eDSL program can use either Felt<F> or Usize<F> as their runtime behavior is the same.

An example of a Fibonacci program is provided in benchmarks/programs/test_kernel: benchmarks/programs/test_kernel/src/main.rs is the Rust program containing the macro invocation, and benchmarks/programs/test_kernel/kernel/src/main.rs is the program containing the eDSL code which is run to produce benchmarks/programs/test_kernel/compiler_output.txt. You can run benchmarks/src/bin/test_kernel.rs to compile and execute this program.

[lispc]

i am going to develop some examples tomorrow where a const size array of Felts is converted to array of u32s. That is needed in my case (verify root proof inside a native-rv32 hybrid program).

[lispc]

I would suggest split this pr into multi parts:
first merge the transpiler relared pr, to allow hybrid rv32 instructions and long form instructions.
when other parts.

[jonathanpwang]

we can address later, I would slightly prefer if this was a string with " " to match include_bytes!

[jonathanpwang]

what is the purpose of VARIABLE_REGISTER_INDICATOR? it seems weird to have here in this general processing function

[TlatoaniHJ]

It's discussed at the top of page 3 in https://docs.google.com/document/d/1_NMUff1OcJK8uf4RNgPs1xv55thyTMZ4OadqY2SFVrs/edit?usp=sharing

[jonathanpwang]

I somehow missed this doc :O

[jonathanpwang]

We can merge after addressing comments.

Will merge this version first, but noting that we already had some discussions that the "transport" layer should be moved more into something that can be done in the eDSL implementation itself, and we can layer macros on top later to help with more complicated struct transport later.

Can you add documentation around the different opcode indicators for LongForm, Gap, VariableRegister? It's hard to follow what their role is.

[github-actions]

group	app.proof_time_ms	app.cycles	app.cells_used	leaf.proof_time_ms	leaf.cycles	leaf.cells_used
verify_fibair	(+33 [+0.9%]) 3,827	746,108	30,009,664	-	-	-
fibonacci_program	(-26 [-0.4%]) 6,077	1,500,137	51,505,102	-	-	-
regex_program	(-209 [-1.1%]) 18,327	4,190,904	165,028,173	-	-	-
ecrecover_program	(+4 [+0.2%]) 2,607	285,401	15,092,297	-	-	-

Commit: 53475d1

Benchmark Workflow

[lispc]

one thing missed i think, is the "castf" extension handling in sdk configs