S5: Large Bulk Loads into an S5 DB faile due to limited node cache

[HerbertJordan]

Right now, large bulk loads need to be split into smaller bulk loads distributed over multiple blocks because Carmen S5 has a limit number of modifications per block (see #686). This is not documented in the Carmen Facade.

Ideally this limitation is removed by enabling arbitrary large bulk load operations on a single block on S5 structures.

[HerbertJordan]

Here is a test to reproduce the issue (assuming to be located somewhere in the carmen package):

func TestDatabase_CanRunLargeBulkLoads(t *testing.T) {
	db, err := OpenDatabase(
		t.TempDir(),
		GetCarmenGoS5WithoutArchiveConfiguration(),
		Properties{
			LiveDBCache: "10000", // bytes
		},
	)
	if err != nil {
		t.Fatalf("failed to create database: %v", err)
	}
	defer db.Close()

	load, err := db.StartBulkLoad(0)
	if err != nil {
		t.Fatalf("failed to start bulk load: %v", err)
	}

	for i := 0; i < 10_000; i++ {
		load.SetNonce(Address{byte(i >> 8), byte(i)}, 12)
	}

	if err := load.Finalize(); err != nil {
		t.Fatalf("failed to finish bulk load: %v", err)
	}
}

[cabrador]

This code will become deprecated soon, as genesis files are being introduced for priming and its only usage is in Aida.