v2.1 Ethereal Sky

v2.1 Ethereal Sky

This release comes with two main features:

• pot now supports CLI flags as well as environment variables (run pot --help)
• pot now support Open Telemetry integration by passing (--metrics and --tracing). You need to setup a default OTEL collector.

Usage: main --bucket=STRING

Flags:
  -h, --help                Show context-sensitive help.
      --log-level="info"    debug | info | warn | error ($LOG_LEVEL)
  -b, --bucket=STRING       bucket name ($BUCKET)
      --zip=STRING          zip is the path where the zip file is stored ($ZIP)
      --distributed-lock    distributed-lock enables distributed locking of the pot
                            ($DISTRIBUTED_LOCK)
      --tracing             tracing enables tracing ($TRACING)
      --metrics             metrics enables metrics ($METRICS)

Running Elections on Pot

Another major feature is a support for elections. You can check the example to see how you can hold elections between multiple servers to decide which one should be and keep being a leader. This is an incredibly simple algorithm mainly for use in situations where one needs a lightweight, almost-no-cost replacement to Raft or Paxos. The algorithm was inspired by Raft, but runs fully on a bucket as a backend.

slog.Info("attempting election", slog.String("id", id), slog.Bool("primary", primary))
res, err := client.Create("test/election", []Leader{{ID: id}}, pot.WithNoRewrite(time.Second*10))
if err != nil {
	if pot.IsNoRewriteViolated(err) {
		primary = false
	} else {
		slog.Error("failed", slog.String("err", err.Error()))
	}
}

if !primary && err == nil {
	primary = true
	slog.Info("became primary", slog.String("id", id), slog.Int64("generation", res.Generation))
}

Full Changelog: v2.0...v2.1

v2.0 Azure Elk

Election Support

Pot supports election support from version 2.0. This can be seeon on the test for election and re-election. These functionalities help with a simple implementation for when multiple replicas need to decide on the primary/secondary states.

Code Example:
The code example below runs in an infinite loop. The WithNoRewrite flag disables writing for all clients but the current one unless the provided time of 10 seconds has passed. In case the object is stall for 10 seconds, another such loop can can take over the leadership and will be able to write and rewrite the object.

for {
  _, err := client.Create(testPath, []testStruct{obj}, WithNoRewrite(time.Second * 10))
  if errors.Is(err, pot.ErrNoRewriteViolated) {
    slog.Info("I became secondary")
  } else if err != nil {
    slog.Error("Whoopsie", slog.String("err", err.Error()))
  } else {
    slog.Info("I am primary!")
  }

  time.Sleep(time.Second * 5)
}

What's Changed

• feat: add possibility to lock certain objects for primary / replica elections by @petomalina in #9

Full Changelog: v1.1...v2.0

v1.1 Aqua Deer

What's Changed

• feat: add experimental distributed lock by @petomalina in #4
• feat: apiclient draft by @petomalina in #7

Full Changelog: v1.0...v1.1

API Client Example

type testStruct struct {
	ID         string   `json:"id"`
	Age        int      `json:"age"`
	Path       []string `json:"path"`
	NiceThings []struct {
		Name string `json:"name"`
	}
}

func (t testStruct) Key() string {
	return t.ID
}

func newTestAPIClient() *APIClient[testStruct] {
	return NewAPIClient[testStruct]("http://localhost:8080")
}

func main() {
  client := newTestAPIClient()

  // first make sure there is nothing stored on the path
  content, err := client.Get(testPath)
  if err != nil {
    t.Fatal(err)
  }

  // store an object on the path
  obj := testStruct{
    ID:  "test",
    Age: 10,
    Path: []string{
      "test", "path", "to", "test",
    },
  }

  err = client.Create(testPath, obj)
  if err != nil {
    t.Fatal(err)
  }
}

v1.0 Gray Turtle

This is the first release of Pot, a server and a Go library that helps handle common use-case of editing structured files on Cloud Storage.

The primary motivation behind Pot was to create an interface to handle changes compatible with Open Policy Agent. It provides the main CR(U)D operations that update and manage the datasets on arbitrary paths.

Note on performance
Pot was not designed to match performance of general-use databases. It's performance is currently bound by an internal mutex (will be a mutex per path in the future), and so fully depends on the latency and performance of the underlying bucket. Writing with zipping especially, is an operation that takes up to 1s and locks down the whole bucket - so use with caution for now.