feat: make PML compatible with OE by staging in-memory CLOB side effects (backport #2447)

[mergify]

Changelist

PML supports user transaction that creates new ClobPair (both in state and in memory objects). Currently there are two issues:

• The user txn can be reverted later, but the in-memory changes are not discarded
• The block may be processed optimistically (OE) and discarded later, but the in-memory changes are not discarded

To solve these issues we use the FinalizeBlockEventStager to store a ClobPair creation event during DeliverTx, and apply the in-memory side effects later in Precommit.

Test Plan

[Describe how this PR was tested (if applicable)]

Author/Reviewer Checklist

• If this PR has changes that result in a different app state given the same prior state and transaction list, manually add the state-breaking label.
• If the PR has breaking postgres changes to the indexer add the indexer-postgres-breaking label.
• If this PR isn't state-breaking but has changes that modify behavior in PrepareProposal or ProcessProposal, manually add the label proposal-breaking.
• If this PR is one of many that implement a specific feature, manually label them all feature:[feature-name].
• If you wish to for mergify-bot to automatically create a PR to backport your change to a release branch, manually add the label backport/[branch-name].
• Manually add any of the following labels: refactor, chore, bug.

Summary by CodeRabbit

• New Features

  • Introduced the ClobStagedFinalizeBlockEvent interface for handling CLOB events during block finalization.
  • Added a new Protocol Buffers file defining the ClobStagedFinalizeBlockEvent message type.
  • Enhanced the Precommit function to process staged finalize block events.

• Changes

  • Updated method signatures to enhance functionality and return types in mock implementations.
  • Reorganized module imports for improved structure and clarity.
  • Updated method calls to reflect the new CreatePerpetualClobPairAndMemStructs functionality.

• Bug Fixes

  • Adjusted the return value handling in the MaybeCreateOrderbook method to prevent potential issues.

---

This is an automatic backport of pull request #2447 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of 161c34d has failed:

On branch mergify/bp/release/protocol/v7.x/pr-2447
Your branch is up to date with 'origin/release/protocol/v7.x'.

You are currently cherry-picking commit 161c34dd.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   indexer/packages/v4-protos/src/codegen/dydxprotocol/bundle.ts
	new file:   indexer/packages/v4-protos/src/codegen/dydxprotocol/clob/finalize_block.ts
	modified:   indexer/packages/v4-protos/src/codegen/gogoproto/bundle.ts
	modified:   indexer/packages/v4-protos/src/codegen/google/bundle.ts
	new file:   proto/dydxprotocol/clob/finalize_block.proto
	modified:   protocol/mocks/MemClob.go
	modified:   protocol/testutil/keeper/clob.go
	modified:   protocol/x/clob/abci.go
	modified:   protocol/x/clob/abci_test.go
	modified:   protocol/x/clob/genesis.go
	modified:   protocol/x/clob/keeper/clob_pair.go
	modified:   protocol/x/clob/keeper/clob_pair_test.go
	modified:   protocol/x/clob/keeper/deleveraging_test.go
	modified:   protocol/x/clob/keeper/get_price_premium_test.go
	modified:   protocol/x/clob/keeper/keeper.go
	modified:   protocol/x/clob/keeper/liquidations_test.go
	modified:   protocol/x/clob/keeper/mev_test.go
	modified:   protocol/x/clob/keeper/msg_server_place_order_test.go
	modified:   protocol/x/clob/keeper/orders_test.go
	modified:   protocol/x/clob/keeper/process_operations_test.go
	modified:   protocol/x/clob/memclob/memclob.go
	modified:   protocol/x/clob/memclob/memclob_get_impact_price_subticks_test.go
	new file:   protocol/x/clob/types/finalize_block.pb.go
	modified:   protocol/x/clob/types/keys.go
	modified:   protocol/x/clob/types/memclob.go
	modified:   protocol/x/listing/keeper/listing_test.go
	modified:   protocol/x/listing/types/expected_keepers.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   protocol/x/listing/keeper/listing.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.