Add custom space serialization tutorial #151

enerrio · 2023-10-11T05:09:22Z

Description

This PR adds a new tutorial that goes over serializing custom Gym spaces and shows an example of how to apply a custom observation space to the CartPole environment.

Type of change

Please delete options that are not relevant.

This change requires a documentation update

Screenshots

Screenshot of tutorial page:

Checklist:

I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
I have run pytest -v and no errors are present.
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

younik

It looks good, thank you a lot!

I left some minor comments and a major one about the choice of the space.
Also, we may want to link the future page on the documentation in this error:

Minari/minari/serialization.py

Line 14 in c43a612

raise NotImplementedError(f"No serialization method available for {space}")

younik · 2023-10-11T14:04:25Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+=========================================
+"""
+# %%%
+# In this tutorial you'll learn how to serialize a custom Gym observation space and use that


Gym -> Gymnasium

younik · 2023-10-11T14:05:38Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+print(f"Observation space: {env.observation_space}")
+print(f"Observation space: {env.action_space}")


Maybe it is good to add the outputs of this on the doc page?

younik · 2023-10-11T14:07:58Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+        )
+
+# %% [markdown]
+# Now that we have a custom observation space we need to define functions that properly serialize it.


functions that properly serialize it-> functions that serialize and deserialize it or a function that serialize it

younik · 2023-10-11T14:28:35Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+#
+# When creating a Minari dataset, the space data gets `serialized <https://minari.farama.org/content/dataset_standards/#space-serialization>`_
+# to a JSON format when saving to disk. The `serialize_space <https://github.com/Farama-Foundation/Minari/blob/main/minari/serialization.py#L13C5-L13C20>`_
+# function takes care of this conversion for various supported Gym spaces. To enable serialization for a custom space we can register 2 new functions that


Gym -> Gymnasium

younik · 2023-10-11T14:49:18Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+# delete the test dataset if it already exists
+local_datasets = minari.list_local_datasets()
+if dataset_id in local_datasets:
+    minari.delete_dataset(dataset_id)


I would place it at the end of the tutorial (just remove the dataset without if, with a comment saying you remove it as you created it only for the purpose of the tutorial) to avoid interrupting the flow with something not really related

younik · 2023-10-11T14:52:13Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+custom_observation_space = CartPoleObservationSpace(
+    low=np.array([0, -3.4028235e+38, -4.1887903e-01, -3.4028235e+38], dtype=np.float32),
+    high=np.array([4.8, 3.4028235e+38, 4.1887903e-01, 3.4028235e+38], dtype=np.float32),
+    shape=(4,),
+    dtype=np.float32
+)
+
+
+class CustomSpaceStepDataCallback(StepDataCallback):
+    def __call__(self, env, **kwargs):
+        step_data = super().__call__(env, **kwargs)
+        step_data["observations"][0] = max(step_data["observations"][0], 0)
+        return step_data


I don't know if doing it with a StepDataCallback is the best way for the user to understand (it also introduces the StepDataCallback that may not be known by the user). What about creating a very simple environment (subclassing CartPole), defining step, and reset functions?

younik · 2023-10-11T14:59:36Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+class CartPoleObservationSpace(Space):
+    def __init__(
+        self,
+        low: NDArray[Any],
+        high: NDArray[Any],
+        shape: Sequence[int],
+        dtype: type[np.floating[Any]] = np.float32,
+    ):
+        self.low = np.full(shape, low, dtype=dtype)
+        self.high = np.full(shape, high, dtype=dtype)
+        super().__init__(shape, dtype)
+
+    def sample(self) -> NDArray[Any]:
+        """Sample a random observation according to low/high boundaries"""
+        sample = np.empty(self.shape)
+        sample = self.np_random.uniform(low=self.low, high=self.high, size=self.shape)
+        return sample.astype(self.dtype)
+
+    def contains(self, x: Any) -> bool:
+        """Return boolean specifying if x is a valid member of this space"""
+        if not isinstance(x, np.ndarray):
+            try:
+                x = np.asarray(x, dtype=self.dtype)
+            except (ValueError, TypeError):
+                return False
+
+        return bool(
+            np.can_cast(x.dtype, self.dtype)
+            and x.shape == self.shape
+            and np.all(x >= self.low)
+            and np.all(x <= self.high)
+        )


This is basically the Box space; it can be confusing for the user. I am wondering if we can have a custom space that makes more sense and it is still simple.
For example, minigrid has the mission space, it is basically a TextSpace where the text is sampled from a set. Do you have other ideas? I will think a bit more about it and let you know if I have something better.

Yeah I thought about making a custom action space instead since that's a little simpler to implement (ex: maybe an action space that only allows moving left) but thought that might not get the idea across of a good example for when someone should create a custom space. I'll take a look at the minigrid environment and see if there's a modification that makes sense and is simple to implement.

To clarify: what I meant is that the MissionSpace of minigrid is not serializable, so this can be a use case for the tutorial

younik

Just two minor changes, then it is ready to be merged. Thanks!

younik · 2023-10-24T06:26:40Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+# supported and if try to serialize it with:
+
+# %%
+serialize_space(env.observation_space['mission'])


It is better to not create code that throws error, as we aim to add automatic testing for docs (so the file should run). You can catch the error and print it or just use text to explain it.

younik · 2023-10-24T06:30:27Z

docs/tutorials/dataset_creation/custom_space_serialization.py

+# %% [markdown]
+# To get an idea of what the serialization is doing under the hood we can directly call
+# the ``serialize_custom_space`` function we defined earlier and see the JSON string it returns.
+
+# %%
+serialize_custom_space(env.observation_space['mission'])


show instead of this block that actually you can reload the dataset

del dataset dataset = minari.load_dataset(dataset_id)

And the dataset.observation_space is correct

younik

LGTM, thank you!

enerrio added 3 commits October 10, 2023 22:02

add space serialization tutorial

3df45c7

update formatting for serialization tutorial

1f24cca

fix pre-commit errors for serialization tutorial

f8da73b

younik requested changes Oct 11, 2023

View reviewed changes

enerrio added 2 commits October 22, 2023 17:39

update tutorial to use MiniGrid MissionSpace

480fcf6

fix formatting

e3d9da0

enerrio requested a review from younik October 23, 2023 00:52

younik requested changes Oct 24, 2023

View reviewed changes

remove line that throws error. add block to load new dataset

4b3645a

enerrio requested a review from younik October 24, 2023 23:28

younik approved these changes Oct 24, 2023

View reviewed changes

younik merged commit 7113906 into Farama-Foundation:main Oct 24, 2023
10 checks passed

enerrio deleted the add_tutorial branch October 24, 2023 23:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add custom space serialization tutorial #151

Add custom space serialization tutorial #151

enerrio commented Oct 11, 2023

younik left a comment

younik Oct 11, 2023

younik Oct 11, 2023

younik Oct 11, 2023

younik Oct 11, 2023

younik Oct 11, 2023

younik Oct 11, 2023

younik Oct 11, 2023

enerrio Oct 12, 2023

younik Oct 12, 2023

younik left a comment

younik Oct 24, 2023

younik Oct 24, 2023

younik Oct 24, 2023

younik left a comment

		print(f"Observation space: {env.observation_space}")
		print(f"Observation space: {env.action_space}")

Add custom space serialization tutorial #151

Add custom space serialization tutorial #151

Conversation

enerrio commented Oct 11, 2023

Description

Type of change

Screenshots

Checklist:

younik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

younik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

younik left a comment

Choose a reason for hiding this comment