Move AdvantageBuffer to its own file and switch to tensordict #9

josiahls · 2023-11-03T19:32:36Z

I'm going to switch to using tensordict as the base data structure in hopes that it cleans up a lot of the API
AdvantageBuffer is now separate

- I'm going to switch to using tensordict as the base data structure in hopes that it cleans up a lot of the API - AdvantageBuffer is now separate

- firstlast merger now takes a merging function to support different step merging operations. Might just rename the FirstLastMerger to a StepMerger, but idk.

- API to use tensordict steps instead

- test script

- docker image build cleanup

- profile for the build section so that vscode doesnt try rebuilding it when we can just build

- devcontainer stuff - mojo stuff

- quick updates

- debug env for continuous agents Notes: - advantage makes no sense, so that is definately the place where the bug is located

josiahls and others added 30 commits November 3, 2023 19:31

Changed:

d8f8786

- I'm going to switch to using tensordict as the base data structure in hopes that it cleans up a lot of the API - AdvantageBuffer is now separate

Updated:

d09b40a

- firstlast merger now takes a merging function to support different step merging operations. Might just rename the FirstLastMerger to a StepMerger, but idk.

Updated:

784b5a8

- API to use tensordict steps instead

Update fastrl-docker.yml

3982204

Update setup.py

f1c95e1

Update fastrl-docker.yml

9f3806a

Update fastrl-docker.yml

c3e7793

Update fastrl.Dockerfile

de4bc07

Update fastrl-docker.yml

0721096

Update setup.py

70138d0

update docker image

efe3b40

minor change

53b4bb6

nubir change

da8d5a9

sdmall change

8853183

Workflow updates

0aa031a

update test workflow

1bbb880

Updated:

199eb5c

- test script

smoal updates

b872a54

update

9bb0733

workflow stuff

d649f00

unit test fixes

f28e95b

minor change

96ec74f

minor change

1a55af2

changed a lil something something

f1ee025

updated

878b5bb

update

b04e6d9

updates

a8eb2bc

Added:

aecdafc

- docker image build cleanup

update

54d8514

Added:

a97902e

- profile for the build section so that vscode doesnt try rebuilding it when we can just build

josiahls added 16 commits January 7, 2024 04:43

Minor updates

c1442ea

add mojo to docker file

b8bf80f

Updated:

0ceb206

- devcontainer stuff - mojo stuff

quick update

3440f16

update docker since ubuntu 18 is deprecateds

f5dfc96

qyuc uopdate

e149404

another one

81d69bc

quic updat

93e2d24

omg another update holy shit

ee03446

quakl update

aa96976

sdfg

7ab9ebc

add mojo

81c1091

switch mojo stuff to its own package

2354a84

Changed:

02edf18

- quick updates

Added:

41a59f8

- debug env for continuous agents Notes: - advantage makes no sense, so that is definately the place where the bug is located

Still skeptical whether advantage buffer

dd04f5f

josiahls merged commit e2e5a9e into main Mar 23, 2024
0 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move AdvantageBuffer to its own file and switch to tensordict #9

Move AdvantageBuffer to its own file and switch to tensordict #9

josiahls commented Nov 3, 2023

Move AdvantageBuffer to its own file and switch to tensordict #9

Move AdvantageBuffer to its own file and switch to tensordict #9

Conversation

josiahls commented Nov 3, 2023