Make Cgroups default for all resource types and apply hard mem policy and set 2GB as default reserved memory

[sanjaysrikakulam]

We should also start adding them to the training VMs.

[mira-miracoli]

Maybe we can do it in a more general way and specify it in the userdata (if we want to use it for all)

[sanjaysrikakulam]

Maybe we can do it in a more general way and specify it in the userdata (if we want to use it for all)

Yes, we can add it to the userdata.

[sanjaysrikakulam]

Cgroups are now enabled by default for all VGCN infra resource types, so we can collect job metrics regardless of whether they are training resources (this should have been the case, but we overlooked it somehow, and we have been missing job metrics for the training jobs :(). I also applied the hard memory policy and reserved 2GB of memory.

[mira-miracoli]

Thank you!