Improve management of cloud infrastructure in tests

[pleshakov]

We use shell scripts https://github.com/nginxinc/nginx-gateway-fabric/tree/15bab00d0f31c96eed0fe29f34ed7373fa0b9646/tests/scripts to provision and deprovision cloud infra on GKE for tests.

One problem with shell scripts - handling partial failures: if some resources were created and some note, re-runing a failed script is not possible because it will fail due to attempts to create already existing resources

Consider using declarative infra management tools to manage infra in tests.

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stalled for 14 days with no activity.

[mpstefan]

How often does this error occur? How much time does it cost us?

Placed it in the backlog for now.

[pleshakov]

example of a clean up that wasn't complete -- https://github.com/nginxinc/nginx-gateway-fabric/actions/runs/9306728832/job/25616520648

clean up failed to delete created cluster

[mpstefan]

Moved to #2390