Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kubectl-plugin] Add rayjob yaml generation to ray job submit command #2644

Merged
merged 1 commit into from
Dec 17, 2024

Conversation

chiayi
Copy link
Contributor

@chiayi chiayi commented Dec 12, 2024

Why are these changes needed?

Allows user to not need a pre-written rayjob yaml to submit a rayjob.

Related issue number

Part of #2608

Checks

  • I've made sure the tests are passing.
  • Testing Strategy
    • Unit tests
    • Manual tests
    • This PR is not tested :(

@chiayi chiayi mentioned this pull request Dec 12, 2024
14 tasks
} else if !info.Mode().IsRegular() {
return fmt.Errorf("Filename given is not a regular file. Failed with: %w", err)
}
// Take care of case where no file name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this comment is incorrect, as the next line checks that the filename is not empty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, it should be take care of case where there is a filename

@@ -71,6 +82,8 @@ var (
Submit ray job to ray cluster as one would using ray CLI e.g. 'ray job submit ENTRYPOINT'. Command supports all options that 'ray job submit' supports, except '--address'.
If RayCluster is already setup, use 'kubectl ray session' instead.

If no ray job file is specified, one will be generated.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't generate yaml file; we only apply a default RayCluster object for users, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will generate a basic/ rayjob apply config and apply that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I changed the wording. Please take another look.

Copy link
Member

@MortalHappiness MortalHappiness Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. Here, "one" refers to "RayJob", not "ray job file", right? Wouldn't it be better to replace this word with something more explicit? At first, I thought "one" referred to "ray job file".

Copy link
Contributor Author

@chiayi chiayi Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, one is suppose to refer to RayJob, but you're right, having something explicit is definitely better in case of confusion.

There should be anymore confusion from the wording.

@chiayi chiayi force-pushed the kubectl-plugin branch 2 times, most recently from ee04aaa to 99a45e8 Compare December 13, 2024 21:40
@chiayi chiayi force-pushed the kubectl-plugin branch 3 times, most recently from fe7d2cd to e99e7d7 Compare December 16, 2024 17:31
WorkerMemory: options.workerMemory,
WorkerReplicas: options.workerReplicas,
// This is here to match the existing rayjob sample.
WorkerLifecyclePrestopExecComand: []string{"/bin/sh", "-c", "ray stop"},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this, or at least we've been removing them in sample YAML

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed that the sample rayjob still work after this is removed.

kubectl-plugin/pkg/cmd/job/job_submit.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@andrewsykim andrewsykim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a small nit

kubectl-plugin/pkg/cmd/job/job_submit.go Outdated Show resolved Hide resolved
@andrewsykim andrewsykim merged commit 4021766 into ray-project:master Dec 17, 2024
21 of 23 checks passed
log.Fatalf("Failed to mark flag as required %v", err)
}

cmd.Flags().StringVar(&options.rayjobName, "name", "", "Name of the ray job that will be generated")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way we can consolidate these flags into a single place with the flags also used in kubectl ray create cluster?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it's possible, I think we can try sharing the flags in some way. For example, setting the "parent" command to have these flags and this way all the "child" commands will as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants