Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support submit job with different executor at runtime #57

Open
DashShen opened this issue May 21, 2020 · 0 comments
Open

Support submit job with different executor at runtime #57

DashShen opened this issue May 21, 2020 · 0 comments

Comments

@DashShen
Copy link

DashShen commented May 21, 2020

Currently, flink-sql-gateway can only submit job with a specified executor which defined in flink-conf.yaml。But if someone wants to submit a job to yarn-session, and others want to do it with standalone cluster, then flink-sql-gateway can't satisfy this right now. So could flink-sql-gateway support this feature future。

And I also has a roughly idea that how flink-sql-gateway supports this feature. we can introduce a new component which I called ExecutionManager .ExecutionManager is like SessionManager, but it's main function is to creating a new Execution, each Execution represents an deploy mode(yarn-seesion, yarn-per-job, standalone, local, etc). The Execution will create a new SessionManager and will use user specified execution.target to instance a DefaultContext. By this way, every new session created by SessionManager can submit job to different cluster.

Furthermore, flink-sql-gateway do not need to create SessionManager at start time but instead of ExecutionManager, and the SessionManager will be instance by ExecutionManager.

And, we'll must implement a new netty handler that used to create/delete/get Execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant