Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Results not visible on Hue UI even though job completed successfully on flink cluster #81

Open
s4saurabh opened this issue Jul 23, 2021 · 0 comments
Open

Results not visible on Hue UI even though job completed successfully on flink cluster #81

s4saurabh opened this issue Jul 23, 2021 · 0 comments

Comments

@s4saurabh
Copy link

Hi,

I'm running the following setup:
Hue: 4.9
Flink: 1.12
Flink Sql Gateway: compiled from source based on https://github.com/ververica/flink-sql-gateway/releases/tag/flink-1.12.0

Flink running as a Yarn Session using below command:

flink-yarn-session -n 2 -s 4 -tm 16GB -d

When I submit a sql query from Hue I'm able to see the job running under Flink Dashboard and also shows correct records sent in output back to sql gateway sink but then nothing appears on Hue UI.
I'm running this without passing execution.target: yarn-per-job parameter in $FLINK_HOME/conf/flink-conf.yaml

If I pass that parameter and start flink sql gateway and submit queries I get the following error:
There was a mention about passing application id of flink job on Yarn in flink-conf.yaml but didn't see it documented anywhere. Could you share the details if that is causing this issue or is it something else?

21:55:24.555 [flink-rest-server-netty-worker-thread-1] ERROR com.ververica.flink.table.gateway.operation.SelectOperation - Session: 678f4ec903c4b996b7fe6eadacd3a06e. Error running SQL job.
java.lang.RuntimeException: Could not execute program.
at com.ververica.flink.table.gateway.deployment.ProgramDeployer.deploy(ProgramDeployer.java:86) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.operation.SelectOperation.executeQueryInternal(SelectOperation.java:246) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.operation.SelectOperation.execute(SelectOperation.java:87) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.session.Session.runStatement(Session.java:106) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.StatementExecuteHandler.handleRequest(StatementExecuteHandler.java:81) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:77) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.AbstractHandler.channelRead0(AbstractHandler.java:178) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.AbstractHandler.channelRead0(AbstractHandler.java:75) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:208) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:69) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [flink-dist_2.11-1.12.1.jar:1.12.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
Caused by: java.lang.IllegalStateException
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:177) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.client.deployment.executors.AbstractSessionClusterExecutor.execute(AbstractSessionClusterExecutor.java:72) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
at com.ververica.flink.table.gateway.deployment.ProgramDeployer.deploy(ProgramDeployer.java:84) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
... 47 more
21:55:24.559 [flink-rest-server-netty-worker-thread-1] ERROR com.ververica.flink.table.gateway.rest.handler.StatementExecuteHandler - Unhandled exception.
java.lang.RuntimeException: Error running SQL job.
at com.ververica.flink.table.gateway.operation.SelectOperation.executeQueryInternal(SelectOperation.java:249) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.operation.SelectOperation.execute(SelectOperation.java:87) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.session.Session.runStatement(Session.java:106) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.StatementExecuteHandler.handleRequest(StatementExecuteHandler.java:81) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:77) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.AbstractHandler.channelRead0(AbstractHandler.java:178) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.rest.handler.AbstractHandler.channelRead0(AbstractHandler.java:75) [flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:208) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:69) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [flink-dist_2.11-1.12.1.jar:1.12.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
Caused by: java.lang.RuntimeException: Could not execute program.
at com.ververica.flink.table.gateway.deployment.ProgramDeployer.deploy(ProgramDeployer.java:86) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.operation.SelectOperation.executeQueryInternal(SelectOperation.java:246) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
... 46 more
Caused by: java.lang.IllegalStateException
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:177) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
at org.apache.flink.client.deployment.executors.AbstractSessionClusterExecutor.execute(AbstractSessionClusterExecutor.java:72) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
at com.ververica.flink.table.gateway.deployment.ProgramDeployer.deploy(ProgramDeployer.java:84) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
at com.ververica.flink.table.gateway.operation.SelectOperation.executeQueryInternal(SelectOperation.java:246) ~[flink-sql-gateway-0.3-SNAPSHOT.jar:?]
flink_sql_hue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@s4saurabh