support Apache Spark 3.4

[Ponyo1234]

Is your feature request related to a problem? Please describe.
In Apache Spark 3.4, Spark Connect introduced a decoupled client-server architecture that allows remote connectivity to Spark clusters using the DataFrame API and unresolved logical plans as the protocol

Describe the solution you'd like
support spark-3.4

[GeorgeS2019]

[30thAug2023] .NET for Apache Spark documentation will no longer be updated

[GoEddie]

I’ve been looking into this separately and believe it is possible and probably the right thing to do going forward, we are waiting to find out about the status of the project but I am going to start working on this.

[GeorgeS2019]

@GoEddie

Do follow @luisquintanilla contribution history with Spark.NET before it was even called what we have today. He went all the way back with the project.

Once he is showing interest.

This will be the new beginning!

Thank you for your interest. My role will be more of a morale boosting rather than hands-on.

[GoEddie]

Thanks @GeorgeS2019 Ive let Luis know on the other issue I’m interested in helping with the project!