Julia .NET Runtime

[HyperSphereStudio]

I was wondering what everyone's thought on creating a julia runtime on .NET would be.

This is would be a huge project but I see many advantages to this such as:

Mobile Support
Access to .NET libraries & APIs
.NET Libraries can access Julia Libraries
Code Interpreter (Vastly speed up small script execution)
Continued Improvement by Microsoft

I will create a REPO later with my current prototype which includes:
Partial Julia Parser
Julia IR (Dynamically generates IL on type specialization resolution)

I personally call it julia++ since I plan to extend the language; however, for now the goal is to be able to utilize the julia STL.

[BambOoxX]

This is far above my paygrade, but shouldn't you discuss this with the core julia team ? Also IMHO I think you should try this discussion on the discourse, you will surely reach a lot of people there.

[HyperSphereStudio]

I want to try to get a semi working prototype before I try to propose it to the core team. I find its far more persuasive and shows that the idea can actually be achieved. I have had decent success at static compilation (non macro, no eval calls).

[HyperSphereStudio]

This is going to merely simulate the julia runtime. There are so many differences between this and the official julia runtime. For intance I am not using llvm, I am lowering to MSIL. There entire compilation structure is vastly different.

[damian-666]

i love the idea because i use physics for games and even CFD like stuff.. Cornell asks for code submits along w papers but since almost no simulation code compiles on anyone else's machines they never get many. github links. if you actually type code from a CFD paper you're probably want to use Juila, right?.. im not going to use Cuda because it only would work in my machine what about the kid without an NVIDIA chip?? i cant even build a c++ openGL samel and i worked in c++ for Autodesk for 9 years before going to c#... so i port the c or c++ to c# and run that... I barely know anything about julia, who produces it and how they receive revenue, but to do this proposal , i think you should look also at the Silk project and understand what driving .net to go multiplatform too.. .. what they make are code generators that unify c++ apis. for graphics but their techniques are amazing .. the lead on this part is like this 19 yr old German genius kid probably coded since age 4. sponsore is jetbrains. . thai is all under the Net foundations umbrella. and also CCScript im using it to add live plugins to my tools... there are Microsfot people on there and Jetbrains a competitor is sponsored too... its amazing that theres unification now, because the cloud service , dataminngs business is the biggest pie chart chunk for microsoft.. so when i update to nett6 i run physics at 4000 fps using monogame as a front end for visualization, i get 40% boost in fps.. i go and erase all my old problematic physics optimizatons and see no effect just let bugs.....i dont benchmark because tghe performance gains are just obvious. ...i see so much flux going on in Net 7 what JIT and SIM also.. so i think net 6 was a turning point for .net. I hope i havent written this before , or boring u with what you already know, im in alot of discussions..in lots of people vaults, cant remember what ive shared and been accused of ranting a bit.. after 20 years .net its starting to make good... and now u can monitor the netcore and rosylin vaults and see how they are doing things.. .JIT has a yet unfulfilled promise, the use on individual hardware for the sake of optimization and generalized SIMD intrinscs are finally after 10 years being rolled in my Microsft payrolled staff,, but on open source. so having Julia somehow go under that umbrella would be a wicked casue to pursue. f# is there... but thee is no good scientific language yet my julia s the best one.. but with code that a software company can actually ship and deploy with.. on a billion devices +... because of the limitation of the speed of light and interactivity i probably wont use cloud services much for simulatio ever, if would be impossible to make a multiplayer game with powered ragdolls, so i dont use distributed computing but since thats where the $ is the big companies are sponsoring all this.. its turned out great for me, free tools and efficient high level langage like c#, im just a puppet maker / engineer but at the application level i avoid coding to the metal. so i cant help that much directly... but mhy game is physics based.. and has fluids , and lot of CFD people like Julia , and your studio has a cool name so im intrigued by this idea. the the who MSbuld tooling has changed so radically starting with VS 2022 that everyone is struggling to get there old stuff working but i think its stabilized and on a 2 year LTs plan soo it would be a good time to hop in.. its legacy .net code that just microsoft only, thats sorta killing everyone, im stuck 5 weeks and almost out of this hole but i'm committed to moving to net6..because its just obviously faster when all my stuff is build that what. then i just develop on a touch surface pc ( soon to be running ARM chips) and deploy it on an android phone and its working.. only 6 yeas ago windows phone and silvelight was dying , Nokia was sunk, microsoft was tryng to put windows everywhere, and now things are looking vastly less bleak...now they are trying to put Git on windows, last i spoke to anyone inside, fine with me i guess.. https://github.com/dotnet/Silk.NET https://github.com/oleg-shilo/cs-script.npp, still relevant. https://github.com/dotnet the whole foundation itself has very few political conflicts.. its so transparent you can see in the vaults where they aer. yea some squabbles bettern the tooling jetbrains vs devstudio .... makes but MSFT barely cares about tooling and windows licenses..

…

On Fri, Sep 9, 2022 at 5:51 AM HyperSphereStudio ***@***.***> wrote: This is going to merely simulate the julia runtime. There are so many differences between this and the official julia runtime. For intance I am not using llvm, I am lowering to MSIL. There entire compilation structure is vastly different. — Reply to this email directly, view it on GitHub <#17 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD74XGJR675UEXC6FJ5HIGTV5MQE3ANCNFSM6AAAAAAQISXSVA> . You are receiving this because you are subscribed to this thread.Message ID: <HyperSphereStudio/JULIAdotNET/repo-discussions/17/comments/3606765@ github.com>

[damian-666]

Btw just want to share a bit more my net 6 experience with science code and why I love it now despite all the changes I suffer though for 5weeks liek everyone else. This kind of tooling rework should not happen again for at least 2 years since msft put net6 on long term support. I updated.my own wpf tooling even with legacy close source modules used for docking, somewhow to net 6..all my own core libs from net standard 2 to net 6 shared core libraries. very painful, but net standard 2 isn't really supported well even if officially still allowed Then I have very thin projects that build thin executable launchers, that target desktop open gl and/ or directx via monogame. Thry do any platform specif windows and settings or whatever and launch a monogame window and that's it.. there are alternative to monogame but it's also supporting consoles so they are very restricted..silk is too new and there's not any better options anyways.. to target Android,.linxu, iOS, windows, osx. There's basically zero other options for graphics. without visualization much of simulator code isnt so accessible. I don't wanna see some statistics on some scientists python flavor code, I wanna take his main project build run and see the phenomenon he modelled. I know the researchers business model is to publish papers. But if u don't buy the paper u won't be able to really understand work the code. So companies hire interns to try type.in the pseudcode.whic can take months. Fortunately formme some CFD legends are professors, that just givemme a students GitHub link and thry did it already. Anyways I don't know if you even need be concerned about visualization for this.

So there are ways to take vbcode in thr Csscipt moduel I think, Microsoft code analyser, andaybe a Julia one is possible to implement.

So I took latest Avalon edit..have code completion, csscipt working.for c#..havent integrated Microsoft code analysis but I can make on thr fly pluging script changes to my interfaced game entities like tweak update physics, and draw methods, on different threads, outside of visual studio, have some debug and inspection, and then my plugins go to a folder so that my production code can pick up my prototype plugin file, compile for deployemtb either AOT or JIT, trim unused, pack , sign, and deploy to Android or linx or whatever. Im using a windows surface for my dev tools because it has a touch screen, to prototype touches.on phones on it, but can target anything like an ipad and hope it works..so I get typed scripting,.code completion, even outside of visual studio, at.least for small plugins, one or two classes..linked to my base classes.. the heavy architecture is of course done in visual studio. Shaders in monogame are transpiled.. I barely use them and looking at silk if decide inwant to do gpgpu .. I dont want to deal with compute shaders I would try thr Silk opencl bindings.. instead if I really need that..

[HyperSphereStudio]

The Hello World of the .NET Julia Runtime!

Console.WriteLine(Julia.Eval(@"
                    struct MyTestType end

                    struct MyImplStruct
                        x::MyTestType
                    end
                    "));

Works!!!

What the above code does is the following:

• Generates Dynamic Main Module
• Parses Script to AST
• Generates Value Type MyTestType -> MAIN
• Generates Value Type MyImplStruct -> MAIN With Instance Field Named x with Type MyTestType
• Generates Dynamic Expr Method

									 object EXPR_#(){
                                            Julia.MAIN.InstantiateType("MyTestType"); //Moves type from precompiled table to name table for dynamic lookup
                                             return Julia.MAIN.InstantiateType("MyImplStruct");
                                      }

• Execution of EXPR_# and return of the type MyImplStruct from Eval

[damian-666]

if you can eval an expression with greek letters for a math looking formula in unicode or whatever, in it, i'll yell from the rooftops about it :)

…

On Sat, Sep 10, 2022 at 3:59 PM HyperSphereStudio ***@***.***> wrote: The Hello World of the .NET Julia Runtime! Console.WriteLine(Julia.Eval(@" struct MyTestType end struct MyImplStruct x::MyTestType end ")); Works!!! What the above code does is the following: Generates Dynamic Main Module Parses Script to AST Generates Value Type MyTestType -> MAIN Generates Value Type MyImplStruct -> MAIN With Instance Field Named x with Type MyTestType Generates Dynamic Expr Method csharp object EXPR_#(){ Julia.MAIN.InstantiateType("MyTestType"); //Moves type from precompiled table to name table for dynamic lookup return Julia.MAIN.InstantiateType("MyImplStruct"); } Execution of EXPR_# and return of the type MyImplStruct from Eval — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD74XGI5N65IAZ22FEU6ZLLV5UACRANCNFSM6AAAAAAQISXSVA> . You are receiving this because you commented.Message ID: ***@***.*** .com>

[HyperSphereStudio]

Do you mean like latex? Because Julia supports greek letters last I checked

[HyperSphereStudio]

If what your looking for is latex then thats not julia, its the ide your using that renders the text

[damian-666]

ok, forgive me i am new to julia and only hope to use it in future... see the julia operators are the same as c# normally but wondering if latex can be used as operators and greed symbols as variable rather than 2PI or whatever i was referring to this https://tng-daryl.medium.com/julia-the-greeks-and-math-operators-at-the-command-line-via-latex-1f4e61b51683 it not possible in c# i dont think.. maybe vs code has an extension... can teh latex be used in actual expressions adn in code? os that when you trace the code you see the math not the * / ( ) ^ we aer all used to? i see a few tings in c# for it but never used it.. ints not super important but would be kind of nice as one typo or operator precedence.. can destroy a simulation and a scientist cant just glace over teh algorithm.. https://symbolics.mathdotnet.com/

…

On Sat, Sep 10, 2022 at 7:37 PM HyperSphereStudio ***@***.***> wrote: If what your looking for is latex then thats not julia, its the ide your using that renders the text — Reply to this email directly, view it on GitHub <#17 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD74XGM7CC7R5WKFUWJDDX3V5UZUTANCNFSM6AAAAAAQISXSVA> . You are receiving this because you commented.Message ID: ***@***.*** .com>

[HyperSphereStudio]

This should not be much of a problem I think. Julia only natively provides support for allowing a wide variety of symbols to be used as function/variable names such as greek symbols. Say you want to implement a integral function:

In latex you do \int{}{}

In julia you have to use an IDE or repl to convert \int to the symbol you want. You cannot just have the raw \int be the function name. Internally it will find the char you want and have the be part of the function name. This does limit the latex support julia can offer though as it would not be able to show you a full pretty integral, rather a single line one that still follows julia syntax. It would take ALOT of effort to enable Latex renderable source code in julia. There are libraries such at Latexify that can convert your code to latex. You just cant run said code after the transformation

[damian-666]

i tried a couple LATEX extensions for the md files and the comments in c# code in vs 2022 using an extension and they dont work as they do on websites or html...in vs 2022, after 20 minutes fussing around i gave up for now.. .. in vscode maybe it works in comments out of the box.. well maybe someday ill have readable comments at least, if not code and algorithms that actually get compiled..

…

On Sat, Sep 10, 2022 at 8:08 PM Damian ***@***.***> wrote: ok, forgive me i am new to julia and only hope to use it in future... see the julia operators are the same as c# normally but wondering if latex can be used as operators and greed symbols as variable rather than 2PI or whatever i was referring to this https://tng-daryl.medium.com/julia-the-greeks-and-math-operators-at-the-command-line-via-latex-1f4e61b51683 it not possible in c# i dont think.. maybe vs code has an extension... can teh latex be used in actual expressions adn in code? os that when you trace the code you see the math not the * / ( ) ^ we aer all used to? i see a few tings in c# for it but never used it.. ints not super important but would be kind of nice as one typo or operator precedence.. can destroy a simulation and a scientist cant just glace over teh algorithm.. https://symbolics.mathdotnet.com/ On Sat, Sep 10, 2022 at 7:37 PM HyperSphereStudio < ***@***.***> wrote: > If what your looking for is latex then thats not julia, its the ide your > using that renders the text > > — > Reply to this email directly, view it on GitHub > <#17 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AD74XGM7CC7R5WKFUWJDDX3V5UZUTANCNFSM6AAAAAAQISXSVA> > . > You are receiving this because you commented.Message ID: > <HyperSphereStudio/JULIAdotNET/repo-discussions/17/comments/3616843@ > github.com> >

[BambOoxX]

In general browsers do not render Latex, they render MathJax, which looks similar in terms of syntax, but is far more limited. Even Julia's Documenter.jl uses this kind or rendering IIRC

[damian-666]

well i tried some snippets from this in C#:

https://github.com/weymouth/DataDrivenGMG.jl/blob/main/src/candidate_smoothers.jl

his code does have Greek symbols ( i suppose with onscreen keyboard i can type them i copy pasted them form his) and i'm using Unicode already i guess so some things work in c#..

float ω = 100f;

        var  v = 2*ω;

        Debug.WriteLine(v.ToString());

but not the subscripts and certainly not the ∈ operator.. maybe i could overload the ∈ using something like that though

it doesn't allow a variable with the subscript character either.
float p₀ = 5.666 is taken as p and then the other thing causes an error.

his math is easier to read than c#.. but not quite the fantasy i had in mind.

as for JIT vs the other compiler toolchain i guess watching the flux form the netcore open sources projects in net foundation will tell you if going under that umbrella will be better or a worthy cause. SIMD, GPGPU in silk stuff like that .. the CodeAnalysis and rosylin is how i'm scripting plugins in a custom visual tool , a single interface, so i can experiment with simulations wihtout rebuilding the whole thing, just a little plugin that goes with some visual object that has an Update and Draw method i can tweak on the fly.

so i think Julia code snippets, would be more welcome than visual basic which is allowed in Rosyln and CodeAnalysiss already and CSScript, which is still allowed as a net language, and f# is very cult. however there remains a huge stigma on .net that the evil msft is gonna burn everyone somehow.

maybe the rest below is ranty and TL/DR.. but its my experience and why i use c#.. even after i left Silicon Valley where i learned it ,and we only supported Windows from 2000, to 2012..

i think since they own Git and sell AI and cloud services, they need our data to train their AIs. so im betting not again, once burned twice shy but i kept with it after a long pause when Windows phone died. windows on ARM is going to come this year runinng Android apps natively..

so i wonder why is C# /.net so ignored in academic /science code even now?
i i just read this... 
https://freecontent.manning.com/why-should-you-program-with-julia/ 

what i dont like about c# its the lamda expression and predicate calculus syntax of c#...  i never got used to it after 10 years.  Still, 10 years using it, i just cant type it off my head...    but it works, 20 years later its finally made good, after burning alot of people. (Nokia, windows phone, etc) and its deployable in real production code on so many platforms. Still,   i just cant type it off my head...  but .net and its deployable in real production code on so many platforms.

so after i read this about Julia i wonder why the 20 + year old c# isnt even mentioned.. JUlia has much of has the raison d'etre of c#.. as it is now, with compile time errors, types and scripting.  in .net, there s possibly more, Interfacing with closed sources "assemblies" binaries was part of the original the driving force,it  closed source  c++ libraries are nearly impossible to link and deal with  versioning unless interfaced, which gave rise to COM, then net.. bu thtat was under Microsoflt only.. or the Silverlight which was killed.. and now netcore is multiplatform.  i stay with c# because i learned it in industry and it it takes so little time to covert c and c++ to c# code , usually, and without unsafe pointers or non-negligible performance hits..  so I use it and probably stick to it.  GC can be paused not for realtime use in robots or audio.   im guessing c# is written off by academics , because it had a Windows taboo up until a few years ago when it got open sourced multiplatfrom . but i learned mostly from industry not acedemia, and still now  c#  gives me both static types, low level memory management if i need it, and scripting plugins thanks to CSScipt and the MS codeanalysis, and now multiplatform .

[HyperSphereStudio]

Here Is the link to the extremely new runtime repo: https://github.com/HyperSphereStudio/juliapp

Alot of work is left but if anyone wants to see the progress / offer any input on it

[damian-666]

i gave it a quick try to build , but on net6... only seem to get a few errors that i can fix like ths...

not sure if net5 is a limit i just didnt want to install it.. i had to remove the global.json retarget to net 6 to try..

[StructLayout(LayoutKind.Explicit)]
public struct Union<T1,T2> where T1 : struct where T2 : struct{
**[FieldOffset(0)]**private byte type;
[FieldOffset(1)] private T1 a1;
[FieldOffset(1)] private T2 a2;
public T1 A1 {
get => a1;
set {
a1 = value;
type = 1;
}
}

	public T2 A2 {
		get => a2;
		set {
			a2 = value;
			type = 2;
		}
	}

[HyperSphereStudio]

Yeah I just posted the repo so the progress could be tracked... It has so much work left to be done.

For now I am going to be focused on building the JIL interpreter. This interpreter will read Julia Intermediate Language and be able to run it. The next stage of development is the JIL MSIL compiler which will lower the JIL to MSIL.

I think I will keep the target to .NET 6. This may change to a higher version though as .NET introduces new features.

I have no documentation for it right now so if anyone does want to help I can provide some of my current designs and thoughts but right now I just posted for versioning for myself

[HyperSphereStudio]

*JIL is a form of julia created by the Parser. It is a compressed form of the language as a model that can be used for interpretation

[damian-666]

Net 6 is where all the best optimizatiom came in. My frame rate jumped 40 percemr with a simple rebuild. It's not my area of expertise but I hope something in miscosoft vaults can be of use...I didn't see any references to anthing in code anysis or.existing parsers.. transpilers or such.. I'm working with thr cscscript guy to do a light python like parser plugin with autocomplete just for new c# and it's really hard. But this is at app and UI level..thr compiler woriks it's just to slow for an ide or autocompelte. And complex autocomplete and ide stuff..u might see how Microsoft code analysis also deals with vb code in a general way...or maybe since u have IL already , ur way from scratch is most.appropriate .and u don't need any of this..since there are ide already. he dropped vb support also from his parser ... So if u can build, export net assembly.from Julia it's already useful because python can't be deployed as production code..if u tuned something I guess u have to code it in the core in c++ or laoded it as a parameter file... But for a pde solver I'd wanan ship the Julia code as a Net assembly when the rapid prototype is working ..just export pack and deploy it.

…

On Tue, Sep 13, 2022, 6:20 PM HyperSphereStudio ***@***.***> wrote: *JIL is a form of julia created by the Parser. It is a compressed form of the language as a model that can be used for interpretation — Reply to this email directly, view it on GitHub <#17 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD74XGNNPHJQ7QQ2XMZT4WTV6EK3ZANCNFSM6AAAAAAQISXSVA> . You are receiving this because you commented.Message ID: ***@***.*** .com>

[HyperSphereStudio]

Quick update on my julia runtime in .net project.

Made some pretty good progress! Very far from a working (able to actually run julia code) though. It is quite a massive project that will take a while. I have load out a plan in the readme of the project. Take a look if your curious and/or interested in helping out! This will take a team effort to complete but I think will be worth it. I call it Julia++ as it would be more a fork of Julia since its not official at all.

[BambOoxX]

Wow, I am totally unable to judge your (seemingly very thorough) work.y main concerns are:

• do you think there are a lot of people out there that could help with cush tasks?
• Did you manage to discuss with the core Julia team about this? What is their view on hese developments?
• assuming the base development of Julia ++ is done won't such a development require a lot of maintenance effort to follow Julia update pace?

Still, thanks for the hard work!