c-library dependency of linux RIDs

[tmds]

Lifted from https://github.com/dotnet/coreclr/issues/5853#issuecomment-275344009

What should be the rid of the portable linux runtime and libraries with a glibc dependency?

Perhaps it makes sense to introduce a linux-glibc-x64 rid for this?

This would:

• make it clear what rid other c libraries could use: e.g. linux-musl-x64
• make explicit the glibc dependency of the runtime
• make explicit the glibc dependency of the libraries

\cc @janvorli @jkotas @gkhanna79

[tmds]

Placing linux-glibc-x64 in simplified ubuntu.14.04-x64 graph (https://github.com/dotnet/coreclr/issues/9210):

base <- any <- linux                                       // pure managed libraries
                 ^- linux-x64   <- linux-glibc-x64         // library with native bits
                                       ^- ubuntu.14.04-x64 // runtime end-point

[tmds]

ping @janvorli @jkotas @gkhanna79 @ellismg @Petermarcu @ericstj

[janvorli]

It seems that it would make sense to have the C library (glibc, musl, possibly others in the future) in the RID graph, since binaries built targetting one of these libraries cannot be used on systems with the others. Originally, I was thinking that the glibc would be "implicit" (so we would have linux-x64 - meaning linux-glibc-x64 - and linux-musl-x64 as children of linux). But now seeing the larger picture, it seems it would be confusing.
But logically, also the "unix" node should be split by the C library, as @am11 has commented here: https://github.com/dotnet/coreclr/issues/9210#issuecomment-276922217.
@gkhanna79, @ericstj, @Petermarcu what do you think?

[tmds]

Adding unix-glibc, linux-glibc to the picture:

<- unix <- unix-glibc                                // pure managed libraries
     ^- linux <^- linux-glibc                        // pure managed libraries
          ^- linux-x64   <^- linux-glibc-x64         // library with native bits
                                 ^- ubuntu.14.04-x64 // runtime end-point

also the "unix" node should be split by the C library, as @am11 has commented here

I read this more as a workaround and not as a feature request. It is not clear for me when someone would target unix-glibc.

[janvorli]

@tmds I meant it for FreeBSD, NetBSD and other non-Linux flavors.

[am11]

@tmds, it was supposed to be linux-glibc (unix was a typo). From RID inheritance point of view, it needs some work (reason I am supporting this feature request). However, for our current workaround, given a typical RID structure: Platform ; Version ; Architecture, we have simply appended Platform with 'C' lib to form Platform+libcName ; Version ; Architecture (and more generally, just Platform+libcName ; Architecture).

[tmds]

@tmds, it was supposed to be linux-glibc

That's how I read it :)

@tmds I meant it for FreeBSD, NetBSD and other non-Linux flavors.

e.g. freebsd-libc, netbsd-libc, osx-libc? (libc refers to the BSD c-library)
like this:

<- unix                                              // pure managed libraries
     ^- linux <- linux-glibc
          ^- linux-x64   <^- linux-glibc-x64         // library with native bits
                                 ^- ubuntu.14.04-x64 // runtime end-point
     ^- freebsd <- freebsd-libc
          ^- freebsd-x64 <^- freebsd-libc-x64
                                 ^- freebsd.11-x64

[janvorli]

I was really thinking that the libc would be only in the parent type (unix-libc). I think that e.g. FreeBSD or NetBSD etc. are always based on libc and there are no alternative forks using different libraries. So including the libc in the RID name for FreeBSD is redundant. But I am sure there are other Unix versions that we may support in the future and that use different C library.

[tmds]

I was really thinking that the libc would be only in the parent type (unix-libc). I think that e.g. FreeBSD or NetBSD etc. are always based on libc and there are no alternative forks using different libraries. So including the libc in the RID name for FreeBSD is redundant. But I am sure there are other Unix versions that we may support in the future and that use different C library.

Like this?

<- unix                                              // pure managed libraries
     ^- linux <- linux-glibc
          ^- linux-x64   <^- linux-glibc-x64         // library with native bits
                                 ^- ubuntu.14.04-x64 // runtime end-point
     ^- unix-libc
          ^- freebsd
               ^- freebsd-x64
                      ^- freebsd.11-x64

Does a unix-libc-x64 make sense? I.e. can native binaries be shared between freebsd, netbsd, etc?

[Petermarcu]

I think we need to pick a default that people get when they say "linux-x64". There is a tension between what perfectly represents the possibilities and a simple user experience when targeting linux. I want to have "linux-x64" mean something that runs on most linux's and we can document that if people want to run on a linux that isn't glibc, they can specify a more explicit rid.

So, I think we should keep linux-x64 to mean glibc. This makes it a little less pretty for our RID graph but a lot simpler for a large set of end users targeting Linux.

[janvorli]

@tmds yes, that's what I meant. As for the unix-libc-x64, I really don't know if the binaries would be shareable. It really depends on the platforms. They may possibly use different executable format.

[tmds]

So, I think we should keep linux-x64 to mean glibc. This makes it a little less pretty for our RID graph but a lot simpler for a large set of end users targeting Linux.

@Petermarcu does that mean, something like this?
(linux-glib-x64, linux-x64 become linux-x64, linux-base-x64 respectively)

              <- linux-glibc   <v- linux-x64       <- ubuntu.14.04-x64
unix <- linux <------------- linux-base-x64
              <- linux-musl    <^- linux-musl-x64  <- alpine.x.y-x64

[Petermarcu]

I hadn't thought of adding linux-x64 as a child of linux-glibc but that may be an interesting way to do it. This is more of what I was thinking where glibc was basically implied in the branch of the tree that wasn't specific to non-glibc families. Do you see pro's/con's of either approach?

              <- linux-x64       <- ubuntu.14.04-x64
unix <- linux <------------- linux-base-x64
              <- linux-musl    <^- linux-musl-x64  <- alpine.x.y-x64

[tmds]

Do you see pro's/con's of either approach?

No cons, pro is consistency.

What do you think about using the same naming scheme for all branches and introduce a new RID for portable linux, e.g. linux-portable-x64?

                                                   <- linux-portable-x64
              <- linux-glibc   <v- linux-glibc-x64 <- ubuntu.14.04-x64
unix <- linux <------------- linux-x64
              <- linux-musl    <^- linux-musl-x64  <- alpine.x.y-x64

[Petermarcu]

Not as big a fan of that for a couple reasons. Portable is an overloaded word and because I am also hoping we can create a "portable" story for musl and other c library families as well.

It sounds like you feel that your proposal is more "consistent". Is that because it introduces architecture at the same level of the tree rather than one level higher for glibc?

To me the rids that have no architecture (x64 in this example) aren't real. Meaning, it's very unlikely that they would ever be used in a concrete case. They are only for tying the tree together. Am I wrong in that thinking?

I was talking to @janvorli earlier. In the end, I don't have a super strong opinion of whether linux-x64 is under linux-glibc or not. Nobody should ever specify linux-glibc and the ecosystem will not have any nuget packages with concrete assets in them for that rid so I am not sure why we need it in there but if it makes it easier for us as rid graph maintainers to see it there, that's really the only argument I can see for having it.

As long as in the end, people specify linux-x64 to get x64 glibc based assets from packages, I'm fine :)

[tmds]

Not as big a fan of that for a couple reasons. Portable is an overloaded word and because I am also hoping we can create a "portable" story for musl and other c library families as well.

My aim was to provide an alternative where linux-x64 did not imply glibc, but which would seem user-friendlier since a developer would use linux-portable-x64.

It sounds like you feel that your proposal is more "consistent". Is that because it introduces architecture at the same level of the tree rather than one level higher for glibc?
To me the rids that have no architecture (x64 in this example) aren't real. Meaning, it's very unlikely that they would ever be used in a concrete case. They are only for tying the tree together. Am I wrong in that thinking?

There are use-cases for these RIDs. But I they are perhaps a little to far-fetched to include these RIDs from the start. So, they can be left out and introduced later if there is a strong case for them.
The same applies to the RID unix-libc.

Leaving out the architecture-less c-library RIDs:

                <----v-- linux-x64       <- ubuntu.14.04-x64
unix <- linux   <- linux-base-x64
                <----^-- linux-musl-x64  <- alpine.x.y-x64

     <- freebsd <----- freebsd-x64       <- freebsd.11-x64

As long as in the end, people specify linux-x64 to get x64 glibc based assets from packages, I'm fine :)

I understand the desire to tie linux-x64 to glibc. The main use-case here is when a developer publishes his app, right?

dotnet publish --runtime linux-glibc-x64

vs

dotnet publish --runtime linux-x64

When linux-x64 implies glibc, some things will need to change in the RID graph. For example, Alpine inherits linux-x64 atm. Alpine uses musl (and before 2014, it was using uclibc.)

[Petermarcu]

Yeah, it's both what they type on publish and what they put in their project files. Yeah, we definitely need to move Alpine :)

[tmds]

@Petermarcu 2.0 milestone?

[mellinoe]

This issue also came up regarding RIDs for Android, and whether or not they should inherit from the generic linux-xyz rid. dotnet/corefx#16729

[am11]

With Android, bionic-libc would be the fourth supported standard libc after win-crt, glibc and musl-libc. kernelName (BSD, Darwin, Linux, Win32) + libcName (Bionic, BSD libc, CRT, glibc, musl-libc) + libcVersion + architecture would form versatile runtime identifier than binding it with OS name.

[tmds]

@gkhanna79 @Petermarcu not fixing this for 2.0?

[tmds]

ping @janvorli @jkotas @gkhanna79 @ellismg @Petermarcu @ericstj
I am not sure why this is pushed out of the 2.0 milestone. The new RIDs added to the graph become part of the public API at 2.0. So it seems worth solving this for 2.0?

[gkhanna79]

So far, this has been a discussion but is there any concrete plan around the work to be done? If so, can you please point to the PRs?

I am not aware of any changes happening for RID fallback graph for this.

[tmds]

@gkhanna79 We need a definite answer whether non-glibc based linux distros should move from linux-x64 to their own linux-x64-<clib>-rids. If so, the plan is to move them.

[gkhanna79]

Can you elaborate on what it means "move" them? Is it about updating the RID fallback graph to reflect so?

[tmds]

Can you elaborate on what it means "move" them? Is it about updating the RID fallback graph to reflect so?

Exactly.

[tmds]

@mellinoe removed the alpine RIDs from the graph: dotnet/corefx#18948
There are no more non-glibc based linux distros that inherit linux-x64 (android inherits any).
The issue can be fixed post 2.0.

[kleisauke]

FWIW, introducing the linux-glibc* RIDs would resolve the issue mentioned in dotnet/sdk#4195 (comment).

graph TD;
    unix --> linux;
    linux --> linux-glibc & linux-x64 & linux-arm64 & linux-musl;
    linux-glibc --> linux-glibc-x64 & linux-glibc-arm64;
    linux-musl --> linux-musl-x64 & linux-musl-arm64;
    linux-x64 --> linux-glibc-x64 & linux-musl-x64;
    linux-arm64 --> linux-glibc-arm64 & linux-musl-arm64;

Loading