Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails on macOS arm64 with net472 target #49

Closed
paulirwin opened this issue Jan 25, 2024 · 8 comments
Closed

Build fails on macOS arm64 with net472 target #49

paulirwin opened this issue Jan 25, 2024 · 8 comments

Comments

@paulirwin
Copy link

Working on testing apache/lucenenet#892 with @NightOwl888, and I cannot build a project that targets net472 on macOS arm64 that uses MavenReference.

While that PR branch reproduces it, it is quite large, so here is a minimal reproduction.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net472</TargetFramework>
    <RootNamespace>Bug</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="IKVM.Maven.Sdk" Version="1.6.7" />
  </ItemGroup>
  
  <ItemGroup>
    <MavenReference Include="org.apache.opennlp:opennlp-tools" Version="1.9.4" />
  </ItemGroup>
  
  <ItemGroup>
    <None Update="en-sent.bin">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>

</Project>

(Note that the en-sent.bin bit is only needed if you want to actually run this example to see it working on a different target like .NET 6+. Get the file from OpenNLP and put in directory.)

using System;
using opennlp.tools.sentdetect;

namespace Bug
{
    public class Program
    {
        public static void Main(string[] args)
        {
            using var modelIn = new java.io.FileInputStream("en-sent.bin");

            var model = new SentenceModel(modelIn);
            var sentenceDetector = new SentenceDetectorME(model);

            var sentences = sentenceDetector.sentDetect("  First sentence. Second sentence. ");
            Console.WriteLine(string.Join(";", sentences));
        }
    }
}

I have put this in a gist if it is easier to clone it.

The build output is (paths replaced with $HOME):

$HOME/.nuget/packages/ikvm/8.7.3/buildTransitive/IKVM.Tasks.targets(31,9): error MSB4018: The "IkvmCompiler" task failed unexpectedly. [$HOME/git/ikvm-maven-reference-bug/ikvm-maven-reference-bug.csproj]
$HOME/.nuget/packages/ikvm/8.7.3/buildTransitive/IKVM.Tasks.targets(31,9): error MSB4018: IKVM.MSBuild.Tasks.IkvmTaskException: Missing tool path: '$HOME/.nuget/packages/ikvm.msbuild.tools.runtime.osx-arm64/8.7.3/buildTransitive/../ikvmc/net472/osx-arm64/'. [$HOME/git/ikvm-maven-reference-bug/ikvm-maven-reference-bug.csproj]
$HOME/.nuget/packages/ikvm/8.7.3/buildTransitive/IKVM.Tasks.targets(31,9): error MSB4018:    at IKVM.MSBuild.Tasks.IkvmToolExecTask.Execute() in /_/src/IKVM.MSBuild.Tasks/IkvmToolExecTask.cs:line 79 [$HOME/git/ikvm-maven-reference-bug/ikvm-maven-reference-bug.csproj]
$HOME/.nuget/packages/ikvm/8.7.3/buildTransitive/IKVM.Tasks.targets(31,9): error MSB4018:    at IKVM.MSBuild.Tasks.IkvmCompiler.Execute() in /_/src/IKVM.MSBuild.Tasks/IkvmCompiler.cs:line 239 [$HOME/git/ikvm-maven-reference-bug/ikvm-maven-reference-bug.csproj]
$HOME/.nuget/packages/ikvm/8.7.3/buildTransitive/IKVM.Tasks.targets(31,9): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [$HOME/git/ikvm-maven-reference-bug/ikvm-maven-reference-bug.csproj]
$HOME/.nuget/packages/ikvm/8.7.3/buildTransitive/IKVM.Tasks.targets(31,9): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [$HOME/git/ikvm-maven-reference-bug/ikvm-maven-reference-bug.csproj]
    0 Warning(s)
    1 Error(s)

Note that the use case here is not to be able to run .NET Framework 4.7.2 code on macOS arm64, but merely to build.

@wasabii
Copy link
Collaborator

wasabii commented Jan 25, 2024

Yeah. We do not have the ability to run builds targeting NetFX without having NetFX.

It's on the list.

@NightOwl888
Copy link
Contributor

Just FYI, I was able to do the build on a macOS x64 build agent, but it doesn't work on arm64. Our project includes a private reference to Microsoft.NETFramework.ReferenceAssemblies so the bits to build exist on the build machine.

@paulirwin
Copy link
Author

How is that working on macOS x64? Just curious.

@wasabii
Copy link
Collaborator

wasabii commented Jan 25, 2024

I'm actually not sure either. Technically it should try to run the framework version of ikvmc..... Which if you have mono installed would try to load it. But then probably fail because it's mono

@NightOwl888
Copy link
Contributor

So what is "officially" supported for building NetFX? Windows only? At present we are doing official builds on Windows x64, so that is fine.

But to merge the pull request, we need to get around the issue with contributors being able to build from source who may be using various platforms to do the build. We can conditionally exclude the net472 target in order for people to contribute, but should the condition be on ARM64 or should we just assume it doesn't work if they are not on Windows?

NightOwl888 added a commit to NightOwl888/lucenenet that referenced this issue Jan 25, 2024
… on Windows due to lack of non-Windows build support in IKVM 8.7.3 (see: ikvmnet/ikvm-maven#49).
@wasabii
Copy link
Collaborator

wasabii commented Jan 25, 2024

The Framework ikvmc.exe executable can't run unless Framework is available. So, that means Windows (or Wine or working Mono or something). So, mostly, you can't run that except on Windows.

And the Framework ikvmc.exe is required to produce Framework assemblies.

So, you can only produce Framework assemblies on Windows.

The stuff I'm doing now is getting us closer (fixing up IKVM.Reflection, etc).

@wasabii
Copy link
Collaborator

wasabii commented Jan 25, 2024

I think there's only one major issue remaining though, in develop, which is that the determination of what counts as the "core lib" is hard coded. The corelib being one of "mscorlib", "System.Runtime" or "netstandard". Basically it needs to know what assembly to look in for System.Object and all that.

With the SRME stuff, we're going to be capable of emitting Framework executables targeting any platform (ARM64, etc) using the Core executable. So, that means the ARM64 Core version of ikvmc will be able to run on OSX and produce Framework executables targeting any arch.

Once it's able to do that, I'm going to have to reevaluate the MSBuild scripts. Right now they determine which IkvmTool to run based on the framework/runtime target and current OS version. That'll need to be changed. Probably to run using the tool corresponding with the current SDK environment (Core SDK or Framework), and ignore the target environment. And then communicate the corelib name to the tool somehow.

NightOwl888 added a commit to apache/lucenenet that referenced this issue Feb 1, 2024
…ng MavenReference (#892)

* upgrade targets to target .net core 6, in addition to .net framework

* update net 4.6 version

* Lucene.Net.Tests.OpenNLP: Patched IDE behavior to use net48 when net461 is selected and net7.0 when net5.0 is selected. In CI, we set IsTestProject=false and IsPublishable=false to skip these tests.

* publish-test-results-for-test-projects.yml: Added support for net7.0 and net6.0 for Lucene.Net.Tests.Analysis.OpenNLP tests.

* .github/workflows: Regenerated to add net7.0 as a test framework for Lucene.Net.Tests.Analysis.OpenNLP

* .build/dependencies.props: Upgrade System.Memory to 4.5.5 to match IKVM 8.2.0

* .build/dependencies.props: Bumped System.Runtime.CompilerServices.Unsafe to 6.0.0 to match IKVM 8.5.0

* Lucene.Net.csproj: Added direct dependency on System.Runtime.CompilerServices.Unsafe for netstandard2.0 and net462 to ensure the version will work with any combination of Lucene.Net components. This is a transitive dependency in a few 3rd party DLLs, but there may be version conflicts if this isn't done on .NET Framework.

* Lucene.Net.Facet.csproj: Added explicit dependency on System.Memory for netstandard2.0 and net462, since it is being used in Lucene.Net.Facet.

* Lucene.Net.TestFramework.csproj: Added dependency on System.Text.Json to pin the version so it matches the reference of IKVM 8.5.0 (6.0.6).

* upgrade targets to target .net core 6, in addition to .net framework

* Lucene.Net.Analysis.OpenNLP.csproj, .build/dependencies.props: Changed to use <MavenReference> to build opennlp-tools instead of using the pre-built OpenNLP.NET NuGet package.

* .build/dependencies.props: bumped IKVM to 8.7.3 and IKVM.Maven.Sdk to 1.6.7

* Lucene.Net.Analysis.OpenNLP.csproj: Removed duplicate TargetFrameworks declaration

* Lucene.Net.Analysis.OpenNLP: Changed target from net462 > net472, the minimum supported by IKVM.

* Directory.Build.targets: Updated FEATURE_OPENNLP to be available on .NET Core

* Lucene.Net.Tests.AllProjects: Updated references so we can successfully compile with IKVM in the mix, both on .NET Framework and .NET Core

* .build/dependencies.props: Added OpenNLP MavenReference version so it can be managed with the other packages

* Lucene.Net.Analysis.OpenNLP: Added Maven dependency on org.osgi.core to eliminate build warnings (at least 1 type is referenced in opennlp-tools)

* Lucene.Net.Analysis.OpenNLP: For now, making net472 conditional based on Windows due to lack of non-Windows build support in IKVM 8.7.3 (see: ikvmnet/ikvm-maven#49).

* .build/dependencies.props: Reverted back to OpenNLP 1.9.1 because of build issues with opennlp-uima on 1.9.4. This aligns with Lucene 8.2.0.

* publish-nuget-packages.yml: Remove forward slash

* .build/dependencies.props: Bumped IKVM to 8.7.5

* Lucene.Net.Analysis.OpenNLP/overview.md: Added missing docs from Lucene and link to MavenReference demo. Fixes #890.

* FEATURE: Lucene.Net.Analysis.Miscellaneous: Added TypeAsSynonymFilter from Lucene 8.2.0 because it is called out in the docs as part of the process of configuring Lucene.Net.Analysis.OpenNLP. Changed CannedTokenStream to set ITypeAttribute.Type because it is required by the tests for TypeAsSynonymFilter.

* Lucene.Net.Analysis.Miscellaneous.TestTypeAsSynonymFilterFactory: Added comment with lucene version compatibility level (to indicate we ported it from Lucene 8.2.0)

* Lucene.Net.Analysis.OpenNLP.overview.md: Corrected information about which filters are included in the package (there is no NER filter in the box)

---------

Co-authored-by: Laimonas Simutis <[email protected]>
@wasabii
Copy link
Collaborator

wasabii commented Jul 12, 2024

This actually should work now if you have Mono installed. With proceeds on getting the Core executable running. But the Framework one did, with mono. So it's doable.

@wasabii wasabii closed this as completed Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants