Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

need to compile your own version of ONNXRuntime #1

Open
flutter-painter opened this issue Nov 18, 2023 · 10 comments
Open

need to compile your own version of ONNXRuntime #1

flutter-painter opened this issue Nov 18, 2023 · 10 comments

Comments

@flutter-painter
Copy link

Hi Nick,

Thank you for inviting me here and providing precious guidelines.
I am stuck at compiling ONNXRuntime
I gathered that it must be fetched here : https://github.com/microsoft/onnxruntime
And the built using the script.
But it failed both for ios and android for me. I don't doubt that persistence and issue opening will solve this.
Yet I fail to understand why we needs to compile our own versions.

Wouldn't it be possible to use the same runtimes as yours ?
If they are too heavy for git, how about hugging face or a good old gdrive ?

@flutter-painter
Copy link
Author

As you know that there are flutter packages for onnx using ffi, such as :

I suppose that they cannot be used here, but who knows..

@nmfisher
Copy link
Owner

nmfisher commented Nov 19, 2023 via email

@flutter-painter
Copy link
Author

Hi Nick,
Thank you for the quick reply,
Understood, I understand this will be needed eventually, but for now stick to running the example.

Cloning flutter_onnx repository fixed it indeed, no more complaints about the missing ONNX runtime.
Note I just had to edit the framework path in ios/flutter_sherpa_onnx.podspec with
"FRAMEWORK_SEARCH_PATHS" => '"${PODS_ROOT}/../../../flutter_onnx/ios/lib" "$(inherited)"',

Yet now I get "undefined symbol" error from xcode.
Hundreds of error on a real device, like :
Undefined symbol: std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >::find(char, unsigned long) const
Only that one on the emulator : Undefined symbol: ___gxx_personality_v0

Any ideas on how to solve them ?
Will pursue test on android later this week..

@nmfisher
Copy link
Owner

nmfisher commented Nov 20, 2023 via email

@flutter-painter
Copy link
Author

Version 13.2.1 (13C100), I presume it is too low...
Will pursue on android soon, I haven't had the time yet

@nmfisher
Copy link
Owner

Yes, the precompiled ONNXRuntime I gave was compiled with 14.3.1, so you probably need to build the Flutter project with that version.

I strongly suggest you figure out how to build your own ONNXRuntime, that would also solve your problem.

@flutter-painter
Copy link
Author

Hi Nick,

Sorry for the delay in my reply,
Blame it on tensorflow lite and this specific issue : tensorflow/flutter-tflite#41
Meaning using a translation model such as openNMT is not possible on Flutter.
As a result I am reverting to native Android to get the project rolled out.
I may pursue the part in Flutter later but will stick to android for now,
Thanks for your support and sorry for not pushing this further,
This could be useful to many as it is

@dkjazz
Copy link

dkjazz commented Mar 3, 2024

I pulled all the required repos and followed @flutter-painter changes as welll and getting an error when trying to run the example (when tapping CreateRecognizer)

[ERROR:flutter/runtime/dart_isolate.cc(1107)] Unhandled exception:
Invalid argument(s): Failed to lookup symbol 'CreateOnlineRecognizer': dlsym(RTLD_DEFAULT, CreateOnlineRecognizer): symbol not found
#0      DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:33:70)
#1      NativeLibrary._CreateOnlineRecognizerPtr (package:flutter_sherpa_onnx/generated_bindings.dart:34:67)
#2      NativeLibrary._CreateOnlineRecognizerPtr (package:flutter_sherpa_onnx/generated_bindings.dart)
#3      NativeLibrary._CreateOnlineRecognizer (package:flutter_sherpa_onnx/generated_bindings.dart:36:40)
#4      NativeLibrary._CreateOnlineRecognizer (package:flutter_sherpa_onnx/generated_bindings.dart)
#5      NativeLibrary.CreateOnlineRecognizer (package:flutter_sherpa_onnx/generated_bindings.dart:26:12)
#6      FlutterSherpaOnnxFFIIsolateRunner._onCreateRecognizerCommandReceived (package:flutter_sherpa_onnx/flutter_sherpa_onnx_ffi_isolate.dart:154:24)
#7      _RootZone.runUnaryGuarded (dart:async/zone.dart:1594:10)
#8      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#9      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#10     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:784:19)
#11     _StreamController._add (dart:async/stream_controller.dart:658:7)
#12     _StreamController.add (dart:async/stream_controller.dart:606:5)
#13     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

I'm using Version 15.2. Do you have any suggestions on how to solve it?

@nmfisher
Copy link
Owner

nmfisher commented Mar 4, 2024

@dkjazz you probably need to turn off dead code stripping and/or change "Strip Style" to "Non-global symbols".

Does the example project work?

@Ityulei
Copy link

Ityulei commented May 15, 2024

Hello, my side is currently not working.
201715762136_ pic_hd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants