Set jiter as optional dependency to support pyodide (~3 lines diff)

[CNSeniorious000]

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

• This is a feature request for the Python library

Describe the feature or improvement you're requesting

Pyodide currently don't support jiter. openai-python use it for partial json parsing. But it is just used in 2 lines.

If we move jiter into optional-dependencies, we will be able to use openai-python in pyodide runtime.

Once upon a time, httpx is blocking openai from pyodide too. But that issue is already resolved. The only barrier is jiter now.

About openai, pyodide and httpx

I've checked these issue:

• Support for Pyodide #815
• Issue with custom transport #960

At that time, openai is not compatible with pyodide because of httpx.
Now there even exist a pyodide-httpx to patch httpx in pyodide

If we can use openai in pyodide, it will be possible to provide interactive python demos in the browser for prompt engineering frameworks, which I think is a great feature to have.

Additional context

Another way is to use another package to parse partial json. There is a package called partial-json-parser which did the almost same job as jiter.from_json, but also providing more flexibility on specifying which types are allowed to be incomplete. And it keep types too. For the latter one, let me present an example:

from jiter import from_json
from_json(b'{"a": [1', partial_mode=True)  # {'a': [1]}
from_json(b'{"a": [1.', partial_mode=True)  # {'a': []}

In the example above, tokens increase but parsed value disappeared.

Plus, partial-json-parser's API is consistent among its Python/JavaScript/Go implementations.

I tried a bit to replace jiter by partial_json_parser:

7419b70#diff-08dc4c3c3e8e145eec1fd0b6a4577f0bce73567d4da3460e08dd4c2d34b27915

[RobertCraigie]

Thanks for the report, would it be enough to just lazily import jiter instead? Or does simply listing it in dependencies cause issues?

Additionally, have you opened an issue with jiter to see if the pydantic team can do anything to make it Pyodide compatible? I'm sure they'd be interested in making that work.

[CNSeniorious000]

Listing it in dependencies should still cause issues. Because installing openai will try to install its dependencies, and jiter is non-optional dependency of openai. Pyodide only supports pure-python wheels and emscripten wheels, but jiter don't have any of these, so resolving jiter will fail, causing failure on resolving openai.

Thanks for advices. I've opened an issue with jiter:

• Can you build an emscripten wheel to support pyodide runtime? pydantic/jiter#151

[anointingmayami]

This is great.

Integrating with Pyodide would allow the OpenAI library to be used in web applications without needing a backend server. This could open up new opportunities for educational tools, interactive demos, and user-driven applications that leverage the OpenAI API.

The innovation could provide significant benefits, especially for web-based applications, but it requires a thoughtful approach to assess compatibility, potential costs, and benefits. The total cost will vary based on the project's scope, the existing codebase's complexity, and the resources available for development. Planning and phased implementation may be beneficial to manage these efforts effectively.

Give us some time to review this update.

Furthermore, could you specify how you would like to use Pyodide in OpenAI?

[CNSeniorious000]

There are some personal factors, such as I am more familiar with Python, so most of my prompt engineering is written in Python. I am working on an educational tools and interactive demos platform, so I integrates with pyodide a lot. Another reason is the ability to use Pyodide as a code interpreter, providing it as a tool for the LLM. (Here is a prototype)

And, I am planning to provide interactive docs for every supported python library, with LLM copilot built-in, which is a long-term target.

[anointingmayami]

This happen to be the engagement we are looking for in this AI Generation. According to the report release by IBM Seven Bets, we are considering Gen AI as the optimal breakthrough in the AI commercialization to build sustainability and profitability using Data to People (D2P) Model.

[maxschulz-COL]

Hey guys 👋

Stumbled on this, and there is a built wheel of jiter can be found at https://py.cafe/maartenbreddels/jiter-demo.

This in turn allows openai to run in pyodide (and by extension amazing tools like PyCafe). See here the toy example where I am using openai in a Vizro app to generate nice looking charts: https://py.cafe/maxi.schulz/vizro-ai-UI

Maybe that helps?

[CNSeniorious000]

Wow great! Can you upstream that wheel to pyodide so that others can install it directly?

[narumichiaki]

To use jiter's wheel with Pyodide, it seems that the version of Emscripten used to compile jiter must be the same as the one used to compile Pyodide. py.cafe explains how to build jiter into wheel, and and this method worked fine for me.
https://py.cafe/docs/howto/build
I will also share the Dockerfile I used to compile it.
https://gist.github.com/narumichiaki/0cb32c5b6db64493d19066a9ca3aeccb