Skip to content

Latest commit

 

History

History

batch_processing

Introduction

This folders contains instructions and example code for using OpenAI's batch API to process large amounts of data with reduced cost. The idea is simple, you put all your prompts in a file, upload the file to OpenAI's server, and wait for up to 24 hours the responses (my tests show that it's much faster than that actually, but this might change).

Here, I'm going to use the same sentiment analysis example for demonstration. Since the operation requires a lot of interactive commands, I'll use a Jupyter notebook this time. You can check it out here.

Note that OpenAI's website provides UI to control the batch processing. So you could use a script to generate the batch file (the prompts), and then use the website UI to upload the file, create batch job, check the status, and download the results. Here is a script to help you create the batch file.

Finally, OpenAI has a nice demonstration on using the batch API, and you should check it out.