Skip to content

Latest commit

 

History

History
76 lines (47 loc) · 3.51 KB

ai21.md

File metadata and controls

76 lines (47 loc) · 3.51 KB
Raw

AI21 Studio

AI21 Studio

AI21 Studio is a platform developed by AI21 Labs that provides developers with access to powerful language models like jamba-instruct through APIs. These models enable various text generation and comprehension features for numerous applications. AI21 Studio emphasizes flexibility, allowing developers to fine-tune models for specific tasks, and cost-effectiveness, with its unique tokenization that offers more text per token compared to other providers. Their comprehensive suite includes tools for summarization, paraphrasing, grammar correction, and more, making it a valuable resource for developers seeking to integrate advanced language capabilities into their projects.

Interface Name

  • ai21

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'ai21': process.env.AI21_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('ai21', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: jamba-instruct
  • large: jamba-instruct
  • small: jamba-instruct
  • agent: jamba-instruct

Embeddings Model Aliases

  • Model aliases not provided. Please check the AI21 Studio documentation for more information.

Options

The following parameters can be passed through options.

  • logprobs: Includes the log probabilities of the most likely tokens, providing insights into the model's token selection process.
  • max_tokens: The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
  • n: Specifies the number of responses to generate for each input message. Note that costs are based on the number of generated tokens across all choices. Keeping n as 1 minimizes costs.
  • stop: Up to 4 sequences where the API will stop generating further tokens.
  • stream: If set, partial message deltas will be sent, similar to ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.
  • temperature: Controls the randomness of the AI's responses. A higher temperature results in more random outputs, while a lower temperature makes the output more focused and deterministic. Generally, it is recommended to alter this or top_p, but not both.
  • top_logprobs: Details not available, please refer to the LLM provider documentation.
  • top_p: Controls the cumulative probability of token selections for nucleus sampling. It limits the tokens to the smallest set whose cumulative probability exceeds the threshold. It is recommended to alter this or temperature, but not both.

Features

  • Streaming
  • Embeddings

Getting an API Key

Commercial with Free Trial: The AI21 API is a commercial product but offers a free trial with $90 in credits. No credit card is required initially.

To get an API key, first create an AI21 Studio account, then visit the link below.

AI21 Studio Documentation

AI21 Studio documentation is available here.