AWS Bedrock support

[mnicstruwig]

Hi Jack 👋 ,

Probably a long-shot (given the large API differences), but are there any plans to support AWS Bedrock?

Given Amazon's investment into Anthropic, we've seen an uptick in interest in using Bedrock to run LLMs on their existing cloud infrastructure, and we'd like to continue using magentic.

Probably something better for Litellm?

[jackmpcollins]

You could try it via the LitellmChatModel since they seem to support it https://docs.litellm.ai/docs/providers/bedrock

From the docs here https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html , the request (including messages) and response are different between the different models hosted on Bedrock. So magentic would need to know how to map from/to each of these, which would probably be easiest as a different ChatModel for each.

If Anthropic on Bedrock is all you need, you could create a copy of the AnthropicChatModel and just replace the Anthropic client with AnthropicBedrock client. https://docs.anthropic.com/en/api/claude-on-amazon-bedrock#making-requests
here:

magentic/src/magentic/chat_model/anthropic_chat_model.py

Lines 315 to 318 in d7ae5c2

	self._client = anthropic.Anthropic(api_key=api_key, base_url=base_url)
	self._async_client = anthropic.AsyncAnthropic(
	api_key=api_key, base_url=base_url
	)

Other models on Bedrock might similarly be accessible through their provider's SDK, which could be a nice way to implement this because then those providers would also be supported in magentic. This is currently only the case for OpenAI on Azure.