Use of AWS STS Temporary Credentials Not Working for Native Bedrock Endpoint

[ldavis9000aws]

What happened?

I've setup Librechat on a Windows 11 machine using docker and did not specify any AWS related credentials in the .env file except for "BEDROCK_AWS_DEFAULT_REGION=us-east-1 ". After opening a command line, I type: "setx AWS_PROFILE myprofilename" with the understanding that the environment variable should be used by default. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-configure-files-using-profiles.

As a test, I type "aws s3 ls" to validate I'm authenticated and can see S3 buckets in the us-east-1 region. This tells me that, by default, the named profile is being used since I didn't need to specify a profile. I run "docker-compose up", attempt to use Amazon Bedrock, then receive the attached error:

LibreChat | 2024-09-20 19:41:10 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type Could not load credentials from any providers
LibreChat | 2024-09-20 19:41:10 error: [handleAbortError] AI response error; aborting request: Could not load credentials from any providers

Steps to Reproduce

1. Install the AWS CLI v2 on the Windows machine.
2. Configure AWS temporary credentials and a named profile then use "setx AWS_PROFILE user1" as defined here: https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html#cli-configure-files-using-profiles and define them in your user environment:

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html

Example (fake creds):

SET AWS_ACCESS_KEY_ID=ASIAIOSFODNN7EXAMPLE
SET AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
SET AWS_SESSION_TOKEN=AQoDYXdzEJr...<remainder of token> 

4. Next, validate that you have programmatic access to AWS without having to specify the named profile.
4. Uncomment the "BEDROCK_AWS_DEFAULT_REGION=us-east-1" line in the .env file
5. Add the following enviornment variables to the docker-compose.override.yml file to pull them from the host into the LibreChat docker container:

Example:

services:
  api:
    container_name: LibreChat
    environment:
      **- AWS_ACCESS_KEY_ID
      - AWS_SECRET_ACCESS_KEY
      - AWS_SESSION_TOKEN**

6. Run Librechat via "docker-compose up" from within that same CLI session and try to use Amazon BedRock. This assumes you've already setup Bedrock model access beforehand in AWS.

What browsers are you seeing the problem on?

Not a browser issue

Relevant log output

The error will be similar to:

LibreChat         | 2024-09-22 08:50:33 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type The security token included in the request is invalid.
LibreChat         | 2024-09-22 08:50:33 error: [handleAbortError] AI response error; aborting request: The security token included in the request is invalid.

###screenshot###

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Docker is using a separate OS from windows. You would need to mount the credentials to the container setup to load as expected.

Even if you use WSL2 and use the AWS CLI from the Linux terminal, the containers wouldn’t have access unless the container has those “host” credential paths mounted.

[ldavis9000aws]

It's not an issue with the named profile as I originally thought The actual problem is that when you deal with temporary AWS credentials that expire every X number of hours, there is an "AWS_SESSION_TOKEN" environment variable you need to pass with each AWS request. The "api\server\controllers\agents\client.js" is only using "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY", which results in the failure. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html

I got this to work by doing the following:

1. Took all 3 settings in the AWS credentials file and defined them in environment variables on my local Windows host using the set command for the CLI session.
2. Updated docker-compose.override.yml and added the following environment variables to the LibreChat container:
   • AWS_ACCESS_KEY_ID
   • AWS_SECRET_ACCESS_KEY
   • AWS_SESSION_TOKEN
3. Updated the client.js file to conditionally include AWS session tokens if defined in the environment variables (see attached).
4. The only AWS variable defined in the .env file is "BEDROCK_AWS_DEFAULT_REGION=us-east-1".

Now I can work with my temporary AWS credentials running Librechat locally on my Windows laptop (tested with both temporary AWS credentials and non-expiring credentials). When my AWS session token expires I have to stop the Librechat docker container, update the AWS credentials on my lcoal host , then add them back to the environment variables using the setx command, restart the CLI, and finally restart the containers with docker-compose.

client.js.txt

[ldavis9000aws]

Additional reference material on the various AWS STS auth methods and how litellm handles them: https://docs.litellm.ai/docs/providers/bedrock

[danny-avila]

If temporary sessions are required, you could automate this a bit by including aws/cli as an added container, have it run a script to populate environment variables, and have the librechat API container depend on this, something like here: https://demo.librechat.ai/share/twcMojNFOLnDTab3w24Cw

Updated the client.js file to conditionally include AWS session tokens if defined in the environment variables (see attached).

You shouldn't need to do this. The aws dependency being used searches for credentials from the following sources (listed in order of precedence):

Environment variables exposed via process.env
SSO credentials from token cache
Web identity token credentials
Shared credentials and config ini files
The EC2/ECS Instance Metadata Service

The default credential provider will invoke one provider at a time and only continue to the next if no credentials have been located.

This should include the variables you named.

[ldavis9000aws]

Thanks @danny-avila. As far as semi-automating the process, I'm already using a menu-based Powershell script that renews the AWS STS credentials from the Windows host, sets the AWS environment variables, then restarts the docker container and pulls in the AWS environment variables to the LibreChat container via the docker-compose.overide.yaml file. Moving the AWS authentication from the Windows host to a container that will specifically handle AWS authentication seems like another solution also worth exploring, but it's another container that needs to managed and maintained.

In regards to the client.js file, it currently doesn't use the "AWS_SESSION_TOKEN" environment variable needed to support temporary AWS STS credentials. All 3 (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN) are required for use with AWS STS authentication. So even if the client.js does try to use AWS credentials from a list in order of precedence, the missing session token environment variable causes calls to AWS to fail.

Let me know if I'm overlooking or misunderstading how client.js works, but I've tried passing in all 3 AWS environment variables via docker-compose.overide.yaml, connected to the LibreChat container and validated all 3 AWS environment variables were present, then tried to initiate communication to Amazon Bedrock from within LibreChat and it fails. Once I made the modification I attached in my previous post, I was able to connect.

[ldavis9000aws]

I'm closing this for now and opening a new issue based on a more recent install of Librechat v0.7.5. Same issue, different day.