[Question]: Unexpected, phantom calls to Claude API

[serefarikan]

What is your question?

How can I check if a call to Claude API actually came from my Librechat deployment?

More Details

I have LibreChat running via the docker compose deployment. It is running under Ubuntu linux, behind nginx, which does the ssl termination.

Sometimes, but not always, I see entries in anthropic dashboard's log section with time stamps corresponding to times when I was not using Librechat. During some of these even the computer is not on, but the requests are there and they are costing me. The API key used by LibreChat is not used anywhere else. I deleted the key, recreated it and done this a few times now.

Some of the log entries are pointing at claude-3-haiku-20240307 model, which is not even configured in my Librechat deployment!

My Librechat deployment will ask for a login when the deployment url is visited, so I cannot see how some random bot can find it and use it.

This is driving me mad. Is is the anthropic dashboard that is lying to me? Am I being charged by them incorrectly, even if the requests are not made by my Librechat deployment?

I'd love to know how I can keep track of my calls on the Librechat side so that I can at least confirm the calls were made from Librechat. Any other suggestions are most welcome.

What is the main subject of your question?

Other

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

There are a couple of ways to check if this is actually from LibreChat or not.

For quick details, you can check the debug logs within ./logs directory at project root, in particular the logs starting with debug- in the name. These logs are created by date and are timestamped to help you correspond what Anthropic is reporting. Maybe a CTRL+F of claude-3-haiku-20240307 may good to start off with.

Some of the log entries are pointing at claude-3-haiku-20240307 model, which is not even configured in my Librechat deployment!

claude-3-haiku-20240307 is the default title model, it is used if ANTHROPIC_TITLE_MODEL is not set; in my opinion, I believe all that's happening is that your conversations are being titled, and there may be some reporting issue with the exact time Anthropic is recording this.

It may be useful to run a quick snapshot of user activity with npm run user-stats, see here how to run scripts inside the docker containers: https://www.librechat.ai/docs/configuration/authentication#user-management-scripts

Lastly, you can take a look at the transactions collection either by mongo CLI within the mongo container or setting up Mongo Express, as this records all AI provider API calls made by LibreChat, for even more precision: https://www.librechat.ai/blog/2023-11-30_mongoexpress

Please make sure to secure the configuration of LibreChat if you are accessing it remotely, there is some discussion on this topic here: #4620 (comment)

[serefarikan]

It was my complete lack of consideration for new registrations. I added mongo express as per the provided link, then checked out the users collection, and there were many users that were obviously created by some bot or another.
I deleted these from the users collection, then disabled user registration from the .env file. I think this should be sufficient to stop bots hijacking my deployment, but if there's something else obvious I'm missing, please let me know.

I cannot believe I ignored something so fundamental. Thanks a lot for the hints @danny-avila

[danny-avila]

I'm glad you figured it out, sorry you went through this.

[serefarikan]

Thanks @danny-avila , it was my doing entirely: lesson learned.

[ve7mjc]

Interesting! It sounds like your usage surged shortly after I locked the actor out of my instance.

As you saw me discussing over in #4620 - my instance was discovered through a simple Certificate Transparency public log search.

A quick look over the http request logs and LibreChat db (user, conversations, messages) shows the actor was relaying requests through a socks proxy and started with the creation of multiple "random" users, followed by automated user chat text submissions of "Hello" in each combination of AI API and model combination (ie. openai/gpt-*, anthropic/claude-*, etc). I suspect here the actor(s) was qualifying the instance for its abilities. It was not until ~ 2 months later until one of these dormant user accounts began making high volume calls. For reference, many of the messages became millions of input tokens.

My actor seemed to prefer the Anthropic engine - which I found strange (OpenAI, Gemini, and Anthropic were available). It sounds like you examined your instance and found similar. I will send you some indicator/patterns in a DM - I am curious if we had the same actor at play.

[serefarikan]

@ve7mjc thanks. I did not go into diagnostics as much as you did, and frankly, I don't have the time 😃 As long as they're out of my system, I'm good with that.