Skip to content

Commit

Permalink
Merge pull request #42 from startersclan/enhancement/add-http-server-…
Browse files Browse the repository at this point in the history
…to-forward-srcds-cs2-logs-to-daemon

Enhancement: Add HTTP server to forward `srcds/cs2` logs to daemon
  • Loading branch information
leojonathanoh authored Nov 15, 2023
2 parents c161898 + f1c9ff8 commit 450085a
Show file tree
Hide file tree
Showing 9 changed files with 461 additions and 195 deletions.
13 changes: 13 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Launch main.go",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "cmd/source-udp-forwarder/main.go",
"env":{
"GOCACHE": "${workspaceFolder}/.go/.cache/go-build",
"GOMODCACHE": "${workspaceFolder}/.go/pkg/mod"
},
"showLog": true,
"trace": "error"
},
{
"name": "Launch file",
"type": "go",
Expand Down
35 changes: 18 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@
[![codecov](https://codecov.io/gh/startersclan/source-udp-forwarder/branch/master/graph/badge.svg)](https://codecov.io/gh/startersclan/source-udp-forwarder)
[![go-report-card](https://goreportcard.com/badge/github.com/startersclan/source-udp-forwarder)](https://goreportcard.com/report/github.com/startersclan/source-udp-forwarder)

A simple UDP forwarder to the HLStatsX:CE daemon.
A simple HTTP and UDP log forwarder to the [HLStatsX:CE daemon](https://github.com/startersclan/hlstatsx-community-edition).

## Agenda

The [HLStatsX:CE perl daemon](https://github.com/startersclan/hlstatsx-community-edition/tree/master/scripts) infers a gameserver's IP:PORT from the client socket from which it receives (reads) the gameserver's `logaddress_add` logs. This means both the daemon and the gameservers have to run on the same network.
The [HLStatsX:CE perl daemon](https://github.com/startersclan/hlstatsx-community-edition/tree/master/scripts) infers a gameserver's IP:PORT from the client socket from which it receives (reads) the gameserver's `logaddress_add_http` or `logaddress_add` logs. This means both the daemon and the gameservers have to run on the same network.

This UDP forwarder eliminates this need by leveraging on an already built-in proxy protocol in the daemon - It simply runs as a sidecar to the gameserver, receives logs from the gameserver, prepends each log line with a spoofed `IP:PORT` as well as a [`proxy_key`](https://github.com/startersclan/hlstatsx-community-edition/blob/1.6.19/scripts/hlstats.pl#L1780) secret only known by the daemon, and finally sends that log line to the daemon. The daemon reads the gameserver's `IP:PORT` from each log line, rather than the usual inferring it from the client socket.
This log forwarder eliminates this need by leveraging on an already built-in proxy protocol in the daemon - It simply runs as a sidecar to the gameserver, receives logs from the gameserver, prepends each log line with a spoofed `IP:PORT` as well as a [`proxy_key`](https://github.com/startersclan/hlstatsx-community-edition/blob/1.6.19/scripts/hlstats.pl#L1780) secret only known by the daemon, and finally sends that log line to the daemon. The daemon reads the gameserver's `IP:PORT` from each log line, rather than the usual inferring it from the client socket.

`source-udp-forwarder` uses less than `3MB` of memory.

## Install
## Usage

### Binaries

Expand All @@ -32,15 +32,15 @@ To run the latest stable version:
docker run -it startersclan/source-udp-forwarder:latest
```

To run a specific version, for example `v0.1.0`:
To run a specific version, for example `v0.3.0`:

```sh
docker run -it startersclan/source-udp-forwarder:v0.1.0
docker run -it startersclan/source-udp-forwarder:v0.3.0
```

## Demo

1. Start the gameserver with cvar `logaddress_add 0.0.0.0:26999` for `srcds` (`srcds` refuses to log to `logaddress_add 127.0.0.1:<PORT>` for some reason) or `logaddress_add 127.0.0.1 26999` for `hlds` servers, to ensure the gameserver send logs to `source-udp-forwarder`.
1. Start the gameserver with cvar `logaddress_add_http "http://0.0.0.0:26999"` for Counter-Strike 2, `logaddress_add 0.0.0.0:26999` for `srcds` (`srcds` refuses to log to `logaddress_add 127.0.0.1:<PORT>` for some reason), or `logaddress_add 127.0.0.1 26999` for `hlds` servers, and cvar `log on`, to ensure the gameserver send logs to `source-udp-forwarder`.

2. Start `source-udp-forwarder` as a sidecar to the gameserver (both on localhost), setting the follow environment variables:

Expand All @@ -52,14 +52,15 @@ docker run -it startersclan/source-udp-forwarder:v0.1.0

3. Watch the daemon logs to ensure it's receiving logs from `source-udp-forwarder`. There should be a `PROXY` event tag attached to each log line received from `source-udp-forwarder`.

Here are some `docker-compose` examples demonstrating a gameserver UDP logs being proxied via `source-udp-forwarder` to the HLStatsX:CE perl daemon:
See `docker-compose` examples:

- [Counter-Strike 1.6](docs/hlds-cstrike-example/docker-compose.yml) - This will work for all [GoldSource](https://developer.valvesoftware.com/wiki/GoldSrc) games, such as Half-Life and Condition Zero
- [Half-Life 2 Multiplayer](docs/srcds-hl2mp-example/docker-compose.yml). This will work for all [Source](https://developer.valvesoftware.com/wiki/Source) games, such as Counter-Strike Global Offensive and Left 4 Dead 2.
- [Counter-Strike 2](docs/srcds-cs2-example/docker-compose.yml) - Works for Counter-Strike 2 and all games that sends logs using HTTP
- [Counter-Strike 1.6](docs/hlds-cstrike-example/docker-compose.yml) - This will work for all [GoldSource](https://developer.valvesoftware.com/wiki/GoldSrc) games which sends logs using UDP, such as Half-Life and Condition Zero
- [Half-Life 2 Multiplayer](docs/srcds-hl2mp-example/docker-compose.yml). This will work for all [Source](https://developer.valvesoftware.com/wiki/Source) games which sends logs using UDP, such as Counter-Strike Global Offensive and Left 4 Dead 2.

## Usage
## Configuration

Configuration is done via (from highest priorty to lowest priority):
Configuration is done via (from highest to lowest priority):

1. Command line
2. Environment variables
Expand All @@ -74,11 +75,11 @@ Run `source-udp-forwarder -help` to see command line usage:

| Environment variable | Description |
|---|---|
| `UDP_LISTEN_ADDR` | `<IP>:<PORT>` to listen on for incoming packets. Default value: `:26999` |
| `UDP_FORWARD_ADDR` | `<IP>:<PORT>` or `<HOSTNAME>:<PORT>` to which incoming packets will be forwarded. Default value: `127.0.0.1:27500` |
| `FORWARD_PROXY_KEY` | The [`proxy_key`](https://github.com/startersclan/hlstatsx-community-edition/blob/1.6.19/scripts/hlstats.pl#L1780) secret defined in the HLStatsX:CE Web Admin Panel. Default value: `XXXXX` |
| `FORWARD_GAMESERVER_IP` | IP that the sent packet should include. Default value: `127.0.0.1` |
| `FORWARD_GAMESERVER_PORT` | Port that the sent packet should include. Default value: `27015` |
| `LISTEN_ADDR` | `<IP>:<PORT>` to listen for incoming HTTP and UDP logs. Default value: `:26999` |
| `UDP_FORWARD_ADDR` | `<IP>:<PORT>` of the `daemon` to which incoming packets will be forwarded. Default value: `127.0.0.1:27500` |
| `FORWARD_PROXY_KEY` | The [`proxy_key`](https://github.com/startersclan/hlstatsx-community-edition/blob/1.6.19/scripts/hlstats.pl#L1780) secret defined in the HLStatsX:CE Web Admin Panel. Default value: `XXXXX` |
| `FORWARD_GAMESERVER_IP` | IP that the sent packet should include. Default value: `127.0.0.1` |
| `FORWARD_GAMESERVER_PORT` | Port that the sent packet should include. Default value: `27015` |
| `LOG_LEVEL` | Log level. Defaults to `INFO`. May be one of the following (starting with the most verbose): `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, `FATAL`. Default value: `INFO`|
| `LOG_FORMAT` | Log format, valid options are `txt` and `json`. Default value: `txt` |

Expand Down
34 changes: 20 additions & 14 deletions cmd/source-udp-forwarder/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@ import (
"flag"
"fmt"
"os"
"os/signal"
"regexp"
"strconv"
"strings"
"syscall"
"time"

log "github.com/sirupsen/logrus"

Expand All @@ -28,16 +31,17 @@ func getEnvBool(key string) (envValBool bool) {
return
}

func run() error {
func main() {
customFormatter := new(log.TextFormatter)
customFormatter.TimestampFormat = "2006-01-02 15:04:05"
customFormatter.FullTimestamp = true
log.SetFormatter(customFormatter)
// log.Info("Hello Walrus before FullTimestamp=true")

var (
listenAddress = flag.String("udp.listen-address", getEnv("UDP_LISTEN_ADDR", ":26999"), "<IP>:<Port> to listen on for incoming packets.")
forwardAddress = flag.String("udp.forward-address", getEnv("UDP_FORWARD_ADDR", "127.0.0.1:27500"), "<IP>:<Port> of the daemon to which incoming packets will be forwarded.")
listenAddress = flag.String("listen-address", getEnv("LISTEN_ADDR", ":26999"), "<IP>:<PORT> to listen for incoming HTTP and UDP logs.")
udpListenAddress = flag.String("udp.listen-address", getEnv("UDP_LISTEN_ADDR", ":26999"), "<IP>:<PORT> to listen for incoming HTTP and UDP logs. (deprecated, use -listen-address instead)")
forwardAddress = flag.String("udp.forward-address", getEnv("UDP_FORWARD_ADDR", "127.0.0.1:27500"), "<IP>:<PORT> of the daemon to which incoming packets will be forwarded.")

proxyKey = flag.String("forward.proxy-key", getEnv("FORWARD_PROXY_KEY", "XXXXX"), "The PROXY_KEY secret defined in HLStatsX:CE settings.")
srcIp = flag.String("forward.gameserver-ip", getEnv("FORWARD_GAMESERVER_IP", "127.0.0.1"), "IP that the sent packet should include.")
Expand All @@ -50,6 +54,11 @@ func run() error {
)
flag.Parse()

// Support the deprecated flag
if *udpListenAddress != ":26999" {
listenAddress = udpListenAddress
}

switch *logFormat {
case "json":
log.SetFormatter(&log.JSONFormatter{})
Expand All @@ -59,7 +68,7 @@ func run() error {

log.Printf(version.GetVersion())
if *showVersion {
return nil
os.Exit(0)
}

*logLevel = strings.ToUpper(*logLevel)
Expand Down Expand Up @@ -89,18 +98,15 @@ func run() error {
log.Infof("Forward Gameserver IP: %s", *srcIp)
log.Infof("Forward Gameserver Port: %s", *srcPort)

sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGTERM, syscall.SIGINT, syscall.SIGQUIT, syscall.SIGKILL, syscall.SIGHUP)
forwarder, err := udpforwarder.Forward(*listenAddress, *forwardAddress, udpforwarder.DefaultTimeout, fmt.Sprintf("PROXY Key=%s %s:%sPROXY ", *proxyKey, *srcIp, *srcPort))
if forwarder == nil || err != nil {
return err
log.Fatal(err)
}

// Block forever
select {}
}

func main() {
if err := run(); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(1)
}
<-sig
log.Infof("Received shutdown signal. Exiting")
forwarder.Close()
time.Sleep(10000)
}
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ services:
- LOG_LEVEL=DEBUG
- LOG_FORMAT=txt
ports:
- 26999:26999/tcp
- 26999:26999/udp
volumes:
- ${OUTBIN?:err}:/${BIN?:err}
Expand Down
88 changes: 52 additions & 36 deletions docs/hlds-cstrike-example/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,69 +1,85 @@
# In this example, you will see that the Counterstrike 1.6 gameserver sends its UDP logs to `source-udp-forwarder` which then proxies (forwards) logs to the HLStatsX:CE perl daemon.
# In this example, you will see that Counterstrike 1.6 server sends its UDP logs to `source-udp-forwarder` which then proxies (forwards) logs to the HLStatsX:CE perl daemon
# This will work for all GoldSource games (https://developer.valvesoftware.com/wiki/GoldSrc), such as Half-Life and Condition Zero
version: '2.2'
services:
# 1. Counter-Strike 1.6 gameserver sends UDP logs to source-udp-forwarder
# See: https://github.com/startersclan/docker-sourceservers
cstrike:
image: goldsourceservers/cstrike:latest
network_mode: host
volumes:
- dns-volume:/dns:ro
ports:
- 27015:27015/udp
networks:
- default
stdin_open: true
tty: true
stop_signal: SIGKILL
depends_on:
- source-udp-forwarder
entrypoint:
- /bin/bash
command:
- -c
- |
set -eu
exec hlds_linux -console -noipx -secure -game cstrike +map de_dust2 +maxplayers 32 +sv_lan 0 +ip 0.0.0.0 +port 27015 +log on +logaddress_add 127.0.0.1 26999
exec hlds_linux -console -noipx -secure -game cstrike +map de_dust2 +maxplayers 32 +sv_lan 0 +ip 0.0.0.0 +port 27015 +rcon_password password +log on +logaddress_add "$$( cat /dns/source-udp-forwarder )" 26999
# 2. The proxy forwards gameserver logs to the daemon
# source-udp-forwarder: https://github.com/startersclan/source-udp-forwarder
# 2. source-udp-forwarder proxy forwards gameserver logs to the daemon
# See: https://github.com/startersclan/source-udp-forwarder
source-udp-forwarder:
image: startersclan/source-udp-forwarder:latest
environment:
- UDP_LISTEN_ADDR=:26999
- UDP_FORWARD_ADDR=127.0.0.1:27500
- LISTEN_ADDR=:26999
- UDP_FORWARD_ADDR=daemon:27500
- FORWARD_PROXY_KEY=somedaemonsecret # The daemon's proxy_key secret
- FORWARD_GAMESERVER_IP=192.168.1.100 # The gameserver's IP as registered in the HLStatsX:CE database
- FORWARD_GAMESERVER_PORT=27015 # The gameserver's IP as registered in the HLStatsX:CE database
- LOG_LEVEL=DEBUG
- LOG_FORMAT=txt
network_mode: host
volumes:
- dns-volume:/dns
networks:
- default
depends_on:
- daemon
entrypoint:
- /bin/sh
command:
- -c
- |
set -eu
echo "Outputting my IP address"
ip addr show eth0 | grep 'inet ' | awk '{print $$2}' | cut -d '/' -f1 | tee /dns/source-udp-forwarder
exec /source-udp-forwarder
# 3. HLStatsX:CE perl daemon accepts the gameserver logs. Gameserver Logs are parsed and stats are recorded
# The daemon's proxy_key secret can only be setup in the HLStatsX:CE Web Admin Panel and not via env vars
# HLStatsX:CE perl daemon: https://github.com/startersclan/docker-hlstatsxce-daemon
# NOTE: Currently, as of v1.6.19, the daemon crashes upon startup. You will need to fix perl errors and rebuild the image.
# See: https://github.com/startersclan/docker-hlstatsxce-daemon
daemon:
image: startersclan/docker-hlstatsxce-daemon:v1.6.19-geoip-alpine-3.8
environment:
- LOG_LEVEL=1
- DB_HOST=127.0.0.1:3306
- DB_NAME=hlstatsxce
- DB_USER=hlstatsxce
- DB_PASSWORD=hlstatsxce
- DNS_RESOLVE_IP=false
- LISTEN_IP=127.0.0.1
- LISTEN_PORT=27500
- RCON=true
network_mode: host

# 4. The DB for HLStatsX:CE
db:
image: mysql:5.7
environment:
- MYSQL_ROOT_PASSWORD=someverystrongpassword
- MYSQL_USER=hlstatsxce
- MYSQL_PASSWORD=hlstatsxce
- MYSQL_DATABASE=hlstatsxce
volumes:
- db-volume:/var/lib/mysql
network_mode: host
image: startersclan/hlstatsx-community-edition:1.9.0-daemon
ports:
- 27500:27500/udp # For external servers to send logs to the daemon
networks:
- default
command:
- --ip=0.0.0.0
- --port=27500
- --db-host=db:3306
- --db-name=hlstatsxce
- --db-username=hlstatsxce
- --db-password=hlstatsxce
- --nodns-resolveip
- --debug
# - --debug
# - --help

# 5. HLStatsX:CE web UI: https://github.com/NomisCZ/hlstatsx-community-edition/
# Currently, as of the time of writing, there is no docker image. The HLStatsX:CE web frontend must be setup by you.
networks:
default:

volumes:
dns-volume:
db-volume:

82 changes: 82 additions & 0 deletions docs/srcds-cs2-example/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# In this example, you will see that Counter-Strike 2 server sends its HTTP logs to `source-udp-forwarder` which then proxies (forwards) logs to the HLStatsX:CE perl daemon.
version: '2.2'
services:
# 1. Counter-Strike 2 gameserver sends UDP logs to source-udp-forwarder
# See: https://github.com/startersclan/docker-sourceservers
cs2:
image: sourceservers/cs2:latest
volumes:
- dns-volume:/dns:ro
ports:
- 27015:27015/tcp
- 27015:27015/udp
networks:
- default
stdin_open: true
tty: true
stop_signal: SIGKILL
entrypoint:
- /bin/bash
command:
- -c
- |
set -eu
exec game/bin/linuxsteamrt64/cs2 -dedicated -game cs2 -port 27015 +game_type 0 +game_mode 1 +mapgroup mg_active +map de_dust2 +log on +logaddress_add_http "http://$$( cat /dns/source-udp-forwarder ):26999"
# 2. source-udp-forwarder proxy forwards gameserver logs to the daemon
# See: https://github.com/startersclan/source-udp-forwarder
source-udp-forwarder:
image: startersclan/source-udp-forwarder:latest
environment:
- LISTEN_ADDR=:26999
- UDP_FORWARD_ADDR=daemon:27500
- FORWARD_PROXY_KEY=somedaemonsecret # The daemon's proxy_key secret
- FORWARD_GAMESERVER_IP=192.168.1.100 # The gameserver's IP as registered in the HLStatsX:CE database
- FORWARD_GAMESERVER_PORT=27015 # The gameserver's IP as registered in the HLStatsX:CE database
- LOG_LEVEL=DEBUG
- LOG_FORMAT=txt
volumes:
- dns-volume:/dns
networks:
- default
depends_on:
- daemon
entrypoint:
- /bin/sh
command:
- -c
- |
set -eu
echo "Outputting my IP address"
ip addr show eth0 | grep 'inet ' | awk '{print $$2}' | cut -d '/' -f1 | tee /dns/source-udp-forwarder
exec /source-udp-forwarder
# 3. HLStatsX:CE perl daemon accepts the gameserver logs. Gameserver Logs are parsed and stats are recorded
# The daemon's proxy_key secret can only be setup in the HLStatsX:CE Web Admin Panel and not via env vars
# See: https://github.com/startersclan/docker-hlstatsxce-daemon
daemon:
image: startersclan/hlstatsx-community-edition:1.9.0-daemon
ports:
- 27500:27500/udp # For external servers to send logs to the daemon
networks:
- default
command:
- --ip=0.0.0.0
- --port=27500
- --db-host=db:3306
- --db-name=hlstatsxce
- --db-username=hlstatsxce
- --db-password=hlstatsxce
- --nodns-resolveip
- --debug
# - --debug
# - --help

networks:
default:

volumes:
dns-volume:
db-volume:
Loading

0 comments on commit 450085a

Please sign in to comment.