server returned HTTP status 401 Unauthorized

[Tiremisu]

Component(s)

target allocator

What happened?

Description

Steps to Reproduce

When using target allocator + collecotor to scrape a pod or serviceMonitor which endpoint need basic auth,
the collector failed to auth.

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: collector-with-ta
spec:
  mode: statefulset
  targetAllocator:
    enabled: true
  config: |
    receivers:
      prometheus:
        config:
          scrape_configs:
          - job_name: 'prom-with-auth-https'
            scrape_interval: 10s
            static_configs:
            - targets: [ '0.0.0.0:9090' ]
            basic_auth:
              username: admin
              password: password
            scheme: https
            tls_config:
              insecure_skip_verify: true

    exporters:
      # NOTE: Prior to v0.86.0 use `logging` instead of `debug`.
      debug:

    service:
      pipelines:
        metrics:
          receivers: [prometheus]
          processors: []
          exporters: [debug]

Expected Result

Metric collector can scrape the endpoint with auth successfully

Actual Result

TA discover the target and read the auth successfully -> collector failed to scrape the endpoint which return 401

serviceMonitor

TA localhost:80/scrape_config result:

otel version: 0.102.0

TA: 0.95.0

Logs

I added/test out the scrape config using both scrape_config and serviceMonitors, the results are same error 401. Please help investigate. Thanks!

Kubernetes Version

1.28.14

Operator version

0.95.0

Collector version

0.102.1

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

Log output

collector:

Pls ref the Actual Result

Additional context

No response

[jaronoff97]

Hello, have you tried upgrading to a more recent version? We've made a few improvements in the past 10-20 versions around this. Also could you share the full JSON rather than a screenshot of the scrape_configs endpoint?