Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ELB Service: Log Parser skipping lots of lines #243

Closed
alexzon opened this issue Jan 24, 2024 · 1 comment
Closed

ELB Service: Log Parser skipping lots of lines #243

alexzon opened this issue Jan 24, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@alexzon
Copy link

alexzon commented Jan 24, 2024

Describe the bug

When deploying an ELB Service Log Pipeline, a Lambda Function named xxxxxx-LogProcessorFn is deployed with the description: "Function to process and load ELB logs into OpenSearch"

Inside this Lambda, there's the file util/log_parser.py

In the ELB Python class, the method parse (line 88) checks if a line from the log file matches a specific RegEx pattern.
If it doesn't, the line is skipped and not sent to OpenSearch.

The bug is in the block ([A-Z0-9-]+) from the pattern, related to the field ssl_cipher (documentation).

A lot of log entries uses the character _ instead of - in this field. When it happens, the line is skipped.

Expected Behavior

Not to skip log lines that use the character _ on the ssl_cipher field.

Current Behavior

Lines that use the character _ on the ssl_cipher field are skipped by the Log Parser.
Example: TLS_AES_128_GCM_SHA256

Reproduction Steps

Ingest any log file with the character _ in the ssl_cipher field.

Possible Solution

Add the _ character in the RegEx block. Like this: ([A-Z0-9-_]+)
I believe this is the correct file to make the change (line 165).

Additional Information/Context

No response

Solution Version

2.0.1

AWS Region. e.g., us-east-1

No response

Other information

No response

@JoeShi
Copy link
Contributor

JoeShi commented Feb 6, 2024

Will be fix in v2.2

owenCCY pushed a commit to owenCCY/centralized-logging-with-opensearch that referenced this issue Mar 16, 2024
This was referenced Mar 17, 2024
@evalzy evalzy moved this from v2.2.0 (Mar. 2024) to v2.1.X (Nov. 2023) in Centralized Logging with OpenSearch Roadmap Mar 19, 2024
@evalzy evalzy closed this as completed Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

4 participants