Revert "Bugfix for BufferedTokenizer to completely consume lines in c…

[yaauie]

Release notes

• reverts Bugfix for BufferedTokenizer to completely consume lines in case of lines bigger then sizeLimit #16482 due to an encoding bug

What does this PR do?

Reverts #16482, as applied to 8.15

Why is it important/What is the impact to the user?

Strings handled by the buffered tokenizer could be double-encoded, presenting problems.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[mashhurs]

LGTM!

[donoghuc]

Same review as #16688 (review) Will merge once green.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 96f0e32

Failed CI Steps

• 🥼 IT Persistent Queues / part 1