Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement custom bufio.SplitFunc for WebVTT #118

Open
nakkamarra opened this issue Oct 9, 2024 · 1 comment
Open

Implement custom bufio.SplitFunc for WebVTT #118

nakkamarra opened this issue Oct 9, 2024 · 1 comment

Comments

@nakkamarra
Copy link
Contributor

From an excerpt of the WebVTT spec regarding the WebVTT parsing algorithm:

Let input be the string being parsed, after conversion to Unicode, and with the following transformations applied:
Replace all U+0000 NULL characters by U+FFFD REPLACEMENT CHARACTERs.

Replace each U+000D CARRIAGE RETURN U+000A LINE FEED (CRLF) character pair by a single U+000A LINE FEED (LF) character.

Replace all remaining U+000D CARRIAGE RETURN characters by U+000A LINE FEED (LF) characters.

I don't believe the current astisub WebVTT parsing logic is covering this part of the WebVTT parsing algorithm correctly. The bufio.Scanner is using the default which is ScanLines and I'm not sure this is handling the WebVTT parsing correctly (i.e a file with exclusively \r terminations will not be replaced and thus not parsed properly). So I'm thinking maybe implementing a new bufio.SplitFunc and feeding it to the scanner.Split(...)

This somewhat ties into another issue I am currently trying to make a PR to solve #110, where I'd like to:

  1. read the first 6 bytes and make sure they are WEBVTT
  2. read the remainder of the content up until the first occurrence of two successive line terminations
  3. write that content into a new field of the Subtitles struct that has each line

Thoughts? Maybe there's a better way for me to achieve what I'm trying to do.

@asticode
Copy link
Owner

asticode commented Oct 9, 2024

From what I understand, there are 2 problems you're trying to fix:

  1. handle \r
  2. handle optional data after WEBVTT keyword

Regarding 1), implementing a custom bufio.SplitFunc capable of parsing \n\r seems like a good idea
Regarding 2), I would update this logic, trimming BOMBytes prefix, checking the header is there, scanning until 2 empty lines have been found and storing its value in a Subtitles.Comments attribute

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants