Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For anyone looking to migrate to another service... #4461

Open
tdhooten opened this issue Oct 29, 2024 · 13 comments
Open

For anyone looking to migrate to another service... #4461

tdhooten opened this issue Oct 29, 2024 · 13 comments

Comments

@tdhooten
Copy link

Polyglot, my read-it-later conversion service, now supports Omnivore exports and currently allows conversions to GoodLinks, Instapaper, Raindrop.io, and Readwise Reader.

@rochakagrawal
Copy link

How is this a replacement. With no apps ?

@tdhooten
Copy link
Author

@rochakagrawal

It's not a replacement for Omnivore, it's a tool to easily convert your Omnivore library to the import format of the read-it-later apps I mentioned in the original comment.

@devon-research
Copy link

It looks like Readwise Reader built a special tool to migrate since the announcement.

@tdhooten
Copy link
Author

tdhooten commented Oct 30, 2024

It looks like Readwise Reader built a special tool to migrate since the announcement.

If it supports tags, then it's a better solution than my app for Readwise. Currently Readwise doesn't support tags via their CSV import, which is a pain.

@linhavo
Copy link

linhavo commented Oct 31, 2024

I really liked the feature of setting an email address for my newsletters. Really helped unclutter incoming mails. Does anyone, please, have an alternative?

@jacksonh
Copy link
Contributor

I really liked the feature of setting an email address for my newsletters. Really helped unclutter incoming mails. Does anyone, please, have an alternative?

https://meco.app/

@devon-research
Copy link

Readwise Reader also has this feature of generating special email addresses you can use for your feed or library.

@sreenivasanac
Copy link

@jacksonh Thank you for the recommendation. The suggestion Meco.app looks very good

@ocean
Copy link

ocean commented Nov 4, 2024

For anyone trying to process their Omnivore export into something more suitable for import into Raindrop.io, etc, Omnivore suggests using a jq command to convert your files, but it doesn't work very well.

Once you've installed jq, here's a command which creates a nice CSV out of your metadata_*.json files from your Omnivore export, including extracting your tags and cleaning up any non-printable characters in your title and description fields:

jq -r '
  (["url","title","note","tags","created"]), 
  (.[] | [
    .url,
    (.title | gsub("\\n";" ") | gsub("\\r";" ") | gsub("\"";"''") | gsub("[^[:print:]]";" ") | gsub("\\s+";" ")),
    (.description | gsub("\\n";" ") | gsub("\\r";" ") | gsub("\"";"''") | gsub("[^[:print:]]";" ") | gsub("\\s+";" ")),
    ([.labels[]?]|join(",")),
    .savedAt
  ]) | @csv
' metadata_*.json > omnivore-export.csv

Once you've unzipped your Omnivore export .zip file, change into the directory where the metadata_*.json files are, and then you should be able to paste this command into your shell and run it (works for me in zsh and should work in bash as well).

For your edification, learning and enjoyment, here's a detailed breakdown of what this jq command does:

  • First, it outputs a header row with column names: ["url","title","note","tags","created"]
  • Then for each object in the JSON array (.[]), it creates an array with these transformations:
    • .url - Grabs the URL
    • For the .title field, it applies these cleanups in sequence:
      • gsub("\\n";" ") - Replaces newlines with spaces
      • gsub("\\r";" ") - Replaces carriage returns with spaces
      • gsub("\"";"''") - Replaces double quotes with two single quotes
      • gsub("[^[:print:]]";" ") - Replaces any non-printable characters with spaces
      • gsub("\\s+";" ") - Collapses multiple spaces into single spaces
    • Applies the same cleanups to the .description field
    • [.labels[]?]|join(",") - Takes the labels array and joins it into a single comma-separated string
    • .savedAt - Grabs the timestamp
  • Finally, | @csv formats everything as proper CSV, automatically:
    • Adding double quotes around fields that need them
    • Adding commas between fields
    • Creating proper line endings

Edit: I've put this in a gist so it's easy to link to - tell your friends! 😁

@skf-funzt
Copy link

skf-funzt commented Nov 5, 2024

For everyone going to migrate to Raindrop.io, I've created a program to convert the export ZIP to Raindrop's CSV import format.

You can find the project here

@Gitbreast
Copy link

Gitbreast commented Nov 5, 2024

Inspired by @ocean , this is a jq command with a little fine-tune for converting to.xlsx format and .md format, and I add links to local files by =HYPERLINK() command.

Tips: The jq command can directly run in Unix shell such like macOS terminal, but it cannot directly work in windows cause the “quote problem”. You can fine-tune it and give it a try, check: https://jqlang.github.io/jq/manual/

Follow these steps to .xlsx (in macOS):

  1. Follow these official steps. Download your Omnivore ZIP. Unzip it.
  2. Enter the folder, right click the folder name on the bottom bar, choose “open in terminal”.
  3. Paste these code in the terminal and press enter.
jq -r '
  (["Tags","Title","Description","Created","File","URL"]), 
  (.[] | [
    ([.labels[]?]|join(",")),
    (.title | gsub("\\n";" ") | gsub("\\r";" ") | gsub("\"";"''") | gsub("[^[:print:]]";" ") | gsub("\\s+";" ")),
    (.description | gsub("\\n";" ") | gsub("\\r";" ") | gsub("\"";"''") | gsub("[^[:print:]]";" ") | gsub("\\s+";" ")),
    .savedAt,
    ("=HYPERLINK(G1&\"\\content\\" + .slug + ".html\" ,\"Open\")"),
    .url
  ]) | @csv
' metadata_*.json > omnivore-export.csv
  1. You should see the omnivore-export.csv file appears in the same folder. Open it with Numbers App.
  2. In Numbers’s menu, click "Files > Export As > Excel…" , next step, export.
  3. You should see the omnivore-export.xlsx file appears in the same folder.
  4. Transport the folder to a windows PC. Open the .xlsx with Excel.
  5. Copy the folder path in windows, and paste it to the G2 cell in Excel.
  6. Now click a random "Open" button in column E, you should open a local html file.

PS. I cannot make the links in .xlsx work in macOS, but they work in windows, you can give it a try.

Eventually, it looks like this, I covered the text, notice G2:

image

And here’s a command for Markdown .md. Get the .csv file and drop it to an online converter such as this. Markdown is slow at so much data though.

jq -r '
  (["Tags","Title","Description","Created","File","URL"]), 
  (.[] | [
    ([.labels[]?]|join(",")),
    (.title | gsub("\\n";" ") | gsub("\\r";" ") | gsub("\"";"''") | gsub("[^[:print:]]";" ") | gsub("\\s+";" ")),
    (.description | gsub("\\n";" ") | gsub("\\r";" ") | gsub("\"";"''") | gsub("[^[:print:]]";" ") | gsub("\\s+";" ")),
    .savedAt,
    ("[Open](.\\content\\" + .slug + ".html)"),
    .url
  ]) | @csv
' metadata_*.json > omnivore-export.csv

@milosimpson
Copy link

@ocean Thanks for the jq. FYI small issue I ran into is that one of the entries in the meta_X_to_Y.json files had "description" : null and that was breaking the script. Fixed that by making it "description" : "null" and it ran fine, and was able to import to raindrop.

#4461 (comment)

@Soliman2020
Copy link

A simple web app includes all the steps you should do to easily immigrate from Omnivore to Raindrop with all of your articles and their corresponding tags :)
Omnivore JSON to Raindrop HTML web app
Repo and the Python code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests