URL in buffer name, not capturing

[ghost]

This is a similar issue to #26 "Strange emacs buffer instead of capture buffer".

Ubuntu 17.10
Emacs 25.2.2
Chrome and Firefox
emacs -Q

(require 'package)
(let* ((no-ssl (and (memq system-type '(windows-nt ms-dos))
                    (not (gnutls-available-p))))
       (proto (if no-ssl "http" "https")))
  ;; Comment/uncomment these two lines to enable/disable MELPA and MELPA Stable as desired
  (add-to-list 'package-archives (cons "melpa" (concat proto "://melpa.org/packages/")) t)
  ;;(add-to-list 'package-archives (cons "melpa-stable" (concat proto "://stable.melpa.org/packages/")) t)
  (when (< emacs-major-version 24)
    ;; For important compatibility libraries like cl-lib
    (add-to-list 'package-archives '("gnu" . (concat proto "://elpa.gnu.org/packages/")))))
(package-initialize)

(package-install 's)
(add-to-list 'load-path "~/tmp/org-protocol-capture-html")

(require 'org)
(require 'org-capture)
(require 'org-protocol)
(require 'org-protocol-capture-html)
(server-start)


(setq org-capture-templates
      '(
	("l" "Org-protocol link" entry (file "notes.org") (file "~/.org.d/templates/link.tpl"))
	  ("w" "Web site" entry (file "") "* %a :website:\n\n%U %?\n\n%i")
      ) 

This works for the l command:

javascript:(function () {   window.location.href='org-protocol://capture://l/'     +encodeURIComponent(window.location.href)+'/'     +encodeURIComponent(document.title)+'/'     +encodeURIComponent(window.getSelection()); })();

but this one does not:

javascript:location.href = 'org-protocol://capture-eww-readable?template=w&url=' + encodeURIComponent(location.href) + '&title=' + encodeURIComponent(document.title || "[untitled page]");

I just get this buffer name:

?template=w&url=https%3A%2F%2Fgithub.com%2Falphapapa%2Forg-protocol-capture-html&title=alphapapa%2Forg-protocol-capture-html%3A%20Capture%20HTML%20from%20the%20browser%20selection%20into%20Emacs%20as%20org-mode%20content

Tried debugging a bit

This works, (org-protocol-capture-html--url-html "http://github.com").

But not sure what I'm supposed to pass to org-protocol-capture-html--capture-eww-readable, if that's the right way to proceed.

Alternatively, since org-protocol://capture: works, I should be able to just use a %(...) function call in my template file, but I'm not sure what function to put there (or what its arguments should be).

[alphapapa]

What version of Org are you using?

[ghost]

Org mode version 9.1.9 (release_9.1.9-553-gcad9ac)

[ghost]

Er, I'm not sure that's the same version that was running in emacs -Q. I'll update shortly.

[alphapapa]

Er, I'm not sure that's the same version that was running in emacs -Q.

Yeah, probably that was loading the built-in Org version, which is likely older, and may not support the new org-protocol URLs.

[ghost]

Right. That was 8.2.10. I have the same behavior in 9.1.9 and tried using emacs -Q as a debugging tactic.

Any ideas on how I might proceed, either with org-protocol://capture: templating or otherwise?

[alphapapa]

I'm afraid I don't know. It should work properly with new-style org-protocol URLs. I'm still using Org 9.0.5 and I don't have org-protocol working reliably with my browser at the moment, so maybe there are newer bugs that I haven't encountered yet. Maybe in a few weeks I'll be able to look into it, but it might take me a while to find time.

If you are able and willing to debug it, you might have to go as far as stepping through the code in the debugger (e.g. by instrumenting functions with edebug until you find the problem). It's really not fun, but I've had to do that in the past when I ran into some problems with org-protocol.

[ghost]

Thanks. Setting the wrap option defconst org-protocol-capture-html-pandoc-no-wrap-option "--wrap=auto"
and removing eval-when-compile has got it working for me.

[alphapapa]

@apnewberry Thanks, but I don't see how either of those could be causing or fixing the problem. Can you be more specific?

[ghost]

This version is the one that works for me.

I don't think the nil value for org-protocol-capture-html-pandoc-no-wrap-option worked for me, though it's possible I misdiagnosed.

The eval-when-compile prevented it from updating after I changed it, so I removed that.

The html-native_divs works better for me with my current version of pandoc (1.19.2.1), as you discussed in the pandoc issues, if I recall.

[alphapapa]

Okay, so what exactly did you change that fixed the problem you were having in this bug?

[ghost]

Here's the diff. It's a bit hard to read without git diff --color-words because of the indentation changes.

As I understand it, the fixing changes were replacing nil with --wrap=auto and removing eval-when-compile.

The html change is just for cleaning the output.

[alphapapa]

I'm still not sure how either of those would have fixed the problem you seemed to be having.

The --wrap=auto change would suggest a problem with Pandoc, in which case that's an issue that needs to be fixed for users who have that version of Pandoc.

• Which version of Pandoc do you have?
• Would you verify that that fixes the problem by changing it back to nil (and making no other changes)?

The eval-when-compile change, if it fixed anything, would be a different bug altogether.

• Would you verify that that actually fixes something by restoring it (and leaving the --wrap=auto intact)?

Thanks for your help with this issue. It's hard for me to test these issues myself since I have different versions of some of the software.

[ghost]

pandoc 1.19.2.1
Compiled with pandoc-types 1.17.0.5, texmath 0.9.4.1, skylighting 0.3.3.1

I wasn't able to replicate the problem with org-protocol-capture-html-pandoc-no-wrap-option as nil, but I have a fair amount of configuration in place, and I may be missing something. It's always a bit of a struggle to get a clean version when wrestling the builtin and updated versions of org.

Removing eval-when-compile, if I understand correctly, just evaluates the body at "runtime" rather than only while compiling. If this helped, I gather it would've done so by ignoring my previously cached version of org-protocol-capture-html--capture-eww-readable, rather than by making a substantive change.

Sorry I couldn't narrow it down much -- but thanks for following up on the issue nonetheless. :)

[Anton-Latukha]

Just wanted to say and update thread, on installation/configuration I got the same behavior (path+url) in buffer name.

Linux NixOS
GNU Emacs 26.1
pandoc 2.2.1
KDE Plasma 5

This one seems to be the big one.

[alphapapa]

@Anton-Latukha Thank you. A few things:

1. What version of Org are you using?
2. Please try to reproduce the problem by running emacs -q and loading only the necessary packages, without any personal configuration.
3. Please try to reproduce the problem with different settings for the option org-protocol-capture-html-pandoc-no-wrap-option, as @apnewberry said that seems to have made a difference for him.

[Anton-Latukha]

Sorry, I decided back then, that then stripping-down Spacemacs and using classic Emacs binding and somehow gradually debugging - was not a realistic task then.

I do not know now what was not working, maybe I not loaded .el somehow?..

[Anton-Latukha]

Right here #32 - I submit the simple configuration for Spacemacs.

[Anton-Latukha]

Also in #33 I submitted info how to properly make org-protocol handles work on KDE5, KDE4, GNOME & GTK distributions, it also would be healpful to others. And it may be related.

[antoan]

I have the same problem. Url in buffer name, no content in buffer.

Doom Emacs 2.09
Org mode version 9.4
Kubuntu Xenial
Chromium Browser

[alphapapa]

@antoan Unfortunately that is not enough information for me to begin to diagnose the problem.

Anyway, with all the changes that have been made to browsers in the past few years regarding user JavaScript, bookmarklets, etc, I can't confirm that this tool works reliably on any certain browser or configuration. I'd recommend looking into the related browser extensions that perform essentially the same function.

[Anton-Latukha]

I had this syndrome several times. Every time it was a syndrom of configuration not set up by me or that configuration was not loaded properly by applications at runtime.

And every time solution was a case of: reregistering the org-protocol handler properly, restart of the DE, restart of the browser, reloading Emacs.

Maybe it was not put clear. It is a configuration/applicatioin taking configuration issue.
That is why I point to the proper configurations in #27 (comment) and #27 (comment).

Try carefully clean-up config, and reregister/configure stuff, and restart everything.

[Anton-Latukha]

Actually when I was comparing - *this project parsing worked cleaner than the extension solution.

And extension solution can only be clickable by mouse, when this one is indeed modular, so I do not drug & click mouse - I have F2 as a hotkey for page export.

[firmart]

I encountered the same issue, for a reason I didn't investigate, the fork of @apnewberry fix the issue in my side.

[david-wakeo]

Hello, I had this issue as well with:

• emacs 28 master w/ native comp
• pandoc 1.19.2.4
• org 9.4
• latest firefox version
• vanilla bookmarklet, capture template etc...

For me the error occurred in the call-process-region but the message was never triggered because the call never returned normally and raised an exception that was caught by some error handler set up by org-protocol.

;; in org-protocol-capture-html--with-pandoc
      if (not (zerop (call-process-region
                       (point-min) (point-max)
                       "pandoc" t t nil "-f" "html" "-t" "org" org-protocol-capture-html-pandoc-no-wrap-option)))
          (message "Pandoc failed: %s" (buffer-string))    

Funny fact: the error is not specific to pandoc. If i replace it by another command such as echo the call-process-region also fails.

Solution

The fork did not produce consistent result. I made it work once but there were other modifications as well.
What worked for me 100% of the time is to remove the :kill-server in the protocol registration:

  (push '("capture-html"
          :protocol "capture-html"
          :function org-protocol-capture-html--with-pandoc)
          ;;  :kill-client t)
        org-protocol-protocol-alist)

I have no idea why this work or what that option was supposed to do 🤷