oauth_problem=signature_invalid

[reinvented]

I exported 8133 photos from Flickr using the OpenPhoto export-flickr script. No problems encountered and I ended up with 8133 JSON files ready for OpenPhoto import.

Running the import.py script, I'm finding that about 15% of files get rejected with a oauth_problem=signature_invalid error, like this (oauth tokens obfuscated):

Processing 534 of 3582 20090326T113724-3387718988.json ... FAILED: 403 - oauth_problem=signature_invalid&debug_sbs=POST&http%3A%2F%2Fphotos.ruk.ca%2Fphoto%2Fupload.json&dateTaken%3D1237738476%26dateUploaded%3D1238078244%26id%3D3387718988%26license%3DCC%2520BY-NC-SA%26oauth_consumer_key%3DXXXXXXX%26oauth_nonce%3DXXXXXXX%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1335375265%26oauth_token%3DXXXXXXX%26oauth_version%3D1.0%26permission%3DTrue%26photo%3Dhttp%253A%252F%252Ffarm4.staticflickr.com%252F3420%252F3387718988_2d6c8eb77a_o.jpg%26tags%3Dparis%252Cfrance%26title%3DArriving%2520in%2520Gare%2520de%2520l%255C%2527Est
{u'license': u'CC BY-NC-SA', u'title': u"Arriving in Gare de l'Est", u'dateUploaded': 1238078244, u'photo': u'http://farm4.staticflickr.com/3420/3387718988_2d6c8eb77a_o.jpg', u'tags': u'paris,france', u'dateTaken': 1237738476, u'permission': True, u'id': u'3387718988'}

The remaining 85% of photos are imported with no issue at all.

[jmathai]

Can you link to one of the 15%? Can you confirm if the 15% are the ones which contain non-ascii characters?

[reinvented]

Here's the JPEG itself from one of the broken images:

http://farm4.staticflickr.com/3420/3387718988_2d6c8eb77a_o.jpg

And the Flickr record:

http://www.flickr.com/photos/reinvented/3387718988/

[jmathai]

exiftool showed this

Warning                         : [minor] Unrecognized MakerNotes

Unsure if that has anything to do with it. Can you link to an original that
did upload successfully?

[reinvented]

Looking over the 679 JSON files that got dropping into the "errored" directory by the import.py script, I find that:

• 515 of them have single quotes in the name or description field (none of the 4556 files processed without issue in the "processed" directory have any single quotes)
• The remaining 164 files have HTML in the description field -- either links or HTML entities. Again, none of the "processed" files have any HTML in their descriptions.

[reinvented]

Here a JPEG that did import without issue:

http://farm4.staticflickr.com/3633/3644371417_704c05b6d7_o.jpg

And its Flickr record:

http://www.flickr.com/photos/reinvented/3644371417/

[jmathai]

Would you mind doing a test?

Remove the single quote from one of the files as well as the html from
another. Put both of those into the folder for processing and run the
import script again and see what happens.

[reinvented]

I tried the test as you suggested.

In both cases -- single quote removal and HTML removal -- the files imported successfully when the offending content was removed, which appears to suggest that it is, indeed, this that is the issue.

[jmathai]

Thanks! Now to determine if it's in import, openphoto-python or frontend :)

[reinvented]

This is just a shot in the dark after some poking around: I am wondering whether the addition of
unicode-encoding the POST parameters is the cause of this issue?

Perhaps the parameters pre-encoded on POST are not matching the decoded parameters post-POST?

[sneakypete81]

@reinvented: can you please confirm that this is working now?

The openphoto-python library has been renamed to "trovebox", so you'll need to install this first:

   sudo pip install trovebox

See the updated readme for more details.

[mapkyca]

I'm also getting an oauth_problem=signature_invalid in my logs and the import script is spitting back FAILED: HTTP Error 401: Unauthorized. This seems to be for every photo, rather than just a few.

I've installed trovebox via pip just now, so I'm pretty sure it's the latest.

I've tried recreating the application on the site as well, no joy.

Any suggestions?

[mapkyca]

If it helps, I'm also getting PHP Notice: Array to string conversion in /var/www/open-photo/src/libraries/external/epi/EpiCache_File.php on line 57 at the same time...

[sneakypete81]

This sounds a bit like an OAuth problem. You could try a couple of things from the command line:

First, view a list of photos:

trovebox -p -e /photos/list.json

That should return the first page of photos on your site.

If that works, try uploading a photo:

trovebox -p -X POST -e /photo/upload.json -F photo=@/path/to/photo.jpg

If you still get permissions failures, check that your config file (~/.config/trovebox/default) is set up correctly. See the README at https://github.com/photo/openphoto-python for more details.

[mapkyca]

The trovebox client won't work owing to a separate issue (for some reason it won't accept my ssl certificate), so I had to modify import.py as per photo/openphoto-python#50 to get to this stage... is it possible to pass this parameter to trovebox CLI directly?

[sneakypete81]

Indeed, the commandline client doesn't support self-signed SSL certificates. You can test with the Python bindings instead:

python
import trovebox
client = trovebox.Trovebox()
client.configure(ssl_verify=False)

Then try viewing a list of photos:

print(client.photos.list())

Then attempt an upload:

client.photo.upload("/path/to/photo.jpg")

[sneakypete81]

If you still see problems, please try with an account on http://trovebox.com, to see whether this is a problem with your self-hosted installation.

[mapkyca]

Still no joy (yet to try the trovebox.com yet, will do), however looking at EpiCache_File.php, it looks like $value is an array...

Array
(
[1379703081] => Array
(
[153849330830113654701379703081] => 1
)
)

To me this looks like a serialisation problem, or possibly whatever is generating $value is wrong. The values being passed to the endpoint via file_put_contents is Array-1379707386

So I'm guessing the token is invalid here, making the 401 response consistent.

Not sure why $value is an array.