replicate folders + albums from one library to another library (shared library)

[mschindl]

Is your feature request related to a problem? Please describe.
Is there a way replicate albums from one library to another library?
Shared library on both different devices and iCloud accounts in use and 95% of all pictures are stored there.
This feature request came from the use of a shared family library and currently only ONE iCloud has the folder + album structure.
The structure should be available on all devices.

Describe the solution you'd like

1. Using one shared library with a structure by folders and albums
2. Export information about folder + album with related picture link
3. Import information about folder + album with related picture link
4. Second library got the same structure

Describe alternatives you've considered
Nothing found yet

[oPromessa]

Hello @mschindl

See if this ask is identical to yours #887

From it, @RhetTbull built the osxphotos sync option

• Only catch is that it does not (yet!) create folder structure.

[RhetTbull]

Hi @mschindl OSXPhotos has two ways to do what you're looking for. You can use the osxphotos sync or osxphotos batch-edit commands to sync albums between libraries, with some limitations described below.

The first is the osxphotos sync command as @oPromessa pointed out. This can sync albums but unfortunately does not yet sync folders (this will eventually be fixed, see #1639). The sync command matches photos between the libraries via signature (what Photos internally calls a fingerprint). The photos must be exact duplicates to be matched between the two libraries. You can read more about the sync command by running osxphotos help sync. Here's a snippet from the docs. If you only want to sync albums and not other metadata, you'd use --merge albums in place of --merge all. This method requires you to run the sync command on both computers and to share the sync database or copy it between computers.

You can run the --export and --import commands together. In this case, the
  import will be run first and then the export will be run.

  For example, if you want to sync two Photos libraries between users or two
  different computers, you can export the metadata to a shared folder.

  On the first computer, run:

  osxphotos sync --export /path/to/export/folder/computer1.db --merge all
  --import /path/to/export/folder/computer2.db

  On the second computer, run:

  osxphotos sync --export /path/to/export/folder/computer2.db --merge all
  --import /path/to/export/folder/computer1.db

The second method uses the osxphotos batch-edit command to turn albums and folders into specially formatted keywords which are synced across iCloud shared library automatically by Photos. This method does successfully sync folders in addition to albums. Run osxphotos help batch-edit to learn more about the command.

If you run the following command periodically on each user's Mac, it will create keywords in the form of album:Folder/Subfolder/Album for each photo in an album. Then it will find any such keywords that already exist and add each photo to the appropriate album, creating folders and albums as necessary. You must select the photos to apply this to (e.g. all those in an album or all those in the shared iCloud library).

osxphotos batch-edit --verbose --keyword "{album?album:{folder_album}}" --album "{keyword|filter(startswith album:)|sslice(6:)}" --split-folder "/"

This command uses the osxphotos template language to create keywords if an album exists and then uses the template language to extract the folder and album names from the keywords which are then used to create the albums (if needed) and then add the photos to the appropriate albums.

There is a limitation that if you remove a photo from an album, the keywords are not removed nor is the photo removed from the album. AppleScript, which OSXPhotos uses to add photos to the album, is not able to remove photos from albums. Photos will all be tagged with these new keywords with no easy way to remove them so keep that in mind before you run this.

[mschindl]

Great explanation and work. It sounds like osxphotos batch-edit will solve my request.

Does it works also if same Photos located in multiple Albums? I would expect multiple album keyboards per album in this case, right?

I will test it asap and give feedback.
Many thanks

[RhetTbull]

Yes it will work for multiple albums. Each album would result in a new keyword.

[mschindl]

Great, it works good also with special characters in the names.
Is there a way to speed up batch-edit?

I'm using following command and removing "--verbose" will not speed up the process.
"osxphotos batch-edit --verbose --keyword "{album?album:{folder_album}}" --album "{keyword|filter(startswith album:)|sslice(6:)}" --split-folder "/" > ~/file-output.txt

In example it runs for over 10hours for round about 50k pictures/videos.

[RhetTbull]

This is because the AppleScript interface to Photos is really really slow and that's the only way to modify keywords. If Apple provided a better API, I'd use it but they seem uninterested in allowing users more control over the Photos library.

One possible optimization would be to first read all the existing folders and keywords using OSXPhotos then only use AppleScript for the ones that need updating. First run would be slow but subsequent runs would be faster.

Once you do the initial run, you can select only photos you've worked with recently or only albums you've added to then run the command on a smaller set of photos.

[mschindl]

I think all my questions are answered. I will close for now.

Good to know, processing time per object (photo/video) takes 3 seconds which is in sum for 50k objects 42 hours.

Thank you very much for your help!

[mschindl]

I executed the command on an other library (without any folder/album) and if I select only a few objects (pic/vid) then it works well, but if I select the complete library with round about 47k object then I got the message "No photos selected".

Any idea why the error is shown? Is it a scaling issue?

Selecting just the half of objects then it works as well.

[RhetTbull]

My guess is AppleScript is timing out getting the selected objects. You'll have to try with fewer photos to see what the limit is.

[RhetTbull]

It may take me a few days but I'll put together a script to remove the "album:" keywords in case you want to back out of this workflow. AppleScript limitations may prevent this from being tenable.

[mschindl]

If I understand it right then the usage of “album:” keywords will make trouble with future Photos release, right?
Tags can be easily removed with the UI/App Photos in a very fast way.

What is your recommendation to use now?

I used batch-edit successfully now, but of course it tooks time.
50k objects in round about 20hours.

[mschindl]

I want to give some feedback: Unfortunately it does not scale, because after doing the batch job on the 2nd macOS for 30k objects it will not work for more as 150-250 objects now. (20k objects left)
The interesting part is that the count of possible objects decreases with each run. (2500 > 1500 > 500 > 250 > 150 now)

[RhetTbull]

Thanks for the additional data. This is helpful. It will be some time before I can incorporate it but I've been doing some hacking on a side project that uses private APIs to do this type of thing natively. I did a test with ~1000 assets and results are much more promising:

$  python test.py
Added keyword to 1087 assets in 104.11 seconds
Added 1087 assets to album in 1.19 seconds

[mschindl]

I tried the "sync" command today because my folder structure is stable and does not change that often.

Unfortunately, it creates a new/same album at the top level ("My Albums") and does not synchronize the images in the current folder/albumXYZ where the album already exists or needs to be created. I thought there was only an issue with folder creation.

Btw: "-i" is not working and "--import" have to be used, because following error will occur
osxphotos sync -e ~/ZZZ.db --merge all -i ~/YYY.db --dry-run
Error: Got unexpected extra argument (YYY.db)

I appreciate your work and I hope to help you with my tests and reports.

[RhetTbull]

Btw: "-i" is not working and "--import" have to be used, because following error will occur

This is a bug caused because -i is redefined in the arguments:

-i, --ignore-case Case insensitive search for title, description, place, keyword, person, or album.

I will open a new issue for this.

Unfortunately, it creates a new/same album at the top level ("My Albums") and does not synchronize the images in the current folder/albumXYZ where the album already exists or needs to be created. I thought there was only an issue with folder creation.

I think this is the same as issue #1639. It's nearing the top of my queue but I've been blocked by limited time and working on same major changes to the date/time code in OSXPhotos that will improve handling of timezones but has requited extensive testing and updates to the test suite. Currently, the sync code only considers album names, not folders.

[mschindl]

Thanks for the additional data. This is helpful. It will be some time before I can incorporate it but I've been doing some hacking on a side project that uses private APIs to do this type of thing natively. I did a test with ~1000 assets and results are much more promising:

$  python test.py
Added keyword to 1087 assets in 104.11 seconds
Added 1087 assets to album in 1.19 seconds

Have you made progress? Can I give you more information to help you move forward?

[RhetTbull]

This has been on the back burner. I've been working on a rewrite of the date/time handling code to fix some bug and improve some features of the timewarp command. That's almost done and now I've got to switch to python 3.13 support because osxphotos installation in homebrew is now broken as homebrew insists on using python 3.13 for pipx.