Photostructure?

TimHughes

TimHughes

Suspended / Banned
Messages
3,655
Name
Tim
Edit My Images
Yes
Has anyone tried PhotoStructure? https://photostructure.com/

Just started testing this photo/video management app that says it can help organize and deduplicate your entire library across drives/devices. I'm running it via Docker on my NAS to wrangle the legacy project folders, old Lightroom and Capture One catalogs that I have. It tells me I have about 1.3 million image/video files across the drives. Oof. I estimate it will take around two weeks to fully index everything, but once that's done, I'm hoping it’ll help with sorting and culling stuff I don’t actually need to keep.

Curious if anyone else here has tried it, especially on large archives or with RAW workflows.
 
A quick at the website shows that though it is©2024, on the About page the most recent version related entry is early 2021 :thinking:

So, is it still being developed & supported?
 
The release notes look like it’s being maintained about once a year. Seems to be working ok so far. I’ll let you know once the indexing has finished.
 
I don't have much to contribute here but just wanted to say I'm looking forward to your results :notworthy:
 
a little snippet from the author which resonates, fingers crossed it does what is says on the tin

"What I had before I startedPhotoStructure: a pile of hard drives and old backups and duplicates and some files that had succumbed to bit rot. A big,intractable mess.

What I wanted: a deduplicated set of all my photos and videos, in one organized folder hierarchy.

So I wrote it. PhotoStructure can bring some order to your chaos with automatic organization."
 
TimHughes said:
a little snippet from the author which resonates, fingers crossed it does what is says on the tin

"What I had before I startedPhotoStructure: a pile of hard drives and old backups and duplicates and some files that had succumbed to bit rot. A big,intractable mess.

What I wanted: a deduplicated set of all my photos and videos, in one organized folder hierarchy.

So I wrote it. PhotoStructure can bring some order to your chaos with automatic organization."
Click to expand...


I was in a similar situation, and used PhotoMove to sort it all out
 
Howdy! I'm the author of PhotoStructure. A saved search alerted me to this forum post.

The main focus I've had for PhotoStructure is how gracefully deduplicate manual backups, Google Takeouts (some of which have both image and metadata loss) and other software that renamed or somehow mucked with my originals (without me realizing until it was too late).

I want PhotoStructure to be able to run everywhere you've got your photos (or everywhere my wife has stored photos, to be fair)--so on your Mac or Windows desktop, or even on your (intel or ARM) NAS. There are both Desktop and Server “editions” of PhotoStructure available. Unfortunately, the last Desktop build is indeed “stuck” on an old release, as there were breaking changes to the desktop framework that my code relied on. I've rewritten that code over the winter and will be releasing it shortly. Meanwhile, know that there has been a steady release of new versions of the “server” editions (that run in a terminal on in docker, but that can be intimidating for non-nerds).

There's a PhotoStructure forum and discord (links on my website, I would rather not come off as a shill) if anyone needs customer support or wants to chat (mostly about self-hosting and image/video metadata) -- the community there hasn't really talked much about camera hardware, or (arguably) the more fun bits of photography (around technique and tricks to capture compelling images).

Cheers!
 
Last edited:
photostructure said:
Howdy! I'm the author of PhotoStructure. A saved search alerted me to this forum post.

The main focus I've had for PhotoStructure is how gracefully deduplicate manual backups, Google Takeouts (some of which have both image and metadata loss) and other software that renamed or somehow mucked with my originals (without me realizing until it was too late).

I want PhotoStructure to be able to run everywhere you've got your photos (or everywhere my wife has stored photos, to be fair)--so on your Mac or Windows desktop, or even on your (intel or ARM) NAS. There are both Desktop and Server “editions” of PhotoStructure available. Unfortunately, the last Desktop build is indeed “stuck” on an old release, as there were breaking changes to the desktop framework that my code relied on. I've rewritten that code over the winter and will be releasing it shortly. Meanwhile, know that there has been a steady release of new versions of the “server” editions (that run in a terminal on in docker, but that can be intimidating for non-nerds).

There's a PhotoStructure forum and discord (links on my website, I would rather not come off as a shill) if anyone needs customer support or wants to chat (mostly about self-hosting and image/video metadata) -- the community there hasn't really talked much about camera hardware, or (arguably) the more fun bits of photography (around technique and tricks to capture compelling images).

Cheers!
Click to expand...
Hey Matthew, thank you for putting it together and jumping on here.

Personally, I'm using the NAS version with docker and it's chugging away nicely with the indexing. 80k processed 1.2m to go! Should be done in a couple of weeks.

The visualization and tagging already emerging is looking useful and helping me think through the goals for this clean-up and how I'm going to achieve them - something that I've been putting off up to now.

Cheers,

Tim
 
Whilst this looks very interesting, tit is the consolidation of images into a library that spooks me. I'm not sure I have an HD big enough, and I got burned when I took the NAS approach to storage.

Will it just create a catalogue, or does it have to create a new folder structure? I may have to have a play.
 
Pound Coin said:
Whilst this looks very interesting, tit is the consolidation of images into a library that spooks me. I'm not sure I have an HD big enough, and I got burned when I took the NAS approach to storage.

Will it just create a catalogue, or does it have to create a new folder structure? I may have to have a play.
Click to expand...
You can choose either. For the same reason as you I’m starting with indexing only keeping everything in its current place.

My next step will be to use the index to decide what to delete and free up a chunk of space. Once that is done I may automate the reorganization. I’m not sure about that.
 
Pound Coin said:
Whilst this looks very interesting, tit is the consolidation of images into a library that spooks me. I'm not sure I have an HD big enough, and I got burned when I took the NAS approach to storage.

Will it just create a catalogue, or does it have to create a new folder structure? I may have to have a play.
Click to expand...
After having experienced an HDD die only after I tried to write to it (but reading was fine?!?), I decided to force the user to determine what to do regarding “organization,” including “do nothing.” You can also have as many base directories as you want to be part of a single “library.” (and you can have as many libraries as you want).

If you decide to let PhotoStructure sweep all unique files (based on SHA) into a single pile, know that there's a setting (the assetPathnameFormat) that enables very flexible directory and filename structures.

# -----------------------------------------------
# assetPathnameFormat or PS_ASSET_PATHNAME_FORMAT
# -----------------------------------------------
#
# If you opt into "automatic organization" (see the setting
# "copyAssetsToLibrary"), they will be copied into <originals
# directory>/<result of assetPathnameFormat>.
#
# - See the originalsDir system setting for what your <originals directory> is
# (it defaults to your library root directory).
#
# - Please encode this path with forward-slashes, even if you're on Windows.
#
# - If any patterns resolve to including forward-slashes, know that will be
# interpreted as subdirectories.
#
# - If you want to add a static path, escape the pathname with single quotes
# (like "'photos'/y/MM/dd").
#
# - The result of this will always be interpreted as a relative path from your
# PhotoStructure originals directory.
#
# - Use token "BASE" as a shorthand for the original basename ("photo.jpg" for
# "/path/to/photo.jpg").
#
# - Use token "NAME" as a shorthand for the original filename, without the
# file extension ("photo" for "/path/to/photo.jpg").
#
# - Use token "PARENT" as a shorthand for the original file's parent directory
# name ("to" for "/path/to/photo.jpg").
#
# - Use token "GRANDPARENT" as a shorthand for the original file's grandparent
# directory name ("path" for "/path/to/photo.jpg").
#
# - Use token "EXT" for the filename's extension without the "." prefix (like
# "jpg" for "/path/to/photo.jpg").
#
# - Use token "ISO" as a shorthand for "yyyy-MM-dd'T'HH-mm-ss.SSSZZ" (the time
# separators are dashes here as colon is an illegal character for some
# filesystems).
#
# - You can escape other static text by wrapping with single quotes.
#
# - For other tokens, see
# https://moment.github.io/luxon/#/formatting?id=table-of-tokens .
#
# - For more details, see
# https://forum.photostructure.com/t/how-to-change-the-naming-structure/1184/2
#
# Default value:
# assetPathnameFormat = "y/y-MM-dd/BASE"

(If this makes your eyes roll, just ask what you'd like, and I can tell you what you want for the value for this setting).
 
Well, I've bitten the bullet and started the trail, and it is hoovering around finding images now. 23,000 images processed so far and it hasn't got anywhere near my external disks yet!

Thanks.
 
Ha! Only 34k so far...

It is still on the Mac HD. Externals to go yet!
 
I have an estimated completion time estimation on the side.... extending to 3 weeks now. The bottleneck is the HDD read speed on the NAS. All four CPU cores are working hard.
 
So, despite plugging in lots of a few disks, I am at 75k photos. I use a laptop and had to take it away to do work stuff.

I still have Aperture libraries going back far that I've not given it (and don't know how it will respond!) I have a lot of disks left.

It took me a day to find out how to find out how many images are in the library.

onwards and sideways.
 
Pound Coin said:
I still have Aperture libraries going back far that I've not given it (and don't know how it will respond!) I have a lot of disks left.
Click to expand...
If PhotoStructure manages to extract the images from the Aperture libraries (which TBF isn't that hard to do, just quantity I guess), I reckon it's extremely likely they'll be the un-adjusted originals.

If you want the adjusted images from Aperture, AFAIK you'll need to find a Mac with a version of OSX that will still run Aperture, and then run ApertureExporter, which calls Aperture to make the adjustments, and then saves the images according to rules you set.. That can take a long time, mine took over 24 hours for around 30K images. AE offers you the chance to stop and later restart the extraction; I found this a Very Bad Idea, as it slowly goes through the Aperture Library trying to work out where you left off, a process that took nearly as long as doing the first extraction!

The end result for me is that I have the full set of un-adjusted originals and a Capture One catalogue of Aperture-adjusted images. I did try importing the Aperture library into C1Pro, but the latter's attempts to apply its equivalents of Aperture's adjustments were woeful.
 
You must log in or register to reply here.
Back
Top