Changing from LR multiple catalogs to a structured/searchable system - advice please

drounding

drounding

Suspended / Banned
Messages
1,080
Name
Duncan
Edit My Images
Yes
Lightroom Catalogs (LR Classic CC)

I know there have been a lot of postings re LR catalogs but I'm trying to relate to my personal scenario. I've done so much reading about catalogs that my mind is now clouded over :).
For quite some time now I have been creating a new catalog for each day's shooting per event type. I take photos for Parkrun (maybe 1000 at a time) and Hockey (maybe 300-400 at a time) as well as some miscellaneous shots occasionally.

I have been creating individual catalogs so I could archive the catalogs together with the original photos and the exports in one folder. This worked well until I now need to find older photos.

For the Hockey shots in particular I need to change my way of working so that I can go back and locate photos of say the Mens 2 team on date xx or the Ladies 4 team on date yy for example. And perhaps even be able to filter more specifically based on other keywords.

My thoughts are to do the following, please correct me if there are better ways.

In windows create a file structure like this:

For the photos...

-Lightroom Photos (on external USB 3 drive)
-- Hockey
---- Date xx
---- Date yy
-- Parkrun
---- Date xx
---- Date yy
-- Misc
---- Date xx
---- Date yy

And for the catalogs...

-Lightroom Catalogs (on local SSD drive)
-- Hockey
-- Parkrun
-- Misc

I would then copy over my existing photos into the new file structures and then create the three new catalogs and merge the relevant original catalogs.

Does that make sense, is it a good solution or are there better ways?

Also - I presume the preview files will live on the SSD together with the catalog by default? And that I can set the preferences to delete the previews after 30 days or similar to save space?

Hope that makes sense.

TIA
 
I can't really advise you as I have only ever used one catalogue myself. Most people do only use one catalogue, and then have a folder structure within that.

However, if you don't get the advice you need here, you could try the Lightroom Forum as well.
 
One catalogue for all. This umbrellas your personal folder structure which you can make what the hell you like, and your folders will be visible within LR as well as outside it.

You will of course arrange regular backups on separate drives of both your image files and your LR cat ...
 
As I understand it you need to find older images, using the date as one search parameter. Well lightroom does allow you to sort by date as one of the metadata fields you can search by. Also the file structure could be used to segregate into the three categories you want,Thereby eliminating the need to have separate catalogues. Filing by date would work and probably help as a quick sort function The only reason I can see for individual catalogues if the catalogue gets to large and slows things down. I have two catalogues for that reason, and it works fine taking a few seconds to relaunch Lightroom with the new catalogue
Not sure if I would delete the previews. OK yes you can rebuild them but you would need to do this each time you went back and searched for an image. I'd look at keeping all the Lightroom files on the Hard Drive. Yes the SSD would be quicker but I don't think it would save a lot of time to be honest
 
Use one Catalogue, and use the options on Import do the work for you.
For file locations, I use a Year/Date structure - so if I know roughly when I took a shot, I can quickly find things from that.
Then use LR Collections to give you the 'named' info
- You can create a new collection, and have LR automatically add all imported images to it. If you use a structured name for the collections it will be easy to find any you want
EG 'Parkrun Southampton Nov 21 2017'
- Add Keywords Such as Hockey, The Location, Team Names, etc to allow more general searches
 
Thanks for your replies.

I'll create a a sample master catalog and have a play around importing and creating collections etc.

I've got quite entrenched into my current workflow, but I've got to bite the bullet and get this done.
 
See this thread: https://www.talkphotography.co.uk/threads/working-with-multiple-lr-catalogs.665119/#post-7984413

One catalogue. Import your raw shots into year folders. I import by date, standard lightroom function but with a top level folder of the year.
I use keyword tags on import, occasionally it's nice to search through them by event, place.
Sometimes, not always I'll use the map function and tag photos for places as I don't have gps built in my camera so it's a manual process.
Plenty of ways to search for images

I also heavily use collections and collection sets, these are my go to place for finding images. Top level folders for things like cars, sport, family etc, then subdivided under that. For family it's then year, then event, for others it's usually just event for the collection title.

upload_2017-10-23_8-8-30-png.113060
 
drounding said:
Thanks for your replies.

I'll create a a sample master catalog and have a play around importing and creating collections etc.

I've got quite entrenched into my current workflow, but I've got to bite the bullet and get this done.
Click to expand...
To answer a few questions, yes the previews will live on SSD alongside the master catalogue and yes you configure it to discard the full size previews after e..g 30 days to keep the space down here, no there is no downside to doing this in terms of searching or anything else, only regeneration of previews if you ever want to re-edit.

And YES having the catalogue and previews on SSD will make a HUGE difference to the performance of LR in a monolithic catalogue workflow. Even better if it's a separate filesystem even if it's on a shared SSD.
 
Yup, I built my system specifically around Lightroom performance. SSD for the Operating system, SSD for the lightroom cache and catalogue, SSD for this years raws, large fast raid for previous years raws and exported jpgs
 
Thanks for the further comments.
The only drive I have in my machine is a single SSD, so I will create the catalog (and previews - with 30 day expiry) on it and then keep the photos etc on the external usb 3 drive.
Probably the best option i have in terms of speed. I'm only using a laptop with i5 cpu, 16Gb RAM and 240Gb SSD so trying to make the best I can of it.
 
All my personal photos are in a right mess, I have catalogues everywhere, duplicates, drives etc. I keep trying to sort it out but end up with another set of the above.

Fortunately, my work photos are beautifully organised, keyworded, folderised and backed up in three places.

How I untangle this mess I don't know. I keep buying bigger hard drives and say "this one will be a master master copy and will be perfectly ordered "

At least hard drives are cheap now eh :) I suppose it's like the builder's house that needs the extension finishing :banghead:
 
Last edited:
Byker28i said:
Possibly he means partitioning the drive? But that will do nothing as its still disk access. I bought a 250Gb Fast SSD to use solely for catalogue and Cache
Click to expand...
That is what I mean. But it's beneficial because the kernel has a separate parallel buffering path with totally independent filesystem locking for the second filesystem (D:), even if it sits on the same SSD.
 
Obviously it's not as good as buying a second SSD I didn't claim that, but it's better than putting the catalogue on the C drive to compete with the whims of Windows and the page file which locks the entire filesystem. Parallelism is always good and I don't need t convince anyone because it's simply true.

Why do you think VMware says that multiple virtual drives perform better in VMs than a larger single drive, even when they all sit on the same storage underneath?
 
Last edited:
And I would say you'll never make lightroom fly, you just might make it tolerable though...
 
Also - thinking a bit more. Where's the best place to store the exported photos, with the original files, in a separate directory?
 
drdocmatt said:
And I would say you'll never make lightroom fly, you just might make it tolerable though...
Click to expand...
I've said many times the spec I built my PC to. I find no slowness with Lightroom, there are many ways to improve the performance, all fully documented by adobe.

500Gb M2 SSD for OS, blindingly fast, 250Gb SSD for Catalogue and cache only, 500Gb SSD for this years raws. Large raid of WD black 4Tb drives for previous years raws and exported images.

32Gb Ram, i7 Skylake CPU.

OS is Windows 10, with Office 2016, Lightroom 6 standalone and Photoshop CS5.5 (or 6 cant remember) installed and thats about it.

130K images in the catalogue without any noticable slowness. I've just archived about another 80k images out from sunday morning sport and its made no difference to the speed from over 210K images.

Backup is to a HP miniserver running WD Black 5Tb drives in a raid under server 2012 R2 as it also runs the app for the cctv cameras. Backups are scheduled to run every friday night
 
Last edited:
drounding said:
Why do you think VMware says that multiple virtual drives perform better in VMs than a larger single drive, even when they all sit on the same storage underneath?
Click to expand...

We can chat hypervisor systems if you like (Byker28i - VCIP, VCIX etc).
We're talking home based OS system, not on a hypervisor with a VM with many disks, which ultimately are just files onto a disk system. The ultimate speed of which comes from the disk types or number of.

But I'm not getting into whos right or wrong, nor talking theory, just providing answers based on real systems I've built for Lightroom performance on a home PC.
 
Last edited:
drounding said:
Also - thinking a bit more. Where's the best place to store the exported photos, with the original files, in a separate directory?
Click to expand...
I put mine onto a separate drive (raid of large 4tb drives) in a folder structure the same way I have my colletions, so it makes it obvious how to find them. These are finished images so are produced for a reason, putting onto a web page, printing etc so don't tend to be accessed too often so don't need to be on performance drives. I save those for the current years raws, catalogue and cache.
 
Byker28i said:
We can chat hypervisor systems if you like (Byker28i - VCIP, VCIX etc).
We're talking home based OS system, not on a hypervisor with a VM with many disks, which ultimately are just files onto a disk system. The ultimate speed of which comes from the disk types or number of.

But I'm not getting into whos right or wrong, nor talking theory, just providing answers based on real systems I've built for Lightroom performance on a home PC.
Click to expand...
Not quite sure why you're so defensive. It's a real effect, and for those who either can't afford a second drive or don't have the necessary io ports it does provide a real benefit. Just because Adobe don't put it in their wishlist doesn't mean it doesn't work.

Whether using a hypervisor or a kernel running directly on metal the effects of having parallel locking for a separate filesystem are tangible.
 
You must log in or register to reply here.
Back
Top