Interesting, and you are not alone. I find rangefinders very easy to use and have never had a problem seeing the ghost image compared to a split prism for example in an SLR in similar lighting conditions.
RE the expense - true some are extremely expensive and I too think that a lot of that is due to the idea of Leica cameras becoming 'luxury goods' but my M4-P was about £500 and my Voigtlander lens which I use on my R-D1 and my Leica was about £350. There are lots of good priced bodies out there and there are only tiny differences in the models, largely the framelines available (I got the M4-P because I use it with a 28mm lens and it has the right framelines)
I don't know how anyone could describe loading a Leica M as difficult, though. It literally takes 10 seconds and doesn't involve any fiddling or guiding the film into sprockets etc at all. There are several videos on youtube to show how quick it is, although with a little practice.
Different strokes for different folks - for me, checking out a rangefinder was a logical step in my photography hobby. For all it's quirks, It's good fun at the end of the day, and some of my fave images have come out of it's funky idiosyncratic and very possibly outdated and thoroughly surpassed system. Make of that what you will, but hearing and feeling the smooth gears engage as I roll the film forward is as tactile an experience as developing the completed roll, and one which makes me want to shoot more often. I can't say my DSLR is so satisfying to use, though like I said earlier, often it's the only sensible choice to use. But when had being sensible been fun, especially as photography is a hobby for me. I dare say that if photography was my lifeline I wouldn't use a rangefinder and only use it for 'personal' projects and enjoyment, but if photography was my business I'm sure I would use my spare time differently!
ped