My first enlarger covered 6x9, a size I was then using in my Coronet box camera. My second enlarger (and third, for that matter) were 35mm only, although the third could cover 6x7 if the condensers could be found. I had no need when I bought it (new) and the parts seemed unavailable when I moved to roll film again. My fourth enlarger handles up to 6x7, and my fifth 5x4. I still have the 4th and 5th.
Based on my experience, make sure you can handle all the negative sizes you're likely to use at the outset, although I will add the rider that I kept the Durst 6x7 enlarger because
a. It was silent - no transformer
b. It was condensers not diffusion for black and white.
The LPL 5x4 is a diffuser enlarger with a colour head.
On the other hand, the more enlargers the merrier....