You have to have a good look at what the Canon 70-200 f2.8 mkII zoom lens is all about. It's a wide fast aperture
zoom lens that you have compared against a Canon 100mm prime macro lens (known for being very sharp), and it is comparable wide open. This is no easy feat and why the lens commands a premium price. If you want the best, you have to pay the premium. Wait until you start looking at the longer whites, those prices make the 70-200 look great value!
Perhaps your expectations were unrealistic? There is an over-egging of equipment in photography, there is very rarely night and day, life changing differences in kit. CA control, distortion, Bokeh quality, AF speed etc. may all be slightly improved in one lens over another, but when all added together it can make a significant difference for a photographer. For some photographers this 70-200 gives them prime quality at 85mm, 100mm, 135mm and 200mm. Not to mention everything in between. So one quality lens that covers this starts to look much better value. Of course primes can also give you the wider aperture option, which is why I aim to have both.
The Sigma lenses are a good shout, I used the non OS for a number of years and it was sharper than my Canon mkI IS at 70mm but they were about the same by 200mm. There's also the Tamron which is pretty sharp but AF speed is lacking.