I tried a 100-400mm, 70-200mm f2.8, 300mm f4, and 400mm f5.6 at the same time for a weekend.
As you got a 70-200mm already, then get a 400mm f5.6 prime. I found the drop of in IQ with a 70-200mm f2.8 with 1.4tc wsnt really noticeable to me verses the 100- 400mm, and some are saying the the f4 version is as sharp if not sharper then the f2.8.
The 300mm f4 is better then the 70-200, and 100- 400mm i sold it as i found it was getting a great deal of use.
The 400mm prime wipes the floor with the 100-400 at 400mm. Sharper faster better colours etc, comes in to its own in darker light as the 100-400mm can really hunt, where i found the prime didnt. Ok you lose IS on the prime, but its really light enough to hand hold.
The plus point for the 100-400 has to be its a one lens option, so if you worried more about having to take two lenses out then the 100-400, or Sigma as mentioned earlier.
If your after best IQ stick with what got and add the 400mm F5.6 prime. Comes in handy having 2 body's then.