I would say it depends on the body... I use 2.8s (70-200, 120-300) hanging off of the body all of the time w/o issues. The 400/2.8, 500/4 etc are pushing the limits, I once bent the locking pin handling a 500/4 that way (major PITA). Haven't damaged anything w/ the 400/2.8 YET, but I'm a bit more careful w/ it attached. The bodies I use are top line/full magnesium... a plastic body would probably be more questionable.
With a 70-200 the issue is much more likely to be w/ the tripod/head... that's a lot of leverage for an average head to hold. But AFAIK, all of the 70-200/2.8s come with a tripod collar so it should be a non-issue.