I am sure some will say there is, but for me, the content of the photo is more important than absolute sharpness etc. Yes, I want my pictures to look as good as possible, and need all the help I can get, but both are amongst the best, and I would guess that any differences could be actual sample differences. I have not used my f2.8 much, as it is too heavy. Unless you shoot both lenses side by side you would have a really hard job telling which was which. I cannot tell the difference in quality in any of my lenses (all are Canon, but only 1 is L, though I had another L and sold it for a non L, as it suited my needs better.
As Canon Bob, who knows a thing or two (not much more than 2 though..) recently pointed out, we often get caught up in quality of the gear, and maybe for some at the expense of the quality of the photo itself..
I love the f4 as it is small and light enough not to break my back, or to scare my subjects, and will not attract a scumball thief so much as the bigger f2.8.
However, these are my feelings, and you and your needs may be different