We have APS-C (Sony A58) M43 (Olympus E-M10) and Nikon FX (D610) here, so to a degree I can can compare sensor performance for you.
In good lighting with good lenses there's little to choose in image quality between them IF you can keep to base ISO on the smaller sensor cameras and your exposures are good. In these circumstances even viewing 100% crops it's hard to tell.
However there's 2 places the smaller sensors lose out, and that's 1) when lighting is difficult & you need higher ISO and 2) when you need to process an image a bit harder to get the best from it. Smaller sensors seem more prone to finging, halos, artifacts from pushing the processing compared with larger ones, and while it's partly lens dependent, I notice more issues with the smaller formats even using good glass (zeiss) on the Sony compared with more ordinary lenses on the Nikon.
The other thing is that, for me, FX can give a greater sense of depth to an image, probably because of the shallower depth of field that goes with the territory. And control of DoF is more restricted on the smaller formats, especially M43, compared with FX. At the moment I'm happy to carry a medium weight DSLR & bag of lenses around, but there will probably come a time when the weight is more important than image quality.