Neither, I'd use grey if I had no others.
Grey can be lit to make it white.
It can be left unlit to make it very dark (almost black in some situations)
It can be lit to create graduated or spot lit backgrounds
Gels can be used to create either pastel or deeply saturated colours.
Paper is the best bet if the light is coming from an oblique angle, simply because paper is flatter than cloth and smoother than vinyl.