i think it will affect the sound during high speed and also the sound of the door closing will be different. And it get worst, during heavy downpour, water will make their way into your car too.
Than it will spoiled your car carpet, that is the worst scenario laa..... kekeke...