Any noise coming from the compressor is bad, but mechanical noises can sometimes be mimicked by a drive belt that is dry, sticky, or damaged. So, if the compressor is damaged and making a noise, it is likely that the system will not work as well as before.
It may be worth the trouble to pop around to Barry's garage to let him check out the belt, and/or tensioner pulley(s), especially if the A/C system is working as well as it did before the noise started.
It must be coming while driving too, but with road and engine noise you might not be hearing it. Slight extra AC load on engine due to low rev while idle is normal for all cars especially in summer months, as AC work harder and frequent but what grinding noise you get, can you attach an audio file here.