It is probably just soap scum. Scrubbing Bubbles takes care of that.
A quick shot of scrubbing bubbles followed by a hot water rinse and you're good to go. Avoid vinegar on your razor. Vinegar can eat through the plating.
Plain simple household citric acid in water. Mild enough.
We get our water from the Edwards Aquifer, a honeycomb of ancient calcium deposits 600 feet down, and the water here is full of dissolved calcium. I catch rainwater in buckets and transfer it to barrels for dry seasons (and we've had back to back droughts lately separated by maybe six months of normal rain.). In the bathroom, I have an electric teapot appliance that makes enough hot water for a shave, and more besides. I don't get the type of really tough soap scum common locally, made of part calcium, part soap. I drain the wash basin after shaving, add rain water, and rinse everything one last time, including my face.