The thing you are trying to get rid of is mainly mineral deposits from harder water, like soap scum. If you have soft water, you probably won't have a lot of mineral deposits.
People that wash their hair with plain old soap can have a similar problem. Their hair can become flat and dull. They are often advised to rinse their hair with a weak vinegar+water solution after washing it.
Interesting. I learned something. I guess our water is not too hard, never cleaned a brush, ever.