I use an automotive plastic polish, of a type that removes scratches from headlight lens covers. Kit is the brand I currently have but I think I like mothers a little better.
Although I am new in this arena, based on the recommendations from the members of B&B who do brush restorations, I recently purchased a tube of "Flitz" Metal and Plastic Polish for my plastic handled brushes. I found mine at my local Ace Hardware store.
I have also read about some members using a very "soft" grade of wet/dry sandpaper using a successively softer grit as the handle is sanded down. Hopefully one of our resident experts will chime in and offer you more help. Here is a B&B post on brush restorations that may help you: