Otherwise, anything that's always heavily creased and in need of attention after washing, is unlikely to ever be worn, and has been evicted.
With the right clothing items, drip drying on hangers, and a period of wearing and warming to body temp before leaving the house, it's not that noticeable, and certainly doesn't seem to draw disapproving looks.
You seem to have noticed the problem of ironing, and found ways to work around it and still look good.
If you occasionally need to look a bit "dressier" may I suggest that a long sleeved "jumper" (a.k.a. "sweater" over this side of the pond) over top of a white dress shirt looks fantastic and basically means you don't have to iron the shirt! (Maybe just the collar.)
Or sweater-vest to wear under a suit jacket if needed.
Given your retired lifestyle this may not be needed much, but you may like having this in your repertoire just in case. If wearing it under a suit jacket, though, be sure the sweater/jumper is thin rather than bulky/thick.