If they're dead/dying clip them off, otherwise leave them alone. Generally the plant will shed them on it's own when they are damaged beyond usefulness.
People debate over it all the time. I prefer letting nature take its course. So far it is working. I basically view it as "what if it was alone in the woods"...Mother Nature takes care of her kids. But, some say remove it right away as it blocks buds....I on the other hand think that even when they are dying they are still acting as solar panels for buds and whatever else needs something. I'm posting day 23 of flowering of my pics now. Check them out...that is pure mother nature.