I just let nature take its course. Assuming your plants are healthy, yellowing would start from the bottom and work its way up.
These lower leaves do not block light from any other leaves/buds, and even if they have just half of them green, they will photosynthesize.
I just let mine fall off naturally. It then gets mixed into my topsoil, and decays into some food for my plants. Good stuff!