I would. I take off fans blocking bud sites. Also if it is so bushy to cause mold issues.
I use side light now so I take less off but the idea that lower buds can not be like upper ones is wrong. If they get the light they will grow. The fans are not that useful for nutrients as the soil I use is nutrient dense. The leaves on the buds are plenty of 'solar panel' power for me. The fans do not make the buds grow, they just help. The buds require direct intense light to get any quality in my experience. I do not get great buds from shaded buds and a bunch of fan leaves powering them. Simply never happened.
When you take fans the leaves on the buds grow more, so a bit of stunting due to that but I only take a few and not all at once, so that is minimal.