The only yellow growth in a bud that I've seen is a male pollen sack.
It may not be, so don't pull the trigger yet.
Have you taken a close look at it, under magnification? Since you are just starting bud formation, if it is a male flower, then you most likely have a true hermie. If it spews yellow powder, then you'll know.
If it's just a colorful calyx, then you're OK.
