Yeah man it will polloniate the others. Remove it and do a lil backyard surgery to remove the sacks. Be careful not to release any pollen. Once its cleared of nuts, let it grow for a bit longer. Hopefully there wont be anymore bananas but if there is, get rid of it.