Rockwool is made from basalt rock, melted and spun out like cotton candy. It cannot be eaten by anything and won't support mould growth by itself, even if wetted with plain water. However, if you add nutes to the water and algae spores are in the air, like any good plant, algae will grow on rockwool. Rockwool stays intact when wet, even for long periods of time.
Peat is an organic material. It can be broken down and eaten by fungi and moulds. It will also tend to physically break into little bits and generally create muck in a hydroponic system.