All good man. Thermal adhesive has a relatively low thermal conductivity so if you can use a thermal paste you can minimize the effect of the thermal interface and minimize the effect of the relatively poor surface of the aluminum.
Press down for 60 seconds on the corners of the COB until you have a very thin layer of paste. Then use kapton tape or silicon to keep it from moving sideways and you will a very nice thermal interface