It is more complicated than that.

First of all, what you are using in the inside of your cabinet is insulation. You do not want to be trying to keep the heat inside your cabinet. In between grows I would recommend either painting the inside white or using mylar. Let the plywood/osb board breath and help with some of the heat.

Now, how far is the intake ductwork from your grow room and what size diameter is the intake hole?

How about the output?