Ideas for keeping temps down?

I just went out and bought a decent portable AC unit to put in my tarped room. Outside of the room is extremely hot because its on the second floor. The only way air from outside the room is by having a fan pushing air from outside the grow room to inside. I probably shouldn't change that because it's the only way fresh air is getting in the room. While thats bringing hot air in and the AC is trying to cool the room down, there is also my exhaust pulling air out of the room. It's very hard to maintain low temps in this room during hot days. Does anyone have any suggestions?

Would it be a good idea to turn off the fan bringing hot fresh air in the room while its hot during the day? Or maybe just turn off the exhaust pulling all the air out during the day? Maybe both?