In general, people prune some of the larger leaves to increase light and airflow to the bud sites, which are generally around those smaller leaves you propose removing. And they prune to shape the plant to best take advantage of space. Don't do anything yet until you have a good idea of what you're trying to do.
Nice plant!