Rockers, ported charger and some type of exhaust mods. 3.2 is possible, however if you scan and look at timing there is a good chance you will note that your timing is much lower than it should be. I once has the go as small as you can pulley misconception. Through good info etc friends told me I was running on the low octane tables which is what the pcm reverts to when you have enough constant knock.
As you can assume low octane tables mean less timing, less power, less fuel mileage. Oddly enough going to a 3.4 or 3.5 if you have low timing or knock could gain you more timing, more power and better mileage while using less power from the motor to turn the bigger pulley.
Sounds like you are willing to listen from those that blew up motors before you.