Let me first toss off a few generic suggestions that probably won’t help much but are worth doing:

1. Get rid of unused fonts in your font file. MapServer will reach out and touch each one for each map request.
2. If your shapefiles don’t change often, preprocess them to create a separate shapefile for each CLASS, so you don’t have to filter as much.
3. If you must use CLASSes, organize them so the most commonly-used class comes first in each LAYER.
4. If you have a certain number of combinations of shapefiles used in a given request, create multiple map files with only those layers. For example, if your 12 layers are one set of 4 base layers that are always used, and then 8 more layers only one of which is displayed at a time, create 8 map files, each with only 5 layers instead of 12 – the four base layers and one specific overlay layer. Then use your application code to figure out which map file to select.

And make sure you’re using shptree to generate spatial indexes for all your shapefiles!