The idea of swapping wheels with any regularity sounds awful to me :P . Winter tires I can understand; they are expensive and wear quickly in dry conditions.
However, I like math.
Driving 15k miles per year, and getting back 2 MPG at most by only running the MTs exclusively offroad, you'd save $210 per year with gas at $3/gal.
If you would choose to run only high-end MTs and would need replace them at 30k miles for $1000 (4 tires), but are willing to run basic road tires to 45k miles at $600 per set, it certainly makes some difference. After 12 years, you would have saved about $3900 total, including gas (at 15k miles per year). That is $325 per year, or $27 per month. If you go wheeling twice a month and spend 15 mins each time on the swap (including jacking, and especially moving all those wheels around)--for 4 swaps a month--that is $27/hr -- not bad!
Of course, there are plenty of capable MTs in the $700 range, which brings that total 12-year savings down to about $2060. That is about $170 saved per year, $14 per month, or $14/hr for your time if you want to look at it like that.
I dunno, I think I'd happy shell out $15 per month to
1) simply never have to bother with the regular wheel changes,
2) not have to store a huge stack of heavy tires, and
3) Maintain maximum vehicular coolness on every last grocery and mall run.
*edit*
Minor math correction (forgot you would need 4 swaps per month for 2 trips, rather than just 2 swaps :P ).
*edit*
Also, FWIW, going from 33x12.5" MTs on heavy 20x10 rims, to 32" ATs on light-weight aluminum 17s, I saved 50lb per wheel/tire combo. Still, I only picked up 1 MPG. So, 2MPG is, like I said, pretty optimistic except for huge changes like putting 35s on a 2door wrangler that came with 30s or 31s.