I love the E46 as well, but if you're going to go pure Ultimate Driving Machine, then you should go for the purest Ultimate Driving Machine, and AWD is not it.
I'm not crazy about creaky, flexy ragtops, but AirB has a point. Maybe a RWD convertible with a stick and that delicious I6 would make a nice third car?
IMO a BMW with AWD is a concession to practicality, not something to try to justify as an unnecessary, emotional purchase.