I was there a couple months ago, and the drive is not that spectacular. Actually, it's a bit disconcerting given the proliferation of strip malls until you reach Key West proper (that is, the older part of Key West - the southwest tip where most of the action is - the first part you run into is more strip malls and general lameness). The views aren't that great most of the way. I would fly in if I were to go again. It was a pleasant place to be for a few days.
Some of the drive through the Keys isn't all that attractive, but some of it is actually quite pretty, especially at sunset. And the Seven-Mile Bridge is fun. But if you don't feel the need to see that, I recommend flying. It's a lot faster--an hour to fly, versus three or four hours to drive.
But lurkingfear is slightly mistaken: The older part of Key West ("Old Town") is the entire western part of the island, and the area "where most of the action is" is actually the
northwest tip. That's where most of the well-known bars are, and their noise and general rowdiness often spills into the street. The southwest end is actually pretty quiet, even on a Saturday night. All of Old Town is very pretty, very quaint, and, yes, very walkable. You definitely don't need a car, and parking is impossible. Scooters or bicycles are the way to go, once you're there.