Summers was forced to withdraw by a bunch of Democrats who didn’t like Obama’s pick. As far as I can see, there were three reasons for this:
(1) Summers had been insufficiently in favor of financial regulation during the 90s.
(2) He had deeply angered feminists by his Harvard speech about the lack of women at the highest levels of science (I covered this in some detail at the time; see the bottom three posts in this list; the summary version is that I think there was nothing wrong with what Summers said).
(3) From all the rumors I’ve ever heard, Summers tends to rub a great many people the wrong way.
But it was the combination of numbers one and two above that really hurt his chances. Summers is both insufficiently ideologically pure and feminists have had it in for him since at least 2005. What’s more, there’s an available woman for the job to champion, Janet Yellen.
Actually, it was a surprise to me that Obama picked Summers in the first place. Reports now are that Obama is angry at being crossed. I’ll make a prediction, although not one I feel all that strongly about: if his nose is really out of joint about this, Obama won’t choose Yellen either. He’ll go for some third person, one who’s ideologically more to the left and perhaps even a woman, but not Yellen.