The reasons for binary interfaces are good, but the problem with COM is that it is used for everything, and it was a poorly-extended design. It started out mostly-sane, later morphing into the pile of twisted pain it is today. It works, is fairly efficient, but damn ... it is a mess. But it works, which merits some respect.
But I think CORBA stinks too, from a complexity POV. There are places to use such complexity, but complexity has to be placed very carefully. We've probably all seen the results of misplaced complexity, and yet it continues to plague software today (just think of the children!).
Performance is important, as is something actually working ... COM accomplishes these. But simplicity, and some semblance of forethought could have prevented a lot of quirky software, warped developers, and the need to replace it all less than 10-years later.