Bad day yesterday. Lot's of fighting with Ubuntu Precise. Upgraded from Oneiric, but X would freeze regularly. Could still ping/ssh in.
I think it was an issue with the application switcher. I wasn't using the Unity one. There were other issues too. As the system was so unstable, I decided not to take any changes with conflicts, so simply wiped my ~/.config and started over (but copied back settings from apps I wanted to keep). Pretty stable now. The only issue is that I'm using the Unity switcher and I don't like the "Show desktop" option, which you can't remove. Unless you recompile the source I suppose.