Most of this software in and around GNU is written in C. The major problem is that C promotes reuse at the libary level, not the source level. This is not a problem by itself, indeed a reasonable set of libraries can devised and maintained. In systems based on GNU however there often many poorly designed libraries, libraries that duplicate each others functionality, overly generalized libraries, libraries that wrap multiple similar libraries. Rarely do these libraries offer a consistent interface.
Over time, you would think that this bloat would reduce to a very compact set of libaries which contain the core set of functionality. But this is not the case, indeed the greatest freedom of free software is ignored, the ability to modify the source. Why people choose to create new software instead modifing existing is software, I think is not just ego. As dan points out, just trying to build someone else's software today can be daunting. Also within each project are different set of coding conventions, sometimes odd use of language (classes in C?), different platform abstractions, and many other intricacies peculiar to a project. And then there is the sheer size of some projects. Try understanding a project like Mozilla, Linux, or X. The result of this is that factoring code into a consistent highly reusable set of libraries very difficult. The problems with applications are similar.
Another major barrier to reuse is the use of different languages. This division, for example, produced two practically equivalent Unix free software desktops, KDE and Gnome. And even then the division is minor since C functions can be called directly by C++ code. Factoring is still difficult if not impossible between these languages. If the languages are even more different than you can forget reuse altogether.
So I've listed some of the problems that exist. What is the solution? Focus. If the project is a Unix make that the scope of the project. If the project is a desktop, design from the hardware to the user interface. The result may be several operating systems, for different uses. The GNU project lacks this focus, it tries to be everything to everyone, this is the source of much of the bloat.
Beyond this I think better languages are in order, and tighter integration of the language to the system. colorForth address many of the problems I have outlined, and many I didn't even mention. And I think it embodies the spirit of free software better than the GNU project. The system is almost purely represented as source code. Not only does this encourage reuse, understanding of others code, and modification, it also promotes a high degree factoring and tailoring of code.
Oddly our software mirrors many aspects of proprietary software, binary distribution, plugins, many isolated libraries and applications. I think the future of free software should be in source code based systems. This is a more natural paradigm for free software.