3rd-party RPMs of libraries bad?

Posted 4 Nov 2002 at 11:17 UTC by murrayc Share This

Does it make sense for anyone other than the distros themselves to supply RPMs of dependencies, such as C/C++ libraries? Note that this issue does not apply to RPMs of applications.

People often offer to provide binary RPM packages of gtkmm2, but I can't see how they could ever be used by applications, because:

  • If 2 people provide RPMs, and 2 each applications depend on a different one, then those 2 packages will conflict, and it will not be possible to install both applications.
  • There is always a chance that the distro, such as RedHat, will provide their own official RPMs, so no 3rd-party can ever expect to provide the one-and-only RPMs.

I believe that the conflict between Ximian and RedHat GNOME RPMs proves this - you must remove Ximian RPMs before installing RedHat RPMs and vice-versa.

This is all in addition to these other problems with 3rd-party RPMs:

  • Someone provides RPMs for a couple of versions. People get used to using them. Then no RPMs are provided for later versions so people don't get the benefit of bugfixes.
  • RPMs providers are not always competent - they often use non-standard systems, or non-standard compiler options. The people who use these RPMs don't know that application problems are caused by the RPM of a dependency. For instance, Gabber gets swamped by bug-reports that are really caused by dodgy RPMs.
  • Users have no reason to trust binary packages unless they come from well-known providers. There is always the risk of malicious code.

I'm hoping that someone can either

  • persuade me that I'm wrong. Or
  • suggest how we might encourage distros such as Redhat to provide definitive RPMs of dependencies. Or
  • tell me that there is some way to solve this problem by changing the RPM/installation system itself, and that such a change is on the way.

Although I do not use Debian, I am aware that this is not a problem with debian because individual developers are able to provide binary debs that do become definitive debian debs.

What's the alternative?, posted 4 Nov 2002 at 14:32 UTC by neil » (Master)

So would you propose that until Red Hat (or some other user of rpm) supports a given version of a given library, that packagers an application that needs the library should just shun Red Hat?

And what would you have users of Red Hat do if they need an app that is in that situation?

What's the alternative?, posted 4 Nov 2002 at 14:51 UTC by murrayc » (Master)

I don't know. I hope someone will tell me.

Why not a problem with Debian? (And reference specfiles are Good.), posted 4 Nov 2002 at 18:48 UTC by braden » (Journeyer)

I don't understand why this isn't a problem with Debian. If mutually exclusive .rpm's are possible, why aren't mutually exclusive .deb's equally possible?

Is it that they are equally possible, but don't happen so much in practice simply because more people use Red Hat (thus there are more folks creating packages)?

There was a particularly nasty instance of the problem you mention with Mozilla some months back. This was before mozilla.org was including an "authoritative" specfile. Ximian's Mozilla specfile put NSPR in its own package, but Red Hat's lumped NSPR in with the main mozilla package. This meant that different dependency chains were possible depending on whether one was building against Red Hat or Ximian RPMs. Yuck.

Fortunately, now both the Ximian and Red Hat specfiles create the same packages. I think that having a reference specfile available as part of the mozilla.org distribution of Mozilla probably helped matters.

Ultimately, the only thing that will keep things sane is if different vendors of RPMs make an attempt to play nice. But I think package maintainers can help them do so by providing a reference specfile.

Why not a problem with Debian? (And reference specfiles are Good.), posted 4 Nov 2002 at 20:26 UTC by murrayc » (Master)

It's not a problem because regular developers can make definitive debs. You don't need to work for some "Debian Incorporated" to make an official deb. Mutually exclusive debs are still possible, but the situation doesn't provide any incentive to produce them.

Distros need to let go a little more, posted 4 Nov 2002 at 22:27 UTC by garym » (Master)

There's two issues here: One is the RPMs setting up conflicts such as Mozilla being packaged differently and thus causing different dependencies in the RPMs, the other is if a vendor or 3rd-party mod to the actual code produces an incompatible change.

So there are several scenarios here:

  • a 3rd party RPM depends on or provides different packages: This is solved, as with mozilla, by some agreement (or a mandate) on the spec file dividing the files between packages.
  • the 3rd party mods of the components produce API incompatibilities (ie forks the code, for example the ac-series kernel including pre-accepted API changes): This situation, if it can be avoided, would allow vendors and 3rd party providers to interchange RPMs, but where the API must change, well, that's why we have /opt and /usr/local!

Personally, I think the vendors are taking too much on themselves to try and be all things to all people; the should aim to own less and less of each new release, not more and more. "More and more" is an escalating game they cannot possibly win, the costs of it must always increase, the opportunities for errors must grow exponentially, the risk gets higher and that puts the whole fish-tank of Linux in jeopardy.

IMHO, each Linux distro should be only a framework, the minimum to be it's differentiated space, and as much as possible the packages bundled with the distro should be the interchangeable products of 3rd parties in a global Linux ecology.

Is it practical? Probably not. This was the original intent of the Linux Standards Base, but we also all know that in the real world the technical and business needs of each distro require changes not acceptable to authors or the other distros, so we end up with forked RPMs.

The remaining way around this is for package maintainers to avoid --prefix=/usr leaving that to the distros. That way, if you grab gtkmm from the authors, it's going to be the edition in /opt set ahead in your path and LD_LIBRARY_PATH, and any future distro upgrade is not going to hold any code-clobbering surprises.

Distros need to let go a little more,, posted 4 Nov 2002 at 22:43 UTC by murrayc » (Master)

Well, using a separate prefix for 3rd-party RPMs might prevent a conflict with any future distro-owned RPM. But it wouldn't prevent conflicts _between_ 3rd-party RPMs. There aren't enough prefixes for that.

Two options, posted 4 Nov 2002 at 23:49 UTC by nymia » (Master)

First option would be putting away binary install to solve version issues. Maybe a Gentoo-like system might be implemented over and above the standard distribution base. Building from source may be triggered for certain events like app and lib installation/upgrade. While some may just use binary install for certain cases.

The second option, use mimetypes to implement vendor and version ID. Each app can have the ability to identify the package and decide whether it is compatible or not.

Experience with making our own RPMS, posted 5 Nov 2002 at 12:18 UTC by Uraeus » (Master)

I help out with the packaging of GStreamer and its dependencies. GStreamer as you know depends on many other multimedia libraries being present many which isn't packaged by Red Hat distro. Due to this we provide premade RPMS for Red Hat 7.3, 7.3 and now 8.0. I would claim that providing these RPMS has been mostly a blessing since it makes the threshold for testing GStreamer much lower, it solves a lot of build issues for our users and maybe most important; it has helped us clean out many build and packaging issues as they occur. Also since we provide these RPMS as part of the GStreamer project application developers will use them as their reference when/if they produce RPMS for their projects.

I think the same experience would be true for gtkmm, if you provide a set of rpms by an official gtkmm Red Hat packager these packages would be what everyone would use as their dependency until such a time Red Hat ships an official version themselves.

For our own part we do however plan to continue offering GStreamer rpms even if Red Hat eventually will ship their own stuff. But we will make sure our RPMS are compatible with Red Hats in their structure and composition. But since we have so strictly kept our SPEC file up to date and tried making it very similar to Red Hat's own SPEC files already I would be very suprised if Red Hat (and Ximian) doesn't use our SPEC as the basis for their own, avoiding such issues as the Ximian/Red Hat incompatability problems. Linux Mandrake who already ships GStreamer in cooker do this AFAIK.

Summary, posted 5 Nov 2002 at 18:49 UTC by murrayc » (Master)

So, it looks like the long term solution would be: RedHat and other distros could provide a contributors space for 3rd-party RPMs, so that there is at least a central place and a way to say that something is the definitive RPM for that platform. RedHat could use this as a no-warranty testing ground which would provide a shortcut when they decide to package stuff officially.

In the short term, there doesn't seem to be any solution apart from:

  • Provide your own RPM.
  • Remove it when RedHat provide their own
  • Roam the internet looking for false RPMs and asking people to stop.
  • Support less platforms, due to lack of resources. For instance, just RedHat, and just RedHat 8.0.

Of course you could force RedHat to want to package libraries by writing killer apps that happen to use those libraries, but that's not really within the mission-statement of most library authors.

I like the 'Gentoo-like' suggestion..., posted 7 Nov 2002 at 01:33 UTC by crandall » (Journeyer)

I use Red Hat almost exclusively; however, I rarely install binary RPM's unless that's the only choice.

I recompile SRPMS on my system(s) and use only the binary RPM's I've built myself. This enables me to tailor the binary packages for each of my systems (e.g. I have 3 AMD K6-2's, some P-II's and some P-III's, and I've hand-tweaked my rpmrc compilation flags for each arch.)

Having a system similar to 'apt-get source <foo>' (perhaps 'apt-get build <foo>'?) where you only suck-down SRPMS and build them locally seems like it would tend to resolve a lot of people's problems when it comes to disparate compilation environments. In the case of big software projects/packages like GNOME (for example), I don't think there's any kind of solution, unless either:

  • The GNOME project specifies how software should be packages
  • The various major GNOME provideers (Ximian, Linux distros, Sun, etc..) agree on how GNOME is to be packaged
  • The LSB steps-in, and decides how GNOME should be packaged

In any case, I think this problem is mostly academic, and a "geek-centric" problem, as most commercial installations of Linux will be centered around a particular distro, with a particular packaging "style" that the customer will probably emulate, when it comes to in-house RPM generation.

At least, that's how I've seen it, and I've been on both ends...


Companies and communities, posted 8 Nov 2002 at 01:22 UTC by realblades » (Journeyer)

RedHat is a business and do things in their own way. That is probably a way that they see as beneficial to their business (and it seems to be rather successful). They want to make money. They want to (or in a sense, have to) limit the stuff they support and make things fit on a set number of CD's (preferrably as few as possible without hurting the business) etc.

I personally consider RH a very nice and smart company but them being a company and not an organization does make some things (like communities and contributions) difficult.

A (though not neccessarily the) solution? External community. Having the apt tool on rpm has made these very useful and RH and rpm are a fairly clean and neat system to start with. At least after removing most of the helpful gui's and living past Anaconda :)

A top few of the repositories/projects that I use (not mutually exclusive!):

Problem Solved., posted 25 Oct 2003 at 11:27 UTC by murrayc » (Master)

Red Hat's restructure, as the open Fedora project, solves this problem perfectly. At least I know now that other people were thinking the same way.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page