21 Aug 2003 mikehearn   » (Journeyer)

Inspired by the ever interesting Chris Brumme, who blogs for Microsoft on .NET and Windows technical matters, I thought I'd steer this blog more towards technical writings on autopackage and Wine.

Though I do have a personal journal elsewhere, I haven't posted to it for a while, partly because now I'm back at home I'm just a phone call away from all my friends, so we have less need of it to keep in touch, and partly because since my hostelling trip around Europe ended unhappily for various reasons, not much is happening around here.

If you enjoy these mini-articles then please let me know, and if there's something you'd like me to write about, ditto.

Wine and InstallShield

One of the problems that has plagued Wine for a very long time now is (perceived) unreliability. Many people fail to get Wine working with their favourite apps, or even any apps at all. Part of the problem is that we recommend you use a fake windows drive, then install your applications into it, though as far as I know this recommendation is actually buried deep in the users guide.

The reasons for that are pretty simple - typically Windows installations have huge piles of rubbish that get in the way and can confuse Wine, not to mention that it stresses components that generally don't get much testing, like the code which reads and writes to the binary registry format. In the age of NTFS filing systems, the lack of writeability can likewise cause difficulties.

Unfortunately many people fall at the first hurdle, namely installing apps. Thanks to the near total lack of package management on Windows, there is a glut of installers all of which have different behaviour, and many of which are very complex under the hood. InstallShield falls into this category, and is a program that has given Wine problems for many years now. The difficulties with InstallShield basically fall into two areas:

1) Window management
2) DCOM

Window management has long been an achilles heel of Wine, as Win32 window management is close enough to X11 to make a mapping worthwhile and valuable, but different enough to cause serious problems in certain scenarios. The main difficulty lies in the location of the window management code.

In Windows, the application is responsible for performing its own management, or to be accurate it delegates via the default window procedure (message handling code) to libraries provided by Windows. Those libraries draw the window decorations, provide the default move/resize behaviour and so on. That simplifies the architecture somewhat, and means that opaque resizes don't suffer as much from loss of sync between the border and the contents (a problem which Havoc Pennington has been writing a patch for in metacity) but also means that if the app freezes, the window becomes impossible to move, minimize or close. In fact in XP the kernel has a hack for this, such that if a window stops responding it's silently replaced with a faked OS level window so you can still move it and resize it.

In contrast, on X11 based systems like Linux, the window management is performed on behalf of the application by a WM, a dedicated application. This program draws the borders for the app, controls its position, which buttons are available and many other things.

Typically, this distinction does not cause problems, but some apps take advantage of the implementation details. WinAmp for instance, actually creates a standard window then overdraws its own window borders with the skinned versions. Clearly this isn't possible in X11 so for times like this Wine has an "unmanaged mode", where the window is disconnected from the WM and Wine performs all its own management, which is closer to how Windows itself operates.

Unfortunately, that approach has several problems as well. For starters, the windows don't respect the virtual desktop you're on, nor z-ordering. This means that unmanaged Wine windows seem to be "nailed" to the screen, often making the app unusable in the real world.

The problems with InstallShield arise from the fact that typically, for reasons of intense LSD tripping on behalf of the developers, these installers take over your screen with a nice big blue background. Unfortunately, Wine cannot currently make managed full screen windows, as the protocols for that have only recently been agreed upon, so it defaults back to unmanaged mode. We now have a problem - with no real window management in use, the actual installer windows it opens on top of the blue background will appear underneath the background window, and worse there is no way to raise it to the top.

Because of this, for some time Wine has shipped in its default config file AppDefault entries which force all known installshield executables (they have seemingly random names) into "desktop mode". This, in theory, creates a window which contains the virtual Wine desktop, and means that Wine can accurately control and render the Windows without X interfering. Unfortunately for architectural reasons, desktop mode doesn't operate as you might think - each window gets its own desktop.

Until recently this wasn't a problem, but a bugfix elsewhere in the x11 driver now causes Wine to die with an X11 Client error when it attempts to shift multiple windows in desktop mode around. This problem renders InstallShield effectively unrunnable. There are two possible solutions to this problem - to fix desktop mode so all windows appear in the same desktop (which is what you'd expect), and to implement "half managed" mode, which is where the WM controls the window, but Wine is responsible for its position and border rendering. XMMS does something like this, and it works OK. Eventually both approaches will need to be implemented.

This leads to the second problem with InstallShield, namely its extensive usage of DCOM and RPC. DCOM, or the "Distributed Component Object Model", is the rough analogue of CORBA on Microsoft systems. It's been around for a very long time now, and is used by many apps that you would never even suspect. If you've seen a Windows 2000/XP box that's been hit by Blaster, you'll know how vital it is that RPC and DCOM are functioning correctly at all times, even seemingly unrelated things like copy and paste rely on it. That's why XP will reboot in 30 seconds if the RPC daemon terminates.

InstallShield uses DCOM primarily for inter-thread communication. While CORBA is intended for out of process communications, distributed across networks, DCOM is intended also for invoking object methods within a process. The process is fairly simple - you create a COM object that has interfaces, then marshal one of those interfaces into a stream (or alternatively, the global interface table, an object I partially implemented in Wine a few months ago). You then pass that stream to another thread, which unmarshals it. Behind the scenes, DCOM constructs a proxy/stub pair that switch the method call into the first thread context.

Exactly how it does that is a topic for another journal entry, but until recently Wine did not implement this technology. That caused InstallShield to have problems, especially with correctly redrawing its windows, although enough of the OLE/COM architecture was present to enable it to basically work. TransGaming developed a patch for WineX which implemented this part of DCOM, and thanks to the votes of its subscribers this work is now available to WineHQ. Part of my work that I talked about in my first journal entries was merging this work into the (significantly different to winex) WineHQ tree. That merge still isn't complete, but hopefully I'll get it done before going to uni in October.

At that point, the final missing piece of the DCOM puzzle will be the ability to generate binary TLB files using Wines IDL compiler, as Alexandre deemed it unacceptable to take Microsofts version (a large binary file) and stick it into CVS. Wine will then have enough infrastructure to deal with even complex usage of DCOM and OLE.

That's important because many programs use it behind the scenes, not just InstallShield. For instance, I worked on an app that embedded Internet Explorer in Java, then programmed it from another thread. That involves cross-language, cross-apartment method invocations: not a trivial thing to do, as it involves late-bound method calls via IDispatch (where you pass the numeric ID of the method and a variant array of parameters to the IDispatch::Invoke method). Sounds simple, in practice IDispatch is so hard to implement that Microsoft strongly discourage you from trying, you're told to rely on the implementations they provide instead.

Until the work is complete however, you can continue to install Microsofts OLE implementation by running the DCOM98 executable with a recent version of Wine. Make sure your config file is up to date. Fortunately this approach works reasonably well, although as the DCOM updates could be pulled at any time, and cannot be redistributed, it's important that Wine gets its own implementation. You need to run apps like this:

"wine --dll ole32,oleaut32,rpcrt4=n foobar.exe"

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!