Microsoft patents ASF media file format, stops reverse engineering

Posted 5 Jun 2000 at 22:15 UTC by atai Share This

Microsoft patents the ASF media file format and stops the author of VirtualDub, a GPLed video capture and processing program for Windows, from supporting ASF since he reverse-engineered the ASF file spec. The consequences are numerous: Third parties cannot develop their own tools to decode ASF or convert ASF to other formats, and if ASF becomes the dominate media format on the Internet (due to Microsoft's proprietary but high-quality MPEG4 codec and strong marketing), Microsoft gains de facto technical control over the creation and distribution of digital media. And if patenting file formats becomes a common practice, it can have a chilling effect on free software development since the reading/writing of data in popular formats (say, Microsoft Word files or MPEG4 video) would be prohibited, unless one uses designated drivers.

The following is the detailed account from Avery Lee, author of VirtualDub, about how Microsoft forces him to remove the ASF-to-AVI conversion functionality from his program (copied from his website, http://www.geocities.com/virtualdub/virtualdub_news.html):

5/12/2000: VirtualDub 1.3d released; ASF support removed at request of Microsoft

Today I received a polite phone call from a fellow at Microsoft who works in the Windows Media group. He informed me that Microsoft has intellectual property rights on the ASF format and told me that, although I had reverse engineered it, the implementation was still illegal since it infringed on Microsoft patents. I have asked for the specific patent numbers, since I find patenting a file format a bit strange. At his request, and much to my own sadness, I have removed support for ASF in VirtualDub 1.3d, since I cannot risk a legal confrontation. This unfortunately means that I can no longer redistribute versions of VirtualDub older than V1.3d. (I did appreciate, though, that I heard this through the programming staff and not the legal department.) However, when I asked, he did say it was legal to convert from MPEG-4 using the DirectShow system. Therefore, I have decided the following about VirtualDub 2.0:

It will have an external input plugin SDK. It will have support for DirectShow ASF input.

Also, let me plead this case to you:

Please, please do not encode to ASF unless you are absolutely sure of what you are getting into. Microsoft claims it is illegal to decode ASF outside of their drivers and does not allow transcoding compressed MPEG-4 to other formats with the Windows Media Format SDK, making it nearly impossible to do so legally. Before transcoding to ASF, make sure you understand that it will be difficult for anyone to decompress your file, even just to MPEG-1 for better viewing performance. By using ASF, you are trapping your content in a less open format and restricting who can view it, even within the Windows platform. You can encode video files in MPEG-4 V2 with comparable quality and you will still be able to distribute in the open, unrestricted AVI file format.

This is the change log for V1.3d:

Build 10803 (Version 1.3d): [thanks to Microsoft] * Support for ASF and MPEG-4 V3 has been removed at the request of Microsoft. I can't tell you how disappointed I am at this, but Microsoft says they have intellectual property rights, and I can't do anything about it. This makes me very sad.

5/13/2000: VirtualDub and ASF further explained

I've received some email regarding VirtualDub and ASF, and extend my thanks to all who expressed their sorrow. However, I'd like to clarify some things about the current situation.

VirtualDub is GPL. Unless you modify it and redistribute it improperly, redistribution under the terms of the GPL is not piracy; you are allowed to do that. VirtualDub has never been sold in stores, or even online -- at least, not by me. I have never made a cent off this program. I cannot request a Microsoft license for two reasons. The first is that I cannot pay for one, if they request money for it. The second, and more important, reason is that any license I request places further restrictions upon the code that are not allowed by the GPL. The code base for ASF would be undistributable under the GPL because reuse of the code by anyone else would require them to obtain a license from Microsoft themselves. This prohibits obtaining even the Windows Media Format SDK, which is licensed freely by Microsoft under certain restrictions. I cannot, and will not, bastardize my program to the extent necessary to relicense it; VirtualDub 1.x has always been released under the GPL with source code, and this will continue through 2.x.

Microsoft most likely had a bone with VirtualDub because its ASF support had two important features that the people in Redmond did not want available. The first is that, since VirtualDub treated ASF essentially as a poor man's AVI, it was capable of processing ASF files in Direct Stream Copy mode, meaning it could rip the frames out to AVI at over 4,000 frames per second. The DirectShow filters force real-time conversion, and the DirectShow AVI output filter mucks the audio synchronization. The second is that, more generally, it allowed compressed ASF files to be transcoded to other formats. Microsoft is concerned about the rights of content providers, and they see this as prohibited behavior. The individual I talked to over the phone indicated that this would not be allowed under the Windows Media SDK either, although it would be allowed with standard DirectShow linkage (presumably because Windows Media Player has to do this). I consider this kind of restriction to be the rape of fair use and consumer rights in favor of the content producers, but that's my opinion.

Another fact you should keep in mind is that ASF support in its current form was doomed anyway. Starting with Windows Media Player 7, Microsoft is moving away from standard ACM/VCM codecs and going wholly DirectShow. VirtualDub was never able to use DirectShow codecs, which is why you needed to install Windows Media Tools to get the proper codecs even though Windows Media Player could play the files. ASF content created with digital cameras has been off-limits to VirtualDub due to the lack of a MP4S codec in VCM form, and this trend would only have continued through WMP7. It also would have been trivial for Microsoft to change the linkages between NSREX and the existing VCM codecs, and break VirtualDub's ASF support that way. Moving to DirectShow is the right way, being legal in the eyes of Microsoft, and more importantly, would require that they break Windows Media Player to break compatibility with the new import filter.

Some of you believe that I was misappropriating Microsoft's intellectual property in my ASF handler. I don't see it that way. Remember, we're not talking about any sort of compression technology -- VirtualDub has always used the official Microsoft codecs for MPEG-4 V3 or Windows Media Audio decoding. The problem was that I had reverse engineered the ASF file format itself. This was done legally; I created my own content in both AVI and ASF form and compared the results at the byte level, without disassembling the ASF file format drivers themselves. I don't see what intellectual property Microsoft could claim in ASF, it being a universally unimaginative and poorly-designed format, but I respected their dubious claim to IP, and therefore removed ASF support at their request. Let them have it; there are better battles to fight than this one, and remember, this is not the end of ASF support in VirtualDub, since DirectShow is still a valid path, by their own admission.

We're not talking about a codec problem here. Microsoft claims patent protection on the file format. Remember these implications the next time you consider ASF for your content:

A broken ASF file not accepted by the Microsoft parser would be lost; the patent would prevent anyone from writing a byte-level tool to recover the ASF file. A third-party Linux player wouldn't be legal, since there would be no way to legally extract the file data, even if third-party video and audio decoders were available. Attempting to transcode a compressed ASF to another format would be impossible with any Microsoft-licensed tools, even if you have the permission of the copyright owner, or even if you are the copyright owner, because the Windows Media Format SDK license requires programs to actively block this action. For instance, Microsoft compelled Nullsoft to disable DSP plugin support in WinAmp with Windows Media Audio content because the DSP interface could be used to transcode, even though DSP plugins normally just process the audio.

Patent ?, posted 5 Jun 2000 at 23:08 UTC by chbm » (Journeyer)

What exactly is the patent number ?

MS ASF patent number, posted 5 Jun 2000 at 23:17 UTC by atai » (Journeyer)

Avery Lee did not indicate the patent number, but a search of the patent database shows the following:

http://164.195.100.11/netacgi/nph-Parser? Sect1=PTO2&Sect2=HITOFF&p=1&u=/netahtml/search- bool.html&r=1&f=G&l=50&co1=AND&d=curr&s1=ASF&s2=Microsoft&OS=ASF+AND+Mic rosoft&RS=ASF+AND+Microsoft

------------------------------------------------------------------------ --------

United States Patent 6,041,345 Levi , et al. March 21, 2000

------------------------------------------------------------------------ --------

Active stream format for holding multiple media streams

Abstract An active stream format is defined and adopted for a logical structure that encapsulates multiple data streams. The data streams may be of different media. The data of the data streams is partitioned into packets that are suitable for transmission over a transport medium. The packets may include error correcting information. The packets may also include clock licenses for dictating the advancement of a clock when the data streams are rendered. The format of ASF facilitates flexibility and choice of packet size and in specifying maximum bit rate at which data may be rendered. Error concealment strategies may be employed in the packetization of data to distribute portions of samples to multiple packets. Property information may be replicated and stored in separate packets to enhance its error tolerance. The format facilitates dynamic definition of media types and the packetization of data in such dynamically defined data types within the format.

------------------------------------------------------------------------ --------

Inventors: Levi; Steven P. (Redmond, WA); VanAntwerp; Mark D. (Seattle, WA); Dowell; Craig M. (Redmond, WA); Knowlton; Chadd B. (Bellevue, WA) Assignee: Microsoft Corporation (Redmond, WA) Appl. No.: 813151 Filed: March 7, 1997

U.S. Class: 709/217; 445/34.1 Intern'l Class: G06F 017/50 Field of Search: 348/38,441 370/232,17,94.3,94.1,312,474,329 375/46 371/37.4,40 369/275.3 386/111 395/325 709/217 380/14 711/202 455/4.2,34.1

Huau. That must a Bogon Guiness Book entry., posted 6 Jun 2000 at 09:30 UTC by chbm » (Journeyer)

This is one of the most bogus patents i ever seen. Not only it should not acceptable as a patent, since it does nothing, it mostly describes MPEG and H.26[13] and H.323. It has got prior art written all over.
Just one more stone in the "MS owns (part of) the world" central office.

DOJ recommendations, posted 6 Jun 2000 at 14:54 UTC by listen » (Journeyer)

Would the DOJ recommendations stop MS doing this kind of stuff? IIRC, it had stuff about disclosure of interfaces in it, but I don't know if it forbade ridiculous patent claims. If not, the DOJ needs to ammend its recommendations.

This is really related to the DVD issue., posted 6 Jun 2000 at 19:03 UTC by cmacd » (Journeyer)

One of the features of The DVD System is that the player will not transcode the stream into anything that can be re-distibuted.

Micro~01 has been pushing their streaming media as "protecting" the content from being transcoded. One can apperently set some bits in the stream, and the user will be prevented from using the file on the local disk. The Media Giants like this feature, as they can then treat the stream like the radio, where making a useable copy is blocked by having the DJ talk through the first part of the music and or the songs running into each other.

Using a patent to ensure that any use of the format will be closed is probaly seen by micro~01 as needed to allow content providers to buy the MS solution to streaming media. The plan is no-doubt to have ASF replace MP3 as the format of choice for web-content. You will get a stream you can listen to, and if you give your creditcard number you can have the same stream in a format you can use on YOUR computer, at least until you re-install Windows. They may make a Linux and or Mac player downloadable just so that they cannot be accused of only catering to Windows.

This is the sort of danger that was pointed out with the Amazon patent dispute. Microsoft is not going to give up without a fight. They will be using their considerable leverage to try and set as many standards as they can before they are delt with by the DOJ.

Will this affect free software

Probably

Look for streaming audio content in wordprocessing files. It would be quite the commercial advantage for the supplier of the most popular word processor to prevent competitors from being able to import documents. Zigs and Zags have kept WordPerfect and the like one step behind up till now, but the gap is closing.

Now if someone wants to fight, they would be dealing with an organization big enough that the US government has trouble playing the game.

Since the code WAS released under the GPL, perhaps someone outside the reach of the US patent office might want to create a module that could "plug-in" to an interface. The main program would have to use plug-ins for all the converters so that this (and whatever other format that comes with Lawyers) could be pluged in without the interface becoming something that was strictly related to the patented format. This implies that the plug ins would have let the main program know the format(s) that they support, and the main program would have to build a list of what plug-in to use every time.

One of the Main reasons for wanting open software in the first place is to avoid having ones data trapped in a format that will not be supported in the future. Every format should be transcodable if it is to be translated to the next big technology when that comes down the pike.

Just let it go..., posted 7 Jun 2000 at 00:27 UTC by BeeWarlock » (Apprentice)

Obviously, we, the free software community, have to make our own format. MPEG 4 will never be really free, not QT encapsulated or ASF encapsulated, or even not encapsulated at all. No it won't. Both Apple and M$ have huge interests in the Motion Picture Experts Group. Here is were we have a big challenge. This is a point were we really have to innovate, not just when it comes to philosophy (which is also important) but also when it comes to technology.

Redhat, SuSE and all the other big linux companies: work together on this (and make it free), it's one of the things Joe the programmer can't do, because it's really hard stuff.

Going back to reality today, these are the most interesting places concerning free media architectures I found, while looking around after it a few days ago:

GStreamer - Free (like GNU free) streaming media architecture. This projects looks good I think, but it isn't very polished yet. Well, actually it's really experimental... no efforts making a new codec there, of what I've seen, concentrating mostly on the well established formats, like mpeg, but we also need an underlying architecture, right? Anyway, it's (oh course) plugin based, so new codecs can be glued into the project easily.

the Ogg project - This project's goal is making free media codecs. It only got a audio codec for now, Ogg/Vorbis, but it's a well established project, started by the maker of the great CD-ripper cdparanoia.

The rest of the stuff I found at least, looked either too much like vaporware, or was dead - if anyone knows about other interesting projects/links, I would be happy to know. I'm also interested in links about basic video compression techniques, because I know zero about the subject, and it really, really interests me.

Anyway, this is an extremely important thing for the free software community, and I think, instead of making another text editor or hello world program, this is a great thing to "waste your time" on.

Playing in the Courts, the way of the audio industry!, posted 7 Jun 2000 at 00:53 UTC by highgeek » (Master)

As you all are already aware, the entire online music industry is currently tight up in lawyer land. To fight well on that ground you need amunition and this is where the patents, trademarks and such come into play.

Everyone thought that the fraunhofer and thompson patents on MP3 were the big hassle of the online music industry. I am sorry to say, but it really is just beginning and this move by Microsoft is evidence of that.

innocent thought..., posted 7 Jun 2000 at 01:48 UTC by jpayne » (Journeyer)

but isn't this similar to the RSA patent that isn't applicable overseas? Something about it being in use or public knowledge before the patent was filed....

US patent, sure, but what about elsewhere?, posted 7 Jun 2000 at 07:56 UTC by bagder » (Master)

As was indicated by jpayne above, isn't this patent valid in the US only?

That would at least enable non-US citizens to maintain those parts of the program. It would probably best be written as a kind of plugin so that the main program can still be independently authored and developed.

Or am I wrong?

ASF format documentation, posted 7 Jun 2000 at 17:30 UTC by sergent » (Journeyer)

Microsoft did publicly document the ASF format some time back; it's not on their web site any more (has not been for a while). They circulated an internet-draft documenting the format at one point as well...

This, of course, has no relevance to the current situation, considering the format is patented, since you would still have to pay them to sell a product that uses it. The QuickTime format is patented (5751281), but there is lots of software that uses it (including some free software). Apple could get up at any time and decide to start enforcing it, but it's not really in their interest to do so, and I don't think they would. QuickTime is actually publicly documented in full, which is why there are things like xanim that can play it.

Microsoft has the leverage to be able to try to tie their media format to their platform, so they do things like this enforcement action.

It would be really nifty if someone like Apple would do a patent grant to allow free software developers to use the QuickTime patents in general. They do something like this for any software that is part of their public source program, but there are too many strings attached.

IANAL.

ASF has been adjusted with different codecs several times., posted 7 Jun 2000 at 22:16 UTC by highgeek » (Master)

The latest is the MSAudio saga, that later got re-launched an renamed as Windows Media Technologies. I bet the specs you saw were for the earlier ASF format, not the additions that were made with Msaudio. Msaudio was supposed to be the MP3 killer. I think real still has their msaudio sucks pages up. :) I haven't spend too much time investigating the ASF format, since it seems to change or be adjusted just about every minute. It is kind of like a WAV file, which in reality is simply a header on top of just about any codec or raw PCM data.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page