The financial situation for VA Linux looks bleak. Those of us who depend on SourceForge for development are, or should be, getting worried. Already, one of the core services provided by SourceForge has fallen into "no longer being maintained". It seems prudent to develop back-up plans.
Of course, it's possible that SourceForge will continue to function smoothly. It wouldn't be at all unexpected for a company with a large stake in open source to take it over. However, many of the dot-com closings have been abrupt, taking even employees by surprise. Ghostscript, for example, simply cannot afford this disruption.
Free software is robust and decentralized enough that I doubt the closing of SourceForge will have much long-term impact. If nothing else, it will result in a much needed "garbage collection" of hopeless projects. All projects worth their salt will find a new home without much difficulty.
GNU Projects will probably find themselves right at home on Savannah. This system is more or less a clone of SourceForge, though it is lacking such essential features as a bug tracking system. It looks like the team is considering reworking much of the code. We wish them well. As an aside, the comments about the "unmaintainable state of the SourceForge code" do not help to inspire much confidence in VA Linux.
The Ghostscript project has decided to host itself. They've put a new machine into colo, and are bringing up CVS, bugzilla, and the usual services. We've been wanting to migrate to bugzilla for some time because of the shortcomings of the SourceForge bug tracker (again with the confidence). There are other excuses, including a dedicated regression server, the ability to archive test files (not all of which can be made publicly available), and better responsiveness for the ghostscript.com website, which is currently on a DSL line.
One issue is migrating the services from SourceForge. CVS shouldn't be too big a problem, because SourceForge makes nightly CVS tarballs available. However, they do not provide access to the bug tracker database. This is quite important.
One possible solution to migrating the bug database is to use a web crawling script. There's alreadyone written (used for making backups), but it doesn't do any of the work of parsing the HTML or converting it to a format bugzilla can understand. The Ghostscripters would be happy to work with others trying to do the same thing. It would be even better if the SourceForge team would make the bug database available.
Could the SourceForge people help with this? They're certainly in the best position to ease migration, and it would be greatly appreciated.
SourceForge has been an incredibly valuable resource over its lifetime. It would be quite a shame to see it go, and this cat sincerely hopes they find a way to stay afloat in these turbulent seas. That said, it's never a good idea to be vulnerable to the failure of one server, or one company. It's also not the free software way. The need to keep good backups is as strong as it ever was.
What plans does your project have for dealing with the contingency of SourceForge's demise? What kinds of plans are reasonable for doing backups and preparing for a migration, either planned or forced? Your comments are welcome.
But I do miss pipermail.
I always hated the interface on geocrawler. It was great to have all that disk space for archives, and way too painful to use.
Fortunately, mailman archives were also available and we used those for Utah-GLX and other projects I was involved with. The mailman interface is much more convenient for browsing (though you couldn't search; we had to implement that ourselves) and speaking of migration, the accumulated posts are available for download right there on the archive page.
But last December those stopped working. Perhaps it's part of the same problem though. The earliest bug report I found on the issue is resolved "won't fix" in favor of Geocrawler. :)
I guess I'm just venting. But I've always been concerned about Sourceforge's willingness to play nice with the rest of the community. I'd like to add to the call for more backup tools like the cvs repository tarballs.
I was distressed to hear that SourceForge was going to start using proprietary software on the portal upon which they host free projects - it seems unnecessary. Having a public project, or more than one, to fulfill the function of SourceForge would be desirable.
HP has just donated two hosts to FSF. I wonder if they are being used for this project? I'd be inclined to get the folks at HP to continue to support this sort of work.
Thanks
Bruce
Loic or someone on the Savannah team will likely step up to say something about this soon, but I thought that in the mean time I would mention that Savannah is not only for projects that are part of the GNU Project, but indeed for any Free Software.
Let me also say that Savannah is one of the best things since sliced bread in some aspects. In the past few months that the Savannah hackers has worked on the software and installation, it has grown from becoming just another hosting service to being an integrated part of the GNU Project, while still providing services for non-GNU projects.
For the GNU Project, it is not just a hosting service but a framework for many of our operations.
Hi,
You can get an XML export of the tracker data. This is a hidden feature and you'd better use it before it's removed. I'm not aware of any way to export mailing list archives and user lists which is a big annoyance given the fact that geocrawler is unmaintained.
Tracker export (add ?group_id=xxx&atid=xxxx).
The sourceforge site status page says nothing about the Geocrawler outtage (as of Oct 23, 9am EDT).
While I know that VA's economic shape is bad for some time now (first they sell hardware, then open sourceforge to promote the hardware they sell, then they stop selling hardware) and Jacob Moorman is way overloaded with support work, I'd like to get a notice that information is going to be lost with some time to either mirror the service that has gone out or at least cache the lost info on my home box for future usage.
I hope they at least have some ethics before shutting down other services. If they purge the trackers, VA can expect pounds of email hosing them. If they purge the CVS repositories without previous notice, I wouldn't be surprised if VA's employees started to get hunted down and shot.
Now things are looking dark somewhere else too. What next ? We're gonna live out ot kuro5hin ? *scream*
It's unfortunate, but dotcom problems can appear so quickly that employees who would otherwise make notifications may no longer be in a position to do so. The responsible parties might be shifted to another department and no longer involved with the project in question. They might even be out on the street. Be as sympathetic as you can; it's difficult to make the transition from community participant to layoff participant.
Projects die and projects are shuffled around. For-profit corporations lose their funding or are forced to re-evaluate their priorities. So do non-profit corporations. So do individuals. Don't fall in love with a single project hosting service, VA's, FSF's or anyone else's. The world is too unstable a place to depend on a single venue. The safety of redundancy is essential. So is a willingness to provide your participants an easy exit path.
Who else is working on project hosting? This would be a good time to mention your efforts.
Heh. I moved my projects to sourceforge less than six month ago, and I think it would be a pain to move them again. And for now, sourceforge still seems to work fine - including geocrawler.
Obviously you *do* want to keep backups though. Every night I'm backing up the full contents of my projects web pages and cvs repositories. and I'm even thinking about setting these backups online on other hosting sites like savannah.
The $64K question though, is how can I keep mailing lists archives AND lists of all subscribers. I dont have that right now, and it sucks.
I'm also interested in learning how to synchronize two cvs servers (have one of them mirror the other). Obviously I could get the cvs backups and import these on the mirror server but I dont think savannah supports that yet.
Back up your projects while you can.
Hello,
My name is Patrick McGovern and I manage SourceForge.net.
I wanted to take a few moments to reply to some of the comments in this thread.
I think there is quite a bit of confusion.
About SourceForge.net. SourceForge.net is not going away. VA is not going away. The company as well as the technology sector is having a hard time right now, but the company has the money and resources to be around for the long haul.
SourceForge.net is growing. The number of projects, users and pageviews continue to grow. The site is getting additional resources (not less) to be successful. We recently added 70 servers. We've added systems to the compile farm. We are adding to the number of download servers.
We are hiring. I need talented folks to work on the site. If you are a skilled DBA send me your resume. (pat at sourceforge.net)
If you do have concerns, by all means back up your code. I think it's a prudent thing to do anyway. The SourceForge.net team creates tarballs every night that are downloadable.....you can download a copy of your code anytime you want.
About Geocrawler:
Why is VA no longer actively maintaining Geocrawler? The answer is simple, we are creating a replacement that will be available in the next 10 days. It will be integrated directly with SourceForge.net (for sourceforge.net mailing lists) to make searching and archiving easier for the User. We are not spending less resources, as some have suggested, but more to make the experience better for the Open Source community.
If you have any questions about this, please feel free to email me at pat @ sourceforge.net
Thank you.
Pat-
It's nice to know you're listening, and that the rumors were largely unfounded. I really like the services you have on sourceforge and its great to know we can count on still having these in the future.
I'm still doing my backups/mirrors anyways :) because I think that cant hurt. And I'll still be lookingfor a mailing list backup solution too...
I have talked with Pat about this in the recent past and its still a concern. I know that VA believes that they are going to stay in business and I hope and pray that they do. However, as phpGroupWare has grown I am more and more concerned and feel a responsibility to my project to make sure the code, bug tracking, etc... is in a safe place. This http://sourceforge.net/export/sf_tracker_export.php looks great, but doesnt work very well. If I knew I could get a backup of my tracker records, my developer list and permissions, and all other such data on a regular basis then I would feel ALOT ALOT better. I would know that I could take regular backups to my own machine and if worse came true and sf.net was closed I would be able to resurect my project details in short time.
Seek3r
Hopefully, whatever improvements ghostscript creates in bug tracking, project management, etc, can be shared with GNU's project hosting site. Ghostscript's contributions can probably save the GNU site hackers a lot of work.
Anyone who is building software engineering tools is welcome to host their project on tigris.org. It is a small, focused community where all the other projects and developers are also focused on software engineering tools. Also, I work with potential project leads to get their project launched on the right foot. One recent project that has recently switched from sourceforge.net to tigris.org is binarycloud. Other popular projects on tigris include Subversion (a new version control system), Scarab (a new issue tracking system), and ArgoUML (an open source UML tool).
On the tigris.org home page, there is also a list of some other focused communities on various topics. These sites are sponsored by various companies that have a real interest in and commitment to the success of their communities.
The examples that we have looked at in building these communities are the big, successful open source projects: apache, mozilla, gnome, kde, etc. Each of these is a focused community with a lot of overlap among the interests of community members and a sense of stability.
Just wanted to mention that there is another Sourceforge-Clone: Berlios. It uses the Sourceforge Software and currently hosts 167 projects. Berlios is a project of GMD Fokus, a german research institute, and supported/funded by a number of companies and the german ministry of economics and technology.
I have several things to say that are all unrelated...
First off, the idea of a garbage collection kind of disgusts me. I agree that there are tons of things in freshmeat's database and on sourceforge that are dead or unsupported or flat out trash (I suspect that there were a lot of "greps implemented in perl" that were produced when Redhat and others opened IPOs to the community)but a big part of the whole open source thing is that I can look at what bits and pieces there are and make a decision on whether or not they do anything for me. I can list at least 6 projects off the top of my head that are dead and that I've gained from, either by seeing their mistakes or because the few things they did were useful to me. Usually the hardest part is getting started. I think having the full repository is a good thing.
I was also one of the people who didn't like the idea of source forge initially. 2 reasons, a) I didn't think VA was going to last in the PC business, not against Dell, Compaq and HP. b) With one company owning sourceforge, VA, slashdot, freshmeat. OSDN, and other big opensource platforms it has the potential to create politics and bias in the community. So far I have been wrong. I'm still not really fond of centralizing everything and I think that to some degree it degrades the value of their freshmeat business (before sourceforge, freshmeat was more useful) but I hope they keep doing well and as suggested, I would back up everything you've got on there simply because that's a good idea. Perhaps sourceforge could also make available a set of backups or something for a fee. I'd be willing to buy a set of DVD for $100 or something that contain all of sourceforge. I'm not sure how practical that is, maybe a snapshot twice a year could be made. I'm sure people would want this and it would provide a sense of protection should VA decide to pull the plug.
SCM.. To end my ramble. I bought a VA linux rack mount a while back and it has been a fabulous machine. 2 weeks ago they called me to sell their professional services and to sell me a custom sourceforge installation for my development needs. This is something my company has struggled with for 18+ months now. The OSS world is great for so many things except SCM. CVS is wonderful but it doesn't have issue tracking. Bugzilla is great but it isn't connected to the source code. Sourceforge is close to what you'd need to do professional team oriented software development with OSS but it's not there yet. I ended up writing a python script frontend to cvs to integrate it in to bugzilla some. I think sourceforge has a huge potential for this, regardless of the financial situation, if they could provide a complete end-to-end solution for the SCM problems to the community, I know that there are companies that would pay them to customize it and install it for them. I'd think you could make a business from that alone.
I think the real solution to dependency on a hosting provider is to devise a metadata format which can summarise background information about the project (as seen on sites such as freshmeat) as well as information about the hosting facilities used and the developers involved. In my usual style, I'll call this a project-hosting file.
Tools could then be developed to maintain project-hosting files in a nice, easy-to-use way and then use this when setting your project up with hosting providers (if they have a project-hosting-friendly setup service). Basically, there need to be conversion tools to sync between project-hosting files and mailing list config, CVSROOT files, webserver and DNS configs... and probably many more. Most of these will merely do conversion to a format useful for a particular service, but then even if they're not a project-hosting-friendly provider, you can still keep that service up to date with it, hopefully.
My personal aim here, of course, is to distribute a single project across many different hosting providers, to be able to make maximum use of donated services, etc, rather than having a few monolithic "sourceforges" as we do currently. What I'm missing is the time to develop this. My first thoughts are to use XML, guile and gtk to make the tools, but that's just personal preference and I really don't have time to spend on this right now.
I'm more than happy to discuss this with people and I'd *love* it if someone could run with this idea and realise my dream. I think it would be a useful thing for the community to have. The key thing is for it to be able to extend to take in new services which we don't use today.
Anyone up for a challenge? I think the place to discuss this is probably the coopx list -- please come and join!
I believe that the existence of SourceForge is a major reason why Gimp-Print is as successful as it is. I don't have my own hosting facilities handy, and like most administration-intensive things, it has substantial economies of scale. Not only that, but it provides an easy way for people to join the team.
I certainly take a CVS snapshot every night, and I keep 4 backup copies of it. I've been doing this since very early on; the CVS repository is the lifeblood of the project. However, for a non-commercial project there aren't a lot of choices. If Sourceforge were to go under we'd probably have to move to savannah (not that I'd object to it per se, but we need somewhere, and moving a project is a huge pain).
Tjansen mentioned BerliOS as a sourceforge clone. BerliOS' setup has a couple of weaknesses which you should be aware of before moving there. (I also pointed them out to the BerliOS people so they know about it.)
To just go to a development site which offers service gratis for your project might not be enough to secure your freedom in the mid term, you have to choose carefully.
I have LinuxQuality hosted at SunSITE.dk, after I requested a host on the linux-kernel list and they responded with a kind offer.
One very nice advantage of SunSITE.dk is that they have database facilities. They created a MySQL database for LinuxQuality, and told me they would be experimenting with PostgreSQL. I'm not sure if they actually have made PostgreSQL available yet.
It is a volunteer run facility that last year decided they would devote to the promotion of open source.
They're also a bunch of really nice people.
We, at ibiblio (aka metalab aka sunsite) are always happy to host open source projects. We've been at it for a while ;-> and we do offer most of the services as sourceforge and sunsite.dk including mySQL.
We're not down on SourceForge though. In fact, we are one of the official sourceforge ftp sites and are glad that they support the communities.
My Run at OSDN has been very good actually, I built a great network, that pretty much runs itself at this point, and then got sick, I was offered voluntary severance, and I was thinking I want to do so much more now anyway, getting more into code than network engineering and systems admin anyway. I'm looking at moving to MD, the revolution is over, I've made a decent mark on it :)
OSDN generally has had a rough history, when VA took over Andover, things were looking pretty good, but the formula was changed. There was nothing wrong with how Andover was working, however there was a culture clash internally between VA and Andover, Andover was originally a software company, then it was a media company, VA was a hardware company. VA now looks like its becoming a software company. Sourceforge itself needs a lot of work, it has since I became involved, it was kinda clunky, had only two people working on it, and had to manage a very complicated environment that grew organically, rather than planned. By the time VA decided to focus on it, it was a mess, the admins on the west coast and coders did a wonderful job of making it work and keeping it running.
While as of this week, I am no longer involved with VA in anything but a writing capacity (I'm contracted to write several articles over the next year for OSDN sites), I have nothing but good to say about the people there, and jumping ship will not help VA stay running.
Bruce Perens and I disagree with each other here:
There is *nothing* wrong with serving the Open Source community by using proprietary software, many OSS companies have sold proprietary software along side GPL'd software for "value added", the spirit of the BSD license says that you can make changes and sell that code too, I think businesswise, the GPL made little sense to rally around, and VA and RedHat tend to prove this, its amazing that with the excesses at the beginnings, that any of us are still around. Why are we? One answer: Talent.
The people involved with Open Soruce have been extremely talented if excesssive people. I am honored to have worked with you all.
One of the minds behind Berlios is Jörg Schily, author of cdrecord and maintainer of cdda2wav. He is an open source hacker since long and I have no doubt about his attitude.
Regards, Marc
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!
del.icio.us
Digg
Google bookmark
reddit
Simpy
StumbleUpon
Furl
Newsvine
Technorati
Tailrank