Advogato blog for robbat2

Gentoo Linux participates in World IPv6 day

Wed, 8 Jun 2011 11:07:35 GMT

In light of World IPv6 day, the Gentoo Linux Infrastructure team would like to announce new IPv6-availability of several services, and list the existing IPv6 services. Every service listed below is running a dual-stack native IPv4/IPv6 service, no tunnels.

The new services available via IPv6 are:

The existing services available via IPv6 are:

CVS/SVN/Git services for developers
http://sources.gentoo.org/
rsync://rsync.gentoo.org/ - our primary rsync rotation
rsync://${CC}.rsync.gentoo.org - our regional community rsync rotations
A number of our mirrors

All of our IPv6 services will remain online after today, unless serious IPv6 problems (esp. regarding routing) are encountered.

Gentoo would like to extend thanks to all our sponsors & mirrors who have provided IPv6 service, and the servers to make use of it!

Robin's 2011 conferences plans and ideas

Sun, 16 Jan 2011 11:23:56 GMT

Working on my conference travel plans and wishes for the year. I am downgrading OLS to a maybe, the cost is becoming more of a factor. Likewise, while I had incredible fun at FOSDEM last year, and OSCON in 2006, I cannot justify the airfare/hotel expenses for them. I would like to attend SCALE at some point as well, but uncertain for the same cost reason.

Confirmed:

April 11-14, MySQL UC @ Santa Clara, CA, USA [1]
August 17-19, LinuxCon 2011 @ Vancouver, BC, Canada [2]

Maybe:

June 13-15, Linux Symposium @ Ottawa, ON, Canada.

Would like to go, but out of my financial reach:

February 5-6, FOSDEM @ Brussels, Belgium.
February 25-27, SCALE 9x @ Los Angeles, CA, USA.
July 25-29, OSCON @ Portland, OR
(Not yet announced), Linux Plumbers.

Notes

I will be manning the phpMyAdmin booth, like past 5 years.
I have no accommodation yet, I'd love to split a hotel room at the Hyatt (or another spot within walking distance) with somebody.
Local this year, so no travel costs :-)

Complaining at Journalists again: Gentoo Security and the UnrealIRCd backdoor

Thu, 30 Dec 2010 22:14:31 GMT

Those that have followed me for a while might have seen me previously complain at journalism that's misleading, wrong, or outright fictitious. Now I've got another case...
This article by Ed Bott at ZDNet:
Linux infection proves Windows malware monopoly is over; Gentoo ships backdoor? [updated]

The article was first published 2010/06/12 20:37 UTC.
It claims to be "worse" when updated at 2010/06/14 19:30 UTC.

Gentoo had a revision bump to a known good copy of the tarball at 2010/06/12 16:34 UTC (using a different filename, and verified against the GPG signature provided by upstream), so it was ALREADY fixed when the article was published. The old revision was explicitly removed at 2010/06/12 21:18 UTC.
Commit data for fixes:
Changes for unrealircd-3.2.8.1-r1.ebuild
Changes for unrealircd-3.2.8.1.ebuild

The trojaned tarball was then removed from the Gentoo master mirror at 2010/06/13 08:00 UTC, about 11 hours after the article was published. It would have been sooner, but it was a matter of bad timing.

Gentoo bug 323691.

The article also claims: "There’s a great deal of comment in the Talkback section of this post about how official repositories can be trusted. It appears that system broke down thoroughly in this case."
This claim is bogus. The developer that updated the package made perhaps a mistake in trusting that the upstream had not been tampered with. However, in lacking anything to verify against (the upstream apparently did not sign releases at that point), he couldn't have detected the backdoor except by manual inspection of all the code. He downloaded the package AFTER it had been tampered with (2009/11/11 I believe), so he never saw the tamper-free version either.

The entire point of the Gentoo Manifests are to ensure that OUR mirrors are not the point where a compromise is introduced. We can detect upstream changes by this same mechanism, but they mostly tend to be upstream deciding to 'fix' something without bumping the version number. In this regard, they functioned perfectly.

P.S. I'm not saying the existing Gentoo mirroring is perfect either, see my prior writings on tree-signing, and the "Attacks on Package Manager" papers by Cappos et al., which are blocked only with the full tree-signing system.

On Google Summer of Code Applications

Thu, 30 Dec 2010 22:14:31 GMT

(This post inspired by Petteri Räty (betelgeuse)'s similar post

For this year's Gentoo GSoC projects, I'm a mentor on two of our suggested ideas (but also interested in totally new ideas that fit my fields):

upstart on Gentoo
Distfile Fetcher Intelligence

Do you actually understand the project idea?: This is actually a gap that I didn't expect to exist, but I have seen in previous years. This is mainly a difference of expectations between the proposal and what the potential student sees as what the idea really entails.
Using Upstart as an example, it supports an existing init.d compatibility mode, but we're not interested in that. Instead we want our init.d scripts to be treated just like upstart jobs (located in /etc/init/). The init.5 manpage shipped with upstart gives a good start...
Code maintainability: betelgeuse spoke about long-term maintenance, but you should think about it long ahead of that. Some degrees of abstraction, and avoiding difficult to understand logic should be prevalent here. betelgeuse mentioned spaghetti code, but it's important to realize that even well formatted code can impose a much larger mental workload if not well thought out.
Timezones, Timezones!: Most of your project should not be blocking on asking for mentor advice, as timezones and real world pressures often conspire to prevent easy real world communication. I may live in UTC-7, but my hours drift as needed by work but I tend to be online anywhere between 17h00 UTC and 10h00 UTC. If you're trying to communicate with me on a regular basis, this can be tough, so being able work on a problem independently, ask highly directed questions via email can go a long way.

Advice for Google Summer of Code students

Thu, 30 Dec 2010 22:14:31 GMT

Good advice for any prospective GSoC student, regardless of gender

I'm also a mentor for Gentoo again this year, after taking a break last year.
You can find our list of potential ideas here: Google Summer of Code 2010 ideas for Gentoo
But don't limit yourself to them! Creative ideas can get you very far too :-)

I'll also be the infrastructure contact for the accepted SoC students, for any issues you have with the source code repositories (we'll be offering Git again), your shell accounts, and a sounding board on deploying your successful project (for those that hosting or larger resources).

Spamtrap addresses vs. list confirmation emails, or how to lose 2k list emails

Thu, 30 Dec 2010 22:14:31 GMT

In the early hours of this morning, a spammer managed to get the IP of the Gentoo list server on the NiX Spam RBL... simply by spamming the subscribe address :-(. This caused approximately 2000 deliveries of normal list mail to be rejected while the server was present on the RBL.

Notice the subscribe request, line 0004. (whitespace added)

0001 Feb  1 00:15:56 pigeon postfix/smtpd[29314]: 52278E0778: client=unknown[210.212.220.106]
0002 Feb  1 00:15:57 pigeon postfix/cleanup[31589]: 52278E0778:
  message-id=<01caa301$d307f7d0$b173a8c0@ambachglasfaser>
0003 Feb  1 00:15:58 pigeon postfix/qmgr[12260]: 52278E0778:
  from=<ambachglasfaser@test.mailnet.dyndns.biz>,
  size=59874, nrcpt=3 (queue active)
0004 Feb  1 00:15:58 pigeon postfix/local[31581]: 52278E0778:
  to=<gentoo-embedded+subscribe@lists.gentoo.org>,
  orig_to=<gentoo-embedded-subscribe@lists.gentoo.org>,
  relay=local, delay=2.4, delays=2.4/0/0/0.01, dsn=2.0.0, status=sent (delivered to command: ....)
0005 Feb  1 00:15:58 pigeon postfix/local[31716]: 52278E0778:
  to=<gentoo-user-id@lists.gentoo.org>,
  relay=local, delay=2.4, delays=2.4/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to command: ....)
0006 Feb  1 00:15:58 pigeon postfix/local[31509]: 52278E0778:
  to=<gentoo-gwn@lists.gentoo.org>,
  relay=local, delay=2.4, delays=2.4/0.01/0/0.02, dsn=2.0.0, status=sent (delivered to command: ....)
0007 Feb  1 00:15:58 pigeon postfix/qmgr[12260]: 52278E0778: removed

Assuming that the it's a real subscribe request, we send a confirmation request, and promptly get blacklisted for being a good citizen. Line 0013.

0010 Feb  1 00:15:58 pigeon postfix/smtpd[31587]: B6FA9E0778: client=localhost[127.0.0.1]
0011 Feb  1 00:15:58 pigeon postfix/cleanup[31589]: B6FA9E0778:
  message-id=<1264983358-31717-mlmmj-3905840d@lists.gentoo.org>
0012 Feb  1 00:15:58 pigeon postfix/qmgr[12260]: B6FA9E0778:
  from=<gentoo-embedded+bounces-confsub-32dfa15d1a18a7a9-ambachglasfaser=test.mailnet.dyndns.biz@lists.gentoo.org>,
  size=1345, nrcpt=1 (queue active)
0013 Feb  1 00:16:29 pigeon postfix/smtp[31603]: B6FA9E0778:
  to=<ambachglasfaser@test.mailnet.dyndns.biz>,
  relay=mx.dyndns.biz[217.11.54.110]:25, delay=31, delays=0.06/0/30/0.41, dsn=5.7.1,
  status=bounced (host mx.dyndns.biz[217.11.54.110] said:
    554 5.7.1 Service unavailable; Your spam message has been received.
    You will be blacklisted. Thank you (in reply to end of DATA command))
0014 Feb  1 00:16:29 pigeon postfix/bounce[31637]: B6FA9E0778: sender non-delivery notification: B8AE9E089A
0015 Feb  1 00:16:29 pigeon postfix/qmgr[12260]: B6FA9E0778: removed

Why did this happen? I do agree on the importance of spamtrap accounts, but they MUST check the content of their messages. A list confirmation message MUST NOT be considered as spam.

The original subscribe request came from what seems to be a compromised server in Secunderabad, India. So it wouldn't have been detected by RBL focused on modem/dialup addresses.

Short of raising the bar to subscribe (with a specific token that needs to be included, and then it's only a matter of time till spammers include it too), there isn't much we can do to block stuff like this at the list-server level. There is no way to detect than an address is a spamtrap. There cannot be by definition, as the spammers would avoid it themselves otherwise.

Useful and complex /etc/conf.d/net setups (In support of USE=oldnet)

Thu, 30 Dec 2010 22:14:31 GMT

I've been prodding at the concept of the new network script in OpenRC-0.5, and I'm at a loss to try and see why Roy has decided to toss the old network config system away. The new system doesn't have a lot of capabilities, and most significantly totally loses the ability to restart a single interface without affecting the rest of the system. If it's just for a rewrite, then I'm not too worried, but unless all the functionality is still there, I'm worried we are going to move backwards with it.

At the same time, I don't think many people are aware of how powerful the "old" network configuration mechanism is. The net.examples file is only the start, once you start mixing in the pre/post calls, there's a lot of power. It's capable of some feats that I don't see used even in certain parts of the Gentoo documentation^[1]. I've put together some of my gems of conf.d/net, and if you have some, I'd love to hear them. Leave a comment or email me the scripts, along with a description.

Configurations available

Easy to maintain HE.net (Hurricane Electric) IPv6 tunnels - Download
Running two ISPs at home (basic multi-homing) - Download
"Enterprise" multi-homing setup, with 4 paths to the Internet - Download

Hosting

I've also started a bit of storage in my Gentoo webspace for these collected works of network configuration, with a bit more documentation.

Notes

The Gentoo docs have this for IPv6: Gentoo IPv6 Router Guide, Tunnel Configuration. You could bring it up manually, or you could just take the IPv6 config above and use it straight with your variables filled in. Volunteers welcome to help merge that config into the Gentoo IPv6 documentation.

Gentoo release statistics as of 2009/10/09 23h59 UTC

Thu, 30 Dec 2010 22:14:31 GMT

solar was asking about release statistics, so I grabbed the current data from Bouncer. The nearly 34k releases for 10.0 is just in the 5 days that it's been out. I included the various architetures that were part of each released 'product', to make some degree of comparision possible.

What	Hits	Arches
2005.1
installcd-minimum	228561	alpha,amd64,hppa,ia64,ppc,ppc64,sparc64,x86
installcd-universal	374388	alpha,amd64,hppa,ppc,sparc64,x86
packagecd	162537	alpha,amd64,ppc,ppc64,sparc64,x86

2006.0
livecd	242422	x86
minimal	287496	alpha,amd64,hppa,ia64,ppc,ppc64,sparc64,x86
packagecd	42572	amd64,ppc-g4,ppc-ppc,sparc64
packagecd-32ul	10909	ppc64
packagecd-64ul	2981	ppc64
universal	111359	alpha,amd64,hppa,ppc,ppc64,sparc64

2006.1
livecd	307481	amd64,x86
minimal	330505	alpha,amd64,hppa,ia64,ppc,ppc64,sparc64,x86
packagecd	39118	ppc,ppc-g3,ppc-g4,ppc64,ppc64-g5
universal	122280	alpha,hppa,ppc,ppc64,sparc64

2007.0
bt-http-seed	72980	ALL
livecd	411958	amd64,x86
minimal	496943	alpha,amd64,hppa,ia64,ppc,ppc64,sparc64,x86
packagecd	27593	ppc-g4,sparc64
universal	137554	hppa,ppc,ppc64,sparc64

2008.0_beta1
livecd	19426	amd64,ppc64,x86
livedvd	4	amd64,x86
minimal	14069	alpha,amd64,hppa,ia64,ppc64,sparc64,x86
universal	1745	ppc64,sparc64

2008.0_beta2
livecd	37771	amd64,x86
livedvd	17842	amd64,x86
minimal	55745	alpha,amd64,hppa,ia64,ppc,sparc64,x86
universal	3142	ppc,sparc64

2008.0
livecd	477934	amd64,x86
minimal	406531	alpha,amd64,hppa,ia64,ppc,sparc64,x86
packagecd	12308	sparc64
universal	83600	hppa,ppc,sparc64

10.0_pre20090926-1952
livedvd	4870	amd64,x86

10.0
livedvd	33703	amd64,x86

10.1
livedvd	0	amd64,x86

Notes

2008.* has the LiveDVD's pulled from mirrors due to size complaints.
bt-http-seed was an (failed) experiment with a set of mirror URLs for trying to load-balance Bittorrent's HTTP seeding
Bouncer really needs replacing, but there's nothing really good to do so that I'm aware of. mod_sentry isn't nice. Other suggestions welcome. Should support products, architectures within products, seperate check/serve URLs, detailed hit recording for analysis.

Visualizing Gentoo profiles

Thu, 30 Dec 2010 22:14:31 GMT

To add a new USE flag, that's globally enabled for all Linux profiles, what's the minimum set of profiles that need to change? Deprecated profiles must be handled as well, for users that need to migrate still.

I ran into this today, while working on the USE=modules changes for linux-mod.eclass.

As an attempt to solve this, I munged up some GraphViz work to show profile inheritance, pictures as the end. Both sets have the trailing profiles "/desktop", "/developer", "/server" turned off for the 2008.0 and 10.0 releases, to cut down on the noise.

Graphs and script for download.

My answers as to which profiles:

default-linux
default/linux
base
embedded

Odd observations

Several Prefix profiles (linux/{amd64,ia64,x86} link to 2008.0 profiles explicitly instead of the generic architecture)
default/linux does not bring in base. Some profiles at a glance neglect this and might not have base brought in at all.
"embedded" is all alone in the tree.

Thumbnail of one graph

Question for any skilled GraphViz users:

If all nodes in a given subgroup/cluster have an edge going to a single destination node, is there any way to get graphviz to replace them with a single fat edge from cluster to destination node?

Heatwaves lead to hardware failures

Thu, 30 Dec 2010 22:14:31 GMT

So for our Vancouver heatwave (I noted 39C away from the water today, in the shade!), it's finally claimed some of my computer hardware. Most annoying, the battery backup unit (BBU) in the newer fileserver, and 1.5 of the disks of the RAID1 array in the old server...

My website and personal email will be offline for a day or two while I ensure my backups are up to date, and redeploy to the newer fileserver (after I buy a new BBU tomorrow).