Older blog entries for robbat2 (starting at number 30)

Adding 95th Percentile in Munin, without any patches: undocumented setting graph_args_after

Munin is commonly used to graph lots of systems stuff, however it lacks a common piece of functionality: 95th percentile.

The Munin bug tracker has ticket #443 sitting open for 7 years now, asking for this, and proving a not-great patch for it.

I really wanted to add 95th percentile to one of my complicated graphs (4 base variables, and 3 derived variables deep), but I didn't like the above patch either. Reading the Munin source to consider implementing VDEF properly, I noticed an undocumented setting: graph_args_after. It was introduced by ticket #1032, as a way of passing things directly to rrdtool-graph.

Clever use of this variable can pass in ANYTHING else to rrdtool-graph, including VDEF! So without further ado, here's how to put 95th percentile into individual Munin graphs, relatively easily.

# GRAPHNAME is the name of the graph you want to render on.
# VARNAME is the name of the new variable to call the Percentile line.
# DEF_VAR is the name of the CDEF or DEF variable from earlier in your graph definition.
# LEGEND is whatever legend you want to display on the graph for the line.
#   FYI Normal rrdtool escaping rules apply for legend (spaces, pound, slash).
${GRAPHNAME}.graph_args_after \
  VDEF:${VARNAME}=gcdef${DEF_VAR},95,PERCENT \
  LINE1:${VARNAME}\#999999:${LEGEND}:dashes \
  GPRINT:${VARNAME}:\%6.2lf\%s\\j
# Example of the above I'm using
bandwidth1.graph_args_after \
  VDEF:totalperc=gcdeftotal,95,PERCENT \
  LINE1:totalperc\#999999:95th\ Percentile\ (billable\):dashes \
  GPRINT:totalperc:\%6.2lf\%s\\j

Syndicated 2014-01-28 23:51:39 from Move along, nothing to read

python-exec: solutions for package conflicts, and making it easier on users

Running into another system today with the fun python-exec block, I realise that while it has been discussed on the Gentoo mailing lists, and the forums slightly, there's been hardly any posts about it in the blog stream.

I'm not going to go into what caused it, but rather solutions for package conflicts in the short term, and also the long-term. The TL;DR general solution is running "emerge -1 dev-python/python-exec"

Here's the latest conflict I got on it; I wanted to install mirrorselect to compare some hosts

hostname / # emerge -pv mirrorselect

These are the packages that would be merged, in order:
[ebuild  N     ] net-analyzer/netselect-0.3-r3  22 kB
[ebuild     U  ] dev-lang/python-2.7.5-r3:2.7 [2.7.3-r2:2.7] USE="gdbm hardened%* ipv6 ncurses readline ssl threads (wide-unicode) xml -berkdb -build -doc -examples -sqlite -tk -wininst" 10,026 kB
[ebuild     U  ] dev-lang/python-3.2.5-r3:3.2 [3.2.3:3.2] USE="gdbm hardened%* ipv6 ncurses readline ssl threads (wide-unicode) xml -build -doc -examples -sqlite -tk -wininst" 9,020 kB
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[ebuild  N     ] dev-util/dialog-1.2.20121230  USE="nls unicode -examples -minimal -static-libs" 422 kB
[ebuild  N     ] app-portage/mirrorselect-2.2.0.1  PYTHON_TARGETS="python2_7 python3_2 -python2_6 (-python3_3)" 13 kB
[blocks B      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0)

Total: 6 packages (2 upgrades, 4 new), Size of downloads: 19,580 kB
Conflict: 1 block (1 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-python/python-exec-0.2::gentoo, installed) pulled in by
    dev-python/python-exec[python_targets_python2_7(-),-python_single_target_python2_5(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-)] required by (dev-libs/libxml2-2.9.0-r2::gentoo, installed)

  (dev-lang/python-exec-2.0::gentoo, ebuild scheduled for merge) pulled in by
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)]) required by (dev-python/setuptools-0.6.30-r1::gentoo, installed)
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,python_targets_python3_3(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-)]) required by (app-portage/mirrorselect-2.2.0.1::gentoo, ebuild scheduled for merge)
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,python_targets_python3_3(-)?,python_targets_pypy2_0(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)]) required by (virtual/python-argparse-1::gentoo, installed)

For more information about Blocked Packages, please refer to the following
section of the Gentoo Linux x86 Handbook (architecture is irrelevant):

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked

This system has just dev-lang/python-exec-2.0 presently. We can reduce the conflict down to a minimal version as follows:

HOST / # emerge -pv  dev-lang/python-exec

These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[blocks B      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0)

Total: 1 package (1 new), Size of downloads: 79 kB
Conflict: 1 block (1 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-python/python-exec-0.2::gentoo, installed) pulled in by
    dev-python/python-exec[python_targets_python2_7(-),-python_single_target_python2_5(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-)] required by (dev-libs/libxml2-2.9.0-r2::gentoo, installed)

  (dev-lang/python-exec-2.0::gentoo, ebuild scheduled for merge) pulled in by
    dev-lang/python-exec
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)]) required by (dev-python/setuptools-0.6.30-r1::gentoo, installed)
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,python_targets_python3_3(-)?,python_targets_pypy2_0(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)]) required by (virtual/python-argparse-1::gentoo, installed)

For more information about Blocked Packages, please refer to the following
section of the Gentoo Linux x86 Handbook (architecture is irrelevant):

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked

So what do we know?

  1. We have dev-python/python-exec-0.2 installed, it has the default SLOT=0.
  2. Here's what the packages in the tree right now look like:
    $ egrep '^R?DEPEND|^SLOT' dev-{python,lang}/python-exec/*ebuild
    dev-python/python-exec/python-exec-10000.1.ebuild:SLOT="0"
    dev-python/python-exec/python-exec-10000.1.ebuild:RDEPEND="dev-lang/python-exec:0[${PYTHON_USEDEP}]"
    dev-python/python-exec/python-exec-10000.2.ebuild:SLOT="2"
    dev-python/python-exec/python-exec-10000.2.ebuild:RDEPEND="dev-lang/python-exec:0[${PYTHON_USEDEP}]
    dev-lang/python-exec/python-exec-0.3.1.ebuild:SLOT="0"
    dev-lang/python-exec/python-exec-0.3.1.ebuild:RDEPEND="!<dev-python/python-exec-10000"
    dev-lang/python-exec/python-exec-0.9999.ebuild:SLOT="0"
    dev-lang/python-exec/python-exec-0.9999.ebuild:RDEPEND="!<dev-python/python-exec-10000"
    dev-lang/python-exec/python-exec-2.0.ebuild:SLOT="2"
    dev-lang/python-exec/python-exec-2.0.ebuild:RDEPEND="!<dev-python/python-exec-10000"
    dev-lang/python-exec/python-exec-2.9999.ebuild:SLOT="2"
    dev-lang/python-exec/python-exec-2.9999.ebuild:RDEPEND="!<dev-python/python-exec-10000"
    
  3. If we try to bring in dev-lang/python-exec directly, it will trigger the block, because our version of dev-python/python-exec is too old.
  4. This entire problem happens because the python*r1 eclasses bring in dev-lang/python-exec.

This leads to a simple user-actionable solution of "emerge -1 dev-python/python-exec", which will work as follows (notice that portage uninstalls the old version for us):

HOST / # emerge -pv  dev-python/python-exec
These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-0.3.1  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 73 kB
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[uninstall     ] dev-python/python-exec-0.2  PYTHON_TARGETS="(jython2_5) (jython2_7) python2_5 (python2_6) (python2_7) python3_1 (python3_2) -pypy1_9 (-pypy2_0) (-python3_3)" 
[blocks b      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0, dev-lang/python-exec-0.3.1)
[ebuild  NS    ] dev-python/python-exec-10000.2:2 [0.2:0] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 0 kB

Total: 3 packages (2 new, 1 in new slot, 1 uninstall), Size of downloads: 152 kB
Conflict: 1 block

The above is not actually the minimal solution, but it is the best general solution. The minimal solution is to include the slot on the package, but in future if the slots change further and the default slot is removed, this won't work anymore.

HOST / # emerge -pv dev-python/python-exec:0
These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-0.3.1  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 73 kB
[ebuild     U  ] dev-python/python-exec-10000.1 [0.2] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3) (-pypy1_9%) (-python2_5%*) (-python3_1%*)" 0 kB
[blocks b      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-0.3.1)

Total: 2 packages (1 upgrade, 1 new), Size of downloads: 73 kB
Conflict: 1 block

But now the better question, is as developers, can we help users prevent this, and at what cost? If we don't mind new users having an extra placeholder package, then yes, we CAN actually solve it for the users. In all of the dev-lang/python-exec ebuilds we need to make this simple change:

 RDEPEND="!<dev-python/python-exec-10000"
+PDEPEND=">=dev-python/python-exec-10000:$SLOT"

This provides a nice solution as follows:

# emerge -pv dev-lang/python-exec
These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-0.3.1  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 73 kB
[ebuild     U  ] dev-python/python-exec-10000.1 [0.2] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3) (-pypy1_9%) (-python2_5%*) (-python3_1%*)" 0 kB
[blocks b      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0, dev-lang/python-exec-0.3.1)
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[ebuild  NS    ] dev-python/python-exec-10000.2:2 [0.2:0] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 0 kB

Total: 4 packages (1 upgrade, 2 new, 1 in new slot), Size of downloads: 152 kB
Conflict: 1 block

All that remains is convincing the Python team to accept this solution for users...

Syndicated 2013-11-15 21:34:32 from Move along, nothing to read

Looking for a new graphics card

So I'm hunting for a new graphics card, and my set of requirements make this a difficult quest. I welcome all suggestions, either as comments, or as email.

I haven't seen any passive cards in the new R7/R9 lines from ATI, not sure if those will come out later only

Requirements

  • MUST be supported by the open-source Radeon or Noveau drivers.
  • MUST support at least 3 displays
  • MUST be cost less than $300
  • SHOULD occupy only one PCI-e slot (trying to avoid double-height cards)
  • SHOULD be passively cooled, or after-market water-cooled
  • NICE TO HAVE: good 3D performance

Options so far (for further review)

Name & Link Outputs Passive Card height Price
ATI FirePro 2460 4x MiniDP Yes 1 slot $256 NCIX (no PP)
VISIONTEK Radeon HD 7750, 2GB GDDR5, PCIe x16, 6x Mini-DP, Retail 6x MiniDP No 1 slot $273 NCIX (no PP)
Radeon HD 7750 Low Profile - Eyefinity 4 CGAX-7758LM4 - PCI Express 3.0 - 2048 - GDDR5 - 128 BIT 4x MiniDP No 1 slot 136GBP AmazonUK, no NCIX listing
SAPPHIRE FleX 100322FLEX Radeon HD 6450 1GB 64-bit DDR3 PCI Express 2.1 x16 HDCP Ready Low Profile Ready Video Card 1x DVI-D, 1x DVI-S, 1x HDMI Yes 2 slots LP $60 Newegg, $75 NCIX no-PP
PowerColor HD7750 2GB GDDR5 Eyefinity 4 LP Edition (UEFI) AX7750 2GBD5-4DL 4x MiniDP No 1 slot $190 Ebay, no NCIX listing

Syndicated 2013-10-26 01:07:47 from Move along, nothing to read

Tracking all prior open-source contributions

For the many other open-source contributors and developers out there, I'm wondering if anybody has a complete list of all works they have created. It came up recently that such a list would be useful in asserting my own prior copyrights in any future employment, and avoiding claims that I'd taken any code [1].

For version control systems are still accessible, this isn't too much of a problem, but for past historical creations, this is a lot harder. Has anybody else done it? To what level of detail did your listing go?

Here's my initial broad listing(I'm going to come back often to fill it in more)

I realized that this does form a sort of portfolio of work that I've done, and it shows just how flexible I am, esp. if I went and wrote this up better including a blurb about some of the larger or more standalone projects I've done.

  • phpMyAdmin
  • Projects while at university (in addition to the Gentoo stuff I did)
    • diradm (v2)
    • localshell (both the perl & C versions)
    • chpasswd (LDAP-based reimplementation of original core tool)
    • RATS (Research Asset Tracking System)
    • readahead-list
  • Gentoo Linux (Lots of packages and projects here to list later)
    • Ufed
    • Genflags
    • Gentoo Git Migration code
    • Dokuwiki Automation
    • Packages.gentoo.org
    • MANY MANY packages
    • GLI (Gentoo Linux Installer): automated mode
    • Gentoo Infra Installer
  • Linux Kernel
    • 2011/10/24: 99a700bc [SCSI] mv_sas: OCZ RevoDrive3 & zDrive R4 support
    • 2008/09/13: f51ff993 ALSA: HDA: patch_analog: Quirk for Asus P5Q Premium/Pro boards.
    • 2008/09/13: 9cae0c63 ALSA: HDA: patch_analog: Implement multiple outputs for AD1988
    • 2008/09/13: 0481f453 ALSA: HDA: hda_proc: Fix printf format specifier
    • 2008/09/13: 35e8901e ALSA: HDA: hda_local: Less magic numbers.
    • 2008/09/13: e8bfc6c1 ALSA: HDA: patch_analog: Fix SPDIF output on AD1989B
    • 2007/03/28: 12340106 Add notation that the Asus W5F laptop has a short cable instead of 80-wire.
    • 2006, 2007: Apple G5 PowerMac Linux sensors "WindFarm" [1] [2] [3]
    • 2006/12: Porting of SATA Port Multipler code
    • 2006/06/12: cfd95a9c [PATCH] tmpfs: time granularity fix for [acm]time going backwards
    • Port of MIPS AMD Alchemy 1500 code for MyCable XXS1500 from 2.4 to 2.6 [1]
    • Changes for SGI Visual Workstation 320
    • Other work prior to v2.6.12 of the kernel when Git started (TODO)
    • TinCan Nail kernels
    • ???
  • 2004/08: Ksymoops: Cross-platform debug support without a cross-compiler
  • Git (various commits over the years, git-svn fixups, stuff in git commit signing, TODO)
  • MogileFS (Author of the PostgreSQL backend, core improvements
  • CephFS (Packaging fixes)
  • Ruby On Rails - ActiveRecord: Implemented BigDecimal floating-point storage [1]
  • APC PDU & MasterSwitch scripts
  • OpenSSH: HPN (High Performance Networking) patch porting & maintenance [1]
  • OpenSSH: LPK (LDAP key patch) patch porting, maintenance & feature additions [1]
  • Perl CPAN module for Watt's Up Pro USB hardware [1]
  • Ccache [1, utimes truncation]
  • CVS (Custom lockdown code)[1][1]
  • udev (add persistent tape symlinks)[1]
  • Amanda (fix race condition in backup encryption)[1]
  • LVM2 (usb block support [1]), ???
  • OpenSC (GSM SIM card access prototype [1])
  • Perlbal (SSL version selection control [1])
  • Rbot (Penalty calc [1] RPL_WHOISCHANNELS [2] Privacy in !seen [3]
  • 2013: Puppet LVM module [1]
  • 2008/09/13: ALSA (Fix NULL pointers [1])
  • 2006/08/13: GPSD (fix race conditions [1], [2])
  • 2008/01/14: mlmmj (Fix mail loss bug [1])
  • 2009/12/16: mlmmj ([PATCH] Re: requeued mail files going missing from archive/)
  • 2012/11/12: PulseAudio ([PATCH 0-3/3] rtp: Configurable source IP, Cleanup variable naming, rename modarg destination to dstip, Introduce source IP configuration with srcip)
  • 2006/06/19: Qpsmtpd ([PATCH] Require TLS/SSL before offering AUTH, Include SSL level in header, as per qmail-ssl)
  • 2006/12/17: Qpsmtpd (check_badmailfrom patch - per-line rejection messages)
  • 2012/02/25: Qpsmtpd [PATCH] Accept messages with no body and no trailing \n after headers
  • Random open-source fixes (TODO)

And I'm sure that I'm missing many more.

Flattr this

Syndicated 2013-10-25 18:32:55 from Move along, nothing to read

Gentoo Linux participates in World IPv6 day

In light of World IPv6 day, the Gentoo Linux Infrastructure team would like to announce new IPv6-availability of several services, and list the existing IPv6 services. Every service listed below is running a dual-stack native IPv4/IPv6 service, no tunnels.

The new services available via IPv6 are:

The existing services available via IPv6 are:

  • CVS/SVN/Git services for developers
  • http://sources.gentoo.org/
  • rsync://rsync.gentoo.org/ - our primary rsync rotation
  • rsync://${CC}.rsync.gentoo.org - our regional community rsync rotations
  • A number of our mirrors

All of our IPv6 services will remain online after today, unless serious IPv6 problems (esp. regarding routing) are encountered.

Gentoo would like to extend thanks to all our sponsors & mirrors who have provided IPv6 service, and the servers to make use of it!

Syndicated 2011-06-08 11:06:46 from Move along, nothing to read

Robin's 2011 conferences plans and ideas

Working on my conference travel plans and wishes for the year. I am downgrading OLS to a maybe, the cost is becoming more of a factor. Likewise, while I had incredible fun at FOSDEM last year, and OSCON in 2006, I cannot justify the airfare/hotel expenses for them. I would like to attend SCALE at some point as well, but uncertain for the same cost reason.

Confirmed:
  • April 11-14, MySQL UC @ Santa Clara, CA, USA [1]
  • August 17-19, LinuxCon 2011 @ Vancouver, BC, Canada [2]
Maybe:
  • June 13-15, Linux Symposium @ Ottawa, ON, Canada.
Would like to go, but out of my financial reach:
  • February 5-6, FOSDEM @ Brussels, Belgium.
  • February 25-27, SCALE 9x @ Los Angeles, CA, USA.
  • July 25-29, OSCON @ Portland, OR
  • (Not yet announced), Linux Plumbers.
Notes
  1. I will be manning the phpMyAdmin booth, like past 5 years.
    I have no accommodation yet, I'd love to split a hotel room at the Hyatt (or another spot within walking distance) with somebody.
  2. Local this year, so no travel costs :-)

Syndicated 2011-01-16 10:34:46 from Move along, nothing to read

Complaining at Journalists again: Gentoo Security and the UnrealIRCd backdoor

Those that have followed me for a while might have seen me previously complain at journalism that's misleading, wrong, or outright fictitious. Now I've got another case...
This article by Ed Bott at ZDNet:
Linux infection proves Windows malware monopoly is over; Gentoo ships backdoor? [updated]

The article was first published 2010/06/12 20:37 UTC.
It claims to be "worse" when updated at 2010/06/14 19:30 UTC.

Gentoo had a revision bump to a known good copy of the tarball at 2010/06/12 16:34 UTC (using a different filename, and verified against the GPG signature provided by upstream), so it was ALREADY fixed when the article was published. The old revision was explicitly removed at 2010/06/12 21:18 UTC.
Commit data for fixes:
Changes for unrealircd-3.2.8.1-r1.ebuild
Changes for unrealircd-3.2.8.1.ebuild

The trojaned tarball was then removed from the Gentoo master mirror at 2010/06/13 08:00 UTC, about 11 hours after the article was published. It would have been sooner, but it was a matter of bad timing.

Gentoo bug 323691.

The article also claims: "There’s a great deal of comment in the Talkback section of this post about how official repositories can be trusted. It appears that system broke down thoroughly in this case."
This claim is bogus. The developer that updated the package made perhaps a mistake in trusting that the upstream had not been tampered with. However, in lacking anything to verify against (the upstream apparently did not sign releases at that point), he couldn't have detected the backdoor except by manual inspection of all the code. He downloaded the package AFTER it had been tampered with (2009/11/11 I believe), so he never saw the tamper-free version either.

The entire point of the Gentoo Manifests are to ensure that OUR mirrors are not the point where a compromise is introduced. We can detect upstream changes by this same mechanism, but they mostly tend to be upstream deciding to 'fix' something without bumping the version number. In this regard, they functioned perfectly.

P.S. I'm not saying the existing Gentoo mirroring is perfect either, see my prior writings on tree-signing, and the "Attacks on Package Manager" papers by Cappos et al., which are blocked only with the full tree-signing system.

Syndicated 2010-06-15 08:36:34 from Move along, nothing to read

On Google Summer of Code Applications

(This post inspired by Petteri Räty (betelgeuse)'s similar post

For this year's Gentoo GSoC projects, I'm a mentor on two of our suggested ideas (but also interested in totally new ideas that fit my fields):

  • upstart on Gentoo
  • Distfile Fetcher Intelligence
Do you actually understand the project idea?
This is actually a gap that I didn't expect to exist, but I have seen in previous years. This is mainly a difference of expectations between the proposal and what the potential student sees as what the idea really entails.
Using Upstart as an example, it supports an existing init.d compatibility mode, but we're not interested in that. Instead we want our init.d scripts to be treated just like upstart jobs (located in /etc/init/). The init.5 manpage shipped with upstart gives a good start...
Code maintainability
betelgeuse spoke about long-term maintenance, but you should think about it long ahead of that. Some degrees of abstraction, and avoiding difficult to understand logic should be prevalent here. betelgeuse mentioned spaghetti code, but it's important to realize that even well formatted code can impose a much larger mental workload if not well thought out.
Timezones, Timezones!
Most of your project should not be blocking on asking for mentor advice, as timezones and real world pressures often conspire to prevent easy real world communication. I may live in UTC-7, but my hours drift as needed by work but I tend to be online anywhere between 17h00 UTC and 10h00 UTC. If you're trying to communicate with me on a regular basis, this can be tough, so being able work on a problem independently, ask highly directed questions via email can go a long way.

Syndicated 2010-03-30 19:24:24 from Move along, nothing to read

Advice for Google Summer of Code students

Good advice for any prospective GSoC student, regardless of gender

I'm also a mentor for Gentoo again this year, after taking a break last year.
You can find our list of potential ideas here: Google Summer of Code 2010 ideas for Gentoo
But don't limit yourself to them! Creative ideas can get you very far too :-)

I'll also be the infrastructure contact for the accepted SoC students, for any issues you have with the source code repositories (we'll be offering Git again), your shell accounts, and a sounding board on deploying your successful project (for those that hosting or larger resources).

Syndicated 2010-03-26 05:14:46 from Move along, nothing to read

Spamtrap addresses vs. list confirmation emails, or how to lose 2k list emails

In the early hours of this morning, a spammer managed to get the IP of the Gentoo list server on the NiX Spam RBL... simply by spamming the subscribe address :-(. This caused approximately 2000 deliveries of normal list mail to be rejected while the server was present on the RBL.

Notice the subscribe request, line 0004. (whitespace added)

0001 Feb  1 00:15:56 pigeon postfix/smtpd[29314]: 52278E0778: client=unknown[210.212.220.106]
0002 Feb  1 00:15:57 pigeon postfix/cleanup[31589]: 52278E0778:
  message-id=<01caa301$d307f7d0$b173a8c0@ambachglasfaser>
0003 Feb  1 00:15:58 pigeon postfix/qmgr[12260]: 52278E0778:
  from=<ambachglasfaser@test.mailnet.dyndns.biz>,
  size=59874, nrcpt=3 (queue active)
0004 Feb  1 00:15:58 pigeon postfix/local[31581]: 52278E0778:
  to=<gentoo-embedded+subscribe@lists.gentoo.org>,
  orig_to=<gentoo-embedded-subscribe@lists.gentoo.org>,
  relay=local, delay=2.4, delays=2.4/0/0/0.01, dsn=2.0.0, status=sent (delivered to command: ....)
0005 Feb  1 00:15:58 pigeon postfix/local[31716]: 52278E0778:
  to=<gentoo-user-id@lists.gentoo.org>,
  relay=local, delay=2.4, delays=2.4/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to command: ....)
0006 Feb  1 00:15:58 pigeon postfix/local[31509]: 52278E0778:
  to=<gentoo-gwn@lists.gentoo.org>,
  relay=local, delay=2.4, delays=2.4/0.01/0/0.02, dsn=2.0.0, status=sent (delivered to command: ....)
0007 Feb  1 00:15:58 pigeon postfix/qmgr[12260]: 52278E0778: removed

Assuming that the it's a real subscribe request, we send a confirmation request, and promptly get blacklisted for being a good citizen. Line 0013.

0010 Feb  1 00:15:58 pigeon postfix/smtpd[31587]: B6FA9E0778: client=localhost[127.0.0.1]
0011 Feb  1 00:15:58 pigeon postfix/cleanup[31589]: B6FA9E0778:
  message-id=<1264983358-31717-mlmmj-3905840d@lists.gentoo.org>
0012 Feb  1 00:15:58 pigeon postfix/qmgr[12260]: B6FA9E0778:
  from=<gentoo-embedded+bounces-confsub-32dfa15d1a18a7a9-ambachglasfaser=test.mailnet.dyndns.biz@lists.gentoo.org>,
  size=1345, nrcpt=1 (queue active)
0013 Feb  1 00:16:29 pigeon postfix/smtp[31603]: B6FA9E0778:
  to=<ambachglasfaser@test.mailnet.dyndns.biz>,
  relay=mx.dyndns.biz[217.11.54.110]:25, delay=31, delays=0.06/0/30/0.41, dsn=5.7.1,
  status=bounced (host mx.dyndns.biz[217.11.54.110] said:
    554 5.7.1 Service unavailable; Your spam message has been received.
    You will be blacklisted. Thank you (in reply to end of DATA command))
0014 Feb  1 00:16:29 pigeon postfix/bounce[31637]: B6FA9E0778: sender non-delivery notification: B8AE9E089A
0015 Feb  1 00:16:29 pigeon postfix/qmgr[12260]: B6FA9E0778: removed

Why did this happen? I do agree on the importance of spamtrap accounts, but they MUST check the content of their messages. A list confirmation message MUST NOT be considered as spam.

The original subscribe request came from what seems to be a compromised server in Secunderabad, India. So it wouldn't have been detected by RBL focused on modem/dialup addresses.

Short of raising the bar to subscribe (with a specific token that needs to be included, and then it's only a matter of time till spammers include it too), there isn't much we can do to block stuff like this at the list-server level. There is no way to detect than an address is a spamtrap. There cannot be by definition, as the spammers would avoid it themselves otherwise.

Syndicated 2010-02-01 20:25:41 from Move along, nothing to read

21 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!