joey is currently certified at Master level.

Name: Joey Hess
Member since: 2000-03-06 23:42:41
Last Login: 2011-12-31 20:04:52

FOAF RDF Share This

Homepage: http://kitenet.net/~joey

Projects

Recent blog entries by joey

Syndication: RSS 2.0

propelling containers

Propellor has supported docker containers for a "long" time, and it works great. This week I've worked on adding more container support.

docker containers (revisited)

The syntax for docker containers has changed slightly. Here's how it looks now:

example :: Host
example = host "example.com"
    & Docker.docked webserverContainer

webserverContainer :: Docker.Container
webserverContainer = Docker.container "webserver" "joeyh/debian-stable"
    & os (System (Debian (Stable "wheezy")) "amd64")
    & Docker.publish "80:80"
    & Apt.serviceInstalledRunning "apache2"
    & alias "www.example.com"

That makes example.com have a web server in a docker container, as you'd expect, and when propellor is used to deploy the DNS server it'll automatically make www.example.com point to the host (or hosts!) where this container is docked.

I use docker a lot, but I have drank little of the Docker KoolAid. I'm not keen on using random blobs created by random third parties using either unreproducible methods, or the weirdly underpowered dockerfiles. (As for vast complicated collections of containers that each run one program and talk to one another etc ... I'll wait and see.)

That's why propellor runs inside the docker container and deploys whatever configuration I tell it to, in a way that's both replicatable later and lets me use the full power of Haskell.

Which turns out to be useful when moving on from docker containers to something else...

systemd-nspawn containers

Propellor now supports containers using systemd-nspawn. It looks a lot like the docker example.

example :: Host
example = host "example.com"
    & Systemd.persistentJournal
    & Systemd.nspawned webserverContainer

webserverContainer :: Systemd.Container
webserverContainer = Systemd.container "webserver" chroot
    & Apt.serviceInstalledRunning "apache2"
    & alias "www.example.com"
  where
    chroot = Chroot.debootstrapped (System (Debian Unstable) "amd64") Debootstrap.MinBase

Notice how I specified the Debian Unstable chroot that forms the basis of this container. Propellor sets up the container by running debootstrap, boots it up using systemd-nspawn, and then runs inside the container to provision it.

Unlike docker containers, systemd-nspawn containers use systemd as their init, and it all integrates rather beautifully. You can see the container listed in systemctl status, including the services running inside it, use journalctl to examine its logs, etc.

But no, systemd is the devil, and docker is too trendy...

chroots

Propellor now also supports deploying good old chroots. It looks a lot like the other containers. Rather than repeat myself a third time, and because we don't really run webservers inside chroots much, here's a slightly different example.

example :: Host
example = host "mylaptop"
    & Chroot.provisioned (buildDepChroot "git-annex")

buildDepChroot :: Apt.Package -> Chroot.Chroot
buildDepChroot pkg = Chroot.debootstrapped system Debootstrap.buildd dir
    & Apt.buildDep pkg
  where
    dir = /srv/chroot/builddep/"++pkg
   system = System (Debian Unstable) "amd64"

Again this uses debootstrap to build the chroot, and then it runs propellor inside the chroot to provision it (btw without bothering to install propellor there, thanks to the magic of bind mounts and completely linux distribution-independent packaging).

In fact, the systemd-nspawn container code reuses the chroot code, and so turns out to be really rather simple. 132 lines for the chroot support, and 167 lines for the systemd support (which goes somewhat beyond the nspawn containers shown above).

Which leads to the hardest part of all this...

debootstrap

Making a propellor property for debootstrap should be easy. And it was, for Debian systems. However, I have crazy plans that involve running propellor on non-Debian systems, to debootstrap something, and installing debootstrap on an arbitrary linux system is ... too hard.

In the end, I needed 253 lines of code to do it, which is barely one magnitude less code than the size of debootstrap itself. I won't go into the ugly details, but this could be made a lot easier if debootstrap catered more to being used outside of Debian.

closing

Docker and systemd-nspawn have different strengths and weaknesses, and there are sure to be more container systems to come. I'm pleased that Propellor can add support for a new container system in a few hundred lines of code, and that it abstracts away all the unimportant differences between these systems.

PS

Seems likely that systemd-nspawn containers can be nested to any depth. So, here's a new kind of fork bomb!

infinitelyNestedContainer :: Systemd.Container
infinitelyNestedContainer = Systemd.container "evil-systemd"
    (Chroot.debootstrapped (System (Debian Unstable) "amd64") Debootstrap.MinBase)
    & Systemd.nspawned infinitelyNestedContainer

Strongly typed purely functional container deployment can only protect us against a certian subset of all badly thought out systems. ;)

Syndicated 2014-11-21 21:33:06 from see shy jo

on leaving

I left Debian. I don't really have a lot to say about why, but I do want to clear one thing up right away. It's not about systemd.

As far as systemd goes, I agree with my friend John Goerzen:

I promise you – 18 years from now, it will not matter what init Debian chose in 2014. It will probably barely matter in 3 years.

read the rest

And with Jonathan Corbet:

However things turn out, if it becomes clear that there is a better solution than systemd available, we will be able to move to it.

read the rest

I have no problem with trying out a piece of Free Software, that might have abrasive authors, all kinds of technical warts, a debatable design, scope creep etc. None of that stopped me from giving Linux a try in 1995, and I'm glad I jumped in with both feet.

It's important to be unafraid to make a decision, try it out, and if it doesn't work, be unafraid to iterate, rethink, or throw a bad choice out. That's how progress happens. Free Software empowers us to do this.

Debian used to be a lot better at that than it is now. This seems to have less to do with the size of the project, and more to do with the project having aged, ossified, and become comfortable with increasing layers of complexity around how it makes decisions. To the point that I no longer feel I can understand the decision-making process at all ... or at least, that I'd rather be spending those scarce brain cycles on understanding something equally hard but more useful, like category theory.

It's been a long time since Debian was my main focus; I feel much more useful when I'm working in a small nimble project, making fast and loose decisions and iterating on them. Recent events brought it to a head, but this is not a new feeling. I've been less and less involved in Debian since 2007, when I dropped maintaining any packages I wasn't the upstream author of, and took a year of mostly ignoring the larger project.

Now I've made the shift from being a Debian developer to being an upstream author of stuff in Debian (and other distros). It seems best to make a clean break rather than hang around and risk being sucked back in.

My mailbox has been amazing over the past week by the way. I've heard from so many friends, and it's been very sad but also beautiful.

Syndicated 2014-11-13 18:59:39 from see shy jo

continuing to be pleasantly surprised

Free software has been my career for a long time -- nothing else since 1999 -- and it continues to be a happy surprise each time I find a way to continue that streak.

The latest is that I'm being funded for a couple of years to work part-time on git-annex. The funding comes from the DataLad project, which was recently awarded by National Science Foundation. DataLad folks (at Dartmouth College and at Magdeburg University in Germany) are working on providing easy access to scientific data (particularly neuroimaging). So git-annex will actually be used for science!

I'm being funded for around 30 hours of work each month, to do general work on the git-annex core (not on the webapp or assistant). That includes bugfixes and some improvements that are wanted for DataLad, but are all themselves generally useful. (see issue list)

This is enough to get by on, at least in my current living situation. It would be great if I could find some funding for my other work time -- but it's also wonderful to have the flexibility to spend time on whatever other interesting projects I might want to.

Syndicated 2014-11-12 20:33:47 from see shy jo

a programmable alarm clock using systemd

I've taught my laptop to wake up at 7:30 in the morning. When it does, it will run whatever's in my ~/bin/goodmorning script. Then, if the lid is still closed, it will go back to sleep again.

So, it's a programmable alarm clock that doesn't need the laptop to be left turned on to work.

But it doesn't have to make noise and wake me up (I rarely want to be woken up by an alarm; the sun coming in the window is a much nicer method). It can handle other tasks like downloading my email, before I wake up. When I'm at home and on dialup, this tends to take an hour in the morning, so it's nice to let it happen before I get up.

This took some time to figure out, but it's surprisingly simple. Besides ~/bin/goodmorning, which can be any program/script, I needed just two files to configure systemd to do this.

First, /etc/systemd/system/goodmorning.timer

[Unit]
Description=good morning

[Timer]
Unit=goodmorning.service
OnCalendar=*-*-* 7:30
WakeSystem=true
Persistent=false

[Install]
WantedBy=multi-user.target

Second, /etc/systemd/system/goodmorning.service

[Unit]
Description=good morning
RefuseManualStart=true
RefuseManualStop=true

[Service]
Type=oneshot
ExecStart=/bin/systemd-inhibit --what=handle-lid-switch --why=goodmorning /bin/su joey -c /home/joey/bin/goodmorning

After installing these files, run (as root): systemctl enable goodmorning.timer; systemctl start goodmorning.timer

Then, you'll also need to edit /etc/systemd/logind.conf, and set LidSwitchIgnoreInhibited=no -- this overrides the default, which is not to let systemd-inhibit block sleep on lid close.

The WakeSystem=true relies on some hardware support for waking from sleep; my laptop supported it with no trouble but I don't know how broadly available that is.

I don't think this would be anywhere near as easy to do without systemd, logind, etc. Especially the handling of waking the system at the right time, and the behavior around lid sleep inhibiting.

(It would perhaps be better to use the per-user systemd, not the system wide one. Then I could change the time the alarm runs without using root. What's prevented me from doing this is that systemd-inhibit uses policykit, and policykit prevents it from being used in this situation. It's a lot easier to run it as root and use su, than it is to reconfigure policykit.)

Syndicated 2014-10-26 22:00:54 from see shy jo

propellor is d-i 2.0

I think I've been writing the second system to replace d-i with in my spare time for a couple months, and never noticed.

I'm as suprised as you are, but consider this design:

  • Installation system consists of debian live + haskell + propellor + web browser.

  • Entire installation UI consists of a web-based (and entirely pictographic and prompt based, so does not need to be translated) selection of the installation target.

  • Installation target can be local disk, remote system via ssh (wiping out crufty hacked-up pre-installed debian), local VM, live ISO, etc.

  • Really, no other questions. Not even user name/password! The installed system will only allow login via the same method that was used to install it. So a locally installed system will accept console/X login with no password and then a forced password change. Or a system installed via ssh will only allow login using the same ssh key that was used to install it.

  • The entire installation process consists of a disk format, followed by debootstrap, followed by running propellor in the target system. This also means that the installed system includes a propellor config file which now describes the properties of the system as installed (so can be edited to tweak the installation, or reused as starting point for next installation).

  • Users who want to configure installation in any way write down properties of system using a simple propellor config file. I suppose some people still use more than one partiton or gnome or some such customization, so they'd use:

main :: IO
main = Installer.main
    & Installer.partition First "/boot" Ext3 (MiB 256)
    & Installer.partition Next "/" Ext4 (GiB 5)
    & Installer.partition Next "/home" Ext4 FreeSpace
    & Installer.grubBoots "hd0"
    & os (System (Debian Stable) "amd64")
    & Apt.stdSourcesList
    & Apt.installed ["task-gnome-desktop"]
  • The installation system is itself built using propellor. A free feature given the above design, so basically all it will take to build an installation iso is this code:
main :: IO
main = Installer.main
    & Installer.target CdImage "installer.iso"
    & os (System (Debian Stable) "amd64")
    & Apt.stdSourcesList
    & Apt.installed ["task-xfce-desktop", "ghc", "propellor"]
    & User.autoLogin "root"
    & User.loginStarts "propellor --installer"
  • Propellor has a nice display of what it's doing so there is no freaking progress bar.

Well, now I know where propellor might end up if I felt like spending a month and adding a few thousand lines of code to it.

Syndicated 2014-09-08 09:32:25 from see shy jo

560 older entries...

 

joey certified others as follows:

  • joey certified joey as Journeyer
  • joey certified davidw as Journeyer
  • joey certified bombadil as Journeyer
  • joey certified dhd as Journeyer
  • joey certified ajt as Journeyer
  • joey certified chrisd as Journeyer
  • joey certified scandal as Journeyer
  • joey certified lewing as Journeyer
  • joey certified jwz as Master
  • joey certified graydon as Journeyer
  • joey certified cas as Journeyer
  • joey certified garrett as Journeyer
  • joey certified lupus as Journeyer
  • joey certified octobrx as Journeyer
  • joey certified pudge as Journeyer
  • joey certified marcel as Journeyer
  • joey certified ljlane as Journeyer
  • joey certified uzi as Journeyer
  • joey certified quinlan as Journeyer
  • joey certified bribass as Journeyer
  • joey certified jonas as Journeyer
  • joey certified dsifry as Journeyer
  • joey certified plundis as Journeyer
  • joey certified deirdre as Journeyer
  • joey certified crackmonkey as Journeyer
  • joey certified jim as Journeyer
  • joey certified vincent as Journeyer
  • joey certified apenwarr as Journeyer
  • joey certified schoen as Journeyer
  • joey certified CentralScrutinizer as Apprentice
  • joey certified wichert as Master
  • joey certified doogie as Journeyer
  • joey certified espy as Journeyer
  • joey certified omnic as Journeyer
  • joey certified hands as Journeyer
  • joey certified stig as Journeyer
  • joey certified nick as Journeyer
  • joey certified tausq as Journeyer
  • joey certified broonie as Journeyer
  • joey certified dunham as Journeyer
  • joey certified austin as Journeyer
  • joey certified lordsutch as Journeyer
  • joey certified Gimptek as Apprentice
  • joey certified jimd as Journeyer
  • joey certified chip as Master
  • joey certified jgg as Master
  • joey certified branden as Journeyer
  • joey certified z as Journeyer
  • joey certified srivasta as Journeyer
  • joey certified danpat as Journeyer
  • joey certified lilo as Journeyer
  • joey certified seeS as Journeyer
  • joey certified netgod as Journeyer
  • joey certified dres as Journeyer
  • joey certified cech as Journeyer
  • joey certified knghtbrd as Journeyer
  • joey certified calc as Journeyer
  • joey certified ruud as Journeyer
  • joey certified edlang as Journeyer
  • joey certified gorgo as Journeyer
  • joey certified jwalther as Journeyer
  • joey certified bma as Journeyer
  • joey certified claw as Apprentice
  • joey certified hp as Journeyer
  • joey certified esr as Master
  • joey certified tobi as Journeyer
  • joey certified ajk as Journeyer
  • joey certified Joy as Journeyer
  • joey certified ejb as Journeyer
  • joey certified corbet as Journeyer
  • joey certified rcw as Journeyer
  • joey certified woot as Journeyer
  • joey certified bcollins as Journeyer
  • joey certified neuro as Journeyer
  • joey certified biffhero as Journeyer
  • joey certified Trakker as Journeyer
  • joey certified bdale as Journeyer
  • joey certified foka as Journeyer
  • joey certified davem as Master
  • joey certified logic as Journeyer
  • joey certified mstone as Journeyer
  • joey certified drow as Journeyer
  • joey certified clameter as Journeyer
  • joey certified mdorman as Journeyer
  • joey certified bwoodard as Journeyer
  • joey certified JHM as Journeyer
  • joey certified lalo as Journeyer
  • joey certified edb as Journeyer
  • joey certified shaleh as Journeyer
  • joey certified x as Apprentice
  • joey certified stephenc as Journeyer
  • joey certified bodo as Journeyer
  • joey certified jpick as Journeyer
  • joey certified ncm as Journeyer
  • joey certified gord as Journeyer
  • joey certified mpav as Journeyer
  • joey certified lazarus as Apprentice
  • joey certified starshine as Journeyer
  • joey certified che as Journeyer
  • joey certified brother as Journeyer
  • joey certified joeysmith as Journeyer
  • joey certified bod as Journeyer
  • joey certified decklin as Journeyer
  • joey certified gibreel as Journeyer
  • joey certified torsten as Journeyer
  • joey certified alfie as Apprentice
  • joey certified aclark as Journeyer
  • joey certified kju as Journeyer
  • joey certified psg as Journeyer
  • joey certified zed as Journeyer
  • joey certified evo as Journeyer
  • joey certified mbaker as Journeyer
  • joey certified cmr as Journeyer
  • joey certified Tv as Journeyer
  • joey certified xtifr as Journeyer
  • joey certified sstrickl as Journeyer
  • joey certified etbe as Journeyer

Others have certified joey as follows:

  • joey certified joey as Journeyer
  • dhd certified joey as Journeyer
  • ajt certified joey as Master
  • davidw certified joey as Journeyer
  • alan certified joey as Journeyer
  • uzi certified joey as Journeyer
  • caolan certified joey as Journeyer
  • tron certified joey as Master
  • bombadil certified joey as Journeyer
  • cas certified joey as Journeyer
  • garrett certified joey as Master
  • lupus certified joey as Journeyer
  • graydon certified joey as Journeyer
  • marcel certified joey as Journeyer
  • mblevin certified joey as Journeyer
  • bribass certified joey as Master
  • plundis certified joey as Journeyer
  • matias certified joey as Journeyer
  • ajv certified joey as Journeyer
  • crackmonkey certified joey as Master
  • jim certified joey as Master
  • CentralScrutinizer certified joey as Master
  • schoen certified joey as Master
  • pedro certified joey as Master
  • omnic certified joey as Master
  • hands certified joey as Master
  • tausq certified joey as Journeyer
  • suzi certified joey as Master
  • broonie certified joey as Master
  • nick certified joey as Journeyer
  • lordsutch certified joey as Master
  • jimd certified joey as Master
  • chip certified joey as Master
  • jgg certified joey as Master
  • branden certified joey as Master
  • srivasta certified joey as Master
  • danpat certified joey as Master
  • darkewolf certified joey as Master
  • z certified joey as Journeyer
  • cech certified joey as Master
  • dres certified joey as Master
  • gorgo certified joey as Master
  • ruud certified joey as Master
  • kaig certified joey as Master
  • wichert certified joey as Master
  • ajk certified joey as Master
  • ljlane certified joey as Master
  • Joy certified joey as Journeyer
  • andrei certified joey as Master
  • rcw certified joey as Master
  • Trakker certified joey as Master
  • neuro certified joey as Master
  • starshine certified joey as Master
  • seeS certified joey as Master
  • foka certified joey as Master
  • pretzelgod certified joey as Master
  • mstone certified joey as Master
  • bcollins certified joey as Master
  • doviende certified joey as Master
  • dmarti certified joey as Master
  • splork certified joey as Master
  • bdale certified joey as Master
  • drow certified joey as Master
  • edward certified joey as Master
  • ljb certified joey as Journeyer
  • claw certified joey as Master
  • edb certified joey as Master
  • shaleh certified joey as Master
  • jpick certified joey as Master
  • zacs certified joey as Journeyer
  • jae certified joey as Master
  • benson certified joey as Journeyer
  • wardv certified joey as Master
  • jeroen certified joey as Master
  • lazarus certified joey as Journeyer
  • mpav certified joey as Master
  • walken certified joey as Master
  • ncm certified joey as Master
  • Barbwired certified joey as Master
  • kraai certified joey as Master
  • che certified joey as Master
  • lstep certified joey as Master
  • brother certified joey as Master
  • nas certified joey as Journeyer
  • acme certified joey as Master
  • moshez certified joey as Master
  • tca certified joey as Journeyer
  • cord certified joey as Master
  • sethcohn certified joey as Master
  • bod certified joey as Journeyer
  • tripix certified joey as Journeyer
  • jLoki certified joey as Master
  • sh certified joey as Master
  • lerdsuwa certified joey as Master
  • torsten certified joey as Master
  • alfie certified joey as Master
  • mhatta certified joey as Master
  • aclark certified joey as Master
  • kju certified joey as Master
  • psg certified joey as Master
  • zed certified joey as Master
  • karlheg certified joey as Master
  • evo certified joey as Master
  • ole certified joey as Master
  • jfs certified joey as Master
  • bma certified joey as Master
  • jtc certified joey as Master
  • gibreel certified joey as Master
  • Jordi certified joey as Master
  • jhasler certified joey as Master
  • cpbs certified joey as Journeyer
  • ths certified joey as Master
  • decklin certified joey as Master
  • Tv certified joey as Master
  • xtifr certified joey as Master
  • joeysmith certified joey as Master
  • mishan certified joey as Master
  • keverets certified joey as Master
  • pa certified joey as Master
  • Slimer certified joey as Master
  • weasel certified joey as Master
  • technik certified joey as Master
  • baretta certified joey as Master
  • robster certified joey as Master
  • juhtolv certified joey as Master
  • rcyeske certified joey as Master
  • kmself certified joey as Master
  • andersee certified joey as Master
  • asuffield certified joey as Master
  • charon certified joey as Master
  • claviola certified joey as Master
  • chrisd certified joey as Master
  • mdz certified joey as Master
  • buckley certified joey as Master
  • moray certified joey as Master
  • jtjm certified joey as Master
  • mwk certified joey as Master
  • proski certified joey as Master
  • cmiller certified joey as Master
  • pau certified joey as Master
  • rkrishnan certified joey as Master
  • dieman certified joey as Master
  • eckes certified joey as Master
  • fxn certified joey as Master
  • etbe certified joey as Master
  • Sam certified joey as Master
  • fallenlord certified joey as Master
  • hanna certified joey as Master
  • maxx certified joey as Master
  • dopey certified joey as Master
  • tfheen certified joey as Master
  • ttroxell certified joey as Master
  • Netsnipe certified joey as Master
  • quarl certified joey as Journeyer
  • amck certified joey as Master
  • riverwind certified joey as Master
  • pere certified joey as Journeyer
  • NoWhereMan certified joey as Master
  • jochen certified joey as Master
  • faw certified joey as Master
  • mako certified joey as Master
  • Pizza certified joey as Master
  • sysdebug certified joey as Master
  • vern certified joey as Master
  • ctrlsoft certified joey as Master
  • lkcl certified joey as Master
  • hasienda certified joey as Master
  • gesslein certified joey as Master
  • ean certified joey as Master
  • dangermaus certified joey as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page