Ansible and Fedora Infrastructure
I’ve been getting some questions about why and how Fedora Infrastructure is trying out ansible for systems mgmt. So, I thought I’d write up the answers I’m regularly replying with.
https://fedoraproject.org/wiki/User:Skvidal/Ansible
Feel free to add comments, etc there.
func future and maintainer
Over the last year or so development on func has slowed and stopped. I’ve
been working on other projects and while I’m still USING func I’m not
working on it anymore. I asked about new maintainers to all the folks with
commit access and Steve Salevan stepped up! He’s been working on func for
a VERY long time and his new position has opened up opportunities for him
to enhance it further. I am happy that someone else will pick up func and
make it better and I’m also happy it is someone who has worked on it for
so long.
Thank you for being patient while things slumbered.
coprs/buildsystems
If you’ve been following my blog you’ve probably seen most of what I’m working on. However, at the request of my manager (hi Tom!) I wanted to collect what I’ve been working on and why. So I decided to put the pieces together on the fedora wiki:
https://fedoraproject.org/wiki/User:Skvidal/BuildSystem
If you’d like to help or to discuss email me or discuss on the fedora buildsys list.
ansible and builder creation/provisioning etc
I posted this to the ansible list but I thought I’d point it out here, too:
http://fedorapeople.org/cgit/skvidal/public_git/scripts.git/tree/ansible/start-prov-boot.py
For fedora infrastructure we wanted to make our builders trivial to refresh and reinstall. They hadn’t been that easy to do before and ideally I wanted something I could have done without any user intervention at all.
However, I did not want to just instantiate a pre-existing image for multiple reasons:
1. b/c it includes an implicit amount of staleness in the img – and ongoing maintenance
2. b/c it includes a certain amount of dirtiness b/c of the sysprep scripts are not as clean as a brand-new kickstart is
3. in any case I wanted something that could work in other places and with other virt tools.
The above is a first cut – but it mimics the vm kickstarting process we use for a lot of hosts in fedora infrastructure. However, it does it all using ansible to flow-control the whole thing.
I’m positive someone will mention some other mechanism I could have used to do the same thing. I’m not very interested, to be honest. I wanted something that:
1. didn’t require me to do a bunch of special magic in the kickstart %post to be able to molest/handle the instance (in this case I just need to suck down an authorized_keys file)
2. didn’t require any special software installed on the instance (in this case ansible only needs a running sshd – which I cannot imagine installing a system w/o)
3. was relatively easy to work with for the future.
4. something that was so loosely tied to any specific instance-creation program or platform that I could easily swap out one for the other or make it modular w/o a massive restructuring.
5. was in python 
anyway – it worked very nicely to spin up the 35 buildvm instances for the current koji rebuild and if nothing else, I’m quite pleased with that.
mockremote
I’ve been working with ansible and mock to make a simpler tool for building some pkgs on a remote system. I started on ftbfs but I wanted something for a less involved build process. Essentially a ‘build these pkgs over there and fetch back the results as you go’ command. So this is mockremote.
mockremote: mockremote will build a set of packages on a remote machine using mockchain. If you specify more than one package it will build them one at a time on the remote machine allowing for each previous result to be used as a buildreq for the one before. Just like mockchain it will will optionally build them recursively until it can no longer achieve a successful build. It retrieves the results from each build via rsync after each pkg completes. This ensures that in the event of a remote system crash you lose the least number of pkgs. If you restart the build, pointing to the same destdir it will only build those pkgs which did not complete before. - requires mock, mockchain on remote system - requires the user you specify for the remote system be able to run mock/mockchain (defaults to user mockbuilder) - requires ansible and rsync locally - requires you have ssh keys for the host on which you wish to build mockremote -b hostname -u username -r mock_chroot -a url pkg pkg pkg
It’s a pretty straightforward command and works nicely for building
things in a set when your remote system is in a better place or has
better bandwidth or just better cpu/ram combos to build a package
you want built.
I know what you’re thinking: “But can’t I do all this with ssh and running mockchain on the remote host by myself?”
and the answer to that is – yes – you can – this command just means you don’t HAVE to, which is nice.
things I’ve learned today
1. @ IN CNAME someserver.org
is REALLY not allowed and won’t work anytime soon – despite it being reasonable that it should.
2. [[ $a == something* ]] – is a nice shorthand in bash for ‘startswith’ for string comparison
3. [[ requires you use && and || instead of -a and -o
4. A way to use the git pre-receive hook to bludgeon certain kinds of checks everytime (zone files anyone?)
5. how much about dnssec seems to be somewhat ‘magical’ and not in a good way.
mockchain grows a couple more options
mockchain has grown a couple more options:
-r chroot
originally the chrootname was the first argument – but that became an annoyance and limitation for some other plans so I just made it -r. This is now required to make it work
I also made it so the pkgs you specify can be local pkgs or they can be http[s] urls to pkgs. mockchain will fetch and build them for you. This is in order to achieve some other evil plans I have.
spherical cow
I wanted to make sure people knew the background on spherical cow:
http://en.wikipedia.org/wiki/Spherical_cow
having worked as a sysadmin in a physics department this was a bit of a running gag so the name amuses me.
It was good to be amused by things while working in the physics dept
mockchain use cases and examples
I’ve mentioned mockchain a few times here. An interaction with averi today on irc pointed out that I should give some examples of how to use it.
Use case #1:
I have a big pile of srpms – I’m pretty sure they all build, but not sure in what order, I want them all built against fedora-16-x86_64.
mockchain fedora-16-x86_64 -l /tmp/myrepo –recurse *.src.rpm
That will produce a dir at /tmp/myrepo which has a config and results dir filled with: resulting pkgs and the logs from the build process in a subdir per package.
Inside the results dir there will be a fedora-16-x86_64 dir which is a yum repository and can be referred to in a yum repo configuration like:
[myrepo-fedora-16-x86_64]
name= my repo I just built with mockchain
baseurl=file:///tmp/myrepo/fedora-16-x86_64/
It will also try to build each and every package and any failures will be tried again after everything else has been built – and this will continue in a loop until either all the packages are successfully built or the failures are identical on two loops.
Use Case #2:
I have a bunch of src.rpms, I want to build them, one at a time. After each package build I want to rsync my packages up to fedorapeople immediately for everyone to use.
for pkg in ~/pkgs/*.src.rpm
do
mockchain fedora-16-x86_64 -l /tmp/myrepo $pkg
rsync -avH -e ssh /tmp/myrepo/results/ fedorapeople.org:public_html/myrepo
done
Use Case #3:
I have some new pkgs I want to build for fedora-16-x86_64. I have a remote repository that contains additional pkgs which provide some of my build dependencies for these new pkgs.
mockchain fedora-16-x86_64 -a http://servername/path/to/my/remote/repo -l /tmp/myrepo -c *.src.rpm
This will give you a repo in /tmp/myrepo/results/fedora-16-x86_64
Just a few examples of how you can use mockchain in your daily operations.
mockchain.py –recurse
On a suggestion from kevin – I added a –recurse option to mockchain. It builds all the pkgs it can and records the failures. When it gets to the end of the stack it tries to build the failures again and keeps trying until either: everything succeeds or the set of failures is the same – twice in a row.
Works pretty well to build a giant pile of pkgs and walk away.
I also talked to clark about mockchain and mock and he’s going to pull it in as a script in mock itself once I get it ready.
Here’s the latest version of things:
Try it out and let me know what it needs.
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!
