Older blog entries for skvidal (starting at number 444)

FAD: Two-Factor Auth setup

This week we held a Fedora Activity Day in Raleigh at RH HQ to get two-factor auth setup for Fedora Infrastructure. We had a pretty good plan and we knew the pieces we needed to put together – it was just a matter of doing it. We got our goals accomplished but that’s not what I wanted to talk about.

We ended up with 9 people there. I was originally a bit concerned that was too many people, that we would end talking more than accomplishing things and that would suck, especially if we had failed to get things implemented. I was keeping track of what everyone was doing, how they were helping. What I noticed was that everyone contributed in some way. There wasn’t anyone on the sidelines. At one point we had 2 people working on the package reviews of the pkgs we needed to get into fedora (totpcgi and pam_url). We had a person writing the cgi to let us use both yubikeys and google auth, a person working on the provisioning interface to get people setup using google auth, a person working on the puppet config, a person setting up the certs/pki needed to let pam_url connect securely. We had a person setting up cloud instances for us to use to test/blow things up and we had a couple of people writing/rewriting their yubikeys and auth secrets in order to test and retest and reretest.


The FAD just removed all friction (as someone else put it to me yesterday). It meant that instead of waiting a few days or more to solve the problems we only waited 20minutes. Like often said about mediation – good facilities are sometimes all that is required to get things done.

It was great having everyone there and able to work it was great being able to ONLY focus on this one thing. I think we will have this again in the future to help accomplish tasks which are just too involved to bite off a little at a time or will take years to get done at that rate.


Syndicated 2012-11-30 14:38:38 from journal/notes

ansible presentation from @jp_mens

Great presentation slidedeck:




introducing ansible.


Syndicated 2012-11-08 16:29:44 from journal/notes


As I find the need I write functionality I need into the existing euca2ools using their lovely cli python api.

I hate trying to remember an instance id. I know the ip of the host or I know the dns name of the hose. I don’t need to go find the instance id to know which one I want to kill.

But euca-terminate-instances is silly and won’t let me pass in an ip or a hostname. Nor will it let me specify globs :(

So I wrote this


It takes public or private ips, public or private dns names (the ones euca or ec2 has) or instance ids.

It also lets you pass file-globs to them. So you can do things like:

my-terminate-instances i-\*

and kill everything you’re running. Isn’t that fun!



Syndicated 2012-11-02 22:10:32 from journal/notes

ansible and cloud instances

A few days ago I posted about using ansible to provision persistent ip instances in a public or private cloud.

Today I finished up the next piece I needed to make this work for copr builders w/transient ips/instances.

I needed a way to create a new instance, provision it as if it was one of the normal inventory of our systems, and return back the ip of that was given to it via eucalyptus (or ec2) automatically. And when I was done with it – I didn’t want any record of it preserved in our inventory system. I wanted it gone.

The problem was ansible wants to act only on the hosts it knows about in its inventory. This is quite understandable. But since I’m not specifying an ip to this host and I don’t know it in advance I wanted a way, in a single playbook to do this.

So I wrote add_host an action_plugin for ansible. These let you interact with the internal state of the inventory/playbook/etc while executing a playbook.

All it does is to add the host to the in-memory inventory the ansible playbook object has. And it adds that host to a group you specify. This is so in the second play in the playbook you can say ‘oh operate on hosts in this special group name’ and that will be the only host in that group.

I’ve sent in a pull request for it but it’s not been accepted quite yet. :) However, if you want to try it out you can just toss it into your  action_plugins dir in ansible and call it.

Here’s an example playbook. It’s very similar to the one for creating a persistent instance. In the fedora public ansible repo we are, in fact, importing the same set of tasks from another file to set them up the same.

It just means if anyone wants an instance in our private cloud running f17 or el6 it is incredibly trivial to make one available for you.



Syndicated 2012-10-31 05:52:17 from journal/notes

handy ansible action for adding root keys to cloud instances

You’ve just spun up a new instance and you need to give additional people access to the system as root. You have a common IDMS that houses ssh pub keys for your users. You want to trivially specify a list of those users and have their keys show up in root’s .ssh/authorized_keys file.

Here’s what you do:


- name: add root keys for other allowed users
action: authorized_key user=root key=’$PIPE(/path/to/script/for/keys ${root_auth_users})’
only_if: is_set(‘${root_auth_users}’}


In our infrastructure FAS houses all the pubkeys. So Toshio wrote this script: http://infrastructure.fedoraproject.org/infra/ansible/scripts/auth-keys-from-fas

So if you define a hostvar in your ansible inventory for this host – then the above will automatically populate your root authorized_keys with the right pub keys.

Kinda awesome, I think.

Syndicated 2012-10-26 17:42:33 from journal/notes

creating and provisioning euca/ec2/cloud* instances using ansible in one step

Since we’ve been working more and more with our private cloud setups on eucalyptus and openstack I’ve been trying to come up with a semi-sane way of having persistent hosts in the cloudlets.

So that if we shut things down or an instance dies – it can be brought back up automatically without someone having to explicitly say ‘go start this up’. I’ve also been working on setting up ansible such that we can run it via cron on our central command host to maintain our systems.

So, of course, I’ve combined those two things into one task :)

We want certain cloud instances to:

  1. always exist
  2. have the same ip each time they come up.

These are for things like the build master for COPRs/PPAs or for the jenkins-master server or for any number of random services where they need a persistent ip throughout recreations. For the ips we’re using euca-associate-address after allocating a number to our account.

So I started on a mechanism to create those instances and give me results I could use in ansible for other actions. I came up with ec2_create which lets you do that from a local_action call in ansible.

Once you have that module in your ansible library path you just need to:

  • add the host to your ansible inventory file
  • take  this playbook example
  • modify it for your host
  • run it like: ansible-playbook thatplaybook.yml

you’ll need euca2ools installed along with the deps for ansible.

Here’s what the playbook does:

  1. it looks to see if the host is running (by a simple nc connect to port 22)
  2. if it is not running then it will run ec2_create with your args
  3. then associate the ip
  4. wait for the host to get the ip and become available
  5. then it goes on to the next play which is normal ansible provisioning

which is perfect, for the next time you run – steps 2-4 won’t happen – it just goes directly to provisioning. Since ansible playbooks are idempotent you can run them as many times as you want w/o maligning your server.

Now the example playbook is all merged together – but if you use host_vars in your ansible inventory and you setup the check/create tasks as an includable task file – then the playbook for creation and provisioning of any new cloud instance you want becomes VERY short.

More to come as I finish it.

Syndicated 2012-10-25 22:33:37 from journal/notes

ansible and cloud notes

I’ve been working with ansible a lot recently. Fedora has a new public repo:

That’s just the beginnings of our repo for managing systems. There’s a number of details to work out, though and I have to merge over all the non-private things from our builder repo. I’ve started that process it’s just a matter of making reasonable decisions on where the tasks should live.

Recently I’ve been looking at:

and figuring out how a euca2ools-based local_action module would work. I can spin up new instances, if they don’t already exist, and then provision them then I can create an entire network from more or less nothing.

Also working out how best to use euca-allocate-address and and associate-address in our infrastructure to provide some persistent and consistent servers/apps in the cloud. I think I have that bit hashed out but we’re discussing some tracking for cloud instances so we can sensibly shut them down when we need to w/o making everyone angry :)

I’m positive I can do all these things with ansible and euca/openstack. The question now is how long it will take to make them all happen. Anyone interested in helping out, come by #fedora-admin and talk to me.



Syndicated 2012-10-18 15:42:14 from journal/notes

What I want from schwab:

What I want from schwab:

I had told my partner that schwab had always treated me well. That
they had never made me jump through any ridiculous hoops. That I
could do everything via the web. No need for phone calls.

I told her this b/c I thought it was true. I told her this b/c
she’s deaf and it means a lot to her to be able to do things
entirely on her own w/o having to involve anyone else.

So it was a bit shocking when she was asked to call schwab to verify
the deposit amounts and complete the moneylink setup to your
brokerage service.

Worse yet, when I called to deal with the problem, with her at my
side, I was told no one could speak with me, despite having all
the requisite security information. Furthermore, I was told that
if she wanted to complete the process she would need to call in
via TDD or Relay operator.

This was all done in the name of security. Let’s explain a
bit about how security works.

A person is asked a series of questions that only they should know the
answers to. As a result that person is authenticated as actual. Whether
you ask these questions over the web, in person or on the phone, it
provides you with proof that the person you were speaking with is
authentic. Yes, sometimes that data is compromised by people intending
to defraud but if the person presents all the valid information,
it is impossible to know that.

This is the very basics of how it works:
Question, Answer, Authenticated.

In this case the questions were:
1. ssn
2. mother’s maiden name
3. account number
4. the precise amounts deposited into the other account
to moneylink with

Schwab requires that this information be presented ONLY over the phone.
They claim that this is to make it more secure. How or why a telephone
conversation is more secure is never offered. But I will let it go
for the moment their claim that it is. For a hearing person.

For a deaf person who is forced to use a Relay operator it means
connecting to a service, having them dial another number, then giving
all of the above information to a random person with whom you have
no contractual nor social relationship of any kind and virtually no
way to track them down, especially if you use the service more than

Please explain to me how that is MORE secure than having my partner
submit the information over an encrypted network connection?

If this were all done over the web at the end of the day the same
data exchange results but instead of making my partner angry
and resentful, you’ve made her feel happy. And you’ve not made me
into a liar when I told her that schwab wouldn’t make her do all that.

I want this corrected.

Here’s how you correct it:
1. Make the service entirely web-based and web-driven.

2. Provide a mechanism on your website where if a customer
needs to interact with a representative they can do so
in an encrypted chat window.

3. If you are going to try to force people to use the TTY and
or relay operators. Test it out for yourselves and make
your security audit and customer service people use it
with their own information. See how much trust they feel
in that system.

Syndicated 2012-10-12 13:13:29 from journal/notes

Ansible and Fedora Infrastructure

I’ve been getting some questions about why and how Fedora Infrastructure is trying out ansible for systems mgmt. So, I thought I’d write up the answers I’m regularly replying with.


Feel free to add comments, etc there.

Syndicated 2012-08-16 16:11:47 from journal/notes

func future and maintainer

Over the last year or so development on func has slowed and stopped. I’ve
been working on other projects and while I’m still USING func I’m not
working on it anymore. I asked about new maintainers to all the folks with
commit access and Steve Salevan stepped up! He’s been working on func for
a VERY long time and his new position has opened up opportunities for him
to enhance it further. I am happy that someone else will pick up func and
make it better and I’m also happy it is someone who has worked on it for
so long.

Thank you for being patient while things slumbered.

Syndicated 2012-08-03 15:33:55 from journal/notes

435 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!