Advogato blog for Omnifarious

More LVM love

Fri, 19 Sep 2008 18:10:54 GMT

I must say again that I love LVM. I attached my new hard-drives to my fileserver while it was running, then I had to reboot it because the 3ware 9650SE-4LPML card seems to require that I use the 3ware BIOS setup utility to configure my RAID and doesn't seem to have a Linux utility to do it. But, being able to add the drives while the server was on still saved me about 45 minutes of downtime.

After the reboot, I started using LVM in earnest. I added the new array to the volume group, used pvmove to move all the filesystems to the new volume group (while they were in active use mind you) and then removed the old RAID array from the volume group. Standard LVM stuff. But I was able to do it all while the system was up and running.

Also, I had been using 64MiB allocation chunks, but with 3.18TiB of storage, that gets to be an unwieldy number of chunks. But LVM now has a feature that allows you to change the size of the allocation chunks. In my case, all of my physical volumes had an even number of chunks, and all the filesystems also consisted of a contiguous region with an even number of chunks (at least after I moved them they did) and so I was able to move to a more manageable 128MiB chunk size.

All this change only required about 15 minutes of server downtime. I'll likely have another 45 minutes or so as I switch the old array to RAID 0 (to make sure the wipe works thoroughly) wipe it and then remove all the disks and the card. Most of that will likely be taken up by getting the new disks into cages. That makes a total of about 50-60 minutes of downtime.

If 3ware had a RAID management utility that worked in Linux I could've done all this with 0 downtime so far, and likely only about 30 minutes of downtime in the future for physically moving the drives into cages.

If I had 8 hot-swap cages (4 for the old drives, 4 for the new), I could do it with no downtime.

Why the iPhone is evil

Sat, 13 Sep 2008 22:08:16 GMT

I think the iPhone is evil, and doubly so because it is so nice and pretty. The reason I think it is evil is because it is laden with DRM. Apple controls it completely. By putting your life into your iPhone, you've basically sold your soul to Apple.

Apple must approve any application that runs on the iPhone. Any application must be signed by Apple in order to run. So, as an example, Apple is refusing to allow an iTunes competitor to run on the iPhone. Why anybody would want to pay money for a device that basically isn't really their's is beyond me.

I think the iPhone is especially evil because it is such a neat and pretty device. Many people would be tempted to think that really they do actually get the value they pay money for and ignore the fact that really, the device exists only to benefit Apple and any benefit it provides to them is like the lure of a angler fish. The iPhone is basically a trojan horse, and the fact that you are tricked into paying for it only adds insult to injury.

Random thought on SQLite

Mon, 1 Sep 2008 12:08:21 GMT

SQLite is the Unixey equivalent of Microsoft Access. A cheap, simple database for random use. Of course, it has no fancy GUI or integration with Visual Basic to make fancy forms or anything like that. It just does SQL, and does it reliably and well within its problem domain.

So, now I have an answer for some people who want to use Microsoft Access. It would be nice if someone built a VB like tool to sit on top of SQLite, then it would be a complete replacement. SQLite is certainly a great way to play with SQL without having to go through the administrative pain of setting up a database with a normal piece of database software designed for a much heavier load.

Edit 08:56 PDT: I didn't do the research I ought to have. There are apparently a whole bunch of SQLite GUIs. I don't know if any of them do as much as MS Access does, but I suspect at least one of them will actually be nicer to use for a beginner than the command line is.

The SQLite website has a list of SQLite GUI and/or management tools.

As a random aside, my stupid desktop system locked when I tried to send the new xcode I'd downloaded from Apple over to my laptop. It has a hardware/firmware issue on the motherboard that causes it to lock up when disk access occurs during heavy network usage, heavy usually meaning high speed transfers within my gigabit network. This is a probabilistic problem though, so if I use bittorrent or gnutella on that system it will likely eventually freeze, though it will take hours.

Mythbuster's censored

Sun, 31 Aug 2008 01:10:43 GMT

This is an interesting Slashdot article: CC Companies Scotch Mythbusters Show On RFID Security.

It's basically about how incredibly insecure RFID chips are from a whole host of different perspectives, and why you should never, ever let anybody give you something with an RFID chip in it that is supposed to have some sort of legal meaning of some sort, like a credit card, or a passport, though you don't have much choice nowadays with passports. Of course, this information is just too scary for the public to know, so the credit card companies all got together and pressured the Discovery channel into censoring the Mythbuster's episode in which they demonstrate this.

Nobody should be afraid of what Mythbuster's does. Anybody who is definitely has something to hide.

Another small milestone in PersonalJournal project

Sat, 23 Aug 2008 23:07:16 GMT

Now my PersonalJournal project allows people to register. It automatically brings them to a registration page if they log in with an unknown OpenID. It also uses the OpenID SReg (Simple Registration) extension to try to fetch registration information for a new OpenID to auto-fill in many of the form fields when registering them.

I think I'm going to work on a simple posting interface now. I'm going to allow a given PersonalJournal instance to host posts by multiple users, but I'm going to require that they be given permission to post by a site administrator.

One interesting case is mentioning someone in a post using their OpenID when the OpenID's owner hasn't registered at the site yet. I think I will give that OpenID a stub registration that the owner can change if the OpenID owner registers with that OpenID.

Another interesting case is when a posting user believes that several different OpenIDs refer to the same owner. That's tricky. Right now I allow several different OpenIDs to be tied to the same 'user'. And then all access control is on the basis of 'user', ot OpenID.

I think that what I will do is only allow the owner of an OpenID to tie that OpenID to any others. And the user can 'take over' an OpenID from a different user if they can prove they are the id's owner. This will only work if the OpenID being taken over is associate with a 'user' that has one and only one OpenID associated with it. Then the user will be deleted and all other references to the taken over user will be changed to refer to the taking over user.

It is tempting to only allow automatically created user's who have never registered and turned into 'real' users to be taken over, but someone might log in and register using one of their alternate OpenIDs once without realizing what they're doing and then need to suck that OpenID into the main user they are reigistered under. It would be nice to be able to do that without admin intervention.

Small milestone in PersonalJournal project

Thu, 21 Aug 2008 01:09:13 GMT

I managed to put together a TurboGears application that allows logging in via OpenID. Well, OK, not completely, but the hard part of setting up the two step process of verifying an OpenID someone enters is done.

This was one of the small hurdles to putting together my PersonalJournal project.

Some parts of this like the oid_store and a few other components should be split out into a general package so other people can make TurboGears programs that support OpenID as a client. I would like to know how to make TurboGears extensions that can be used when you're starting a TurboGears application to add new model classes, like the identity extension. OpenID needs some model classes for the OpenID store.

I also stuck in a framework for putting mini-sessions around certain tasks within a session. This is so that my site will be resistant to cross-site scripting attacks based on POSTing to a random URL. I intend to make most POST URLs include a sub-session identifier as part of the URL or a required part of the data posted.

I also added secret data to the session object in the database. This is because I needed to have an HMAC key. I wanted to hand the client some data and wanted to make sure that when it handed it back to me that it was exactly the data I gave it. So the data includes an HMAC of itself using the secret as a key. I imagine this secret data will be more widely useful in other parts of the system.

A better name than 'PersonalJournal' might be in order. The WSJ appears to have used this in the past (or possibly even currently) for some feature of theirs. I'm actually OK with colliding with that though.

Bizarre failure mode for my Mac

Tue, 19 Aug 2008 08:08:40 GMT

Today my Mac exhibited a very strange failure mode. It refused to talk on it's Ethernet port and almost completely refused to listen. It was the strangest thing. After I rebooted, it went away. And the wireless connection worked just fine.

This mattered because at home I set things up so almost no networking works except over an encrypted ssh tunnel when I'm wireless. So I generally prefer to be connected with a wire if I can.

I fixed my main workstation

Wed, 6 Aug 2008 19:12:27 GMT

I fixed my main workstation. I've been meaning to do this for a long time. I partly didn't out of fear that the problem wasn't the fairly simple one I thought it was and partly because my main workstation is old enough now to be disappointing.

The problem was the video card, which is what I thought it was. It was bad. I swapped it out for a really cheap video card I had spare and it worked great.

The video card that I swapped in is largely inadequate for what I use that computer for though, so I'm going to have to replace it with a much better video card a soon as I can afford to. *sigh*

The whole workstation should be replaced. It's old, comparatively slow, and the motherboard has a hardware glitch that causes the system to lock during periods of heavy drive write activity and network access.

Algorithm advice?

Fri, 30 May 2008 02:16:09 GMT

I think I would like an algorithm similar to Advogato's trust metric for deciding what people's names should be by default.

But, let me state the problem...

I'm designing a system in which people use OpenID to authenticate eachother on a series of small websites. I also want people to be able to talk about eachother in a linkable way, like they do on LJ.

The problem with using OpenID for this is that an OpenID is a fairly long and somewhat cumbersome identifier. I think most people have nicknames they would like to use to call others. In fact, many people become known by a well known nickname. For example, there are many people in the world who know me as omnifarious and possibly even think of me that way instead of my actual name of Eric Hopper.

I would like a distributed way to allow a group of people to agree about nicknames. If someone new comments on your site, it might be nice if they immediately acquired a nickname that was short and was well known by other people who commented on your site.

One thought is to have people make computer-readable statements associating an OpenID with a nickname and publishing them on their sites. Then you could periodically gather this data from various sites and run an algorithm like Advogato's trust metric algorithm to come up with default nicknames for people who are still two or three degrees away from you on your social graph.

First recorded attempt to attack my systems via IPv6

Fri, 30 May 2008 00:13:15 GMT

Someone just tried to spam my CAKE wiki via IPv6. The attack came from 2002:c26a:c164:0:216:cbff:feab:b3f5 which is a 6to4 address (you can tell from the beginning 2002) meaning that it corresponds to the IPv4 address of c26ac164, also known as 194.106.193.100 which is the address of some computer in Poland.

It also looks like they're on a network that's using EUI-64 based IPv6 address assignment, so the MAC address it came from is 00:16:cb:ab:b3:f5. Looking that up at the MAC Address Vendor lookup page reveals that this MAC address belongs to an Apple.

Someone's poor hacked Mac is trying to spam my wiki, or this is the computer of the hacker who's running the botnet trying to figure out why none of the spam is showing up.