Stevey is currently certified at Master level.

Name: Steve Kemp
Member since: N/A
Last Login: 2013-01-05 09:46:29

FOAF RDF Share This

Homepage: http://www.steve.org.uk/

Notes:

For the curious I live in Edinburgh, Scotland.

I'm a big believer in the benefits of the open source
software. I've written, or contributed, to a large
number of Open Source projects including GNU Emacs,
GNUTella, GoGo, GNUMP3d, MP3Blaster.

Nowadays I guess the most visible thing I do in my spare
time is run a site I created for Debian GNU/Linux system administration:

* http://www.debian-administration.org/

Projects

Articles Posted by Stevey

Recent blog entries by Stevey

Syndication: RSS 2.0

If this goes well I have a new blog engine

Assuming this post shows up then I'll have successfully migrated from Chronicle to a temporary replacement.

Chronicle is awesome, and despite a lack of activity recently it is not dead. (No activity because it continued to do everything I needed for my blog.)

Unfortunately though there is a problem with chronicle, it suffers from a bit of a performance problem which has gradually become more and more vexing as the nubmer of entries I have has grown.

When chronicle runs it :

  • It reads each post into a complex data-structure.
  • Then it walks this multiple times.
  • Finally it outputs a whole bunch of posts.

In the general case you rebuild a blog because you've made a entry, or received a new comment. There is some code which tries to use memcached for caching, but in general chronicle just isn't fast and it is certainly memory-bound if you have a couple of thousand entries.

Currently my test data-set contains 2000 entries and to rebuild that from a clean start takes around 4 minutes, which is pretty horrific.

So what is the alternative? What if you could parse each post once, add it to an SQLite database, and then use that for writing your output pages? Instead of the complex data-structure in-RAM and the need to parse a zillion files you'd have a standard/simple SQL structure you could use to build a tag-cloud, an archive, & etc. If you store the contents of the parsed-blog, along with the mtime of the source file you can update it if the entry is changed in the future, as I sometimes make typos which I only spot once Ive run make steve on my blog sources.

Not surprisingly the newer code is significantly faster if you have 2000+ posts. If you've imported the posts into SQLite the most recent entries are updated in 3 seconds. If you're starting cold, parsing each entry, inserting it into SQLite, and then generating the blog from scratch the build time is still less than 10 seconds.

The downside is that I've removed features, obviously nothing that I use myself. Most notably the calendar view is gone, as is the ability to use date-based URLs. Less seriously there is only a single theme, which is what is used upon this site.

In conclusion I've written something last night which is a stepping stone between the current chronicle and chronicle2 which will appear in due course.

PS. This entry was written in markdown, just because I wanted to be sure it worked.

Syndicated 2014-09-17 17:23:08 from Steve Kemp's Blog

Applications updating & phoning home

Personally I believe that any application packaged for Debian should neither phone home, attempt to download plugins over HTTP at run-time, or update itself.

On that basis I've filed #761828.

As a project we have guidelines for what constitutes a "serious" bug, which generally boil down to a package containing a security issue, causing data-loss, or being unusuable.

I'd like to propose that these kind of tracking "things" are equally bad. If consensus could be reached that would be a good thing for the freedom of our users.

(Ooops I slipped into "us", "our user", I'm just an outsider looking in. Mostly.)

Syndicated 2014-09-16 19:42:11 from Steve Kemp's Blog

Storing and distributing secrets.

I run a number of hosts, and they are controlled via a server automation tool I wrote called slaughter [Documentation].

The policies I use to control my hosts are public and I don't want to make them private because they server as good examples.

Because the roles are public I don't want to embed passwords in them, which means I need something to hold secrets securely. In my case secrets are things like plaintext-passwords. I want those secrets to be secure and unavailable from untrusted hosts.

The simplest solution I could think of was an IP-address based ACL and a simple webserver. A client requests something like:

  • http://secret.example.com/user-passwords

That returns a JSON object, if the requesting host is permitted to read the data. Otherwise it returns a HTTP 403 error.

The layout is very simple:

|-- secrets
|   |-- 206.190.139.148
|   |   `-- auth.json
|   |-- 127.0.0.1
|   |   `-- example.json
|   `-- 80.68.84.109
|       `-- chat.json

Each piece of data is beneath a directory/symlink which controls the read-only access. If the request comes in from the suitable IP it is granted, if not it is denied.

For example a failing case:

skx@desktop ~ $ curl  http://sss.steve.org.uk/chat
missing/permission denied

A working case :

root@chat ~ # curl  http://sss.steve.org.uk/chat
{ "steve": "haha", "bot": "notreally" }

(The JSON suffix is added automatically.)

It is hardly rocket-science, but I couldn't find anything else packaged neatly for this - only things like auth/secstore and factotum. So I'll share if it is useful.

Simple Secret Sharing, or Steve's secret storage.

Syndicated 2014-09-12 20:10:06 from Steve Kemp's Blog

11 Sep 2014 (updated 11 Sep 2014 at 11:16 UTC) »

A small email utility and other updates.

Last night I was looking for an image I knew a model had mailed me a few months ago, as we were talking about rescheduling a shoot at the weekend. I couldn't find it, even with my awesome mail client and filing system.

With some free time I figured I could write a little utility to dump all attachments from email folders, and find it that way.

It did cross my mind that there is the simple mail-utility for dumping headers, etc, called formail, which is distributed alongside procmail, but it doesn't handle attachments ..

I was tempted to write a general purpose script to dump attachments, email header values, etc, etc but given the lack of time I merely solved my own problem.

I suspect there is room for a "mail utilities" package, similar to Joey's "moreutils" and my "sysadmin utils". However I note that there is a GNU Mailutils which does things differently than I'd expect - i.e. it contains a POP3 server.

Still if you want to dump attachments from emails, have GMIME installed, and want to filter by attachment-name, or MIME-type, you might look at my trivial attachment-dump program.

Related to that I spent some time last night updating my photography site, so the animals & pets section has updated images at least.

During the course of that I found a bug in my static-site generator, templer which stopped it from automatically populating image height/widths when called in a glob:

Title: Pets & Animals
Images: file_glob( "*.jpg" )
---

This is the page body, it now has access to a variable called 'images'
which is a HTML::Template loop-structure containing name/height/width/etc
for each image in the current directory.

That should now be resolved, and life should once again be good.

Syndicated 2014-09-11 07:09:57 (Updated 2014-09-11 11:16:19) from Steve Kemp's Blog

kvm-hosting will be ceasing, soon.

Seven years ago I wanted to move on from the small virtual machine I had to a larger one. Looking at the the options available it seemed the best approach was to rent a big host, and divide it up into virtual machines myself.

Renting a machine with 8Gb of RAM and 500Gb of disk-space, then dividing that into eights would give a decent spec and assuming that I found enough users to pay for the other slots/shares it would be economically viable too.

After a few weeks I took the plunge, advertised here, and found users.

I had six users:

  • 1/8th for me.
  • 1/8th left empty/idle for the host machine.
  • 6/8th for other users.

There were some niggles, one user seemed to suffer from connectivity problems more than the others, but on the whole the experiment worked out well.

These days, thanks to BigV, Digital Ocean, and all the new-comers there is less need for this kind of thing so last December I announced that the service would cease - and gave all current users 1 year of free service to give them time to migrate away.

.

The service was due to terminate in December, but triggered by some upcoming downtime where our host would have been moved, in the back of a van, from Manchester to York, I've taken the decision to stop it early.

It was a fun experiment, it provided me with low cost hosting (subsidized by the other paying users), and provided some other people with hosting of their own that was setup nicely.

The only outstanding question is what to do with the domain-names? I could let them expire, I could try to sell them, or I could donate them to other people running hosting setups.

If anybody reading this has a use for kvm-hosting.org, kvm-hosting.net, or kvm-hosting.com, then do feel free to get in touch. No promises, obviously, but it'd be a shame for them to end up hosting adverts in a year or twos time..

Syndicated 2014-09-10 08:17:41 from Steve Kemp's Blog

729 older entries...

 

Stevey certified others as follows:

  • Stevey certified skx as Master
  • Stevey certified zx80user as Journeyer
  • Stevey certified faw as Apprentice
  • Stevey certified Liedra as Journeyer
  • Stevey certified ladypine as Journeyer
  • Stevey certified Sarah as Apprentice
  • Stevey certified Ward as Master
  • Stevey certified chipx86 as Journeyer
  • Stevey certified johnnyb as Journeyer
  • Stevey certified perlpimp as Journeyer
  • Stevey certified CaptainNemo as Journeyer
  • Stevey certified mobius as Apprentice
  • Stevey certified tjansen as Journeyer
  • Stevey certified auspex as Apprentice
  • Stevey certified laymusic as Journeyer
  • Stevey certified apeiro as Journeyer
  • Stevey certified dneighbors as Master
  • Stevey certified alejandro as Journeyer
  • Stevey certified scandal as Master
  • Stevey certified moray as Journeyer
  • Stevey certified sacha as Apprentice
  • Stevey certified Malkin as Journeyer
  • Stevey certified dria as Journeyer
  • Stevey certified Jordi as Journeyer
  • Stevey certified Mysidia as Journeyer
  • Stevey certified dirtyrat as Journeyer
  • Stevey certified bdelacretaz as Apprentice
  • Stevey certified braden as Journeyer
  • Stevey certified pencechp as Apprentice
  • Stevey certified brouhaha as Journeyer
  • Stevey certified fejj as Journeyer
  • Stevey certified hanna as Apprentice
  • Stevey certified aero6dof as Apprentice
  • Stevey certified gman as Journeyer
  • Stevey certified dlecorfec as Apprentice
  • Stevey certified jpr as Journeyer
  • Stevey certified Pizza as Journeyer
  • Stevey certified kroah as Master
  • Stevey certified wingo as Journeyer
  • Stevey certified jwz as Master
  • Stevey certified ploppy as Master
  • Stevey certified nosinut as Journeyer
  • Stevey certified Radagast as Journeyer
  • Stevey certified Jody as Master
  • Stevey certified ariya as Apprentice
  • Stevey certified nausicaa as Journeyer
  • Stevey certified dan as Master
  • Stevey certified kappa as Apprentice
  • Stevey certified slamb as Apprentice
  • Stevey certified stevebaker as Journeyer
  • Stevey certified gregor as Journeyer
  • Stevey certified maelstorm as Apprentice
  • Stevey certified palm as Apprentice
  • Stevey certified Artimage as Journeyer
  • Stevey certified bluets as Apprentice
  • Stevey certified jds as Journeyer
  • Stevey certified steve as Apprentice
  • Stevey certified elanthis as Apprentice
  • Stevey certified afayolle as Journeyer
  • Stevey certified bonzini as Journeyer
  • Stevey certified jml as Apprentice
  • Stevey certified ciphergoth as Journeyer
  • Stevey certified Fefe as Master
  • Stevey certified Denny as Journeyer
  • Stevey certified sethcohn as Journeyer
  • Stevey certified bjf as Apprentice
  • Stevey certified sdodji as Journeyer
  • Stevey certified raph as Master
  • Stevey certified jarod as Apprentice
  • Stevey certified StevenRainwater as Journeyer
  • Stevey certified Surfr as Apprentice
  • Stevey certified rlevin as Journeyer
  • Stevey certified ike as Apprentice
  • Stevey certified ebf as Journeyer
  • Stevey certified chakie as Journeyer
  • Stevey certified gstein as Master
  • Stevey certified dtype as Apprentice
  • Stevey certified pompeiisneaks as Journeyer
  • Stevey certified acme as Master
  • Stevey certified lsdrocha as Apprentice
  • Stevey certified mjg59 as Journeyer
  • Stevey certified squrl as Journeyer
  • Stevey certified physos as Apprentice
  • Stevey certified rasmus as Master
  • Stevey certified jelle as Journeyer
  • Stevey certified chrisime as Journeyer
  • Stevey certified julian as Master
  • Stevey certified eliot as Journeyer
  • Stevey certified sh as Journeyer
  • Stevey certified mascot as Apprentice
  • Stevey certified SyntaxPolice as Journeyer
  • Stevey certified bytesplit as Apprentice
  • Stevey certified nymia as Journeyer
  • Stevey certified thomasvs as Master
  • Stevey certified Bram as Journeyer
  • Stevey certified pfremy as Apprentice
  • Stevey certified todd as Master
  • Stevey certified bma as Journeyer
  • Stevey certified coolvibe as Journeyer
  • Stevey certified deekayen as Journeyer
  • Stevey certified fxn as Journeyer
  • Stevey certified bgeiger as Apprentice
  • Stevey certified angelsun as Journeyer
  • Stevey certified andrelop as Apprentice
  • Stevey certified jono as Master
  • Stevey certified groom as Apprentice
  • Stevey certified axboe as Master
  • Stevey certified jennv as Journeyer
  • Stevey certified tseaver as Journeyer
  • Stevey certified duncanm as Apprentice
  • Stevey certified jc as Apprentice

Others have certified Stevey as follows:

  • fxn certified Stevey as Journeyer
  • wingo certified Stevey as Master
  • faw certified Stevey as Master
  • Liedra certified Stevey as Journeyer
  • ladypine certified Stevey as Journeyer
  • CaptainNemo certified Stevey as Journeyer
  • apeiro certified Stevey as Journeyer
  • dneighbors certified Stevey as Journeyer
  • Jordi certified Stevey as Journeyer
  • Mysidia certified Stevey as Master
  • braden certified Stevey as Journeyer
  • Chicago certified Stevey as Master
  • pencechp certified Stevey as Master
  • brouhaha certified Stevey as Master
  • wardv certified Stevey as Journeyer
  • Pizza certified Stevey as Master
  • jrf certified Stevey as Journeyer
  • ariya certified Stevey as Master
  • maelstorm certified Stevey as Journeyer
  • palm certified Stevey as Journeyer
  • alfie certified Stevey as Master
  • donscarletti certified Stevey as Master
  • afayolle certified Stevey as Journeyer
  • bonzini certified Stevey as Journeyer
  • ciphergoth certified Stevey as Journeyer
  • domi certified Stevey as Master
  • sdodji certified Stevey as Master
  • jarashi certified Stevey as Master
  • jarod certified Stevey as Master
  • ebf certified Stevey as Journeyer
  • lsdrocha certified Stevey as Master
  • alexm certified Stevey as Journeyer
  • richdawe certified Stevey as Journeyer
  • sh certified Stevey as Journeyer
  • mascot certified Stevey as Master
  • realblades certified Stevey as Apprentice
  • lerdsuwa certified Stevey as Master
  • bytesplit certified Stevey as Apprentice
  • thom certified Stevey as Master
  • coolvibe certified Stevey as Master
  • speeder certified Stevey as Master
  • broonie certified Stevey as Journeyer
  • angelsun certified Stevey as Journeyer
  • MAK certified Stevey as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page