Recent blog entries for mikal

Ghost




ISBN: 9781416520870
LibraryThing
Trigger warning, I suppose.











This like a Tom Clancy book, but with weirder sex, much of it non-consensual. Also, not as well thought through or as well researched or as believable. I couldn't bring myself to finish it.

Tags for this post: book john_ringo terrorism nuclear
Related posts: Citadel; Hell's Faire; Princess of Wands; East of the Sun, West of the Moon; Watch on the Rhine; Cally's War


Comment

Syndicated 2014-12-14 15:48:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

14 Dec 2014 (updated 15 Dec 2014 at 00:08 UTC) »

How are we going with Nova Kilo specs after our review day?

Time for another summary I think, because announcing the review day seems to have caused a rush of new specs to be filed (which wasn't really my intention, but hey). We did approve a fair few specs on the review day, so I think overall it was a success. Here's an updated summary of the state of play:



API



API (EC2)

  • Expand support for volume filtering in the EC2 API: review 104450.
  • Implement tags for volumes and snapshots with the EC2 API: review 126553 (fast tracked, approved).


Administrative

  • Actively hunt for orphan instances and remove them: review 137996 (abandoned); review 138627.
  • Check that a service isn't running before deleting it: review 131633.
  • Enable the nova metadata cache to be a shared resource to improve the hit rate: review 126705 (abandoned).
  • Implement a daemon version of rootwrap: review 105404.
  • Log request id mappings: review 132819 (fast tracked).
  • Monitor the health of hypervisor hosts: review 137768.
  • Remove the assumption that there is a single endpoint for services that nova talks to: review 132623.


Block Storage

  • Allow direct access to LVM volumes if supported by Cinder: review 127318.
  • Cache data from volumes on local disk: review 138292 (abandoned); review 138619.
  • Enhance iSCSI volume multipath support: review 134299.
  • Failover to alternative iSCSI portals on login failure: review 137468.
  • Give additional info in BDM when source type is "blank": review 140133.
  • Implement support for a DRBD driver for Cinder block device access: review 134153.
  • Refactor ISCSIDriver to support other iSCSI transports besides TCP: review 130721 (approved).
  • StorPool volume attachment support: review 115716.
  • Support Cinder Volume Multi-attach: review 139580 (approved).
  • Support iSCSI live migration for different iSCSI target: review 132323 (approved).


Cells



Containers Service



Database



Hypervisor: Docker



Hypervisor: FreeBSD

  • Implement support for FreeBSD networking in nova-network: review 127827.


Hypervisor: Hyper-V



Hypervisor: Ironic



Hypervisor: VMWare

  • Add ephemeral disk support to the VMware driver: review 126527 (fast tracked, approved).
  • Add support for the HTML5 console: review 127283.
  • Allow Nova to access a VMWare image store over NFS: review 126866.
  • Enable administrators and tenants to take advantage of backend storage policies: review 126547 (fast tracked, approved).
  • Enable the mapping of raw cinder devices to instances: review 128697.
  • Implement vSAN support: review 128600 (fast tracked, approved).
  • Support multiple disks inside a single OVA file: review 128691.
  • Support the OVA image format: review 127054 (fast tracked, approved).


Hypervisor: libvirt



Instance features



Internal

  • A lock-free quota implementation: review 135296.
  • Automate the documentation of the virtual machine state transition graph: review 94835.
  • Fake Libvirt driver for simulating HW testing: review 139927 (abandoned).
  • Flatten Aggregate Metadata in the DB: review 134573 (abandoned).
  • Flatten Instance Metadata in the DB: review 134945 (abandoned).
  • Implement a new code coverage API extension: review 130855.
  • Move flavor data out of the system_metadata table in the SQL database: review 126620 (approved).
  • Move to polling for cinder operations: review 135367.
  • PCI test cases for third party CI: review 141270.
  • Transition Nova to using the Glance v2 API: review 84887.
  • Transition to using glanceclient instead of our own home grown wrapper: review 133485 (approved).


Internationalization

  • Enable lazy translations of strings: review 126717 (fast tracked).


Networking



Performance

  • Dynamically alter the interval nova polls components at based on load and expected time for an operation to complete: review 122705.


Scheduler

  • A nested quota driver API: review 129420.
  • Add a filter to take into account hypervisor type and version when scheduling: review 137714.
  • Add an IOPS weigher: review 127123 (approved, implemented); review 132614.
  • Add instance count on the hypervisor as a weight: review 127871 (abandoned).
  • Allow extra spec to match all values in a list by adding the ALL-IN operator: review 138698 (fast tracked, approved).
  • Allow limiting the flavors that can be scheduled on certain host aggregates: review 122530 (abandoned).
  • Allow the remove of servers from server groups: review 136487.
  • Convert get_available_resources to use an object instead of dict: review 133728 (abandoned).
  • Convert the resource tracker to objects: review 128964 (fast tracked, approved).
  • Create an object model to represent a request to boot an instance: review 127610 (approved).
  • Decouple services and compute nodes in the SQL database: review 126895 (approved).
  • Enable adding new scheduler hints to already booted instances: review 134746.
  • Fix the race conditions when migration with server-group: review 135527 (abandoned).
  • Implement resource objects in the resource tracker: review 127609.
  • Improve the ComputeCapabilities filter: review 133534.
  • Isolate Scheduler DB for Filters: review 138444.
  • Isolate the scheduler's use of the Nova SQL database: review 89893.
  • Let schedulers reuse filter and weigher objects: review 134506 (abandoned).
  • Move select_destinations() to using a request object: review 127612 (approved).
  • Persist scheduler hints: review 88983.
  • Refactor allocate_for_instance: review 141129.
  • Stop direct lookup for host aggregates in the Nova database: review 132065 (abandoned).
  • Stop direct lookup for instance groups in the Nova database: review 131553 (abandoned).
  • Support scheduling based on more image properties: review 138937.
  • Trusted computing support: review 133106.


Scheduling



Security

  • Make key manager interface interoperable with Barbican: review 140144 (fast tracked, approved).
  • Provide a reference implementation for console proxies that uses TLS: review 126958 (fast tracked, approved).
  • Strongly validate the tenant and user for quota consuming requests with keystone: review 92507.


Service Groups



Sheduler

  • Add soft affinity support for server group: review 140017 (approved).


Tags for this post: openstack kilo blueprint spec nova
Related posts: Specs for Kilo; One week of Nova Kilo specifications; Compute Kilo specs are open; Specs for Kilo; Juno nova mid-cycle meetup summary: slots; Juno nova mid-cycle meetup summary: nova-network to Neutron migration

Comment

Syndicated 2014-12-14 15:07:00 (Updated 2014-12-15 00:08:30) from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

Soft deleting instances and the reclaim_instance_interval in Nova

I got asked the other day how the reclaim_instance_interval in Nova works, so I thought I'd write it up here in case its useful to other people.

First off, there is a periodic task run the nova-compute process (or the computer manager as a developer would know it), which runs every reclaim_instance_interval seconds. It looks for instances in the SOFT_DELETED state which don't have any tasks running at the moment for the hypervisor node that nova-compute is running on.

For each instance it finds, it checks if the instance has been soft deleted for at least reclaim_instance_interval seconds. This has the side effect from my reading of the code that an instance needs to be deleted for at least reclaim_instance_Interval seconds before it will be removed from disk, but that the instance might be up to approximately twice that age (if it was deleted just as the periodic task ran, it would skip the next run and therefore not be deleted for two intervals).

Once these conditions are met, the instance is deleted from disk.

Tags for this post: openstack nova instance delete
Related posts: Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno Nova PTL Candidacy; Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic; Review priorities as we approach juno-3; Thoughts from the PTL

Comment

Syndicated 2014-12-14 13:51:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

Specs for Kilo

We're now a few weeks away from the kilo-1 milestone, so I thought it was time to update my summary of the Nova specifications that have been proposed so far. So here we go...

API



API (EC2)

  • Expand support for volume filtering in the EC2 API: review 104450.
  • Implement tags for volumes and snapshots with the EC2 API: review 126553 (fast tracked, approved).


Administrative

  • Check that a service isn't running before deleting it: review 131633.
  • Enable the nova metadata cache to be a shared resource to improve the hit rate: review 126705 (abandoned).
  • Enforce instance uuid uniqueness in the SQL database: review 128097 (fast tracked, approved).
  • Implement a daemon version of rootwrap: review 105404.
  • Log request id mappings: review 132819 (fast tracked).
  • Monitor the health of hypervisor hosts: review 137768.
  • Remove the assumption that there is a single endpoint for services that nova talks to: review 132623.


Cells



Containers Service



Database



Hypervisor: Docker



Hypervisor: FreeBSD

  • Implement support for FreeBSD networking in nova-network: review 127827.


Hypervisor: Hyper-V

  • Allow volumes to be stored on SMB shares instead of just iSCSI: review 102190 (approved).


Hypervisor: Ironic



Hypervisor: VMWare

  • Add ephemeral disk support to the VMware driver: review 126527 (fast tracked, approved).
  • Add support for the HTML5 console: review 127283.
  • Allow Nova to access a VMWare image store over NFS: review 126866.
  • Enable administrators and tenants to take advantage of backend storage policies: review 126547 (fast tracked, approved).
  • Enable the mapping of raw cinder devices to instances: review 128697.
  • Implement vSAN support: review 128600 (fast tracked, approved).
  • Support multiple disks inside a single OVA file: review 128691.
  • Support the OVA image format: review 127054 (fast tracked, approved).


Hypervisor: ironic



Hypervisor: libvirt



Instance features



Internal

  • A lock-free quota implementation: review 135296.
  • Automate the documentation of the virtual machine state transition graph: review 94835.
  • Flatten Aggregate Metadata in the DB: review 134573.
  • Flatten Instance Metadata in the DB: review 134945.
  • Implement a new code coverage API extension: review 130855.
  • Move flavor data out of the system_metadata table in the SQL database: review 126620 (approved).
  • Move to polling for cinder operations: review 135367.
  • Transition Nova to using the Glance v2 API: review 84887.
  • Transition to using glanceclient instead of our own home grown wrapper: review 133485.


Internationalization

  • Enable lazy translations of strings: review 126717 (fast tracked).


Networking



Performance

  • Dynamically alter the interval nova polls components at based on load and expected time for an operation to complete: review 122705.


Scheduler

  • Add a filter to take into account hypervisor type and version when scheduling: review 137714.
  • Add an IOPS weigher: review 127123 (approved, implemented); review 132614.
  • Add instance count on the hypervisor as a weight: review 127871 (abandoned).
  • Allow limiting the flavors that can be scheduled on certain host aggregates: review 122530 (abandoned).
  • Allow the remove of servers from server groups: review 136487.
  • Convert get_available_resources to use an object instead of dict: review 133728.
  • Convert the resource tracker to objects: review 128964 (fast tracked, approved).
  • Create an object model to represent a request to boot an instance: review 127610.
  • Decouple services and compute nodes in the SQL database: review 126895 (approved).
  • Enable adding new scheduler hints to already booted instances: review 134746.
  • Fix the race conditions when migration with server-group: review 135527 (abandoned).
  • Implement resource objects in the resource tracker: review 127609.
  • Improve the ComputeCapabilities filter: review 133534.
  • Isolate the scheduler's use of the Nova SQL database: review 89893.
  • Let schedulers reuse filter and weigher objects: review 134506 (abandoned).
  • Move select_destinations() to using a request object: review 127612.
  • Persist scheduler hints: review 88983.
  • Stop direct lookup for host aggregates in the Nova database: review 132065 (abandoned).
  • Stop direct lookup for instance groups in the Nova database: review 131553.


Security

  • Provide a reference implementation for console proxies that uses TLS: review 126958 (fast tracked, approved).
  • Strongly validate the tenant and user for quota consuming requests with keystone: review 92507.


Storage

  • Allow direct access to LVM volumes if supported by Cinder: review 127318.
  • Enhance iSCSI volume multipath support: review 134299.
  • Failover to alternative iSCSI portals on login failure: review 137468.
  • Implement support for a DRBD driver for Cinder block device access: review 134153.
  • Refactor ISCSIDriver to support other iSCSI transports besides TCP: review 130721.
  • StorPool volume attachment support: review 115716.
  • Support iSCSI live migration for different iSCSI target: review 132323 (approved).


Tags for this post: openstack kilo blueprint spec
Related posts: Specs for Kilo; One week of Nova Kilo specifications; Compute Kilo specs are open; On layers; Juno nova mid-cycle meetup summary: slots; My candidacy for Kilo Compute PTL

Comment

Syndicated 2014-12-01 20:13:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

The Human Division




ISBN: 9780765369550
LibraryThing
I originally read this as a series of short stories released on the kindle, but the paperback collation of those has been out for a while and deserved a read. These stories are classic Scalzi, and read well. If you like the Old Man's War universe you will like this book. The chapters of the book are free standing because of how they were originally written, and that makes the book a bit disjointed. The cliff hanger at the end is also pretty annoying given the next book hasn't been released.

So, an interesting experiment that perhaps isn't perfect, but is well worth the read.

Tags for this post: book john_scalzi combat aliens engineered_human old_mans_war age colonization human_backup cranial_computer personal_ai
Related posts: The Last Colony ; Old Man's War ; The Ghost Brigades ; Old Man's War (2); The Ghost Brigades (2); Zoe's Tale


Comment

Syndicated 2014-11-25 17:14:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

Fast Food Nation




ISBN: 9780547750330
LibraryThing
I don't read a lot of non-fiction, but I decided to finally read this book having had it sit on the shelf for a few years. I'm glad I read it, but as someone who regularly eats in the US I am not sure if I should be glad or freaked out. The book is an interesting study in how industrialization without proper quality controls can have some pretty terrible side effects. I'm glad to live in a jurisdiction where we actively test for food quality and safety.

The book is a good read, and I'd recommend it to people without weak stomaches.

Tags for this post: book eric_schlosser food quality meat fast industrialized
Related posts: Dinner; Dishwasher Trout; Yum; 14 November 2003; Food recommendation; Generally poor audio quality on pod casts?


Comment

Syndicated 2014-11-16 01:43:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

Specs for Kilo

Here's an updated list of the specs currently proposed for Kilo. I wanted to produce this before I start travelling for the summit in the next couple of days because I think many of these will be required reading for the Nova track at the summit.

API

  • Add instance administrative lock status to the instance detail results: review 127139 (abandoned).
  • Add more detailed network information to the metadata server: review 85673.
  • Add separated policy rule for each v2.1 api: review 127863.
  • Add user limits to the limits API (as well as project limits): review 127094.
  • Allow all printable characters in resource names: review 126696.
  • Expose the lock status of an instance as a queryable item: review 85928 (approved).
  • Implement instance tagging: review 127281 (fast tracked, approved).
  • Implement tags for volumes and snapshots with the EC2 API: review 126553 (fast tracked, approved).
  • Implement the v2.1 API: review 126452 (fast tracked, approved).
  • Microversion support: review 127127.
  • Move policy validation to just the API layer: review 127160.
  • Provide a policy statement on the goals of our API policies: review 128560.
  • Support X509 keypairs: review 105034.


Administrative

  • Enable the nova metadata cache to be a shared resource to improve the hit rate: review 126705 (abandoned).
  • Enforce instance uuid uniqueness in the SQL database: review 128097 (fast tracked, approved).


Containers Service



Hypervisor: Docker



Hypervisor: FreeBSD

  • Implement support for FreeBSD networking in nova-network: review 127827.


Hypervisor: Hyper-V

  • Allow volumes to be stored on SMB shares instead of just iSCSI: review 102190 (approved).


Hypervisor: Ironic



Hypervisor: VMWare

  • Add ephemeral disk support to the VMware driver: review 126527 (fast tracked, approved).
  • Add support for the HTML5 console: review 127283.
  • Allow Nova to access a VMWare image store over NFS: review 126866.
  • Enable administrators and tenants to take advantage of backend storage policies: review 126547 (fast tracked, approved).
  • Enable the mapping of raw cinder devices to instances: review 128697.
  • Implement vSAN support: review 128600 (fast tracked, approved).
  • Support multiple disks inside a single OVA file: review 128691.
  • Support the OVA image format: review 127054 (fast tracked, approved).


Hypervisor: libvirt



Instance features



Internal

  • Move flavor data out of the system_metdata table in the SQL database: review 126620 (approved).
  • Transition Nova to using the Glance v2 API: review 84887.


Internationalization

  • Enable lazy translations of strings: review 126717 (fast tracked).


Performance

  • Dynamically alter the interval nova polls components at based on load and expected time for an operation to complete: review 122705.


Scheduler

  • Add an IOPS weigher: review 127123 (approved).
  • Add instance count on the hypervisor as a weight: review 127871 (abandoned).
  • Allow limiting the flavors that can be scheduled on certain host aggregates: review 122530 (abandoned).
  • Convert the resource tracker to objects: review 128964 (fast tracked, approved).
  • Create an object model to represent a request to boot an instance: review 127610.
  • Decouple services and compute nodes in the SQL database: review 126895.
  • Implement resource objects in the resource tracker: review 127609.
  • Isolate the scheduler's use of the Nova SQL database: review 89893.
  • Move select_destinations() to using a request object: review 127612.


Security

  • Provide a reference implementation for console proxies that uses TLS: review 126958 (fast tracked).
  • Strongly validate the tenant and user for quota consuming requests with keystone: review 92507.


Tags for this post: openstack kilo blueprint spec
Related posts: One week of Nova Kilo specifications; Compute Kilo specs are open; On layers; Juno nova mid-cycle meetup summary: slots; My candidacy for Kilo Compute PTL; Juno nova mid-cycle meetup summary: nova-network to Neutron migration

Comment

Syndicated 2014-10-23 19:27:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

One week of Nova Kilo specifications

Its been one week of specifications for Nova in Kilo. What are we seeing proposed so far? Here's a summary...

API



Administrative

  • Enable the nova metadata cache to be a shared resource to improve the hit rate: review 126705.


Containers Service



Hypervisor: FreeBSD

  • Implement support for FreeBSD networking in nova-network: review 127827.


Hypervisor: Hyper-V

  • Allow volumes to be stored on SMB shares instead of just iSCSI: review 102190.


Hypervisor: VMWare

  • Add ephemeral disk support to the VMware driver: review 126527 (spec approved).
  • Add support for the HTML5 console: review 127283.
  • Allow Nova to access a VMWare image store over NFS: review 126866.
  • Enable administrators and tenants to take advantage of backend storage policies: review 126547 (spec approved).
  • Support the OVA image format: review 127054.


Hypervisor: libvirt

  • Add a new linuxbridge VIF type, macvtap: review 117465.
  • Add support for SMBFS as a image storage backend: review 103203.
  • Convert to using built in libvirt disk copy mechanisms for cold migrations on non-shared storage: review 126979.
  • Support libvirt storage pools: review 126978.
  • Support quiesce filesystems during snapshot: review 126966.


Instance features

  • Allow direct access to LVM volumes if supported by Cinder: review 127318.


Interal

  • Move flavor data out of the system_metdata table in the SQL database: review 126620.


Internationalization



Scheduler

  • Add an IOPS weigher: review 127123 (spec approved).
  • Allow limiting the flavors that can be scheduled on certain host aggregates: review 122530.
  • Create an object model to represent a request to boot an instance: review 127610.
  • Decouple services and compute nodes in the SQL database: review 126895.
  • Implement resource objects in the resource tracker: review 127609.
  • Move select_destinations() to using a request object: review 127612.


Scheduling

  • Add instance count on the hypervisor as a weight: review 127871.


Security

  • Provide a reference implementation for console proxies that uses TLS: review 126958.
  • Strongly validate the tenant and user for quota consuming requests with keystone: review 92507.


Tags for this post: openstack kilo blueprints spec
Related posts: Compute Kilo specs are open; Blueprints to land in Nova during Juno; On layers; My candidacy for Kilo Compute PTL; Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno Nova PTL Candidacy

Comment

Syndicated 2014-10-13 03:27:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

Compute Kilo specs are open

From my email last week on the topic:

I am pleased to announce that the specs process for nova in kilo is
now open. There are some tweaks to the previous process, so please
read this entire email before uploading your spec!

Blueprints approved in Juno
===========================

For specs approved in Juno, there is a fast track approval process for
Kilo. The steps to get your spec re-approved are:

 - Copy your spec from the specs/juno/approved directory to the
specs/kilo/approved directory. Note that if we declared your spec to
be a "partial" implementation in Juno, it might be in the implemented
directory. This was rare however.
 - Update the spec to match the new template
 - Commit, with the "Previously-approved: juno" commit message tag
 - Upload using git review as normal

Reviewers will still do a full review of the spec, we are not offering
a rubber stamp of previously approved specs. However, we are requiring
only one +2 to merge these previously approved specs, so the process
should be a lot faster.

A note for core reviewers here -- please include a short note on why
you're doing a single +2 approval on the spec so future generations
remember why.

Trivial blueprints
==================

We are not requiring specs for trivial blueprints in Kilo. Instead,
create a blueprint in Launchpad
at https://blueprints.launchpad.net/nova/+addspec and target the
specification to Kilo. New, targeted, unapproved specs will be
reviewed in weekly nova meetings. If it is agreed they are indeed
trivial in the meeting, they will be approved.

Other proposals
===============

For other proposals, the process is the same as Juno... Propose a spec
review against the specs/kilo/approved directory and we'll review it
from there.


After a week I'm seeing something interesting. In Juno the specs process was new, and we saw a pause in the development cycle while people actually wrote down their designs before sending the code. This time around people know what to expect, and there are left over specs from Juno lying around. We're therefore seeing specs approved much faster than in Kilo. This should reduce the effect of the "pipeline flush" that we saw in Juno.

So far we have five approved specs after only a week.

Tags for this post: openstack kilo blueprints spec
Related posts: Blueprints to land in Nova during Juno; On layers; My candidacy for Kilo Compute PTL; Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno Nova PTL Candidacy; Juno nova mid-cycle meetup summary: scheduler

Comment

Syndicated 2014-10-12 16:39:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

Lock In




ISBN: 0765375869
LibraryThing
I know I like Scalzi stuff, but each series is so different that I like them all in different ways. I don't think he's written a murder mystery before, and this book was just as good as Old Man's War, which is a pretty high bar. This book revolves around a murder being investigated by someone who can only interact with the real world via personal androids. Its different from anything else I've seen, and a unique idea is pretty rare these days.

Highly recommended.

Tags for this post: book john_scalzi robot murder mystery
Related posts: Isaac Asimov's Robot Short Stories; Prelude To Foundation ; Isaac Asimov's Foundation Series; Caves of Steel; Robots and Empire ; A Talent for War


Comment

Syndicated 2014-10-08 02:43:00 from stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

978 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!