Recent blog entries for elwell

Traffic Light status IoT device

Following on from an all-too-regular "Is the lasercutter working?" mail thread at our local hackerspace, I've decided to come up with a nice simple IoT 'traffic light' device.

Plan is to use KISS principles and have 3 big pushbuttons (Red, Amber, Green) that light up (and stay lit) so that the next person knows if machine needs maintenance/ misbehaving slightly / all good.

Using something like an ESP8266 module (hello Wemos D1), this could trivially publish the status to a broker, and then be acted on elsewhere - updating status on website etc

Next up, finding a local supplier of parts to start a prototype at the next Arduino-U night.

Syndicated 2016-04-01 17:05:00 (Updated 2016-04-01 17:06:23) from Andrew Elwell

23 Jan 2016 (updated 8 Feb 2016 at 11:10 UTC) »

Growatt inverter monitoring with Raspberry Pi

At home we have a small (2.5KW - 10*250w panels) PV system to try and offset our daytime electricity usage. This is connected to a 'Growatt' inverter that handily has both RS485 (wierd 2 pin plugs) and RS232 (9 pin D connector buried under a screwplate) outputs.

With the firmware on ours (installed Sept 2013) it supports modbus-rtu over serial 9600 8N1.

I had done some initial digging and experimentation (as announced on Whirlpool) but never really got sensible values out.When my guruplug (via a long USB to serial adaptor) finally died and I shelved the whole thing. With the completion of the structured wiring though I finally got round to reconnecting it and starting again.

Blue serial cable attached to structured wiring.
Small D9 Gender changer, + cisco console cable (all hail fleabay) gives a nice neat look on the outside, and in the garage I have another console cable plugged into the relevant patch outlet and a cheap usb-serial adaptor in a Raspberry Pi (which also has a GPS module connected, acting as a PPS NTP master)

Anyway, in the intervening time, someone had worked on my hacky scripts and wrapped the posting to PVoutput in an exec(curl) call -- first up I ripped that out and swapped for python requests.

I then went through the growatt modbus guide and made sure that it correctly calculated high and low byte values where these are split. The resulting script can be found on github,  and todays output can be seen on pvoutput. - a couple of charts are duplicated below.
Todays output v insolation prediction
As you can see, we had a couple of clouds going overhead today, so only generated 13KWh  vs 13.7 yesterday. Also the pvoutput fields are somewhat vague - 'Voltage' I've chosen to upload the array DC voltage rather than the grid AC volts (actually, I upload that and the grid frequency as extended data), and 'temperature' - I'd ideally like to have the panel temperatures, but upload the inverter temp so I can see if it's getting toasty. These can be seen on the 'all info' plot below

The observant of you will notice that the Etoday figure was slow to take off - this is because I didn't RTFM and discover that it's uploaded in watts, not kW...

Update 2016-02-08

If you pulled an early version of my code, please grab a new version - I realised the total lifetime generated (and any other 2*2word values) were off as I was doing thing[1]<<8+thing[2] and it should be thing[1]<<16+thing[2]. Ahem. 

The new version also just runs once in the background rather than being called from a cron entry every 5 mins, - it still publishes every 5 mins to pvoutput, but publishes all the messages (in json format) onto my message broker (MQTT) so I can draw a spiffy html5 canvas + websockets graph of whatever I fancy from 

solar/json {"Status": "Normal", "Etotal": 8705.3, "Tinverter": 46.1, "Pac1": 305.1, "ttotal": 32957749.5, "Vac1": 242.1, "PV1Curr": 1.1, "Etoday": 14.4, "Iac1": 1.3, "Pac": 305.1, "Ppv": 339.0, "Fac": 50.02, "PV1Watt": 339.0, "Vpv1": 303.8}

It also monitors the status, and if it changes to 'Fault' it'll look up the fault code and send an alert via pushover. 

Syndicated 2016-01-23 15:35:00 (Updated 2016-02-08 10:35:39) from Andrew Elwell

Publishing DHT22 data via MQTT with an ESP8266

Some time ago I picked up a couple of ESP-01 modules with the intention of using them as wireless temperature/humidity sensors coupled with a DHT22.

Initial investigations took place at the Perth Artifactory "Arduino-U" evenings - I managed to put on a nodemcu lua firmware and found a few (varying) dht22 libraries. however I couldn't ever manage to get it to consistently publish the information to my message broker - it'd do one or two and then lock up. I dug it out again recently and decided to have another go - especially as Pete Scargill seemed to be having success with them (running native C).

So trying to 'revert' to a newer espressif release turned out to be non-trivial - installing the relevant toolchain needs multiple bits. I gave up and noticed that there was a newer (0.9.6-dev_20150704) nodemcu release, so I gave that a try.

First discovery - There's native support for the dht sensors in the firmware, so to get the current values all you need is a

So, I trivially modified the example mqtt.client from the docs and uploaded it with luatool. I'd already set my wifi parameters by hand, so it was connecting to the network at power on automatically. Once I was happy that my 'pub.lua' was publishing OK, I added a trivial init.lua (also available on the gist)

Some points to note from that gist:

  • IP address of broker is hard coded. The nodemcu example uses a non standard port - be aware if you copy n paste
  • Although I have a last will, there's no status setting on successful connect. needs fixing.
  • I'd rather have temp / humidity as two separate topics, but I merged them into one json string to avoid having to worry about mqtt publishing concurrently (used to be fragile on the esp8266)
  • I publish every 5s (tmr.alarm) - 2s is the min recommended for the dht22
so I now see a pretty
sensors/ESP-10264640/json { "temp": 16.8, "humidity": 53.4 }
every 5 s ("mosquitto_sub -h broker -t '#' -v" makes for great sanity checking), which is great but not terribly pretty. Thankfully I have a websocket enabled mosquitto broker running, so I'd already dabbled at some html display. so I took jpmens' dial example and altered it to one of the other gauge styles. and lo...

Syndicated 2015-09-06 17:34:00 (Updated 2015-09-06 17:35:40) from Andrew Elwell

Satellite Tracking / New rotor controller

(it appears I'm about due for my annual blog post entry). Those of you who follow me on twitter will be aware that I've just acquired an old Kenpro 5400 (this is roughly the same as the Yaesu G5500) Azimuth / Elevation rotator, that I plan to use to track cubesats and play with for ham radio.

On opening the control unit (I wanted to see if there were any other primary taps on the transformer, as it's a 110v controller) it was evident it had been 'altered' in the past. To quote someone on #highaltitude "that wiring job is responsible for a thousand dead kittens",

hence a plan was developed to leave the existing controller as an emergency spare and build a fresh 1U rack version instead. The ideas (such as they are) are on a github gist that I'll keep updated with plans. The rough idea being to have a decent embedded board (probably a beaglebone black as a Raspberry Pi depends on an SD card) controlling the relays for output directly, and reading in the potentiometer values to calc position. Using a more powerful microprocessor than say a pic or atmega (arduino) means I can update TLE's automatically and offload much of the tracking directly to the controller - meaning any SDR receivers can concentrate on the signal alone.

I'm also going to house in a GPS module (most likely another one from upu) so that it doubles as a stratum 1 NTP server as well as having accurate position to calculate passes from.

Syndicated 2015-06-14 15:08:00 (Updated 2015-06-14 15:08:13) from Andrew Elwell

Fedora Catchup

The couple of packages I maintain in Fedora have been sitting stable for so long that I've not really had much to do with Fedora recently (that, and getting a mac laptop for $work), but I've just discovered so it's now time to claim a few extras (oh, and push an update to PyEphem while I'm at it...)

Syndicated 2014-07-10 07:10:00 (Updated 2014-07-10 07:10:49) from Andrew Elwell

Happy Cheeks vs SuSE Chameleon

Happy Cheeks imprisoned in a Cray

Kids today. Joyriders in the supercompute cell
Friday Afternoon, and Operations staff are summoned to the Pawsey Centre as the evil SuSE chameleon has been chasing Happy Cheeks (the iVEC Quokka) around the supercomputing cell.

Happy Cheeks and Chameleon explore 'Magnus' 
Happy Cheeks and Chameleon on Galaxy
Inspecting the layout for the Petascale expansion

Being Chased

Syndicated 2014-06-06 06:47:00 (Updated 2014-06-06 06:47:59) from Andrew Elwell

Pretty Colours via MQTT

What does a geek do when they have some spare RGB LED strip (addressable WS2812B) and some cheap nasty LED devices? LED transplant time...

So, first to go was the LED glass prism stand received as a christmas present - out went the potted pcb with three fading LEDs, and in went a single piece of RGB strip fixed in place with a hot glue gun.
wire comes out the bottom and goes to a nanode.

So far so good, but I don't just want fixed or fading colours so time to revisit an IoT idea: Cheerlights

The cheerlights API defines 10 colors that can be set, but I want the possibility of sending any RGB value, so I created @FakeCheerlights as an MQTT series of topics on the broker


which contain the hex RGB value, the identified colour name and the raw tweet.

A separate script (running on the NAS) uses the twitter API via tweepy to follow the twitter stream search for 'cheerlights' and 'fakecheerlights' mentioned in a tweet. If a colour name (matched from the X11 rgb.txt) is found then it publishes the corresponding hex value to the broker

Since fakecheerlights uses a publish/subscribe model, it's *much* faster to react than the original cheerlights protocol which relies on a client polling the server API. The downside is there's no nice fade time between colours.

The nanode (I have one of the earlier batches) was designed as a low-cost ethernet enabled arduino, so uses the Microchip ENC28J60 ethernet rather than the wiznet of the arduino shield. Thanks to UIPEthernet.h and PubSubClient.h it's just possible to code in a basic subscriber which sets the strip output to match.

Since I plan to use the pub/sub model at work for monitoring the machine status and batch queues, I gutted an old ikea childs lamp and replaced the LED with another WS2812B and hooked that up to a freetronics etherten with a PoE daughterboard attached. Sadly the hardware revision I had didn't include the MCP 24AA025E48 that my old nanode did, so my sketch had to include a hard-coded MAC address.

A short youtube video demonstrates the reaction time, and all the source code is on github. (with the exception of the nanode sketch as I didn't save it before closing the arduino ide)

Syndicated 2014-05-19 17:08:00 (Updated 2014-05-19 17:08:23) from Andrew Elwell

The Internet of Meh

Yes, I know there are some very talented, hardworking people busy making "The Internet of Things" (IoT) a reality, and yes, there's even some govt money available just now. However there's still a huge hurdle to overcome before it will ever progress beyond a geek hobby.

Example 1: We have a solar PV system installed (Peak output 2500W) and happen to live in a Damn Sunny place. Great. So far it's chucked out ~2400kWh. Our daytime loads are the evaporative aircon (variable speed 1500w motor) and the pool pump (1600W, fixed speed, needs to run for ~8h day in summer). Together with 'normal' modern household appliances of Washing machine and dishwasher we *should* be able to balance our house load so we're roughly running "grid neutral" -- ie hardly importing or exporting during day. With the current tarrifs (feed in 8c/unit, consumption, 28c unit) it's just not cost effective to put a huge array up (ignoring the fact that we're restricted to 5kW max feed in anyway, and don't have unlimited roof space).

What I want is some smart magic load shedding intelligence - if we have sun, make sure pool pump has run for minimum amount of time. Look at weather forecast - no clouds predicted, take a chance on there being sun up till sunset (so we can predict insolation and expected output) and run for 4h morning, 4h afternoon. Spare capacity? sure - lets ramp up aircon and cool the house a little more comfortably. Oh wait - unexpected load? (dishwasher / WM on a daytime cycle) - back off the aircon, if its the dishwasher, we know that heating cycle will be over in X mins, so suspend the pump.

Sounds lovely right? sure it can be manually done watching the display on the PV inverter and the main electricity meter, but there's gotta be a better way. Well Tough. In Australia a standard domestic fusebox lurks outside, and has max 30 ways. Unless you're a licenced electrician, any poking in here is illegal, and so will cost you contractor rates. How many hours of sparky work (excluding any materials) will it take to make any cost savings negligible. Not Many. Boo.

So, what about appliance control at the "heavy users" directly? RJ12 connector on the bottom of pool controller -- lets ask the manufacturer - do they make an interface? No. Can I get the spec?
"All of our equipment uses an dedicated communication system designed purely for our own use.  Our comms system is not compatible with any other equipment and here is no protocol that we can release for external use."
Maybe not then. Perhaps try Breezair? Similar story.

So, everything has to be reverse engineered (needing skills and tooling) or you pay an integrator (no there aren't any) who will charge $$$$ to do this based on previous work. Current state: Ongoing as a spare time project. Mutter.

Example 2: Air Quality Egg.
Launched with a great flourish and fanfare 2 years ago. OK there were issues with producing the prototypes, but judging from the map, there's now quite a few of us with shields, eggs and more up and running.

Now what? where's the "one year on... report with a few pretty graphs. Even if its a case of "hmm variation is too high to make out anything reasonable" at least let your users know what's going on. Otherwise they'll get bored.

How can I compare my results with others around Perth? Not easily -- takes some hacking of the site to get the raw pachube^Wcosm^Wxively feed ID and then poll it and feed into gnuplot. End-User friendly? I think not.

I *like* the idea of IoT -- it does have the potential to help do clever things - but unless the glue and costs to end user are reasonable it's just not going to take off. Yet.

Syndicated 2014-03-10 09:36:00 (Updated 2014-03-10 09:36:48) from Andrew Elwell

Low-Level hardware hacking

In our 3 shiny new computer rooms (supercompute, IO, Tape - Each with slightly different environmental and cooling requirements), the architects have fitted a single temperature sensor on the back wall. Out of the airflow. This isn't giving us a terribly accurate picture of the supply air going into each rack, and so an alternative DIY method is underway.

A Raspberry Pi in each room (small, low enough power that it can be fed from PoE via a splitter) acts as the local 'head' and grabs the data to send it to a central message broker. From there, we can choose what to do and how to display it.

The sensor hardware consists of: A single DHT22 attached near to the Pi - As useful as these are on an arduino, on a R-Pi they are a bit harder due to the need for timing. The Adafruit interface for these grabs directly from memory address and seems well, a bit hacky. As suggested on IRC, a proper kernel module may be helpful. hint for someone else to complete :-)

The main sensor array is a set of DS18B20 sensors from fleabay (cheap if you're ordering 50 at a time) attached to the sheepwalk i2c interface 'RPI2'. These are then strung around to suit depending on location (ie, for our watercooled racks I'm going to affix the sensors directly to the flow/return lines on the watercooled racks to give us the delta-T across each set of door radiators (The BMS only gives flow/ret temp overall) which when tied to the internal temp sensors in the back of the racks should give us an idea of how well we're shifting the heat away from the machines. All this hardware comes in at less than 100 AUD per room - and some soldering time to make up the sensor chains to suit. To get an idea of the stratification (and therefore how do we position floor tiles relative to the overhead extraction vents) I'm going to put 15-20 sensors on a piece of plastic waste pipe to get a nice vertical profile - cue trip to bunnings (local DIY store) later this weekend.


Hat-tip to the excellent MQTT people - nice n simple to use a python publisher script to send to a mosquitto broker on the lan. From there we can republish to Xively (useless as it doesn't respect timezones in the plotting so I can only see data I sent 8h ago...), Google spreadsheet (in a refactor of the original adafruit idea, and currently to a simple python curses console
Once we go into production and kit is running properly I plan to use the broker for all the machine status info and we'll dispatch accordingly for alarming.

More info and code will appear at

Syndicated 2013-05-25 07:06:00 (Updated 2013-05-25 07:06:57) from Andrew Elwell

Internal Gravatar type service

Since arriving at my new job (excellent ta, thanks for asking) I've once again come across some internal pages that it'd be nice to associate mugshots with (thumbnail is fine) in a service just like gravatar offer. In fact, exactly like gravatar offer... then it could be used with some minimal URL re-writing and minor code changes in applications

IMHO when you have a corporate photo-id upon your person, then this should be available on the intranet - it shouldn't be *that* hard to use the same algorithm on a small webserver vhost to select a suitably sized photo (since its corporate, these can be mapped automatically to LDAP or some other identity management service) entries.

Perhaps I'll have a code, but not next week as we have new toys arriving from our vendor (rhymes with  'play') finally :-)

Syndicated 2013-05-16 15:05:00 (Updated 2013-05-16 15:05:56) from Andrew Elwell

104 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!