My analysis of I2C's error density has revealed some rather interesting and unanticipated results. First, I2C does have an extremely low error density, even when you push it beyond the maximum capacitance the spec allows. The errors also tend to be clustered, rather than obeying a nice, spread out, Poisson style distribution. This indicates errors are mostly due to noise in the environment. This was also expected. Now the unexpected stuff. Within an I2C frame, if any bytes are corrupted, the most likely scenario are that two bytes are corrupted. I have no good explanation of this situation, other than perhaps the noise creating the errors is spanning across two bytes. I also observed that occasionally frames are missing bytes. This is most likely due to noise interferring with the ACK from the slave to the master, changing an ACK to a NACK, and causing the master to abort the transfer.
Now for the really weird stuff. I was also observing the throughput of I2C over the different cable lengths. In each experiment, everything was exactly same, except for the length of the cable. As the cable length (and thus the cable capacitance) increased, the throughput decreased. Dramatically. A 100 ft cable only had 43% of the throughput of 2 ft cable. The master was supposed to be transmitting with a 400 kHz SCL rate in all the setups. Upon close examination of the I2C specification, I2C has a scheme on the SCL line to allow slow slaves to slow down the transfer rate to the maximum speed they support, or even pause the transfer if, for instance, they need to service an interrupt. It also serves to synchronize the clocks between multiple masters on the bus before arbitration kicks them off. This SCL handling scheme is referred to as clock stretching/clock synchronization, depending on whether slave or master nodes are triggering it.
In my scenario, the bus was triggering it. With the pull-up resistors, the bus lines act as an RC circuit. Thus signal transitions don't occur simultaneously, but obey RC charge and discharge curves. As the line capacitance increases (without the pull-up resistors changing), the RC constant increases and the curves become more gradual. This increased slew time on the bus lines appears like another master performing clock synchronization, or a slave performing clock stretching. This results in the automagically slowing down the SCL clocking rate in response to the bus, rather than nodes attached to the bus. Furthermore, I believe the SCL rate which is converged upon (and thus the throughput rate) is the maximum that the bus lines are capable of reliably supporting. This auto-throttling effect is not documented anywhere that I can find, nor is it predicted by anything that I can find.
On less therotical AKO items, Chris and I discovered the LTC4300-1 2-Wire Hot Swappable Bus Buffer. We believe this chip will alleviate all the flakiness we were having with our hot swap setup. It will also allow us to create an active hub, as opposed to our current passive hub which merely connects all the ports onto the same backplane. With the LTC4300, there shouldn't be any backplane disruptions when a port is connected. Also, if any component connected to a port becomes faulty and pulls a bus line or lines low, a microcontroller on the hub can detect the probability of this situation, and begin disconnecting ports, until it has disconnected the offending port and returned the bus to a usable state. It can also force a STOP condition on the bus if that is required to return it to an idle state after disconnecting the faulty port. We're also going to put transistors on the power lines of all the ports, so that the uC can attempt to reset faulty components. The one thing is that to make it really work right, we need to know at any given point in time if a device is plugged into a port on the hub. The only way we've really figured out how to do this is with a sense line, which means 4-conductor flat satin cable won't cut it anymore. However, I don't think we need to move onto a IEEE1394 style connector. By using the LTC4300's on an active hub, I think that we can hot-swap fine just using 5/6-conductor flat satin cable with RJ type connectors.
We have samples of the LTC4300-1's on the way, and we have a prototype active hub designed, utilizing an ATmega103 as the uC. I think we can get an active hub working pretty quickly.
Yesterday, I was working on some routines to transmit alphanumeric messages to POCSAG pagers. My routines had some serious "issues", it would appear. I attempted to transmit "HELLO WORLD" to a pager. Instead, the pager started dumping out a bunch of its internal canned messages, all garbled up. "CALL OFFICE NOW CALL OFFICE NOW COME BACK HOME NOW LOVE YOU SO MUCH TAKE CARE MEET YOU AT THE SAME PLACE HAVE A GOOD TRIP APPOINTMENT IS CANCELLED CALL BACK..." and on and on. Eventually, the pager crashed and had to be reset. I think I'm going to keep these routines around ;)