Recent blog entries for benad

Running Final Fantasy VII (Steam) on Mac

The recent re-release of Final Fantasy VII on PC and Steam at last made the game easily accessible on modern computers. No need for MIDI drivers or the Truemotion codec, since they were replaced with Ogg-vorbis music and On2 VP8 video files. As an added bonus, the game can sync the save files "to the cloud", and has a way of "boosting" the character stats in those save files to make the game easier if necessary.

But then, the game is made only for Windows, and I only have a Mac. Sure, I can use Bootcamp or a virtual machine, but I'd rather play it on the Mac itself than install and maintain a Windows machine. And not everybody has spare Windows licenses anyway.

So I attempted to use Wine, the "not a Windows emulator". I don't know why each time I use Wine, I'm still pleasantly surprised at how well it works. Sure, the very early versions of Wine were quite unstable, but that was a decade ago. Nowadays, at least 90% of Windows software can run quite well in Wine, without too much effort or hacking, and it just keeps getting better each time I try it.

Now, the latest version of Wine can run Steam quite well. Inside of that Steam, installing and running Final Fantasy VII worked flawlessly.

Setting up Wine and Steam on Macs is quite easy:

  1. Install Xcode. Go in Xcode, Preferences, Downloads and install the command-line tools.
  2. Install MacPorts.
  3. In the Terminal, run sudo port install wine-devel winetricks.
  4. Run winecfg, and close the window.
  5. Run winetricks steam.
  6. Run env WINEPREFIX="$HOME/.local/share/wineprefixes/steam" wine C:\\windows\\command\\start.exe /Unix ~/.local/share/wineprefixes/steam/dosdevices/c:/users/Public/Start\ Menu/Programs/Steam/Steam.lnk

And that's it. When installing the game I checked the option to install a shortcut in the Start menu, and I've made a small tool called to launch the game directly from the Mac. For example, I would start it with perl ~/.local/share/applications/wine/Programs/Steam/FINAL\ FANTASY\ VII.desktop.

Syndicated 2014-07-12 17:50:24 from Benad's Blog

Resuming Final Fantasy with VII

I've never played Final Fantasy VII. Why? Simply put, I've never owned a PlayStation. Considering that countless players say it is the best Final Fantasy game, if not the dubious claim that it's the best video game ever made, I have to try it. Since both FF VII and FF VIII were released on Steam (though Windows-only), they are now finally easily (and legally) accessible to me without having to own a PlayStation console.

I'm skipping too much context. FF VII was a highly influential game. Not only was it the first 3D Final Fantasy, but also one of the first (Japanese) RPG played by a new generation of video game players. It was one of the highest sold video game and was hugely popular outside of Japan. So how come I avoided it for all those years?

Back then, I played pretty much all Final Fantasy games available in North America up to Final Fantasy VI, and since then I played the previously unreleased ones (FF II, III and V). I even played many of the side-franchises of Final Fantasy (Legend, Mystic Quest, and so on), and pretty much all RPGs made by Square available on Nintendo consoles.

But then, in a shrewd move, Square became Sony-exclusive. In exchange, Sony would create a huge marketing campaign to promote FF VII across all of its product lines (movies, music, publications, electronics, DVDs...). At the same time, it heralded a generation of early 3D games that were mixed with pre-rendered backdrops (akin to classical animation) and FMV cut scenes, making video games more accessible, movie-like and appealing to non-gamers. Basically, it became the forbearer of everything I hated about high-budgeted video games that were more movies "for the masses" than games.

Over the years, I couldn't escape its influence, even in other media. I had the misfortune of seeing the overly pretentious Final Fantasy movies (Spirits Within and that FF VII sequel/thing), and the effect it had on Square porting their Nintendo-based RPGs to the PlayStation by replacing important cut scenes with ill-conceived FMVs. Not playing anything else from the Final Fantasy series was my own kind of "rebellion".

What made it infuriating to me was that so many people considered FF VII the "best video game ever", but without having played any previous game in the series, summarily dismissed because they don't contain FMVs or because they aren't in 3D. Those that would defend FF VI as being objectively the better game would also be dismissed using the dubious argument that people prefer the first Final Fantasy they played. I'm fortunate enough that the first Final Fantasy game I played was the original game in the series (the one released in the NES in North America), and I clearly don't consider it the best, far from it. At the same time, I do hold Final Fantasy VI as the best Final Fantasy game up to that point in the series, the best RPG that I've ever played and a masterpiece. I'd be pleasantly surprised if FF VII was better than VI and lived up to its hype and marketing.

Now, 15 years later, I'm emotionally distanced enough from that teen rebellion to attempt to play this game more objectively and on its own merit, apart from the Sony hype (and my hate of Sony) and the opinion of other uneducated players. I already played a few hours of the game and I already have lots to say, but I reserve judgment until I complete the game. For now, I'll follow up on how I'm able to play that game on my Mac in the first place.

Syndicated 2014-07-12 17:23:36 from Benad's Blog

Google Play Music in Canada

One of the things I noticed when I got the Nexus 5 phone was that it pales as an audio player compared to the iPhone. What I didn't mention is also how much the built-in Google Play Music app was spartan. All I could do to play music was to transfer audio files by a USB cable. While it is nice that it auto-scans the storage for music file, I would have preferred a cleaner approach where I could manually add to my music collection specific audio files.

Since I recycled my iPhone 4S as my main music and podcast player, I seldom looked at the Play Music app, until in early June I casually looked at it after an app update and noticed it had quite a few more options. Basically, "Google Play Music All Access" was now available in Canada.

The first thing that jumped at me is that you can upload your music files in your Play Music library, for free. It can even automatically import your entire iTunes library. I tried it with mine, which has about 3800 songs and two dozen playlists, and it worked quite well. There was a minor issue with the file importer ignoring all music files that had accents in their file names, so I had to import those files manually, but still, compared to the non-free iTunes Match service, the experience was amazingly smooth. To compare, Play Music never, ever had any issue importing and playing back my non-matched audio files (meaning, songs not part of the Play Music store), while I constantly run into issues and general slowness with iTunes Match. Yes, the $30 / year service from iTunes is worse than the free Google service, and the Google service doesn't require you to install iTunes in the first place, for all you iTunes haters out there.

So, for the "All Access" thing, it is quite similar to Rdio. I talked about music streaming services in the past, but to give you a summary of the what happened since in Canada, not much. There's still only Rdio, Deezer, and a few small ones with even smaller libraries. So, doing the legal thing with copyright in Canada still sucks.

After using Rdio for a few years, there are still a few things that annoy me, especially given that it's quite an expensive service ($10 per month):

  • By default, Rdio is still highly "social", with everything shared and public by default. And by that I mean your playback history, playlists, listening habits, what you're listening to right now, and more. They compromised by introducing a "private" mode where things are shared with your Rdio friends, but still that sucks.
  • The music collection oddly seems to be shrinking over time. Sure, I noticed that a lot of albums have exact duplicates, with the only distinction being the label that published the album. This creates the weird effect that if you add to your collection one of those albums, but chance it may get delisted and you'll have to hunt the new "owner". Still, half the time stuff gets delisted and never comes back, almost as often as those "1-year contracts" you see on Netflix Canada.
  • Playlist management sucks. Sure, they just added the functionality to manage playlists on iOS, but still, on all platforms you cannot manipulate more than a song at the same time. For example, breaking the album "Mozart: The Complete Operas" into manageable playlists took hours, moving songs one by one. Shift-click?
  • Search, and pretty much the entire GUI on desktop or iOS, is somehow slow. Looking for "that song" doesn't work well and can be frustrating.
  • If you mark songs to be synchronized to mobile devices, it will be so for all your devices. Doesn't make sense if you only carry one device with you, and on top of that the massive bandwidth usage this can generate if you have four devices downloading songs at the same time.

So, I had to try "All Access" from Google, at least to compare. Also, because the first month is free and if you sign up before June 30 (meaning, today or tomorrow), it is $8 instead of $10 per month, for life.

So, how does it stack up compared to Rdio? Let's start with the cons first, to be fair.


  • The user reviews from the Play Music store don't show up in the player view. I miss the inline user reviews from Rdio.
  • No global playback history. Sure, history of radios show up in the current queue, but once the queue is cleared, it is gone. Also, playback queues aren't shared across devices, unless you save them as playlists each time.
  • Mobile support is limited to iOS and Android.
  • The "Thumbs up" automatic playlist isn't sorted the same way across devices for some reason.


  • All your music is there. Personal, bought on Play Music, or "All Access".
  • Search. It's Google. Search is amazing and instant.
  • The GUI is amazingly fast on all platforms.
  • The album library is cleaner (no label duplicates).
  • The library is somewhat bigger than Rdio. For every one "vanishing" album on Rdio, there are three or four that exist only on All Access.
  • Playlist and metadata editing is great. Shift-click to select multiple songs works, and it quite powerful.
  • Song caching for offline playback is done per-device, not as a global setting as on Rdio.

Overall, Google Play Music All Access, apart from its stupid name ("Rdio" has four letters) wins hands down. I think I'll transition from Rdio to All Access in the next few months. Pricing is identical, for feature-wise, All Access is much cleaner, faster and bigger that Rdio. And the (amazingly) completely free "My Library" music upload service is just icing on the cake.

As for iTunes, Apple should get their act together. iTunes Match is plain buggy and slow, searching (local or in the store) is even slower, their Podcasting support is broken beyond repair (avoid it completely), and everybody on Windows hate iTunes, for good reason. Sure, iTunes Radio, Beats, and so on, but they're all US-only, so why should I care if it's going to take until 2016 until we see anything of them in Canada? Same can be said of Spotify, Pandora, Amazon Music, and so on: Until you show up internationally outside of that Silicon Valley bubble, put up or shut up. Google Play Music All Access is there internationally now.

Syndicated 2014-06-29 00:05:25 from Benad's Blog

iOS and Android Development Tools

As I previously mentioned, I've started doing some iOS and Android development. While I haven't done any App of reasonable size in either, I've read a few books and developed some "beginner's software in both, namely "Android Programming: The Big Nerd Ranch Guide" and "Learn iOS 7 App Development". I've also delved a little bit deeper with both platforms, specifically Core Animation and Android NDK.

And, being opinionated about everything, I quickly formed an opinion of the development tools for iOS, with Xcode, and Android, with Eclipse ADT and Android Studio.

Xcode for iOS

Xcode does represent well Apple's aesthetics in software: Over-simplified GUI, limited, but if you can live within those limits it is highly efficient. The GUI is hit-and-miss, and it lacks a ton of features you can now expect from modern IDEs, for example refactoring. If you want a more advanced IDE for Objective-C, you may want to look at AppCode from JetBrains, the makers of IntelliJ IDEA.

Its build system is similar to Microsoft Visual Studio, in the sense that it has projects with various settings, and it compiles your code with its own proprietary system. You can use the command-line xcodebuild to build an Xcode project from the command-line. If you have to use Makefiles to build some cross-platform code, you may want to look at MacPorts, or use the xcrun command to look for the compiler tools specific to an SDK (xcrun --sdk iphoneos ...). Of course, you can add custom build steps that run custom scripts that in turn will execute your external Makefiles.

Still, you won't want to leave Xcode much. Everything is well integrated in there, including the "Quick Help" in the side bar, simple packaging and signing of the iOS app packages, running and debugging with an emulator or a real device plugged in with a USB cable, and so on. It is a complete, self-contained environment, with no surprises. Considering that most of it is closed-source and that you'd have no other option if it didn't work well, it is comforting that it is quite stable.

It should be noted that Xcode includes a superbly packaged set of documentation. The quality of the documentation is very high and comes very close to MSDN.

ADT and Android Studio

If I were asked a few years ago to develop a mobile OS, I would have surely done something quite similar to Android. Based of Java (at least, its API, so Dalvik), Linux (but with a simplified user-space API, so Bionic), a bunch of XML files, a hacked version of Eclipse, and so on. This sounds like praise, but it's not, considering how wrong I were and how much I didn't know any better back then.

Let's start with the Eclipse-based Android Developer Tool. Whatever was there on Google's web site was broken out of the box. Its update mechanism had missing update server sources, so updating it would break its Android 4.4 support. Usability is garbage, but that's expected from Eclipse. The emulator is shockingly slow, even on my bleeding-edge Intel Haswell i7 with 16 GB or RAM. Of course, you can use the virtualized Android VM for Intel chips, but the version that was there a few months ago would crash Mac OS X 10.9, and once you fix it with a patch from Intel, it's still twice slower than the iPhone simulator. Oh, and rotations in the Android 4.4 simulator don't work. Go figure.

Of course, I could switch one beta-quality IDE, ADT, to another beta-quality IDEA-based one, Android Studio. It sucks and it's buggy, but just slightly less so than ADT. It insists on converting the integrated build system of ADT to a bunch of equally obscure Gradle plugins. In theory this is more flexible, but editing a Gradle build file is as intuitive as Maven, meaning not at all.

Oh, and if after all those dire warnings that you should not attempt to develop native, non-Java code on Android you still do so with the Native Development Kit (NDK), you'll be slapped in the face with a horrible hacked build system built atop Makefiles (ndk-build). Of course, NDK with its Makefiles hack doesn't integrate well with ADT, and can't seem to be integrated at all with Android Studio. You can also forget about your slightly faster virtualized Intel Android VM: Back to the slow ARM emulator.

Basically, Android development tools suck. Plan ahead a few days of work to set it up.

Hey, I could have been a video game programmer for game consoles, so I should stop complaining about crappy development environments.

Syndicated 2014-05-27 00:27:14 from Benad's Blog

OpenSSL: My Heart is Bleeding

After a week, I think I can comfortably explain what happened with this "heartbleed" OpenSSL bug. Now, everybody make mistakes. Especially programmers. Especially me. But at least my errors didn't create a major security hole in 20% of the Internet. Let's review some basic tenets of Software Engineering:

  1. All code (of minimal size and complexity) has bugs. Less code and complexity (and functionality) means less bugs.
  2. Software should be made resilliant against errors. If it can't, it should at least halt (crash).
  3. Software should be designed for Humans, both the code and user interface.

Out of hubris, excess and arrogance, the OpenSSL developers managed to do the opposite of all of these tenets. To quote Theo de Raadt:

OpenSSL is not developed by a responsible team.

Why? Let's do some investigation.

First, Robin Seggelmann had this idea to add a completely unnecessary "heartbeat" feature to TLS. Looking at the protocol design alone, the simple fact that the size of the payload exists in two different places (TLS itself and Heartbeat) is pretty bad and begs for a security hole. Anyway, tenet one.

Still, Seggelmann went ahead and sent working code a year later, on December 31st, at 11:59 PM, the best time for a code review. Of course, the code is filled with non-descriptive variable names which hide the error in plain sight during the ineffective code review, but given the poor quality of the OpenSSL code, they find this acceptable. That's tenet three.

At this point, you may ask: "Shouldn't most modern malloc implementations minimally protect software against buffer overflows and overreads?" If you did, you are correct. But then, years ago, OpenSSL implemented their own memory allocation scheme. If you try to revert that back to plain malloc, OpenSSL doesn't work anymore because its code has bugs that depends on memory blocks being recycled in LIFO fashion. That's tenet two.

The result is bad, and very, very real. In Canada, nearly a thousand Social Insurance Numbers were leaked. And that doesn't count or even start to imagine how many private keys and information leaked like that over the past two years.

By the way, this kind of mess have been my experience with cryptographic software. The usability problem with cryptography isn't just for end users, but also the code itself. Using single-letter variables in a mathematical context where each variable is described at length may be acceptable, but meaningless variable letters without comments in code isn't. While I don't mind much about such "math code" in data compression, for security this makes the code less likely to be secure. Basically, everybody think that being smart is sufficient for writing good code, so of course they would be offended if a software engineer would recommend writing the code from their specs instead of letting them do it themselves. No wonder the worst code always comes from university "careerists".

Personally, I'd stop using OpenSSL as soon as possible.

Syndicated 2014-04-15 01:29:07 from Benad's Blog

A Week with Android

Given my reputation as a "Mac Guy", one would expect me to be ignorant of other platforms. It's actually quite the opposite since, at work, I am a well-experienced Windows and Linux developer. I've only recently started programming on iOS, so I thought it would be a good idea to also learn to develop on the Android platform. While not strictly necessary for software development, I bought an Android phone, to get used to its environment and idioms. To get the best possible experience of the platform, I bought a Nexus 5, which I received by mail a little bit over a week ago. Already, I observed a few differences with iOS beyond the typical "iOS versus Android" feature comparisons.

I quickly noticed how much "Android versus iOS" has parallels with "PC versus Mac" in terms of the software platform. Android is highly customizable, though it's unclear if that was done for the users or to appease the manufacturers and carriers. If I haven't gotten a phone directly from Google, I suspect that buying a typical non-Google, bundled-with-a-plan Android phone would have given me something filled with crapware I couldn't uninstall. Like a PC. That, and given how much havoc rogue Apps can do, I immediately installed an anti-virus, firewall and backup software. Again, like a PC.

Then, there's the screen. It's spectacular. At 445 dpi, it may be the first screen I've ever used where I can't distinguish its pixels at any distance. Cramming a full-sized 1080p screen in 5 inches is amazing. The colours are great too. Still, the screen is physically too large for casual one-hand use. It almost feels like a mini-tablet rather than a phone. Also, its large screen size has an impact on battery life.

Speaking of battery life, when the screen is off, battery life is spectacular. Sure, comparing any new device against the aging battery of a 30-month-old iPhone is unfair. But this Nexus 5 can be casually used for days before fully draining its battery. Oh, and that induction charging is really nice too, compared to fighting with those asymmetrical micro-USB cables… Of course, its battery life depends on well-behaved Apps. As an example, a runaway Songza App leaked a CPU spinlock, causing it to keep the CPU fully powered for hours. Even in this extreme scenario, the battery life could still be compared to my iPhone.

Speaking of music Apps, audio on the Nexus 5 suck. Using the exact same high-quality headphones, I can tell that the headphone jack (or whatever else in the audio chain) is significantly worse than the iPhone. There are many Apps to add equalizers and bass boosters, but even then it still doesn't sound as good. Also, volume controls from my headsets don't work. Well, for now all of my music is in the Apple ecosystem, so I don't mind using my SIM-card-less iPhone as an iPod.

I'll be comparing iOS and Android development once I'm more experienced in both. For now, I'll start getting used to it, both from a user and developer perspective.

Syndicated 2014-04-03 00:25:41 from Benad's Blog

Going Root

So, I've just changed the host for my web site. Not because I was unhappy with the excellent Fused, but because, well, I've outgrown shared hosting. If my web site ought to be some kind of professional image, then at the very least I should own the entire software stack, and not be restricted with whatever versions of Apache or PHP the host decide to rent me.

A decade ago, having a web site meant choosing between shared hosting and "bare metal", but since then VPS (virtual private servers) became a viable option that sits in between. For the past few years I was put off VPS because of the costs, using Amazon EC2 as a reference. Considering that I have to maintain a complete server, paying $50 a month for the privilege to do so was just too much for me.

Luckily, pricing for VPS became quite competitive, especially for a web site as simple as mine. So I tried out RamNode, and after a few weeks of use I trust it enough to move my web site to it, and save a hundred dollars per year in the process.

Migrating to a Debian 7 environment was a bit of a challenge. My background, due to my workplace experience, is mostly with RedHat environments, so it took a little while to adapt to Debian. I really like how lean and fast a well-configured Linux server can be, and having automated security updates on a locked-down stable Debian environment is comforting. I still don't host any dynamic content on my site, but if I ever do, it will be my code, not some 3rd-party PHP script that will become another attack vector.

In the end, nobody will notice, but it really like the idea that I now fully own "my web site", from the IP address to the domain name, down to the version of the web server and operating system, short of the hardware. In a world where most people's online presence is done entirely through social networks that treat your content at their whim, owning a web site is the closest thing to complete free speech one can have. That, and it's a cool thing to add to my résumé.

Syndicated 2014-03-30 20:06:56 from Benad's Blog

The Syncing Problem

Conflicts Everywhere

For the past few months I've been using a simple iOS app to track the episodes I've seen in the TV shows I follow. One of its advertised features is its ability to synchronize itself to all of your iOS devices, in my case my iPhone and iPad. Well, that feature barely works. Series that I would remove from my watch list would show up again the next time I launch the app. Or I would mark an episode as watched and within a few seconds it would show up again.

I can't blame the developer: Apple promised that the iOS Core Data integration with iCloud would make synchronization seamless, and yet until iOS 7 it would regularly corrupt your data silently. Worse, it's over-simplistic API hide the countless pitfalls about its use, and I suspect the developer of my simple TV show tracker fell in every single one of them.

Synchronizing stuff across devices isn't a particularly new problem. Personally, I've never experienced synchronizing a PDA with a dock and some proprietary software, but I've experienced a bit the nightmare of synchronizing my old Motorola RAZR phone with iSync and various online address book services. It wasn't just that the various software interpreted the address book structure differently, causing countless duplicated entries, but also that if I dared modifying my address book from multiple devices before a sync, it would surely create conflicts that I would have to fix everywhere.

Nowadays, synchronization issues still loom over every "cloud-based" service, be if for note-taking like Evernote or seamless file synchronization like Dropbox and countless others. In every case, synchronization conflicts are delegated to an "exceptional scenario" for which the usability is horrendous: If you're lucky, elements are renamed, if not, data gets deleted. In almost every case, you'll have to find back previous versions of your data, while keeping older versions of your data is considered a "premium" feature.

This is frustrating. Data synchronization across electronic devices is not something new, and this is now becoming commonplace. Even with a single client device, the server could use a highly-scalable "NoSQL" database that can make the storage service out-of-sync with itself. Imagine how surprised I was when I learned that the database developer is responsible to handle document conflicts in CouchDB, and that those conflicts would "naturally" happen if the database is replicated across multiple machines! The more distributed the data, be it on client-side devices or on the server side, the more synchronization conflicts will inevitably happen.

And yet, for several years, programmers have been daily using tools that solved this problem...


Going back to the example of the TV show tracker, one of the problem with most synchronization systems is that they synchronize documents rather then the user action themselves. If I marked a TV show episode as "watched", then, unless for some reason I marked it as "unwatched" on another device, the show tracker should never come up with the action of "unwatching" an episode.

Similarly, if on one device I mark an episode as "watched", then revert it back to "unwatched", yet on another device that is unaware of those action I completely remove the TV series, then the synchronization should prioritize the latter. Put another way, conflicts are often resolved by taking into context on what state of the data the action was made.

In addition to storing the data model, each device should also track the sequence of actions done on each device. Synchronization would value the user actions first, and then resolve back those actions into the data model. In effect, the devices should be tracking divergeant user action histories.

Yeah, that's what we call "Distributed Version Control Systems". Programmers have been using DVCS like git, Mercurial and many others for over a decade now. If the solution have been a part of our programming tools for so long, why are we still having problems with automated synchronization in our software?

It's a Plain Text World

Sadly, DVCS are made almost solely for plain-text programming source code. Not only that, but marking user actions as "commits" is carefully done manually, and so are resolving synchronization conflicts. Sure, they can support binary files, but they won't be merged like other text files.

As for how DVCS treat a group of files, it went to the complete opposite of older version control systems. In the past, each file had its own set of versions. This is an issue with source code, since usually files have interdependencies that the version control system isn't aware of. Now, with DVCS, a "version" is the state of all the files in the "repository", so changing two different files on two different devices require user intervention as this could cause a conflict from the point of view of those implicit interdependencies. Sure, you can make each file its own repository with clever use of "sub-repositories", but the overhead of doing so in most DVCS makes this impractical.

For individual text files, DVCS (and most VCS in general) treat them as a sequence of text lines. A line of text is considered atomic, and computed deltas between two text files will tend to promote large chunks of sequential line modifications over individual changes. While this may work fine for most programming languages, a few cases cause issues. As an example, in C, it is common practice to indent lines with space characters to make the code more readable, and to increase indentation for each increasing level on nesting. Changing the code a little bit could affect the nesting level of some code block, and by good practice, the indentation level. The result, from a DVCS perspective, is that the entire code block changed because every line had space characters prepended, even if in reality the code semantic is identical for that block.

Basically, DVCS are really made only for plain text documents. And yet, with some effort, some of their underlying design could be used to build a generic and automated synchronization system.

The Long and Snowy Road

Having a version control system on data structures made by software rather than plain text files made by humans isn't particularly new. Microsoft Word does it internally. Photoshop also, through it's "infinite undo" system.

Computing the difference between two file structures is surely something that was solved ages ago by computer scientists. In the end, if each file is just a combination of well-known structures like lists, sets, keyed sets and so on, then one could easily make a generic file difference engine. Plain text files, viewed as simple lists of lines, or even to a certain extent binary files, viewed as lists of bytes, could easily fit in such an engine.

There would be a great effort required to change any existing "programmer's DVCS" into a generic synchronization engine. Everything is oriented towards user interaction, source code text files, deltas represented as the result of a diff command, a single history graph for all the files in the repository, and so on. But on the other way, a generic DVCS can be used as a basis for a programmer's DVCS without too much issue. It would be even quite amazing for it to version control not only the source code, but also the source annotated with compiler output, so that the conflict resolution system would be dealing with the semantics of the source code and not just plain text.

There are a few novel issues that are specific to device synchronization that would affect a generic DVCS, for example having a mechanism to safely prune old data history that is not needed anymore because all devices are up-to-date (to a known point). In fact, unlike existing DVCS, maybe not all devices need to support peer-to-peer synchronization.

But implementing a generic DVCS for automated synchronization do feel like reinventing the wheel. At minimum, things that were already implemented in today's DVCS have to be redone in almost the same way, but with a few important changes to make them work on generic data structures.

On top of that, I have the nagging feeling somebody, somewhere already did this. Or maybe some asshole patented it and then nobody can implement it anymore for fear of being sued into oblivion. This isn't a novel idea by that much (so it shouldn't be patented anyway). Many already considered backing up their entire computers with version control meant for source code, and while impractical, it kind of worked. The extra step of integrating that into a software's data model serialization isn't a big mental leap either. Apple could have done it with Core Data on iCloud, but, like everybody else, synchronization conflicts was an afterthought.

Right now, if I had to write some software that synchronize data with other devices, there would be no generic solution I could use that would implement a generic DVCS, so I would surely do my best to ignore the hard problem of data conflicts until enough users complain. Or maybe by then I'll get fed up and write my own generic DVCS, if nobody else did it first.

Note to self: I may want to post this as an article on my web site.

Syndicated 2014-02-17 00:00:49 from Benad's Blog

Back to Objective-C

In lieu of the usual annual resolutions, this year I made use of some of those vacation days to plan for what programming languages I will learn in the following months. In the past few years, I learned Python, Erlang and D, but sadly I couldn't come up with any excuse to use them in a programming project, be it at home or at work.

A factor that I glossed over when choosing a programming language is its usefulness in the "work market", be it my current work, seeking a new job, or even self-employment. Another factor is what other non-programmers expected out of me. And, of course, being a user of many Apple products, people wonder when they'll see my iPhone App.

I did learn a little bit of Objective-C about a decade ago, at the beginnings of Mac OS X. I did some very basic "Cocoa" desktop development, though back then the programming environment for it was more NextStep's than Apple's. Since then, I focused so much on UNIX, Linux and server-side development that it felt weird to be using a Mac yet forgot how to write desktop software for it.

Unlike Erlang and D, there are surely tons of books for iOS development, and a few for Mac. I will have a shock using the latest XCode, but I can deal with that. While Mac development is unrestricted, my greatest worry is about iOS', with its restrictive API and closed App Store. Do I need to have a corporation to publish an App? Will it be restricted to the Canadian store? If I want to place ads in the App, how do I deal with the revenue?

Of course, I could just do stuff "in-house", be it with their method to deploy Apps for testing purposes, or by using a jailbroken device. That might be preferable in my case, since I'm not doing this to make money with a "crazy idea" but simply for practice. Still, there are some fees ($100 a year?).

But there is the greater issue that developing for iOS feels like developing for the Java virtual machine, with none of the benefits of the virtual machine. I always liked the idea of developing as close to the OS' kernel as possible, and not be at the mercy of some restrictive APIs. Over the years, I became closer to a Linux developer than Windows, Mac, Android or iOS. In an ideal world, I guess I'd be an open-source developer.

Still, I think it's a good time to jump back in Mac and iOS development. The Mac Cocoa APIs surely matured in the past decade, without the cruft of the "Carbon" API from its Pascal days and the cross-compilation compatibilities with PowerPC. As for iOS, the new visual style of iOS 7 is perfect for a drawing-inept developer like me, as all I need is good and clean spacing of tasteful fonts in hand-picked colours, and not the high-resolution pixellated Photoshop mess of the past few years.

As for what software I'll write, I don't know. Very likely it will be a wholly unoriginal idea that I will do for myself first because the currently existing solutions either do it poorly or not to my taste. Then release them as free and add that to a proper "software portfolio". Compared to the crazy challenges I've had in the past few years, iOS development shouldn't be that difficult…

Syndicated 2014-01-05 19:48:27 from Benad's Blog

Exceptional Exceptions

When I learned the C++ programming language in 1997, I assumed that exception handling was a completely optional language feature, since it was presented in the latter chapters of the C++ books, and up to that chapter none of the code examples would use exceptions. I was wrong, of course, but this mentality is what partly caused my biggest programming mistake of 2013.

Exception handling is a critical language feature in programming that you simply can't ignore, for at least two reasons.

First, in languages compiled for a run-time virtual machine, exceptions are part of the execution environment. In "system programming compiled languages" like C, C++ or D, many operations, like dereferencing NULL references or dividing an integer by 0, cause the operating system to throw a signal to the process that makes it crash. In VMs, they would cause an exception to be thrown at that point in the process. But then, in all VM-based languages I've seen, simply not attempting to catch any of these VM-level exception would cause the default exception handler to crash the process, which is equivalent to system languages.

Second, exceptions are "implicit", meaning they can be thrown from anywhere without warning. Sure, in Java "normal" exceptions are part of the function signatures, forcing exception handling to be handled properly for these at compilation time, but run-time exceptions can still happen. In C++, C# and most other modern programming languages, function signatures don't carry exception types as part of their interfaces.

This would be fine if all exceptions were "bad" to the extent that they would warrant crashing the entire process. But that's not what happened over the years. In Object-Oriented design, object constructors would often be used to perform both the object creation and its initialization to a fully working state. While languages guarantees object creation under all circumstances (actually, before the constructor function is called), initialization can fail. Since the only functional interface of the constructor is to return an object instance, the only other obvious way to report an initialization error is to throw an exception. (Of course, the object could remain in an "uninitialized" state and return an "uninitialized" error for every other function call, but most developers are too lazy to do that.)

For objects that wrap system resources, initialization can fail under normal execution. For example, an object that wraps a file could produce an error "file not found". Personally, I don't think the whole process should crash for something like that. But then, if opening a file is part of the object's constructor, then throwing an exception is the de-facto way of reporting that system error.

The lack of "finalize" statements in C++ kind of forces "Resource Acquisition Is Initialization" design for objects that wrap system resources, which in turns forces reference counting, and so on. It is typical that in most C++ software and libraries, exceptions are avoided as much as possible. Personally, I just avoided C++ altogether.

But for C# and .NET, the result is nearly catastrophic: Anything can and will throw exceptions, without any compilation warning, and not handling those exceptions will crash the entire process. And not just "exceptional exceptions" that ought to crash the process, but for simple things like attempting to open a file that doesn't exist.

Sure, you can add "try / catch" blocks everywhere, but things can get complicated if your code doesn't own all the execution threads. Combine COM objects, callbacks from foreign DLLs and using delegates with WPF, and now there is no simple "catch-all" solution to exception handling. Sure, there are last-chance exception handlers like Application.DispatcherUnhandledException and AppDomain.UnhandledException, but when was the last time you saw any book or sample code using these? So, of course, by ignorance I didn't set up those exception handlers in my code before it was too late.

My point is that exceptions should be avoided in programming interfaces as much as possible. Separating object creation and initialization would be a good step in that direction. Also, throwing exceptions should be reserved for "fatal" run-time errors that normally would be impossible to recover from and ought to crash the entire process.

As for the "other" part of my "biggest programming mistake", it's all about what various versions of the Windows kernel do when a process crashes. Coming from UNIX with its simple "fork / exec" and signal handling, what Windows does is just plain disgusting. But then, it's Windows, so even if Windows 7 is "less crap", I don't know what I expected from its programming environment when you look under its rug…

Syndicated 2013-12-29 19:30:01 from Benad's Blog

96 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!