bagder is currently certified at Master level.

Name: Daniel Stenberg
Member since: 2000-05-10 09:34:05
Last Login: 2009-12-04 19:23:29

FOAF RDF Share This

Homepage: http://daniel.haxx.se/

Notes:

My blog is on daniel.haxx.se/blog

Projects

Articles Posted by bagder

Recent blog entries by bagder

Syndication: RSS 2.0

The TLS trinity dance

In the curl project we currently support eleven different TLS libraries. That is 8 libraries and the OpenSSL “trinity” consisting of BoringSSL, libressl and of course OpenSSL itself.

You could easily be mislead into believing that supporting three libraries that all have a common base would be reallytrinity easy since they have the same API. But no, it isn’t. Sure, they have the same foundation and they all three have more in common that they differ but still, they all diverge in their own little ways and from my stand-point libressl seems to be the one that causes us the least friction going forward.

Let me also stress that I’m but a user of these projects, I don’t participate in their work and I don’t have any insights into their internal doings or greater goals.

libressl

Easy-peacy, very similar to OpenSSL. The biggest obstacle might be that the version numbering is different so an old program that might be adjusted to different OpenSSL features based on version numbers (like curl was) needs some adjusting. There’s a convenient LIBRESSL_VERSION_NUMBER define to detect libressl with.

OpenSSL

I regularly build curl against OpenSSL from their git master to get an early head-start when they change things and break backwards compatibility. They’ve increased that behavior since Heartbleed and while I generally agree with their ambitions on making more structs opaque instead of exposing all internals, it also hurts us over and over again when they remove things we’ve been using for years. What’s “funny” is that in almost all cases, their response is “well use this way instead” and it has turned out that there’s an equally old API that is still there that we can use instead. It also tells something about their documentation situation when that is such a common pattern. It’s never been possible to grasp this from just reading docs.

BoringSSL

BoringSSL has made great inroads in the market and is used on Android now and more. They don’t do releases(!) and have no version numbers so the only thing we can do is to build from git and there’s no install target in the makefile. There’s no docs for it, they remove APIs from OpenSSL (curl can’t support NTLM nor OCSP stapling when built with it), they’ve changed several data types in the API making it really hard to build curl without warnings. Funnily, they also introduced non-namespaced typedefs prefixed with X509_* that collide with other common headers.

How it can play out in real life

A while ago we noticed BoringSSL had removed the DES_set_odd_parity function which we use in curl. We changed the configure script to look for it and changed the code to survive without it. The lack of that function then also signaled that it wasn’t OpenSSL, it was BoringSSL

BoringSSL moved around things that caused our configure script to no longer detect it as “OpenSSL compliant” because CRYPTO_lock could no longer be found by configure. We changed it to instead search for HMAC_Init and we were fine again.

Time passed and BoringSSL brought back DES_set_odd_parity, so our configure script no longer saw it as BoringSSL (the Android fixed this problem in their git but never sent as the fix). We changed the configure script accordingly to properly use OPENSSL_IS_BORINGSSL instead to detect BoringSSL which was the correct thing anyway and now as a bonus it can thus detect and work with both new and old BoringSSL versions.

A short time after, I again try to build curl against the OpenSSL master branch only to realize they’ve deprecated HMAC_Init that we just recently switched to for detection (since the configure script needs to check for a particular named function within a library to really know that it has detected and can use said library). Sigh, we switched “detect function” again to HMAC_Update. Hopefully this exists in all three and will stick around for a while…

Right now I think we can detect and use all three. It is only a matter of time until one of them will ruin that and we will adapt again.

Syndicated 2015-09-02 07:05:26 from daniel.haxx.se

Blog refresh

Dear reader,

If you ever visited my blog in the past and you see this, you should’ve noticed a pretty significant difference in appearance that happened the other day here.

When I kicked off my blog here on the site back in August 2007 and moved my blogging from advogato to self-host, I installed WordPress and I’ve been happy with it since then from a usability stand-point. I crafted a look based on an existing theme and left it at that.

Over time, WordPress has had its hefty amount of security problems over and over again and I’ve also suffered from them myself a couple of times, and a few times I ended up patching it manually more than once. At one point when I decided to bite the bullet and upgrade to the latest version it didn’t work to upgrade anymore and I postpone it for later.

Time passed, I tried again without success and then more time passed.

I finally fixed the issues I had with upgrading. With a series of manual fiddling I finally managed to upgrade to the latest WordPress and when doing so my old theme was considered broken/incompatible so I threw that out and started fresh with a new theme. This new one is based on one of the simple default ones WordPress ships for free. I’ve mostly just made it slightly wider and edited the looks somewhat. I don’t need fancy. Hopefully I’ll be able to keep up with WordPress better this time.

Additionally, I added a captcha that now forces users to solve an easy math problem to submit anything to the blog to help me fight spam, and perhaps even more to solve a problem I have with spambots creating new users. I removed over 3300 users yesterday that never posted anything that has been accepted.

Enjoy.  Now back to our regular programming!

Syndicated 2015-09-01 09:30:01 from daniel.haxx.se

Content over HTTP/2

cdn77 logoRoughly a week ago, on August 19, cdn77.com announced that they are the “first CDN to publicly offer HTTP/2 support for all customers, without ‘beta’ limitations”. They followed up just hours later with a demo site showing off how HTTP/2 might perform side by side with a HTTP/1.1 example. And yes, the big competitor CDNs are not yet offering HTTP/2 support it seams.

Their demo site initially got critized for not being realistic and for showing HTTP/2 to be way better in comparison that what a real life scenario would be more likely to look like, and it was also subsequently updated fairly quickly. It is useful to compare with the similarly designed previously existing demo sites hosted by Akamai and the Go project.

NGINX logocdn77’s offering is built on nginx’s alpha patch for HTTP/2 that was anounced just two weeks ago. I believe nginx’s full release is still planned to happen by the end of this year.

I’ve talked with cdn77’s Jakub Straka about their HTTP/2 efforts, and since I suspect there are a few others in my audience who’re also similarly curious I’m offering this interview-style posting here, intertwined with my own comments and thoughts. It is not just a big ad for this company, but since they’re early players on this field I figure their view and comments on this are worth reading!

I’ve been in touch with more than one person who’ve expressed their surprise and awe over the fact that they’re using this early patch for nginx to run in production. So I had to ask them about that. Below, Jakub’s comments are all prefixed with his name and shown using italics.

nginx

Jakub: “Yes, we are running the alpha patch, which is basically a slightly modified SPDY. In the past we have been in touch with the Nginx team and exchanged tips and ideas, the same way we plan to work on the alpha patch in the future.

We’re actually pretty careful when deploying new and potentially unstable packages into production. We have separate servers for http2 and we are monitoring them 24/7 for any exceptions. We also have dedicated developers who debug any issues we are facing with these machines. We would never jeopardize the stability of our current network.

I’m not an expert on neither server-side HTTP/2 nor nginx in particular , but I think I read somewhere that the nginx HTTP/2 patch removes the SPDY support in favor of the new protocol.

Jakub: “You are right. HTTP/2 patch “rewrites” SPDY into the HTTP/2, so the SPDY is no longer supported after applying the patch. Since we have HTTP/2 running on separate servers, we still have SPDY support on the rest of the network.”

Did the team at cdn77 at all consider using something else than nginx for HTTP/2, like the promising newcomer h2o?

Jakub: “Not at all. Nginx is a clear choice for us. Its architecture and modularity is awesome. It is also very reliable and it has a pretty long history.

On scale

Can you share some of the biggest hurdles you had to overcome to deploy HTTP/2 on this scale with nginx?

Jakub: “Since nobody has tried the patch in such a scale ever before, we had to make sure it will hold even under pressure and needed to create a load heavy testing environment. We used servers from our partner company 10gbps.io and their 10G uplinks to create intensive ghost traffic. Also, it was important to make sure that supporting tools and applications are HTTP/2 ready – not all of them were. We needed to adjust the way we monitor and control servers in few cases.

There are a few bugs in Nginx that appear mainly in association with the longer-lived connections. They cause issues with the application layer and consume more resources. To be able to accommodate HTTP/2 and still keep necessary network redundancies, we needed to upgrade our network significantly.

I read this as an indication that the nginx patch isn’t perfected just yet rather than signifying that http2 is special. Perhaps also that http2 connections might use a larger footprint in nginx than good old http1 connections do.

Jakub mentions they see average data bandwidth savings in the order of 20 to 60 percent depending on sites and contents with the switch to h2, but their traffic amounts haven’t been that large yet:

Jakub: “So far, only a fraction of the traffic is running via HTTP/2, but that is understandable since we launched the support few days ago. On the first day, only about 0.45% of the traffic was HTTP/2 and a big part of this was our own demo site. Over the weekend, we saw impressive adoption rate and the total HTTP/2 traffic accounts for more than 0.8% now, all that with the portion of our own traffic in this dropping dramatically. We expect to be pushing around 1.2% – 1.5% of total traffic over HTTP/2 till the end of this week.

Understandably, it is ramping up. Still, Firefox telemetry is showing at least 10% of the total traffic over HTTP/2 already.

Future of HTTPS and HTTP/2?

Whttp2 logohile I’m talking to a CDN operator, I figured I should poll their view on HTTPS going forward! Will the fact that all browsers only support h2 over HTTPS push more of your customers and your traffic in general over to HTTP, you think?

Jakub: “This is not easy to predict. There is encryption overhead, but HTTP/2 comes with header compression and it is binary. So at this single point, the advantages and disadvantages zero out. Also, the use of HTTPS is rising rapidly even on the older protocol, so we don’t consider this an issue at all.

In general, from a CDN perspective and as someone who just deployed this on a fairly large scale, what’s your general perception of what http2 means going forward?

Jakub: “We believe that this is a huge step forward in how we distribute content online and as a CDN company, we are especially excited as it concerns the very core of our business. From the new features, we have great expectations about cache invalidation that is being discussed right now.

Thanks to Jakub, Honza and Tomáš of cdn77 for providing answers and info. We live in exciting times.

Syndicated 2015-08-27 20:56:25 from daniel.haxx.se

One year and 6.76 million key-presses later

I’ve been running a keylogger on my main Linux box for exactly one year now. The keylogger logs every key-press – its scan code together with a time stamp. This now allows me to do some analyses and statistics of what a year worth of using a keyboard means.

This keyboard being logged is attached to my primary work machine as well as it being my primary spare time code input device. Sometimes I travel and sometimes I take time-off (so called vacation) and then I usually care my laptop with me instead which I don’t log and which uses a different keyboard layout anyway so merging a log from such a different machine would probably skew the results a bit too much.

Func KB-460 keyboard

What did I learn?

A full year of use meant 6.76 million keys were pressed. I’ve used the keyboard 8.4% on weekends. I used the keyboard at least once on 298 days during the year.

When I’m active, I average on 2186 keys pressed per hour (active meaning that at least one key was pressed during that hour), but the most fierce keyboard-bashing I’ve done during a whole hour was when I recorded 8842 key-presses on June 9th 2015 between 23:00 and 24:00! That day was actually also the most active single day during the year with 63757 keys used.

In total, I was active on the keyboard 35% of the time (looking at active hours). 35% of the time equals about 59 hours per week, on average. I logged 19% keyboard active minutes, minutes in which I hit at least one key. I’m pretty amazed by that number as it equals almost 32 hours a week with really active keyboard action.

Zooming in and looking at single minutes, the most active minute occurred 15:48 on November 10th 2014 when I managed to hit 405 keys. Average minutes when I am active I type 65 keys/minute.

Looking at usage distribution over week days: Tuesday is my most active day with 19.7% of all keys. Followed by Thursday (19.1%), Monday (18.7%), Wednesday (17.4%) and Friday (16.6%). I already mentioned weekends, and I use the keyboard 4.8% on Sundays and a mere 3.6% on Saturdays.

Separating the time-stamps over the hours of the day, the winning hour is quite surprising the 23-00 hour with 11.9% followed by the more expected 09-10 (10.0%), 10-11 and 14-15. Counting the most active minutes over the day shows an even more interesting table. All the top 10 most active minutes are between 23-00!

I’ll let the keylogger run and see what else I’ll learn over time…

Syndicated 2015-08-19 05:58:23 from daniel.haxx.se

curl me if you can

I got this neat t-shirt in the mail yesterday. 3scale runs a sort of marketing campaign right now and they give away this shirt to the ones who participate, and they were kind enough to send one to me!

Curl me if you can

Syndicated 2015-08-04 08:57:53 from daniel.haxx.se

895 older entries...

 

bagder certified others as follows:

  • bagder certified shughes as Journeyer
  • bagder certified andrei as Master
  • bagder certified kbob as Apprentice
  • bagder certified mbp as Master
  • bagder certified shughes as Journeyer
  • bagder certified sussman as Journeyer
  • bagder certified mpawlo as Apprentice
  • bagder certified BrucePerens as Master
  • bagder certified rmk as Master
  • bagder certified Fefe as Journeyer
  • bagder certified gstein as Master
  • bagder certified robey as Master
  • bagder certified edd as Journeyer
  • bagder certified ask as Journeyer
  • bagder certified joe as Master
  • bagder certified alan as Master
  • bagder certified pawal as Apprentice
  • bagder certified stone as Apprentice
  • bagder certified sej as Journeyer
  • bagder certified fxn as Apprentice
  • bagder certified forrest as Apprentice
  • bagder certified wsanchez as Master
  • bagder certified zagor as Journeyer
  • bagder certified ben as Master
  • bagder certified kfogel as Master
  • bagder certified orabidoo as Master
  • bagder certified linas as Master
  • bagder certified jas as Master

Others have certified bagder as follows:

  • ib certified bagder as Master
  • chipx86 certified bagder as Master
  • rupert certified bagder as Master
  • larsu certified bagder as Master
  • mvw certified bagder as Journeyer
  • neurogato certified bagder as Journeyer
  • whytheluckystiff certified bagder as Master
  • andrei certified bagder as Journeyer
  • jbowman certified bagder as Journeyer
  • alexr certified bagder as Journeyer
  • pretzelgod certified bagder as Journeyer
  • thallgren certified bagder as Journeyer
  • execve certified bagder as Master
  • pelleb certified bagder as Master
  • GJF certified bagder as Master
  • kroah certified bagder as Master
  • jooon certified bagder as Master
  • nixnut certified bagder as Journeyer
  • jLoki certified bagder as Master
  • mpawlo certified bagder as Journeyer
  • technik certified bagder as Master
  • highgeek certified bagder as Journeyer
  • Stab certified bagder as Master
  • TheCorruptor certified bagder as Master
  • sethcohn certified bagder as Master
  • elho certified bagder as Master
  • monkeyiq certified bagder as Master
  • ebf certified bagder as Master
  • jmg certified bagder as Master
  • robey certified bagder as Master
  • edd certified bagder as Master
  • jbontje certified bagder as Journeyer
  • khazad certified bagder as Master
  • walken certified bagder as Master
  • ask certified bagder as Journeyer
  • xsa certified bagder as Master
  • shlomif certified bagder as Master
  • pawal certified bagder as Master
  • CarloK certified bagder as Master
  • stone certified bagder as Master
  • lerdsuwa certified bagder as Master
  • sej certified bagder as Master
  • nny certified bagder as Journeyer
  • ks certified bagder as Journeyer
  • fxn certified bagder as Master
  • jono certified bagder as Master
  • forrest certified bagder as Master
  • rw2 certified bagder as Master
  • wsanchez certified bagder as Master
  • dlc certified bagder as Journeyer
  • zagor certified bagder as Journeyer
  • ncm certified bagder as Master
  • redi certified bagder as Master
  • ianweller certified bagder as Master
  • bwy certified bagder as Master
  • badvogato certified bagder as Journeyer
  • mnot certified bagder as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page