Recent blog entries for mbrubeck

Let's build a browser engine! Part 3: CSS

This is the third in a series of articles on building a toy browser rendering engine. Want to build your own? Start at the beginning to learn more:

This article introduces code for reading Cascading Style Sheets (CSS). As usual, I won’t try to cover everything in the spec. Instead, I tried to implement just enough to illustrate some concepts and produce input for later stages in the rendering pipeline.

Anatomy of a Stylesheet

Here’s an example of CSS source code:

    h1, h2, h3 { margin: auto; color: #cc0000; }
div.note { margin-bottom: 20px; padding: 10px; }
#answer { display: none; }

  

Now I’ll walk through some the CSS code from my toy browser engine, robinson.

A CSS stylesheet is a series of rules. (In the example stylesheet above, each line contains one rule.)

    struct Stylesheet {
    rules: Vec<Rule>,
}

  

A rule includes one or more selectors separated by commas, followed by a list of declarations enclosed in braces.

    struct Rule {
    selectors: Vec<Selector>,
    declarations: Vec<Declaration>,
}

  

A selector can be a simple selector, or it can be a chain of selectors joined by combinators. Robinson supports only simple selectors for now.

Note: Confusingly, the newer Selectors Level 3 standard uses the same terms to mean slightly different things. In this article I’ll mostly refer to CSS2.1. Although outdated, it’s a useful starting point because it’s smaller and more self-contained than CSS3 (which is split into myriad specs that reference both each other and CSS2.1).

In robinson, a simple selector can include a tag name, an ID prefixed by '#', any number of class names prefixed by '.', or some combination of the above. If the tag name is empty or '*' then it is a “universal selector” that can match any tag.

There are many other types of selector (especially in CSS3), but this will do for now.

    enum Selector {
    Simple(SimpleSelector),
}

struct SimpleSelector {
    tag_name: Option<String>,
    id: Option<String>,
    class: Vec<String>,
}

  

A declaration is just a name/value pair, separated by a colon and ending with a semicolon. For example, "margin: auto;" is a declaration.

    struct Declaration {
    name: String,
    value: Value,
}

  

My toy engine supports only a handful of CSS’s many value types.

    enum Value {
    Keyword(String),
    Color(u8, u8, u8, u8), // RGBA
    Length(f32, Unit),
    // insert more values here
}

enum Unit { Px, /* insert more units here */ }

  

All other CSS syntax is unsupported, including at-rules, comments, and any selectors/values/units not mentioned above.

Parsing

CSS has a regular grammar, making it easier to parse correctly than its quirky cousin HTML. When a standards-compliant CSS parser encounters a parse error, it discards the unrecognized part of the stylesheet but still processes the remaining portions. This is useful because it allows stylesheets to include new syntax but still produce well-defined output in older browsers.

Robinson uses a very simplistic (and totally not standards-compliant) parser, built the same way as the HTML parser from Part 2. Rather than go through the whole thing line-by-line again, I’ll just paste in a few snippets. For example, here is the code for parsing a single selector:

        /// Parse one simple selector, e.g.: `type#id.class1.class2.class3`
    fn parse_simple_selector(&mut self) -> SimpleSelector {
        let mut result = SimpleSelector { tag_name: None, id: None, class: Vec::new() };
        while !self.eof() {
            match self.next_char() {
                '#' => {
                    self.consume_char();
                    result.id = Some(self.parse_identifier());
                }
                '.' => {
                    self.consume_char();
                    result.class.push(self.parse_identifier());
                }
                '*' => {
                    // universal selector
                    self.consume_char();
                }
                c if valid_identifier_char(c) => {
                    result.tag_name = Some(self.parse_identifier());
                }
                _ => break
            }
        }
        result
    }

  

Note the lack of error checking. Some malformed input like ### or *foo* will parse successfully and produce weird results. A real CSS parser would discard these invalid selectors.

Specificity

Specificity is one of the ways a rendering engine decides which style overrides the other in a conflict. If a stylesheet contains two rules that match an element, the rule with the matching selector of higher specificity can override values from the one with lower specificity.

The specificity of a selector is based on its components. An ID selector is more specific than a class selector, which is more specific than a tag selector. Within each of these “levels,” more selectors beats fewer.

    pub type Specificity = (uint, uint, uint);

impl Selector {
    pub fn specificity(&self) -> Specificity {
        // http://www.w3.org/TR/selectors/#specificity
        let Simple(ref simple) = *self;
        let a = simple.id.iter().len();
        let b = simple.class.len();
        let c = simple.tag_name.iter().len();
        (a, b, c)
    }
}

  

[If we supported chained selectors, we could calculate the specificity of a chain just by adding up the specificities of its parts.]

The selectors for each rule are stored in a sorted vector, most-specific first. This will be important in matching, which I’ll cover in the next article.

        /// Parse a rule set: `<selectors> { <declarations> }`.
    fn parse_rule(&mut self) -> Rule {
        Rule {
            selectors: self.parse_selectors(),
            declarations: self.parse_declarations()
        }
    }

    /// Parse a comma-separated list of selectors.
    fn parse_selectors(&mut self) -> Vec<Selector> {
        let mut selectors = Vec::new();
        loop {
            selectors.push(Simple(self.parse_simple_selector()));
            self.consume_whitespace();
            match self.next_char() {
                ',' => { self.consume_char(); }
                '{' => break, // start of declarations
                c   => fail!("Unexpected character {} in selector list", c)
            }
        }
        // Return selectors with highest specificity first, for use in matching.
        selectors.sort_by(|a,b| b.specificity().cmp(&a.specificity()));
        selectors
    }

  

The rest of the CSS parser is fairly straightforward. You can read the whole thing on GitHub. And if you didn’t already do it for Part 2, this would be a great time to try out a parser generator. My hand-rolled parser gets the job done for simple example files, but it has a lot of hacky bits and will fail badly if you violate its assumptions. Eventually I hope to replace it with a “real” parser built on something like rust-peg.

Exercises

As before, you should decide which of these exercises you want to do, and skip the rest:

  1. Implement your own simplified CSS parser and specificity calculation.

  2. Extend robinson’s CSS parser to support more values, or one or more selector combinators.

  3. Extend the CSS parser to discard any declaration that contains a parse error, and follow the error handling rules to resume parsing after the end of the declaration.

  4. Make the HTML parser pass the contents of any <style> nodes to the CSS parser, and return a Document object that includes a list of Stylesheets in addition to the DOM tree.

Shortcuts

Just like in Part 2, you can skip parsing by hard-coding CSS data structures directly into your program, or by writing them in an alternate format like JSON that you already have a parser for.

To be continued…

The next article will introduce the style module. This is where everything starts to come together, with selector matching to apply CSS styles to DOM nodes.

The pace of this series might slow down soon, since I’ll be busy later this month and I haven’t even written the code for some upcoming articles. I’ll keep them coming as fast as I can!

Syndicated 2014-08-13 19:30:00 from Matt Brubeck

Let's build a browser engine!

This is the second in a series of articles on building a toy browser rendering engine:

This article is about parsing HTML source code to produce a tree of DOM nodes. Parsing is a fascinating topic, but I don’t have the time or expertise to give it the introduction it deserves. You can get a detailed introduction to parsing from any good course or book on compilers. Or get a hands-on start by going through the documentation for a parser generator that works with your chosen programming language.

HTML has its own unique parsing algorithm. Unlike parsers for most programming languages and file formats, the HTML parsing algorithm does not reject invalid input. Instead it includes specific error-handling instructions, so web browsers can agree on how to display every web page, even ones that don’t conform to the syntax rules. Web browsers have to do this to be usable: Since non-conforming HTML has been supported since the early days of the web, it is now used in a huge portion of existing web pages.

A Simple HTML Dialect

I didn’t even try to implement the standard HTML parsing algorithm. Instead I wrote a basic parser for a tiny subset of HTML syntax. My parser can handle simple pages like this:

    <html>
    <body>
        <h1>Title</h1>
        <div id="main" class="test">
            <p>Hello <em>world</em>!</p>
        </div>
    </body>
</html>

  

The following syntax is allowed:

  • Balanced tags: <p>...</p>
  • Attributes with quoted values: id="main"
  • Text nodes: <em>world</em>

Everything else is unsupported, including:

  • Namespaces: <html:body>
  • Self-closing tags: <br/> or <br> with no closing tag
  • Character encoding detection.
  • Escaped characters (like &amp;) and CDATA blocks.
  • Comments, processing instructions, and doctype declarations.
  • Error handling (e.g. unbalanced or improperly nested tags).

At each stage of this project I’m writing more or less the minimum code needed to support the later stages. But if you want to learn more about parsing theory and tools, you can be much more ambitious in your own project!

Example Code

Next, let’s walk through my toy HTML parser, keeping in mind that this is just one way to do it (and probably not the best way). Its structure is based loosely on the tokenizer module from Servo’s cssparser library. It has no real error handling; in most cases, it just aborts when faced with unexpected syntax. The code is in Rust, but I hope it’s fairly readable to anyone who’s used similar-looking languages like Java, C++, or C#. It makes use of the DOM data structures from part 1.

The parser stores its input string and a current position within the string. The position is the index of the next character we haven’t processed yet.

    struct Parser {
    pos: uint,
    input: String,
}

  

We can use this to implement some simple methods for peeking at the next characters in the input:

    impl Parser {
    /// Read the next character without consuming it.
    fn next_char(&self) -> char {
        self.input.as_slice().char_at(self.pos)
    }

    /// Do the next characters start with the given string?
    fn starts_with(&self, s: &str) -> bool {
        self.input.as_slice().slice_from(self.pos).starts_with(s)
    }

    /// Return true if all input is consumed.
    fn eof(&self) -> bool {
        self.pos >= self.input.len()
    }

    // ...
}

  

Rust strings are stored as UTF-8 byte arrays. To go to the next character, we can’t just advance by one byte. Instead we use char_range_at which correctly handles multi-byte characters. (If our string used fixed-width characters, we could just increment pos.)

        /// Return the current character, and advance to the next character.
    fn consume_char(&mut self) -> char {
        let range = self.input.as_slice().char_range_at(self.pos);
        self.pos = range.next;
        range.ch
    }

  

Often we will want to consume a string of consecutive characters. The consume_while method consumes characters that meet a given condition, and returns them as a string:

        /// Consume characters until `test` returns false.
    fn consume_while(&mut self, test: |char| -> bool) -> String {
        let mut result = String::new();
        while !self.eof() && test(self.next_char()) {
            result.push_char(self.consume_char());
        }
        result
    }

  

We can use this to ignore a sequence of space characters, or to consume a string of alphanumeric characters:

        /// Consume and discard zero or more whitespace characters.
    fn consume_whitespace(&mut self) {
        self.consume_while(|c| c.is_whitespace());
    }

    /// Parse a tag or attribute name.
    fn parse_tag_name(&mut self) -> String {
        self.consume_while(|c| match c {
            'a'..'z' | 'A'..'Z' | '0'..'9' => true,
            _ => false
        })
    }

  

Now we’re ready to start parsing HTML. To parse a single node, we look at its first character to see if it is an element or a text node. In our simplified version of HTML, a text node can contain any character except <.

        /// Parse a single node.
    fn parse_node(&mut self) -> dom::Node {
        match self.next_char() {
            '<' => self.parse_element(),
            _   => self.parse_text()
        }
    }

    /// Parse a text node.
    fn parse_text(&mut self) -> dom::Node {
        dom::text(self.consume_while(|c| c != '<'))
    }

  

An element is more complicated. It includes opening and closing tags, and between them any number of child nodes:

        /// Parse a single element, including its open tag, contents, and closing tag.
    fn parse_element(&mut self) -> dom::Node {
        // Opening tag.
        assert!(self.consume_char() == '<');
        let tag_name = self.parse_tag_name();
        let attrs = self.parse_attributes();
        assert!(self.consume_char() == '>');

        // Contents.
        let children = self.parse_nodes();

        // Closing tag.
        assert!(self.consume_char() == '<');
        assert!(self.consume_char() == '/');
        assert!(self.parse_tag_name() == tag_name);
        assert!(self.consume_char() == '>');

        dom::elem(tag_name, attrs, children)
    }

  

Parsing attributes is pretty easy in our simplified syntax. Until we reach the end of the opening tag (>) we repeatedly look for a name followed by = and then a string enclosed in quotes.

        /// Parse a single name="value" pair.
    fn parse_attr(&mut self) -> (String, String) {
        let name = self.parse_tag_name();
        assert!(self.consume_char() == '=');
        let value = self.parse_attr_value();
        (name, value)
    }

    /// Parse a quoted value.
    fn parse_attr_value(&mut self) -> String {
        let open_quote = self.consume_char();
        assert!(open_quote == '"' || open_quote == '\'');
        let value = self.consume_while(|c| c != open_quote);
        assert!(self.consume_char() == open_quote);
        value
    }

    /// Parse a list of name="value" pairs, separated by whitespace.
    fn parse_attributes(&mut self) -> dom::AttrMap {
        let mut attributes = HashMap::new();
        loop {
            self.consume_whitespace();
            if self.next_char() == '>' {
                break;
            }
            let (name, value) = self.parse_attr();
            attributes.insert(name, value);
        }
        attributes
    }

  

To parse the child nodes, we recursively call parse_node in a loop until we reach the closing tag:

        /// Parse a sequence of sibling nodes.
    fn parse_nodes(&mut self) -> Vec<dom::Node> {
        let mut nodes = vec!();
        loop {
            self.consume_whitespace();
            if self.eof() || self.starts_with("</") {
                break;
            }
            nodes.push(self.parse_node());
        }
        nodes
    }

  

Finally, we can put this all together to parse an entire HTML document into a DOM tree. This function will create a root node for the document if it doesn’t include one explicitly; this is similar to what a real HTML parser does.

    /// Parse an HTML document and return the root element.
pub fn parse(source: String) -> dom::Node {
    let mut nodes = Parser { pos: 0u, input: source }.parse_nodes();

    // If the document contains a root element, just return it. Otherwise, create one.
    if nodes.len() == 1 {
        nodes.swap_remove(0).unwrap()
    } else {
        dom::elem("html".to_string(), HashMap::new(), nodes)
    }
}

  

That’s it! The entire code for the robinson HTML parser. The whole thing weighs in at just over 100 lines of code (not counting blank lines and comments). If you use a good library or parser generator, you can probably build a similar toy parser in even less space.

Exercises

Here are a few alternate ways to try this out yourself. As before, you can choose one or more of them and ignore the others.

  1. Build a parser (either “by hand” or with a library or parser generator) that takes a subset of HTML as input and produces a tree of DOM nodes.

  2. Modify robinson’s HTML parser to add some missing features, like comments. Or replace it with a better parser, perhaps built with a library or generator.

  3. Create an invalid HTML file that causes your parser (or mine) to fail. Modify the parser to recover from the error and produce a DOM tree for your test file.

Shortcuts

If you want to skip parsing completely, you can build a DOM tree programmatically instead, by adding some code like this to your program (in pseudo-code; adjust it to match the DOM code you wrote in Part 1):

    // <html><body>Hello, world!</body></html>
let root = element("html");
let body = element("body");
root.children.push(body);
body.children.push(text("Hello, world!"));

  

Or you can find an existing HTML parser and incorporate it into your program.

The next article in this series will cover CSS data structures and parsing.

Syndicated 2014-08-11 15:00:00 from Matt Brubeck

Let's build a browser engine!

I’m building a toy HTML rendering engine, and I think you should too. This is the first in a series of articles describing my project and how you can make your own. But first, let me explain why.

You’re building a what?

Let’s talk terminology. A browser engine is the portion of a web browser that works “under the hood” to fetch a web page from the internet, and translate its contents into forms you can read, watch, hear, etc. Blink, Gecko, WebKit, and Trident are browser engines. In contrast, the the browser’s own UI—tabs, toolbar, menu and such—is called the chrome. Firefox and SeaMonkey are two browsers with different chrome but the same Gecko engine.

A browser engine includes many sub-components: an HTTP client, an HTML parser, a CSS parser, a JavaScript engine (itself composed of parsers, interpreters, and compilers), and much more. The many components involved in parsing web formats like HTML and CSS and translating them into what you see on-screen are sometimes called the layout engine or rendering engine.

Why a “toy” rendering engine?

A full-featured browser engine is hugely complex. Blink, Gecko, WebKit—these are millions of lines of code each. Even younger, simpler rendering engines like Servo and WeasyPrint are each tens of thousands of lines. Not the easiest thing for a newcomer to comprehend!

Speaking of hugely complex software: If you take a class on compilers or operating systems, at some point you will probably create or modify a “toy” compiler or kernel. This is a simple model designed for learning; it may never be run by anyone besides the person who wrote it. But making a toy system is a useful tool for learning how the real thing works. Even if you never build a real-world compiler or kernel, understanding how they work can help you make better use of them when writing your own programs.

So, if you want to become a browser developer, or just to understand what happens inside a browser engine, why not build a toy one? Like a toy compiler that implements a subset of a “real” programming language, a toy rendering engine could implement a small subset of HTML and CSS. It won’t replace the engine in your everyday browser, but should nonetheless illustrate the basic steps needed for rendering a simple HTML document.

Try this at home.

I hope I’ve convinced you to give it a try. This series will be easiest to follow if you already have some solid programming experience and know some high-level HTML and CSS concepts. However, if you’re just getting started with this stuff, or run into things you don’t understand, feel free to ask questions and I’ll try to make it clearer.

Before you start, a few remarks on some choices you can make:

On Programming Languages

You can build a toy layout engine in any programming language. Really! Go ahead and use a language you know and love. Or use this as an excuse to learn a new language if that sounds like fun.

If you want to start contributing to major browser engines like Gecko or WebKit, you might want to work in C++ because it’s the main language used in those engines, and using it will make it easier to compare your code to theirs. My own toy project, robinson, is written in Rust. I’m part of the Servo team at Mozilla, so I’ve become very fond of Rust programming. Plus, one of my goals with this project is to understand more of Servo’s implementation. (I’ve written a lot of browser chrome code, and a few small patches for Gecko, but before joining the Servo project I knew nothing about many areas of the browser engine.) Robinson sometimes uses simplified versions of Servo’s data structures and code. If you too want to start contributing to Servo, try some of the exercises in Rust!

On Libraries and Shortcuts

In a learning exercise like this, you have to decide whether it’s “cheating” to use someone else’s code instead of writing your own from scratch. My advice is to write your own code for the parts that you really want to understand, but don’t be shy about using libraries for everything else. Learning how to use a particular library can be a worthwhile exercise in itself.

I’m writing robinson not just for myself, but also to serve as example code for these articles and exercises. For this and other reasons, I want it to be as tiny and self-contained as possible. So far I’ve used no external code except for the Rust standard library. (This also side-steps the minor hassle of getting multiple dependencies to build with the same version of Rust while the language is still in development.) This rule isn’t set in stone, though. For example, I may decide later to use a graphics library rather than write my own low-level drawing code.

Another way to avoid writing code is to just leave things out. For example, robinson has no networking code yet; it can only read local files. In a toy program, it’s fine to just skip things if you feel like it. I’ll point out potential shortcuts like this as I go along, so you can bypass steps that don’t interest you and jump straight to the good stuff. You can always fill in the gaps later if you change your mind.

First Step: The DOM

Are you ready to write some code? We’ll start with something small: data structures for the DOM. Let’s look at robinson’s dom module.

The DOM is a tree of nodes. A node has zero or more children. (It also has various other attributes and methods, but we can ignore most of those for now.)

    struct Node {
    // data common to all nodes:
    children: Vec<Node>,

    // data specific to each node type:
    node_type: NodeType,
}

  

There are several node types, but for now we will ignore most of them and say that a node is either an Element or a Text node. In a language with inheritance these would be subtypes of Node. In Rust they can be an enum (Rust’s keyword for a “tagged union” or “sum type”):

    enum NodeType {
    Text(String),
    Element(ElementData),
}

  

An element includes a tag name and any number of attributes, which can be stored as a map from names to values. Robinson doesn’t support namespaces, so it just stores tag and attribute names as simple strings.

    struct ElementData {
    tag_name: String,
    attributes: AttrMap,
}

type AttrMap = HashMap<String, String>;

  

Finally, some constructor functions to make it easy to create new nodes:

    impl Node {
    fn new(children: Vec<Node>, node_type: NodeType) -> Node {
        Node { children: children, node_type: node_type }
    }
}

fn text(data: String) -> Node {
    Node::new(vec!(), Text(data))
}

fn elem(name: String, attrs: AttrMap, children: Vec<Node>) -> Node {
    Node::new(children, Element(ElementData {
        tag_name: name,
        attributes: attrs,
    }))
}

  

And that’s it! A full-blown DOM implementation would include a lot more data and dozens of methods, but this is all we need to get started. In the next article, we’ll add a parser that turns HTML source code into a tree of these DOM nodes.

Exercises

These are just a few suggested ways to follow along at home. Do the exercises that interest you and skip any that don’t.

  1. Start a new program in the language of your choice, and write code to represent a tree of DOM text nodes and elements.

  2. Install the latest version of Rust, then download and build robinson. Open up dom.rs and extend NodeType to include additional types like comment nodes.

  3. Write code to pretty-print a tree of DOM nodes.

References

Here’s a short list of “small” open source web rendering engines. Most of them are many times bigger than robinson, but still way smaller than Gecko or WebKit. WebWhirr, at 2000 lines of code, is the only other one I would call a “toy” engine.

You may find these useful for inspiration or reference. If you know of any other similar projects—or if you start your own—please let me know!

Syndicated 2014-08-08 16:40:00 from Matt Brubeck

Better automated detection of Firefox performance regressions

Last spring I spent some of my spare time improving the automated script that detects regressions in Talos and other Firefox performance data. I’m finally writing up some of that work in case it’s useful or interesting to anyone else.

Talos is a system for running performance benchmarks; we use it to run a suite of benchmarks every time a change is pushed to the Firefox source repository. The Talos test harness reports these results to the graph server which stores them and can plot the recorded data to show how it changes over time.

Like most performance measurements, Talos benchmarks can be noisy. We need to use statistics to separate signal from noise. To determine whether a change to the source code caused a change in the benchmark results, an automated script takes multiple datapoints from before and after each push. It computes the average and standard deviation of the “before” datapoints and the “after” datapoints, and uses a Student’s t-test to estimate the likelihood that the datasets are significantly different. If the t-test exceeds a certain threshold, the script sends email to the author(s) of the responsible patches and to the dev-tree-management mailing list.

By nature, these statistical estimates can never be 100% certain. However, we need to make sure that they are correct as often as possible. False negatives mean that real regressions go undetected. But false positives will generate extra work, and may cause developers to ignore future regression alerts. I started inspecting graph server data and regression alerts by hand, recording and investigating any false negatives or false positives I found, and filed bugs to fix the causes of those errors.

Some of these were straightforward implementation bugs, like one where an infinite t-test score (near certain likelihood of regression) was treated as a zero score (no regression at all). Others involved tuning the number of datapoints and the threshold for sending alerts.

Some fixes required more involved changes to the analysis. For example, if one code change actually caused a regression, the pushes right before or after that change will also appear somewhat likely to be responsible for the regression (because they will also have large differences in average score between their “before” and “after” windows). If multiple pushes in a row had t-test scores over the threshold, the script used to send an alert for the first of those pushes, even if it was not the most likely culprit. Now the script blames the push with the highest t-test score, which is almost always the guilty party. This change had the biggest impact in reducing incorrect regression alerts.

After those changes, there was still one common cause of false alarms that I found. The regression analyzer compares the 12 datapoints before each push to the 12 datapoints after it. But these 12-point moving averages could change suddenly not just at the point where a regression happened, but also at an unrelated point that happens to be 12 pushes earlier or later. This caused spooky action at a distance where a regression in one push would cause a false alarm in a completely different push. To fix this, we now compute weighted averages with “triangular” weighting functions that give more weight to the point being analyzed, and fall off gradually with increasing distance from that point. This smooths out changes at the opposite ends of the moving windows.

There are still occasional errors in regression detection, but as far as I can tell most of them are caused by genuinely misleading random noise or bimodal data. If you see any problems with regression emails, please file a bug (and CC :mbrubeck) and we’ll take a look at it.

Syndicated 2013-11-10 19:53:00 from Matt Brubeck

A good time to try Firefox for Metro

“Firefox for Metro” is our project to build a new Firefox user interface designed for touch-screen devices running Windows 8. (“Metro” was Microsoft’s code name for the new, touch-friendly user interface mode in Windows 8.) I’m part of the small team working on this project.

For the past year we’ve been fairly quiet, partly because the browser has been under heavy construction and not really suitable for regular use. It started as a fork of the old Fennec (mobile Firefox) UI, plus a new port of Gecko’s widget layer to Microsoft’s WinRT API. We spent part of that time ripping out and rebuilding old Fennec features to make them work on Windows 8, and finding and fixing bugs in the new widget code. More recently we’ve been focused on reworking the touch input layer. With a ton of help from the graphics team, we replaced Fennec’s old multi-process JavaScript touch support with a new off-main-thread compositing backend for the Windows Direct3D API, and added WinRT support to the async pan/zoom module that implements touch scrolling and zooming on Firefox OS.

All this work is still underway, but in the past week we finally reached a tipping point where I’m able to use Firefox for Metro for most of my everyday browsing. There are still bugs, and we are still actively working on performance and completing the UI work, but I’m now finding very few cases where I need to switch to another browser because of a problem with Firefox for Metro. If you are using Window 8 (especially on a touch-screen PC) and are the type of brave person who uses Firefox nightly builds, this would be a great time to try Metro-style Firefox and let us know what you think!

Looking to the future, here are some of our remaining development priorities for the first release of Firefox for Metro:

  • Improve the installation and first-run experience, to help users figure out how to use the new UI and switch between “Metro” and desktop modes. (Our UX designer has user testing planned to help identify issues here and throughout the product.)

  • Fix any performance and rendering issues with scrolling and zooming, and add support for double-tap to zoom in on a specific page element.

  • Make the Metro and desktop interfaces share a profile, so they can seamlessly use the same bookmarks and other data without connecting to a Firefox Sync account.

And here are some things that I hope we can spend more time on once that work has shipped:

  • Improve the experience on pages with plugins, which currently require the user to switch to the desktop Firefox interface (bug 936907).

  • Implement a “Reader Mode,” like Firefox for Android. (A pair of students have started working on this project, and their work should also be useful for adding Reader Mode to Firefox for desktop.)

  • Add more features, and more ways to customize and tweak the Metro UI.

If you want to contribute to any of this work, please check out our developer documentation and come chat with us in #windev on irc.mozilla.org or on our project mailing list!

Syndicated 2013-11-10 17:20:00 from Matt Brubeck

Congratulating the IE10 team

Back when Firefox 2 was released (six years ago this week!), the Internet Explorer team started a friendly tradition of sending Mozilla a cake as congratulations. This continued for Firefox 3 and Firefox 4. After Firefox switched from major releases once or twice a year to incremental updates every six weeks, they sent us cupcakes for the next few updates instead. :)

Since IE10 for Windows 8 is shipping today, I thought it would be fun to revive the tradition by delivering a cake to congratulate the IE team. Here’s the cake right after I picked it up from Baked Custom Cakes, with the Firefox logo in painted fondant:

Fellow Mozilla developer Eitan Isaacson drove with my wife Sarah and me to Microsoft Building 50 in Redmond, where program manager Jacob Rossi helped us deliver the cake to a group of IE team members:

The IE team posted their thanks through their official Twitter account. (As you can see from their picture, the bottom border of the cake got sligthly restyled in transit, but it still looks quite edible.) Less than 30 minutes later, Microsoftie Michael Bolan tweeted that cake was already gone. (I hear that the sugary Firefox logo was also eaten shortly afterward.)

So congratulations to the Internet Explorer team on your latest release, and we hope you enjoyed the cake!

Syndicated 2012-10-26 21:38:00 from Matt Brubeck

Metro Firefox without Windows 8

A few weeks ago I started working on the Firefox “Metro UI” project, for Windows 8’s Metro (or Modern) touch-screen environment. While we’re still working on getting our first preview builds ready for Windows 8 users to try out, you can already check out the current source code from the elm branch and build it yourself if you want to get involved and help us fix some bugs.

What you might not know is that you can run “Metro” Firefox even if you don’t have Windows 8. It’s been possible for a while to build and run on older versions of Windows using the -metrodesktop flag. Today I landed a patch to make this work on other platforms too. To build the latest elm source code on Linux or Mac OS X, follow these instructions:

  1. Clone the elm repo: hg clone http://hg.mozilla.org/projects/elm/
    (If you have already cloned mozilla-central or some other repo that shares with it, there’s a faster way to do this.)

  2. Create a .mozconfig file with ac_add_options --enable-metro

  3. Build Firefox as you normally would.

  4. From your objdir, run dist/bin/firefox -metrodesktop (Linux)
    or dist/Nightly.app/Contents/MacOS/firefox -metrodesktop (Mac)

  5. You can visit about:config and enable metro.debug.treatmouseastouch (then restart the browser) to simulate touch interaction with the mouse. Right-click to simulate the Windows 8 edge-swipe gesture, which displays the toolbars.

This is still experimental and mostly untested. Elm might accidentally break on non-Windows platforms from time to time (because of course we are doing all our main development and testing on Windows). While it’s not a perfect replacement for running in the real Windows 8 environment, I hope this is a useful option for adventurous Firefox contributors who want to experiment with the Metro code but don’t have convenient access to Windows 8.

Syndicated 2012-09-20 00:42:00 from Matt Brubeck

Mobile web developers: Your users hate it when you do this

Mobile Firefox beta releases include a “Feedback” add-on (like the one in Firefox 4 beta for desktop), which lets users tell us what they think about the new browser. Based on a sample of feedback from mobile beta testers, the most common complaints are about:

  1. Speed
  2. Fitting text to the screen when zoomed in
  3. Mobile vs. desktop versions of web sites

The first two are straightforward, though not necessarily easy. We’re always working on performance, and we have experimental text reflow code (currently available in the Easy Reading add-on). But the last item is more complicated…

Browser detection pitfalls

Web sites can read the User-Agent header sent by your browser to see what browser and OS you are using. Some sites use that information to decide whether to send a “full” version of a web page, or a version formatted for mobile devices.

This can go wrong in several ways. If your browser or device is new, or wasn’t tested when a site was developed, that site has no way of knowing whether it is “mobile.” Users may also change their User-Agent to work around content restrictions or access different media formats. And some sites make incorrect assumptions, like that all browsers with “Android” in their User-Agent string are based on WebKit.

Even when the browser is known, readers and publishers might not agree about whether the mobile or desktop version is better. Based on our feedback, some users want to switch from full sites to mobile sites while others want just the opposite. And some devices, like large touch-screen tablets, combine aspects of handheld and desktop computers.

Solutions

Looking through these complaints, many people are under the mistaken impression that the browser, rather than the web site, decides whether to display mobile-formatted pages. Even the New York Times' David Pogue gets this wrong in his Galaxy Tab review:

When you visit sites like nytimes.com, CNBC.com and Amazon.com, the Galaxy’s browser shows the stripped-down, mobile versions of those sites. According to Samsung, there’s no way to turn that feature off and no way to visit the full-size sites. You can delete the little “m.” in the Web address until you’re blue in the browser, but the Galaxy always puts it right back.

Web developers: your readers are begging us to display your content in their preferred format. We want to help them, but we can’t do it alone.

(I wrote an add-on called Phony that lets mobile Firefox impersonate the User-Agent strings of other browsers. While this improves the experience on some sites, it breaks it on others. Masquerading as another browser can lead sites to serve non-standard markup that do not work in Firefox.)

Because browser detection is never perfect, web sites should let readers choose between mobile and full content. They can try to guess the right version by default, but please let users opt in or out.

Best practices for web developers

Here are some first steps typical mobile web sites can take to make their readers happier:

  • When possible, serve the same content to all browsers. You can use stylesheets and scripts to customize your layout for different display sizes, as in this beautiful site by Jon Hicks.

  • There are valid reasons to use User-Agent sniffing. But if you must use it, test in as many browsers and devices as possible and learn the correct way to detect various browsers. For example, you can detect Gecko-based browsers by looking for Gecko and rv:, and you can detect mobile Firefox by looking for Fennec/.

  • If a “mobile” user requests a page that isn’t available on your mobile site, serve the full version to them anyway, rather than redirecting them to an unrelated mobile landing page.

  • Let users switch from your mobile site to your full site and back. You may remember users' previous choices for convenience, but let them change their minds again.

Further reading

For much more comprehensive development advice, see Yiibu’s thoughtful and practical approach to building sites that work across many different browsers and mobile devices.

Coming from a different perspective, Andrea Trasatti (developer of the device-detection library WURFL) talks about problems in mobile User-Agent strings and how they could be more useful for device detection.

Syndicated 2010-11-19 15:15:00 from Matt Brubeck

What's different about Firefox for Android

I've been working for the last six months on Firefox for Android (also known as "Fennec"). Here are some thoughts about the challenges in building a mobile browser, and the particular choices we've made.

Along the way, I'll try to answer some frequently-asked questions, like "Why is Firefox so huge on Android?" and "Why should I care?"

Why

People often ask us why Android needs another web browser. These are a few things Firefox does that other Android browsers don't:

  • Syncs bookmarks, tabs, history, passwords, and form data to and from your phone. Firefox Sync and the Firefox Awesomebar help you enter URLs and passwords with less typing, and move seamlessly between your desktop and your mobile phone.

  • Allows extensions to customize every part of the user interface. Adblock Plus and NoScript are two mobile Firefox add-ons that take advantage of this deep extensibility. (Note: both are compatible with the last stable release of Firefox for Nokia Maemo; they'll need to be updated to support the pre-release Android versions.)

  • Uses the Jaegermonkey JIT, which is getting faster all the time. It runs JavaScript much faster than the Android 2.1 browser, and is starting to overtake the Android 2.2 browser on the benchmarks in SunSpider and similar suites.

  • Supports web technologies like SVG, ECMAScript 5, WebM, and HTTP Strict Transport Security. Firefox for Android currently scores 217 points plus 9 bonus points on html5test.com. (Warning: Those tests can be deceptive; use them as a starting point for comparison only.)

Another difference is that Firefox is built by Mozilla, a non-profit organization with a mission to promote openness, innovation, and opportunity on the web. We want our work on the mobile web to benefit everyone, not just Firefox users - just as Firefox on the desktop helped create a new era of innovation and standards for users of all web browsers.

Competition and choice

There are many other browsers for Android, but all of them use the built-in WebKit rendering engine (except Opera Mini, which uses a proxy server for rendering). The same is true for Apple iOS, which is also based on WebKit – as are the latest versions of BlackBerry, Symbian, and Palm webOS.

There's nothing wrong with WebKit. It's a great project. But a growing number of mobile sites work only on WebKit (or even just on iOS or Android). This is dangerously similar to the web ten years ago, when Internet Explorer had an overwhelming market share and many sites used IE-specific markup. This made it very hard for other browsers to compete, which killed the incentive for the dominant browser to keep improving.

Upcoming platforms like MeeGo and Windows Phone may give WebKit some real mobile competition - but many users still won't be able to choose new browser technology without buying new hardware (and often new service contracts). We think people should have a meaningful choice of browsers on their existing phones, just like they do on their computers.

Reusing vs. extending

Part of the point of Firefox is to provide an alternative to the built-in browser engine. Firefox for Android is built on the same Gecko engine as Firefox 4 for desktop. That's how it can add new capabilities like Sync, SVG, and ES5.

Many mobile platforms do not allow browsers to include low-level components like JIT compilers. On platforms like BlackBerry that support only "managed" languages like Java, this is true for technical reasons. On others like iOS, it is forbidden purely as a policy decision. Fortunately there are still platforms like Android, webOS, and Maemo that let apps bundle any libraries they want.

Although Android allows us to distribute our own rendering engine and JavaScript compiler, it really is not built with applications like Firefox in mind. Many Android phones were built with around 64 MB to 512 MB for apps. Users who think nothing of a 12 MB download to install Firefox or Chrome on a laptop will certainly think twice before installing it on one of these phones! Fortunately storage space is much larger on most new phones, but this is still an issue on existing hardware.

Android NDK packaging: Problems and solutions

Android's WebKit libraries are installed on the system partition, and are not part of any app. Firefox doesn't have that luxury; its must include the Gecko libraries in its APK file.

Due to a quirk of the Android NDK, apps' native libraries are saved in two places - compressed inside the APK, and extracted to a folder for loading. For apps like Firefox that are mostly native code, this more than doubles the installation size. Current pre-release versions of Firefox use 30 to 40 MB of storage. Other NDK apps like Google Earth pay the same double storage penalty.

To solve this problem, Mozilla's Michael Wu is writing a custom dynamic linker, so Firefox can load libraries directly from the APK without installing them to a folder. This cuts the installed size in half, but increases the startup time slightly. For newer phones with 1 GB or more of internal storage, we might choose to let Firefox take more space but start faster. On phones with less storage, we can use the custom linker to save space. We're also working on other ways to make startup faster.

System components have another advantage: They can be optimized for specific hardware. In contrast, apps usually come in a single flavor for all devices. Firefox for Android can use ARMv7 features like Thumb-2 and NEON to run as fast as possible on high-end Android phones - but when it's built with these optimizations it can't run at all on low-end hardware. To run optimally on all current hardware, we'd need different builds for different devices. For now, we are focusing on the current high-end phones, which will likely be next year's mainstream hardware.

Try it out

To check if your phone is compatible and download a test build, see the Firefox for Android web page. Our pre-beta nightly builds are already much faster than the alpha release from a few weeks ago. This is still pre-release software, and we aren't done stabilizing and optimizing it - but we are working hard. Let us know what you think!

If you don't want to mess with nightly builds, look for our first beta release very soon now. Beta 1 will include our first batch of speed and stability improvements. And beta 2 will include even more exciting changes like the new Android skin, reduced installation size, and OpenGL-accelerated compositing.

If Fennec doesn't work on your phone, you can also test it on other platforms. And we hope increased choice will encourage all browsers to innovate and learn from each other, so your mobile experience will improve no matter which browser you use.

Syndicated 2010-10-04 23:00:00 from Matt Brubeck

124 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!