rodrigo is currently certified at Master level.

Name: Rodrigo Moya
Member since: 2000-05-16 09:22:05
Last Login: 2008-06-26 12:06:00

FOAF RDF Share This

Homepage: http://rodrigo.gnome-db.org/

Projects

Recent blog entries by rodrigo

Syndication: RSS 2.0

Xamarin

After some well deserved relaxing time, I am glad to announce that, since last Tuesday, I am a happy member of the Xamarin crew, where I’ll be working on the awesome cross-platform development tools the company makes.

This is indeed a change from what I have been doing in the last years, as I will be focusing on platforms other than Linux, as has been the case for the last 15 years, but to be honest, and after thinking a lot about it during my “well deserved relaxing time”, this is something that I really needed to boost my career. Not working on other platforms, but moving out of my comfort zone and find something new and exciting to work on. And Xamarin really seems to be the perfect place for this: the company is full of ex-Ximianites (Ximian is, till now, the best company I have ever worked for), lead by great people and focused on making great development tools/APIs, which is something that I have always liked but haven’t had the chance to work on as much as I would have wanted.

Of course, this doesn’t mean I will stop caring/using/developing GNOME. On the contrary, I think, and based on what I have been working on during my vacation, this will help me in making a better contribution to the project, as a lot of things I have been/will be learning, I think, are great for making GNOME better. But will talk about all that in separate blog posts, as things progress.

Many thanks to my girlfriend Yolanda and my friend Ismael Olea, who were the ones to pinpoint what I needed, which was to look for some exciting new stuff, rather than only wanting to stay on my comfort zone, and to Miguel, for the great coding challenge he got me to do, which made me learn exciting stuff and convince me finally that Xamarin was the best place for me.

Syndicated 2014-01-10 15:07:58 from Rodrigo Moya

Farewell Collabora

Today has been my last day at Collabora, where I’ve spent the last 2 years having a great time and working on some very cool and some not that cool projects, but always with great people as teammates. On the cool projects front, AF_BUS wins. Even though it didn’t gain upstream acceptance, at least it opened the race to bring D-Bus into the kernel, which I think is a great thing (based on real numbers :-D ).

A big hug to all collaborans I’ve met in these 2 years!

Not sure yet what’s coming up next, but first thing I will do, for the first time in aeons, is to get a few weeks of well deserved relax, work on some personal stuff I haven’t had much time to work on lately, and have fun!

Syndicated 2013-11-08 16:04:03 from Rodrigo Moya

desktop-webapp-browser-extension

A few months ago I started work on a Google Chrome/ium extension for integrating Chrome apps into GNOME Shell. The idea is that, whenever you install a Chrome app, a .desktop file is created in ~/.local/share/applications, for them to show up as normal applications.

Also, for desktop shortcuts for “normal” web pages, since Chrome/ium uses the favicon from the pages, the icons look really ugly on GNOME Shell’s overview. So, this extension also tries to retrieve a higher resolution icon and uses it if found, and if not, retrieves a snapshot from the page and uses that as the icon, making it look much nicer in the overview.

So, nothing really magic, but discussing it with some team mates, I thought it could be helpful for other people, so hence this public announcement :-)

The code can be found here.

Next step, when I have time and find out how, is to submit this to the Google Chrome store, but for now, you can just build it and install the .crx file into your Chrome/ium.

Syndicated 2013-06-13 10:42:12 from Rodrigo Moya

Netlink-based D-Bus

As stated in my last blog post, we have been looking at different options for optimizing D-Bus. After some internal discussion and reviewing of the feedback we got, we think the best solution is to get the best ideas from AF_DBUS, but without having to create a new socket family, which wasn’t very well welcomed by the Linux kernel developers. This brought us to having to choose a transport that allowed us to do that, so we decided on Netlink (an IPC mechanism for kernel to user-space communications).

Below is a detailed description of the architecture we are planning.

Netlink sockets
The Netlink protocol is a family of socket based IPC mechanism that can be used to communicate between the kernel and user-space processes and between user-space processes themselves. It was created as a replacement for ioctl and to receive messages sent by the kernel. It is a datagram-oriented service with both SOCK_RAW and SOCK_DGRAM valid socket types. It is based on the Berkeley sockets API and uses the AF_NETLINK address family. Netlink supports different Netlink families such as NETLINK_ROUTE, NETLINK_FIREWALL and NETLINK_SELINUX, each of which is used to communicate with a specific kernel service.

Since Netlink is used as an IPC mechanism for processes (and the kernel) on the same machine, its address only has a port number that identifies each peer (nl_pid). Since Netlink supports both unicast and multicast communication, a message to a group (nl_groups) can also be sent but only process with uid 0 are allowed to send multicast messages from user-space. A Netlink address is represented using the sockaddr_nl data structure:

struct sockaddr_nl {
        __kernel_sa_family_t    nl_family;      /* AF_NETLINK   */
        unsigned short  nl_pad;         /* zero         */
        __u32           nl_pid;         /* port ID      */
        __u32           nl_groups;      /* multicast groups mask */
};

A Netlink message header consists of the fields:

struct nlmsghdr {
        __u32           nlmsg_len;      /* Length of message including header */
        __u16           nlmsg_type;     /* Message content */
        __u16           nlmsg_flags;    /* Additional flags */
        __u32           nlmsg_seq;      /* Sequence number */
        __u32           nlmsg_pid;      /* Sending process port ID */
};

The Netlink protocol is explained in detail here.

Generic Netlink subsystem
Every Netlink family is identified by an integer number that allows using different Netlink services. Currently there are 21 assigned Netlink families from a maximum of 32. To avoid a shortage of Netlink families the Generic Netlink subsystem was created.

The Generic Netlink subsystem can multiplex different communication channels on a single Netlink family NETLINK_GENERIC. Generic Netlink subsystem is not only a simplified Netlink usage but also the communication channels can be registered at run-time without modifying core kernel code or headers.

The Generic Netlink subsystem is implemented as a service bus inside the kernel and users communicate with each other over it. The users can reside both in user-space or inside the kernel. The bus supports a number of Generic Netlink communications channels that are dynamically allocated by a Generic Netlink controller. This controller is a kernel Generic Netlink user itself, that listens on a special pre-allocated Generic Netlink channel “nlctrl” (GENL_ID_CTRL) where users send requests to create, remove and learn about available channels.

Communication channels are uniquely identified by a channel number that is allocated by the Generic Netlink controller. Users that want to provide services over Generic Netlink bus have to communicate with the Generic Netlink controller and ask it to create a new communication channel. Also, users that want to access those services have to query the Generic Netlink controller to know if these services are available and which channel number are currently using.

Every channel is identified by a Generic Netlink family and defines a set of commands that users can trigger. Each command is associated with a function handler that gets executed when a user sends a message specifying this command.

A Generic Netlink message header consists of the fields:

struct genlmsghdr {
        __u8    cmd;
        __u8    version;
        __u16   reserved;
};

Generic Netlink uses the standard Netlink system as a transport so its message format is defined as follows

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                Netlink message header (nlmsghdr)              |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |           Generic Netlink message header (genlmsghdr)         |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |             Optional user specific message header             |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |           Optional Generic Netlink message payload            |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The family (communication channel) used is specified using the Netlink message header (nlmsghdr) type field (nlmsg_type).

Each Generic Netlink family can use a family specific header to be used by the service provided in that channel.

D-bus as a Generic Netlink service
D-Bus can be implemented as a Generic Netlink service by creating a new Generic Netlink family (communication channel) “dbus”. Applications will use this communication channel to send and receive D-Bus messages.

In this scenario, most of the work that is currently done by the dbus-daemon will take place in the D-Bus Netlink service, such as adding applications to the bus when they gain ownership of a name (NameAcquired signal), route the messages to the destination based on the application’s unique name and maintaining match rules (AddMatch method).

The D-bus daemon will only be a special user of the Generic Netlink D-bus service, although it will still have some responsibilities such as authentication and, of course, implementing org.freedesktop.DBus service.

The other D-Bus users (apart from dbus-daemon itself), will just work as they do now, using the D-Bus wire protocol on top of the Netlink transport, although they will have to do some extra step, as explained below.

Genetlink D-bus will provide to applications the following services:
Mechanism to create and delete D-Bus buses: Since we need to separate the traffic for the different buses (system, user, etc) in the kernel module, we need a way for dbus-daemon instances to register buses in the kernel module. To do so, we can specify D-Bus family commands DBUS_CMD_NEWBUS and DBUS_CMD_DELBUS. The process who creates the bus will be the D-bus daemon implementing that bus and all the messages that have org.freedesktop.DBus as destination will be routed to it.

Besides the commands, we have to define a way to specify the name and type of the bus to be added. We can either store that information in a user defined header or define Generic Netlink family attributes to pass that information to the D-bus Generic Netlink service. In any case, the dbus-daemon will be the responsible for choosing a unique name (in the form netlink:name=unique_name), so that the kernel doesn’t have to read any configuration at all, and just has to associate the unique addresses with each bus.

Another option would be to map a D-Bus bus to a multicast group and use the Generic Netlink controller CTRL_CMD_NEWMCAST_GRP and CTRL_CMD_DELMCAST_GRP commands. But we need more fine-grained control over the routing of the messages. We can’t just use genlmsg_multicast() and send the message to every application in the bus. A signal message sent to a bus is not received by all the applications since AddMatch rules can prevent some applications to receive the message. So, we have to maintain our own multicast group based on match rules.

Connect and disconnect from buses: To allow applications to connect to a bus we can define another set of D-bus family commands DBUS_CMD_CONN_BUS and DBUS_CMD_DISC_BUS. When an application wants to connect to a bus, first the bus type is checked, if the type is a
session bus, then only processes that are executed with the same uid as the one for the D-bus daemon are allowed. This restriction is not true for system bus, which allows connection from processes running as any user. Connection requests are routed to the D-bus daemon who does the authentication.

As the create/delete group case, we need to specify to which bus we are trying to connect. We can also store that information on the user defined header or define a set of Generic Netlink family attributes.

Transport to send and receive messages: Receiving messages is straightforward. You only have to create a socket with:

int sd = create_nl_socket(NETLINK_GENERIC, 0);

and use standard BSD socket API such as recv().

To send messages to the bus we have to define a Generic Netlink D-bus family command DBUS_SEND_MSG and fill the Netlink header, Generic Netlink header and if applicable a D-bus service specific header:

struct sockaddr_nl nladdr;
struct {
	struct nlmsghdr n;
	struct genlmsghdr g;
	char buf[256];
} req;
char *dbus_message;

memset(&nladdr, 0, sizeof(nladdr));
nladdr.nl_family = AF_NETLINK;

req.n.nlmsg_len = NLMSG_LENGTH(GENL_HDRLEN);
req.n.nlmsg_type = dbus_family_id;
req.n.nlmsg_flags = NLM_F_REQUEST;
req.g.cmd = DBUS_SEND_MSG;

na = (struct nlattr *) GENLMSG_DATA(&req);
na->nla_type = DBUS_ATTR_PAYLOAD;
na->nla_len = NLA_HDRLEN + strlen(dbus_message);
memcpy(NLA_DATA(na), &dbus_message, strlen(dbus_message));
req.n.nlmsg_len += NLMSG_ALIGN(na->nla_len);

ret = sendto(sd, (char *)&req, req.n.nlmsg_len, 0, (struct sockaddr *) &nladdr,
	     sizeof(nladdr));

All this is even easier when using libnl, a library that simplifies a lot the use of Netlink in user-space applications. This library is used in other system services, like NetworkManager, so adding a dependency on it to D-Bus shouldn’t be a problem.

The Genetlink D-bus service will parse the D-bus message, add the sender field and route to the correct destination in the case of a unicast message. If the message is a signal, the service will get the recipients list according to the match rules.

Also, it will process the NameAcquired and NameLost signals as well as the AddMatch method calls, so that it can keep track of where the messages need to go to.

Security framework: In the previous sections, authentication was mentioned as one of the responsibilities of the dbus-daemon itself. This is indeed what it does right now, but with the kernel Netlink module doing the routing based on user id, as explained above, maybe no authentication is needed on the dbus-daemon side. The question is whether the dbus-daemon should trust all that comes from the kernel or just do an extra check.

For some more fine-grained security, D-Bus services can use PolicyKit to prompt the user requesting the operation for extra authentication.

Support sending large messages: Some D-Bus users complain about bad performance from D-Bus when sending large chunks of data over it, that being the reason for file descriptor passing being available on D-Bus. It is true, though, that one can argue that those applications shouldn’t be sending that much data over the bus, and that it is the application’s responsibility, but the truth is that the problem exists.

Netlink provides the ability to send large messages by using multipart messages, so that the data to be sent can be sliced into chunks (no bigger than the kernel socket buffers’ size), resulting in better performance.

Implementation details
All this needs a bigger change to libdbus/bindings as in our initial plan, since the Netlink messages, as explained before, carry on an extra header that needs to be parsed before the real D-Bus message is processed.

So, for libdbus, we will implement DBusServerNetlink object for the implementation of a Netlink-based D-Bus server, and DBusTransportNetlink for the actual implementation of the wire protocol to be used when using Netlink as a transport. DBusTransportNetlink will be responsible for getting and parsing messages from the Netlink D-Bus kernel service.

For bindings, similar work will be needed to add support to reading and writing Netlink messages, but with the use of the libnl library, this should make it easier, and anyway, it is part of our plan to add whatever code is needed to the most popular bindings.

And that’s all for now, any comments/feedback is appreciated.

Syndicated 2012-03-20 15:34:10 from Rodrigo Moya

D-Bus optimizations II

As explained in my previous post, we are working on optimizing D-Bus for its usage on embedded systems (more precisely on GENIVI).

We started the discussion on the Linux-netdev mailing list about getting our patch to add multicast on UNIX sockets accepted, but, unfortunately, the feedback hasn’t been very good. So, since one of the premises from GENIVI is to get all the work we are doing accepted upstream, we have been thinking in the last few days about what else to use for, as stated in my previous post, fixing the main issue we have found in D-Bus performance, which is the number of context switches needed when getting all traffic on the bus through dbus-daemon. So, here’s a summary of the stuff we have been looking at:

  • Use TIPC, a socket family already available in modern kernel versions and which is targeted at clustering environments, although it can be used for communications inside a single node.
  • Use ZeroMQ, which is a library that, from a first look, provides the stuff we need for D-Bus, that is multicast on local sockets.
  • Provide the multicast on UNIX sockets as a new socket family (AF_MCAST), although this wasn’t well received neither on the linux-netdev discussion. This will contain a trimmed down version of AF_UNIX with only the stuff needed for doing multicast.
  • Extend the AF_DBUS work from Alban to include what we have learnt from the multicast AF_UNIX architecture. This would mean having a patch to the kernel that, as with the AF_MCAST solution, would have to be maintained by distributors, as the linux-netdev people didn’t like this solution neither.
  • Use Netlink, which has all that we need for D-Bus, that is, multicast and unicast, plus it is an established IPC mechanism in the kernel (from kernel space to user space), and is even used for other services similar to D-Bus. We would create a new Netlink subfamily for D-Bus, that would contain code to do the routing, as Netlink, for security reasons, does not allow direct connection between user space apps.
  • Use KBUS, which is a lightweight messaging system, provided as a kernel module.

Right now, we have working code for AF_MCAST, and are looking at Netlink, TIPC and KBUS, so will be blogging more details on what we find out in our experiments. But any feedback would be appreciated since, as stated before, we want to have all this work accepted upstream. So, comments, suggestions?

Syndicated 2012-03-07 18:08:22 from Rodrigo Moya

58 older entries...

 

rodrigo certified others as follows:

  • rodrigo certified xose as Journeyer
  • rodrigo certified LotR as Master
  • rodrigo certified mjs as Master
  • rodrigo certified spong as Journeyer
  • rodrigo certified wichert as Master
  • rodrigo certified martin as Master
  • rodrigo certified alvaro as Journeyer
  • rodrigo certified bertrand as Master
  • rodrigo certified Telsa as Journeyer
  • rodrigo certified jamesh as Master
  • rodrigo certified miguel as Master
  • rodrigo certified timj as Master
  • rodrigo certified lauris as Journeyer
  • rodrigo certified tigert as Master
  • rodrigo certified mathieu as Master
  • rodrigo certified terral as Journeyer
  • rodrigo certified sopwith as Master
  • rodrigo certified tml as Master
  • rodrigo certified daniel as Master
  • rodrigo certified kmaraas as Journeyer
  • rodrigo certified federico as Master
  • rodrigo certified hp as Master
  • rodrigo certified ettore as Master
  • rodrigo certified rodrigo as Apprentice
  • rodrigo certified alan as Master
  • rodrigo certified chema as Journeyer
  • rodrigo certified ole as Journeyer
  • rodrigo certified cuenca as Master
  • rodrigo certified jpick as Master
  • rodrigo certified Uraeus as Journeyer
  • rodrigo certified jacob as Master
  • rodrigo certified juantomas as Journeyer
  • rodrigo certified rakholh as Journeyer
  • rodrigo certified cdwiegand as Journeyer
  • rodrigo certified toledo as Journeyer
  • rodrigo certified cactus as Journeyer
  • rodrigo certified notzed as Journeyer
  • rodrigo certified dwmw2 as Master
  • rodrigo certified DV as Master
  • rodrigo certified rms as Master
  • rodrigo certified dneighbors as Master
  • rodrigo certified listen as Journeyer
  • rodrigo certified Barbwired as Journeyer
  • rodrigo certified ben as Master
  • rodrigo certified dsevilla as Master
  • rodrigo certified olea as Master
  • rodrigo certified lxhispano as Master
  • rodrigo certified clahey as Master
  • rodrigo certified fejj as Master
  • rodrigo certified Jody as Master
  • rodrigo certified jpr as Master
  • rodrigo certified rodo as Master
  • rodrigo certified timur as Journeyer
  • rodrigo certified bit as Journeyer
  • rodrigo certified jjamor as Master
  • rodrigo certified alo as Journeyer
  • rodrigo certified gman as Master
  • rodrigo certified murrayc as Master
  • rodrigo certified hadess as Journeyer
  • rodrigo certified dsandras as Master
  • rodrigo certified csv as Journeyer

Others have certified rodrigo as follows:

  • rodrigo certified rodrigo as Apprentice
  • lauris certified rodrigo as Journeyer
  • alvaro certified rodrigo as Master
  • mathieu certified rodrigo as Journeyer
  • listen certified rodrigo as Journeyer
  • ole certified rodrigo as Master
  • Uraeus certified rodrigo as Journeyer
  • jpick certified rodrigo as Master
  • cuenca certified rodrigo as Journeyer
  • cdwiegand certified rodrigo as Journeyer
  • juantomas certified rodrigo as Master
  • toledo certified rodrigo as Journeyer
  • dneighbors certified rodrigo as Master
  • nixnut certified rodrigo as Journeyer
  • olea certified rodrigo as Master
  • dsevilla certified rodrigo as Master
  • lxhispano certified rodrigo as Master
  • fejj certified rodrigo as Journeyer
  • monk certified rodrigo as Journeyer
  • Jordi certified rodrigo as Journeyer
  • Barbwired certified rodrigo as Master
  • grex certified rodrigo as Master
  • Jody certified rodrigo as Journeyer
  • davefx certified rodrigo as Master
  • ViGueTo certified rodrigo as Master
  • ricardo certified rodrigo as Master
  • timur certified rodrigo as Master
  • fxn certified rodrigo as Journeyer
  • jjamor certified rodrigo as Master
  • pau certified rodrigo as Master
  • wacky certified rodrigo as Master
  • alo certified rodrigo as Master
  • murrayc certified rodrigo as Master
  • gman certified rodrigo as Master
  • hadess certified rodrigo as Master
  • dsandras certified rodrigo as Master
  • mhz certified rodrigo as Master
  • Fabian certified rodrigo as Master
  • jfs certified rodrigo as Journeyer
  • bit certified rodrigo as Master
  • apg certified rodrigo as Master
  • alejandro certified rodrigo as Master
  • ariya certified rodrigo as Master
  • acero certified rodrigo as Journeyer
  • gpoo certified rodrigo as Master
  • strider certified rodrigo as Master
  • pasky certified rodrigo as Master
  • tanis certified rodrigo as Master
  • xose certified rodrigo as Master
  • carlosgc certified rodrigo as Master
  • freax certified rodrigo as Master
  • pvanhoof certified rodrigo as Master
  • gicmo certified rodrigo as Master
  • behdad certified rodrigo as Master
  • roozbeh certified rodrigo as Master
  • iagorubio certified rodrigo as Master
  • lucasr certified rodrigo as Master
  • loopback certified rodrigo as Master
  • midal certified rodrigo as Master
  • jaime certified rodrigo as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page