gary is currently certified at Master level.

Name: Gary Benson
Member since: 2001-05-31 00:39:54
Last Login: 2012-10-24 12:18:42

FOAF RDF Share This



I work for Red Hat.


Articles Posted by gary

Recent blog entries by gary

Syndication: RSS 2.0

Infinity compiler update

I had hoped to make a first release of i8c today but I still have one open issue so you’ll have to wait until next week. Until then here is some note source for you to ponder:

define test::factorial returns int
    argument int x
    extern func int (int) factorial

    load 1
    bne not_done_yet

    load 1

As hints I will a) point you at section 2.5 of the DWARF spec, and b) mention that argument and extern directives instruct the caller and runtime respectively to push entries onto the stack.

Your homework is to see how much of the language and runtime you can infer from this one function.

You may ask questions.

Syndicated 2015-10-02 16:27:05 from

Infinity compiler

When I said I’d never written a compiler, that’s actually a lie: I did some work on the Ceylon compiler a few years back. It wasn’t a very enriching experience but at least I know what ASTs and visitors are.

Anyway, the C-like syntax I talked about Friday might have been a little optimistic. I wanted the source to not be full of stack-shuffling but that’s not going to happen. Partly because I’d been assuming that the problem of compiling to a stack-based instruction set was solved: I was thinking of Java, but of course Java has local variables so it’s not exactly a stack-based instruction set, it’s stack and 65,536 registers–kind of the opposite problem! I didn’t find much online about compiling to stack-based instruction sets other than “it’s hard”.

I could probably manage something, but given that the total output of the compiler for glibc will likely be a hundred or so bytes of instructions, it doesn’t make sense to spend a huge amount of time on it.

The Infinity function compiler is therefore going to be something of a helpful assembler.

Syndicated 2015-09-15 09:59:36 from

Infinity function compiler

Yesterday I reworked the note format to allow functions as first-class objects, and then I had an idea about how to implement the note compiler.

I’d already thought I could have the compiler emit assembly to avoid having to think about endianness, but I couldn’t figure out how to do other platform-specific stuff like 32- vs 64-bit, or choosing which of the three LWP ID to thread descriptor mapping styles to use. My idea was to not only output to the assembler but to input from the C preprocessor as well. That way I can do all the conditional stuff in there, driven by glibc’s include files and CPPFLAGS.

Yeah, I think I can write a compiler that knows nothing about the platform it’s compiling for.

So, I spent all morning messing around with glibc’s build system. I’ve come up with this:

  1. Each machine’s tls.h will have some Infinity definitions following the ones for libthread_db. Here’s the one for x86_64:
    /* Magic for libthread_db to know how to do THREAD_SELF.  */
    #define DB_THREAD_SELF_INCLUDE  /* For the FS constant.  */
    /* Magic for Infinity to know how to do THREAD_SELF.  */
    #define I8_TS_CTA_VALUE FS

    I could probably generate the libthread_db ones from the Infinity ones… but… later.

  2. glibc has a script to generate assembler include files, to get constants and offsets from C headers into the assembler. I wrote a tls-infinity.sym file that the build system turns into tls-infinity.h for me.
  3. The source for map_lwp2thr is infinity_map_lwp2thr.i8. That includes tls-infinity.h and, on x86_64, conditionally includes infinity_lookup_th_unique_cta.i8 to perform the mapping in the correct way for that platform.
  4. infinity_map_lwp2thr.i8 gets fed into the C preprocessor which a) does all the conditionality and b) expands all the constants (as a total bonus!)

This leaves me with this lovely file to write a compiler for.

I’ve never written a compiler.

Syndicated 2015-09-11 13:56:04 from

Infinity progress update

Yesterday I had my first Infinity milestone. I have a glibc with notes implementing libthread_db’s td_ta_map_lwp2thr and a GDB that loads and executes them. I did a side-by-side test calling the Infinity libpthread::map_lwp2thr function every time libthread_db’s td_ta_map_lwp2thr was called and comparing the results… and it worked! You could see the DWARF if you’re interested.

I’ve had something of a look at all the five functions I need to implement now and I’m pretty sure I have a handle on what infrastructure is needed to support them. The one thing I’m missing at present is function calls (infinity-infinity and infinity-other), which I’ll likely handle with a new instruction (DW_OP_GNU_i8call or something). That’s probably the only extension I need. It’s definitely necessary because one of the arguments to td_ta_thr_iter is a function that td_ta_thr_iter then calls. Infinity basically needs first-class functions for this.

I also have a pretty good idea of how to hook it into GDB’s Linux thread_db code. I’ll write a bunch of functions in GDB’s common code with the same signatures as libthread_db’s functions and pass them to the existing thread_db code by pointer. Doing it that way means the existing thread_db code in GDB and gdbserver can be left unmodified. I’ll implement the bookkeeping stuff there too, probably using the thread_agent type to pass around whatever context I need.

At the moment I’m using GDB’s existing DWARF interpreter to execute the notes, but I may well end up writing a new one for it. Firstly because the existing interpreter is deeply embedded into the GDB side: it uses gdbarch, values, types, etc, none of which are currently in common code. And secondly because the existing interpreter is written to evaluate lots of different DWARF expressions once (or, a small number of times). It’s input is raw DWARF, which it decodes instruction-by-instruction. The use case for the Infinity interpreter is executing a small number of expressions lots of times, so some pre-analysis would be beneficial. The instructions could be expanded, with operands decoded and possibly translated, and common multi-instruction cases could be combined. Stuff like calls to builtin functions (ps_get_thread_area, ps_getpid) could be replaced with intrinsics. And, with some restrictions on the bytecode, I could pre-compute the stack depth and do one overflow check at function entry rather than overflow checks for every push. That kind of thing.

I’m expecting the addition of a second DWARF interpreter to GDB to be contentious, but they’ll optimized for different things and doing different things. For example, Infinity could work better with some type tracking (and will likely need it to make function calls secure) but it’s different from what GDB’s existing interpreter needs and it’s difficult to see how to combine the two without ending up with something that’s not very good at either. Not to mention that getting it to a point it can be moved to common code would likely slow it down a ton.

Anyway, that’s where I’m at.

Syndicated 2015-09-10 12:10:23 from


To debug live processes on modern Linux GDB needs four libthread_db functions:

  • td_ta_map_lwp2thr (required for initial attach)
  • td_thr_get_info (required for initial attach)
  • td_thr_tls_get_addr (not required for initial attach, but required for “p errno” on regular executables)
  • td_thr_tlsbase (not required for initial attach, but required for “p errno” for -static -pthread executables)

To debug a corefile on modern Linux GDB needs one more libthread_db function:

  • td_ta_thr_iter

GDB makes some other libthread_db calls too, but these are bookkeeping that won’t be required with the replacement. So, the order of work will be:

  1. Implement replacements for the four core functions.
  2. Get those approved and committed in GDB, BFD and glibc (and in binutils, coreutils readelf).
  3. Replace td_ta_thr_iter too, and get that committed.
  4. Implement runtime-linker interface stuff to allow GDB to follow dlmopen.

The first (non-bookkeeping) function GDB calls is td_ta_map_lwp2thr and it’s a pig. If I can do td_ta_map_lwp2thr I can do anything.

When you call it, td_ta_map_lwp2thr has four ways it can proceed:

  1. If __pthread_initialize_minimal has not gotten far enough we can’t rely on whatever’s in the thread registers. If this is the case, td_ta_map_lwp2thr checks that the LWP is the initial thread and sets th->th_unique to NULL. (Other bits of libthread_db spot this NULL and act accordingly.) td_ta_map_lwp2thr decides whether __pthread_initialize_minimal has gotten far enough by examining in the inferior. If it’s NULL then __pthread_initialize_minimal has not gotten far enough.
  2. On ta_howto_const_thread_area architectures (x86_64, aarch64, arm)
    [glibc/sysdeps/*/nptl/tls.h has
      #define DB_THREAD_SELF CONST_THREAD_AREA(bits, value)
    which exports
      const uint32_t _thread_db_const_thread_area = value;
    from glibc/nptl_db/db_info.c]:
    • td_ta_map_lwp2thr will call ps_get_thread_area with value

    to set th->th_unique.

    ps_get_thread_area (in GDB) does different things for different

    1. on x86_64, value is a register number (FS or GS)
      ps_get_thread_area returns the contents of that register.
    2. on arm, GDB uses PTRACE_GET_THREAD_AREA, NULL and subtracts value from the result.
    3. on aarch64, GDB uses PTRACE_GETREGSET, NT_ARM_TLS and subtracts value from the result.
  3. On ta_howto_reg architectures (ppc*, s390*)
    [glibc/sysdeps/*/nptl/tls.h has
      #define DB_THREAD_SELF REGISTER(bits, size, regofs, bias)...
    which exports
      const uint32_t _thread_db_register32[3] = {size, regofs, bias};
      const uint32_t _thread_db_register64[3] = {size, regofs, bias};
    from glibc/nptl_db/db_info.c]:

    td_ta_map_lwp2thr will:

    • call ps_lgetregs to get the inferior’s registers
    • get the contents of the specified register (with _td_fetch_value_local)


    • SUBTRACT bias from the register’s contents

    to set th->unique.

  4. On ta_howto_reg_thread_area architectures (i386)
    [glibc/sysdeps/*/nptl/tls.h has
      #define DB_THREAD_SELF REGISTER_THREAD_AREA(bits, size, regofs, bias)...
    which exports
      const uint32_t _thread_db_register32_thread_area[3] = {size, regofs, bias};
      const uint32_t _thread_db_register64_thread_area[3] = {size, regofs, bias};
    from glibc/nptl_db/db_info.c]:

    td_ta_map_lwp2thr will:

    • call ps_lgetregs to get the inferior’s registers
    • get the contents of the specified register (with _td_fetch_value_local)
    • RIGHT SHIFT the register’s contents by bias


    • call ps_get_thread_area with that number

    to set th->unique.

    ps_get_thread_area (in GDB) does different things for different

    1. on i386, GDB uses PTRACE_GET_THREAD_AREA, VALUE and returns the second element of the result.

Cases 2, 3, and 4 will obviously be hardwired into the specific architecture’s libpthread. But… yeah.

Syndicated 2015-08-25 15:24:47 from

276 older entries...


gary certified others as follows:

  • gary certified mjcox as Master
  • gary certified joe as Master
  • gary certified tromey as Master
  • gary certified Anthony as Master
  • gary certified leonardr as Journeyer
  • gary certified pedro as Apprentice
  • gary certified nutella as Apprentice
  • gary certified robilad as Master
  • gary certified mjw as Master
  • gary certified timp as Journeyer
  • gary certified aph as Master
  • gary certified twisti as Master
  • gary certified avdyk as Journeyer
  • gary certified gnuandrew as Journeyer

Others have certified gary as follows:

  • chromatic certified gary as Apprentice
  • olandgren certified gary as Apprentice
  • MikeGTN certified gary as Apprentice
  • hub certified gary as Journeyer
  • jao certified gary as Apprentice
  • uweo certified gary as Apprentice
  • sye certified gary as Master
  • cannam certified gary as Journeyer
  • jooon certified gary as Apprentice
  • mike750 certified gary as Journeyer
  • hacker certified gary as Apprentice
  • sad certified gary as Journeyer
  • cdent certified gary as Apprentice
  • opiate certified gary as Journeyer
  • johnnyb certified gary as Apprentice
  • slef certified gary as Apprentice
  • jarod certified gary as Journeyer
  • maelstorm certified gary as Journeyer
  • mjcox certified gary as Journeyer
  • Ausmosis certified gary as Journeyer
  • monk certified gary as Journeyer
  • jono certified gary as Journeyer
  • fxn certified gary as Journeyer
  • ignatz certified gary as Journeyer
  • sethcohn certified gary as Journeyer
  • rupert certified gary as Journeyer
  • larry certified gary as Journeyer
  • sdodji certified gary as Journeyer
  • StevenRainwater certified gary as Journeyer
  • mharris certified gary as Journeyer
  • TheCorruptor certified gary as Journeyer
  • Ilan certified gary as Master
  • criswell certified gary as Master
  • DV certified gary as Journeyer
  • sprite certified gary as Master
  • richdawe certified gary as Journeyer
  • mulix certified gary as Journeyer
  • jrf certified gary as Journeyer
  • rkrishnan certified gary as Journeyer
  • nixnut certified gary as Journeyer
  • jcv certified gary as Journeyer
  • Omnifarious certified gary as Journeyer
  • e8johan certified gary as Journeyer
  • mjw certified gary as Journeyer
  • badvogato certified gary as Master
  • tromey certified gary as Master
  • robilad certified gary as Master
  • reenoo certified gary as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

Share this page