Infinity Full-System Preview NOW AVAILABLE
If you’ve been following Infinity and would like to, you know, download some code and try it out… well, now you can!
Here I am casually using GDB with Infinity
Full-system Infinity preview coming up
I’ve released bits and pieces of Infinity over the past year, but nothing that really brings everything together. Right now I’m working on an initial full-system release of everything to do with Infinity so far. It’s codenamed “First Flight”, and you’ll be able to actually use it, see how everything hangs together, and generally get a better idea of what the point of it all is.
First Flight will be just enough for GDB to attach to a live process or start with a core file. “
info threads” should work, “
print errno” will not. First Flight will comprise:
git cloneand build to get a libpthread.so with Infinity notes for
The libthread_db.so shim won’t be used in future releases—they’ll have support for Infinity built into GDB—but it’ll likely remain as a nice example of how to use libi8x.
Infinity client library
This past few weeks I’ve been working on an Infinity client library. This is what GDB will use to execute notes it finds. It’s early days, but it executed its first note this morning so I thought I’d put something together so people can see what I’m doing. Here’s how to try it out:
tlsdumpexample program needs it:
sudo yum install elfutils-libelf-devel # Fedora, RHEL, etc... sudo apt-get install libelf-dev # Debian, Ubuntu, etc...
git clone -b libi8x-0.0.1 https://github.com/gbenson/libi8x.git libi8x-0.0.1 cd libi8x-0.0.1 ./autogen.sh ./configure --enable-logging --enable-debug make
tlsdumpexample program built:
bash$ ls -l examples/tlsdump -rwxr-xr-x. 1 gary gary 5540 Apr 20 12:52 examples/tlsdump
Yeah, there it is! (if it’s not there go back to step 0)
gcc -o tests/ifact tests/ifact.S tests/main.c
bash$ tests/ifact &  8301 Hello world I'm 8301
tlsdumpexample program with the test program’s PID as it’s argument:
$ examples/tlsdump 8301 0! = 1 1! = 1 2! = 2 3! = 6 4! = 24 5! = 120 6! = 720 7! = 5040 8! = 40320 9! = 362880 10! = 3628800 11! = 39916800 12! = 479001600
What just happened? The executable
test/ifact you built contains a single Infinity note,
test::factorial(i)i, the source for which is in
tlsdump example located the
ifact executable, loaded
test::factorial(i)i from it, and ran it a few times printing the result:
err = i8x_ctx_get_funcref (ctx, "test", "factorial", "i", "i", &fr); if (err != I8X_OK) error_i8x (ctx, err); err = i8x_xctx_new (ctx, 512, &xctx); if (err != I8X_OK) error_i8x (ctx, err); for (int i = 0; i
To see some debug output try this:I8X_LOG=debug examples/tlsdump PID
I8X_DEBUG=truein addition to
I8X_LOG=debugto trace the bytecode as it executes.
I’m winding down for a month away from Infinity. The current status is that the language and note format changes for 0.0.2 are all done. You can get them with:
git clone https://github.com/gbenson/i8c.git
There’s also the beginnings of an Emacs major mode for i8 in there too. My glibc tree now has notes for
td_ta_thr_iter as well as
td_ta_map_lwp2thr. That’s two of the three hard ones done. Get them with:
git clone https://github.com/gbenson/glibc.git -b infinity2
td_thr_get_info is just legwork and
td_thr_tls_get_addr is just a wrapper for
td_thr_tlsbase is the other hard note.
All notes have testcases with 100% bytecode coverage. I may add a flag for I8X to make not having 100% coverage a failure, and make glibc use it so nobody can commit notes with untested code.
The total note size so far is 720 bytes so I may still manage to get all five libpthread notes implemented in less than 1k:
Displaying notes found at file offset 0x00018f54 with length 0x000002d0: Owner Data size Description GNU 0x00000063 NT_GNU_INFINITY (inspection function) Signature: libpthread::__lookup_th_unique(i)ip GNU 0x00000088 NT_GNU_INFINITY (inspection function) Signature: libpthread::map_lwp2thr(i)ip GNU 0x000000cd NT_GNU_INFINITY (inspection function) Signature: libpthread::__iterate_thread_list(Fi(po)oipii)ii GNU 0x000000d2 NT_GNU_INFINITY (inspection function) Signature: libpthread::thr_iter(Fi(po)oiipi)i
In case you hadn’t noticed, I8C 0.0.1 was released last week. As well as the note compiler, this package contains I8X, an interpreter for unit testing compiled notes.
In case you had noticed and you’ve been working with it, please note there’s some compatibility-breaking source language and note format changes in the pipeline.
Infinity has a mailing list, and I want you to subscribe to it. Infinity isn’t just my project–anybody who might either produce or consume Infinity notes has a stake in this:
I don’t know every issue that Infinity will have to solve, and you know things I need to know. So don’t sit on the sidelines! Send an empty message to email@example.com and get involved.
Infinity compiler update
I had hoped to make a first release of i8c today but I still have one open issue so you’ll have to wait until next week. Until then here is some note source for you to ponder:
define test::factorial returns int argument int x extern func int (int) factorial swap dup load 1 bne not_done_yet return not_done_yet: dup load 1 sub swap rot swap call mul
As hints I will a) point you at section 2.5 of the DWARF spec, and b) mention that
extern directives instruct the caller and runtime respectively to push entries onto the stack.
Your homework is to see how much of the language and runtime you can infer from this one function.
You may ask questions.
When I said I’d never written a compiler, that’s actually a lie: I did some work on the Ceylon compiler a few years back. It wasn’t a very enriching experience but at least I know what ASTs and visitors are.
Anyway, the C-like syntax I talked about Friday might have been a little optimistic. I wanted the source to not be full of stack-shuffling but that’s not going to happen. Partly because I’d been assuming that the problem of compiling to a stack-based instruction set was solved: I was thinking of Java, but of course Java has local variables so it’s not exactly a stack-based instruction set, it’s stack and 65,536 registers–kind of the opposite problem! I didn’t find much online about compiling to stack-based instruction sets other than “it’s hard”.
I could probably manage something, but given that the total output of the compiler for glibc will likely be a hundred or so bytes of instructions, it doesn’t make sense to spend a huge amount of time on it.
The Infinity function compiler is therefore going to be something of a helpful assembler.
Infinity function compiler
Yesterday I reworked the note format to allow functions as first-class objects, and then I had an idea about how to implement the note compiler.
I’d already thought I could have the compiler emit assembly to avoid having to think about endianness, but I couldn’t figure out how to do other platform-specific stuff like 32- vs 64-bit, or choosing which of the three LWP ID to thread descriptor mapping styles to use. My idea was to not only output to the assembler but to input from the C preprocessor as well. That way I can do all the conditional stuff in there, driven by glibc’s include files and
Yeah, I think I can write a compiler that knows nothing about the platform it’s compiling for.
So, I spent all morning messing around with glibc’s build system. I’ve come up with this:
tls.hwill have some Infinity definitions following the ones for libthread_db. Here’s the one for x86_64:
/* Magic for libthread_db to know how to do THREAD_SELF. */ #define DB_THREAD_SELF_INCLUDE
/* For the FS constant. */ #define DB_THREAD_SELF CONST_THREAD_AREA (64, FS) /* Magic for Infinity to know how to do THREAD_SELF. */ #define I8_THREAD_SELF I8_TS_CONST_THREAD_AREA #define I8_TS_CTA_VALUE FS
I could probably generate the libthread_db ones from the Infinity ones… but… later.
tls-infinity.symfile that the build system turns into
infinity_map_lwp2thr.i8. That includes
tls-infinity.hand, on x86_64, conditionally includes
infinity_lookup_th_unique_cta.i8to perform the mapping in the correct way for that platform.
infinity_map_lwp2thr.i8gets fed into the C preprocessor which a) does all the conditionality and b) expands all the constants (as a total bonus!)
This leaves me with this lovely file to write a compiler for.
I’ve never written a compiler.
Infinity progress update
Yesterday I had my first Infinity milestone. I have a glibc with notes implementing libthread_db’s
td_ta_map_lwp2thr and a GDB that loads and executes them. I did a side-by-side test calling the Infinity
libpthread::map_lwp2thr function every time libthread_db’s
td_ta_map_lwp2thr was called and comparing the results… and it worked! You could see the DWARF if you’re interested.
I’ve had something of a look at all the five functions I need to implement now and I’m pretty sure I have a handle on what infrastructure is needed to support them. The one thing I’m missing at present is function calls (infinity-infinity and infinity-other), which I’ll likely handle with a new instruction (
DW_OP_GNU_i8call or something). That’s probably the only extension I need. It’s definitely necessary because one of the arguments to
td_ta_thr_iter is a function that
td_ta_thr_iter then calls. Infinity basically needs first-class functions for this.
I also have a pretty good idea of how to hook it into GDB’s Linux thread_db code. I’ll write a bunch of functions in GDB’s common code with the same signatures as libthread_db’s functions and pass them to the existing thread_db code by pointer. Doing it that way means the existing thread_db code in GDB and gdbserver can be left unmodified. I’ll implement the bookkeeping stuff there too, probably using the
thread_agent type to pass around whatever context I need.
At the moment I’m using GDB’s existing DWARF interpreter to execute the notes, but I may well end up writing a new one for it. Firstly because the existing interpreter is deeply embedded into the GDB side: it uses gdbarch, values, types, etc, none of which are currently in common code. And secondly because the existing interpreter is written to evaluate lots of different DWARF expressions once (or, a small number of times). It’s input is raw DWARF, which it decodes instruction-by-instruction. The use case for the Infinity interpreter is executing a small number of expressions lots of times, so some pre-analysis would be beneficial. The instructions could be expanded, with operands decoded and possibly translated, and common multi-instruction cases could be combined. Stuff like calls to builtin functions (
ps_getpid) could be replaced with intrinsics. And, with some restrictions on the bytecode, I could pre-compute the stack depth and do one overflow check at function entry rather than overflow checks for every push. That kind of thing.
I’m expecting the addition of a second DWARF interpreter to GDB to be contentious, but they’ll optimized for different things and doing different things. For example, Infinity could work better with some type tracking (and will likely need it to make function calls secure) but it’s different from what GDB’s existing interpreter needs and it’s difficult to see how to combine the two without ending up with something that’s not very good at either. Not to mention that getting it to a point it can be moved to common code would likely slow it down a ton.
Anyway, that’s where I’m at.
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!