Older blog entries for mslicker (starting at number 22)

mpr, If I knew specifically what I wanted, a small hack might do. I'm not too skilled with Unix, so small hacks are probably out of my reach. Language analysis I know very well, so this seems like the right aproach for me.

Once I have the language in an abstract form, then I can do sofisticated transformations. Probably some experimentation is required to figure out exactly what I want to do with the information once parsed. I have several ideas of what makes the source hard to read and how I can alleviate.

If sucessful, this will be a useful tool to mine C source code, not just the Linux kernel.

To me, C is very much a legacy language. It should have left behind long ago, or perhaps should have never been created.

tk, I've read a lot of the spec. Pretty much every detail is required if you want to really make the most use of the information you are parsing.

C arcana

In the process of writing a parser, I found some little known (or at least little used) facts.

One interesting one, the following two lines mean the same thing syntactically.

<:  :> <%  %> %: %:%:
[   ]  {   }  #  ##

Using this fact, the following is perfectly valid C:

%:include <stdio.h>

 int main ()
<%
    int a<::> = <%1, 2, 3%>;

     return 0;
%>

Linux deobfuscator

This is comming along. tk said writing a C parser is "quite tractable". That is quite an understatement. The language is itself described in no less than 160 pages of consise ANSI verbiage. Add to this the GNU C extentions, which there is no small number.

And I am now quite used to a language which is described in one page. So this quite a lot of things to keep strait.

To bad there is not more interest in this. Probably mostly of interest to people creating systems of which there are few.

Might be of use to people who want to get involved with linux too. However, I don't know how much information I will keep around which would be required for someone to get involved with the development.

I expect understanding a part of linux will take %10 of effort that it would take just using a editor. Most of this will just be in reducing the amount you need to read, some additional gains will be in locating information quickly. I think it would be good to put things in pop up windows. Then, you can build a context for understanding a function, by viewing the related data structures and functions.

Posted my previous entry as an article at the request of tk.

Linux deobfuscation project

Reguardless of the particular merrits of Linux, a large body of device drivers has been written for it. So much so, that when designers of new operating systems embark on creating a system they often op for building on top Linux, despite the incredible drawbacks that come from this approach. Furthermore, for some devices, Linux may be the only reference for programming a device. This is due partly to the fact that manufacturers' support of "open source" means the development of a Linux driver, not releasing the specifications as they should have done.

Reading source can be quite difficult for all but the most seasoned Linux hackers. This is due to many factors. A certain coding style, ubiquitous comments, and the general clumsyness of the C language.

My idea, is to mechanically convert the Linux sources to a form that at once represents their true structure and more directly represents their semantics. A particular machine configuration will be assumed. Therefore preprocessing can be done in it's entirety. Comments and macros will be left as hypertext annotations. I think file structure should be completely broken down, leaving just a call graph (or forest). Types will will be completely striped and left as an annotation, with a similar graph representation.

Currently this only exists as an idea. If you are interrested in funding this work please contact me. I say this because it is not something easily done. It would require some time to do right, and that is perhaps the only thing preventing me from doing so.

--- Update Grammatical corrections.

tk wrote:

Is there a non-OO system architecture which can facilitate the creation of object-like interfaces for the entire system? This is exactly what I've been trying to figure out.

I can name three: Lisp, Postsript, and Forth. Lisp on Lisp Machines aparently had an object-like interface. There was the Postcript News windowing system, which facilitated object-like interfaces. Postscript is of course a special case of Forth, so such interfaces could easily be developed in Forth. I have even heard of Smalltalk like GUIs being developed from members of the forth community, although I've never seen these in person.

One common thread among Lisp and Forth, besides the simple syntax, is the extensibility of the language. This is the key feature which allows these languages to be applied easily to any problem.

The Postcript News system on the other hand, was far more isolated most of the facilities were built into the window server and were hence unchangable.

nymia, I noticed there is a lot of activity hacking the Xbox. Seems quite close to what you want, commodity hardware, PC like. And people seem to be making progress hacking the hardware.

tk, A bit belated. Just to note, that the user interface may be object-like, and user friendly, without the operating system necessarily being object-oriented.

Applications and processes are probably left over from the old time sharing systems, from which unix is derived. There is no reason to organize the system in this particular way, except prehaps lack of imagination on the part of modern os designers. Also the notion of an application has strong favor with proprietary software companies. Kind of a little fortress seperated from the rest of the computer, which they can control, and resell to you and indefinite number of times.

Can't say an OO operating system is worth it. There doesn't seem to be any concrete reason to organize the system in this way except for the designers preference. New systems certainly have a lot of potential if only the hardware barier can be broken through.

vsync

If anyone is familer with programming the vertical sync interrupt of video cards, please contact me. I'm having a bit of trouble. This is a more modern card, the Matrox G400.

raph, I have to ask, what problem are you trying to solve?

If you care about performance, you should know that the fastest display engines know specifically what can change, how it can change, what needs to be redrawn, the best way to redraw, ect.

Also this generic tree structure will of course add much bloat to any application willing to use it. An optimized representation will always beat it, sometimes by orders of magnitude. Creating such library is a sure way to encourage bloatware, and mediocrity.

I believe this line of thinking, and similar thinking, is perhaps the true source of bloat and complexity in todays software. If people would only address the problem at hand...

tk, Are we really doomed to Windows and Unix forever? Or is there a way out? I feel that latter is true. If didn't I'd probably give up programming. You must too, or else why would you work on Delitalk? colorForth, Chuck Moore, and others I believe offer the intellectual strenth to break through the bloatware stranglehold. Not to mention the C stranglehold, The language that has held so much back.

tk, At that resolution, if you intend to program in colorForth, you might want to create your own font. The font that comes with colorForth is quite large, even for 1024x768. One block of source code (1024 bytes) completely filled, fills up roughly the entire screen with the default resolution and font. So you want to scale the font down at least a third in each direction.

Chuck Moore has created a nice manual.

If you want to write a simple app to get a feel for it, first open the editor (type 'e' 'space') find and empty block. Then type some thing like:

ok show black screen green 120 75 at 200 125 box keyboard ;

Where 'ok' is red, the rest are green. Load the block ('xx load '), type 'ok', and you should have a green box on the screen. colorForth comes with other examples, but these might not fit on your screen.

tk, Actually I have converted thte colorForth sources to GNU assembler. You can find the sources and some other things at my colorForth directory (I should probably make web page for this). Some people have created colorForth's with a resolution of 800x600 and some other modifications to get it working on their system (colorForth index).

Here are some screen shots of the editor (full size). In lower left corner you will see a visual display of the stack, in the right corner the function keys (these change depending on the application). 'full' is the last comand typed, which took the screen shot. Of course, the rest is the source code. The funny red shape in the source code is the cursor.

I'm not sure if you need any built-in skills to do colorForth well. Mainly you have to be willing to do things in very different way than you may have done before. Usually we try to break problems into tiny pieces. The smaller the piece, the easier to optimize. Optimization in colorForth is not as complex as register-based machine code optimization. It's kind of like untangling rope. You try to find the most direct computation, with the least number of stack manipulations. Simple RPN arithmetic expressions are examples of optimized colorForth.

I found my own skill has grown quite a bit, from being a colorForth newbie to gaining a certain level of competence. I'm not yet close to the mastery of Chuck Moore.

tk, To me, Smalltalk is clearly one of the more elegant object-oriented languages I've seen.

However, I dislike object-oriented programming in general.

• Rarely are objects and the relationships bewteen them meaningful to the problem being solved.
• The problem can always be solved in a more efficient way using the von Neumann model. The dominant archiecture of computers today.
• I don't like being limited to a single mode of programming.

As a system, Smalltalk is not very promising. The prominant Squeak system is quite large and not that responsive compared to existing systems. Improvement from the users perspective requires consuming less resources, and a faster response, among other things. It could be that the squeak developers went a little feature crazy. Squeak doesn't seem to have a purpose, other than to show how various things might be coded in Smalltalk. Maybe it is possible to have a much smaller smalltalk with adequate performance, and usable applications.

Impovement though, is very much possible. Just look at the slop that passes as functioning C code these days. Impovements to existing systems can be made in speed, resource consumpion, code size, simplicity, and the user interface. All of these can be improved by orders of magnitude, without coding everything in assembly. However it is required that you start from scratch, in an extremly capable language.

In this reguard, the most promising system to me is the colorForth system. The philosophy of this system is to cast off as much bagage as possible and start fresh. The system is %100 original code by Chuck Moore. Performance is excelent. Although the code is not purely optimized for speed like compiled C. colorForth chooses a much simpler compiler instead. In fact the compiler and the entire core software is under 2k. Extensibility is at the heart of the lanugage, so the large range in semantics required by a modern system can be expresed elegantly in the language.

The reason why colorForth can achieve such improvement, is that it relies purely on human ingenuity. The only limit in how good the code is, is in the programmer writing the code. The reason why the compiler does so little, is that the code is already human optimized.

For software to improve, I believe we must rely primarily on our own creative abilility. Automation of creative tasks makes us the slave to the machine.