Older blog entries for lkcl (starting at number 291)

27 Jun 2006 (updated 27 Jun 2006 at 23:01 UTC) »

hey remi,

no "threshold=4" doesn't work for me: i note that on return to a page where i have noted a spammer's diary with a rating of "1", the value is not re-presented to me. perhaps i do not understand how the diary-rating system works (or it's on a 15 min calculation cycle, like the trust metric?)

update: yep, it works. thanks remi. curious: where the heck can i set that? and why isn't the default on recentlog set to very low like... oh... thresh=2?

hi steven, good to hear from you.

the assumption that we made was that all contributions would be of zero or more value, and that there would be nobody stupid enough to endeavour to provide contributions of negative value.

the trust metric keeps people off the front page.

it doesn't keep them off the diary entries.

i'm thinking out loud again.

"negative" certs were discussed many times: i loudly resisted the calls for addition of "negative certs", on the basis that if you haven't got anything good to say, don't say anything.

however, when people _deliberately_ go out of their way to say value-less things, then that's a completely different ballgame.

and advogato is not equipped to deal with that (wrt diary).

here's one possibility to consider:

if an observer's diary _contains_ < img > or < a > tags, then it is simply.... made invisible, and the user is warned that, as an observer, they are not allowed to use < a > or < img >.

also it's important to check for < xxx style="..." > as that can be used to embed images via inline stylesheets.

the reason for making it invisible is that the content pretty much is rendered useless without some convenient markup and images.

i'd far rather we pulled their teeth and made it pointless for peole to post irrelevant content than to encourage people to make "negative" assertions about the content itself.

the "bad neighbourhood" idea - sounds scarily complex, and also sounds like a rewrite of mod_virgule into python is sorely needed (which will reduce the amount of code by approximately 70% by the way).

it sounds to me like the "bad neighbourhood" idea requires "consensus". i.e. if there are less than a certain number of people agreeing that someone is Certified, then that person is "tainted".

i really think it's time to rewrite mod_virgule - especially as the ford-fulkersson depth-first algorithm means that you have to load eeevvveerything into memory to do the trust metric calculation.

if you use a breadth-first algorithm, then 1) you don't have to load the entire graph into memory 2) you can automatically stop at a certain number of degrees 3) you can also work out "consensus" along the way - namely you can check, at each "depth", that the number of people Certifying each of the users at the new "depth", is greater than the minimum number of required Certs (e.g. 3 for Master; 2 for Apprentice and Journeyer).

the min number required for Master definitely needs to be less than the total number of top-level-seeds, btw!!! unless of course you have different rules for top-level-seeds, saying "a top level seed doesn't need "consensus" - their opinions are implicitly trusted and absolute).

a lot of people would be very pissed at the change. well... tough!

p.s. steven: are there backups of the XML files cos my profile has been truncated to zero twice now: the first time lost about a hundred Certs; the second time lost the entire content of my personal details.

21 Jun 2006 (updated 21 Jun 2006 at 10:06 UTC) »
Diary "Rating" system

ok. two very simple things required to ensure that inapproprirate diary entries are not viewed:

1) set the diary "rating" on a user to "1". bring back the version of mod_virgule which allows you to set the viewing threshold (it was an experiment that appears to have been lost somewhere somehow).

2) put in a trust-metric check such that only apprentice-and-above (not observer) diary entries appear in the recent log.

v. simple.

CSS styles

verry very interesting: i've been exploring CSS styles, and with some careful egg-shell-treading work, it's possible to have fluid styles - of menus and of layout - that look reasonable, irrespective of the screen width.

it's taken me about a month to find appropriate articles and learn from them. now that _has_ to be a pretty hard-core technology if it takes _me_ that long to get used to :) [actually, i've been trying to grok css for a hell of a lot longer than that].

one funny thing: i put the menu option items on the right-hand-side, by using "float:right;". i looked at the options and went "huhn? there's something odd about those menu items" - of course, they were in reverse order :)

by adding "float:right;" they get right-floated in priority that they're added from your page!

the neat thing about using float:right; is that when you shrink the screen - even down to 600 or less - the menu options all kinda... jumble and cascade onto the page, at the top.

here's the thing: i _could_ put a border round them, i _could_ shrink the text size, i _could_ set a fixed width... but i'm not going to.

i'm pissed off that i can't resize google's advertising as it is. hm, maybe i will do some auto-screen-width detection some day...

ACM article on corba's decline

by far-and-above the most fascinating and insightful article on software development that i have ever read. excerpt (conclusions) from michi henning's article:

Standards consortia need iron-clad rules to ensure that they standardize existing best practice. There is no room for innovation in standards. Throwing in ``just that extra little feature'' inevitably causes unforeseen technical problems, despite the best intentions.

No standard should be approved without a reference implementation. This provides a first-line sanity check of what is being standardized. (No one is brilliant enough to look at a specification and be certain that it does not contain hidden flaws without actually implementing it.)

No standard should be approved without having been used to implement a few projects of realistic complexity. This is necessary to weed out poor APIs: Too often, the implementers of an API never actually use their own interfaces, with disastrous consequences for usability.

Interestingly, the open source community has done a much better job of adhering to these rules than have industry consortia.

Open source innovation usually is subject to a Darwinian selection process. Different developers implement their ideas of how something should work, and others try to use the feature and critique or improve it. That way, the software is extensively scrutinized and tested, and only the ``fittest'' version survives. (Many open source projects formalize this process with alternating experimental and production releases: The experimental releases act as the test bed and evolutionary filter.)

To create quality software, the ability to say ``no'' is usually far more important than the ability to say ``yes.'' Open source embodies this in something that can be called ``benevolent dictatorship'': Even though many people contribute to the overall effort, a single expert (or a small cabal of experts) ultimately rejects or accepts each proposed change. This preserves the original architectural vision and stops the proverbial too many cooks from spoiling the broth.

At the heart of these open source practices are two essential prerequisites: cooperation and trust. Without cooperation, the evolutionary process cannot work; and without trust, no cabal of experts can act as an ultimate arbiter. This, however, is precisely where software consortia find their doom. It is naïve to put competing vendors and customers into a consortium and expect them to come up with a high-quality product--commercial realities ensure that cooperation and trust are the last things on the participants' minds.

Of course, software consortia contribute to an evolutionary process just as much as open source projects do. But it is the commercial marketplace that acts as the test bed and evolutionary filter, and it is the customers who, with their wallets, act as the (usually not so benevolent) dictator. This amounts to little more than an industry that throws up silver bullets and customers who leap after them like lemmings over a cliff. Until we change this process, the day of universal e-commerce middleware is as far away as ever.

16 Jun 2006 (updated 16 Jun 2006 at 13:32 UTC) »

... but ultimately, in the case of GNOME, i am not particularly worried by flame-wars. GNOME is sick - very sick - and its developers should give up before they too become too sick to continue living decent lives.

halcy0n - yup. there are areas where project developers focus on "their thing" - forgetting that there are other projects out there, forgetting that they are part of a larger picture.

to date, those things which have been forgotten are, that i can recall:

* samba, wine, freedce all are non-interoperable multi-hundred-thousand-line projects which essentially do the same thing: DCE/RPC.

* those crack-heads at deadrat who came up with d-bus _without_ looking at the DCE/RPC specification (the spec for D-BUS is near-identical to The Open Group's DCE/RPC specification - with 80% of the required _useful_ functionality _removed_).

* the linux kernel not embracing-and-incorporating the oskit project and the l4linux project

* nobody funding exchange for unix - not in six years since i attempted (three times) to start the reverse-engineering (and got about 15% or so of the way there, the last time i tried).

* nobody funding xanadux (the most likely chance for a community-owned linux mobile phone) despite the fact that its successful launch would be a _major_ PR coup.

here's the thing: i agree with what you say _except_ the bit about individual developers and groups-of-developers "changing" expectations. there's absolutely no chance that one person can handle the kinds of necessary projects that will _help_ people: the level of complexity is _just_ not practical to handle "part-time" and it's too _much_ for even one "full-time" developer to do.

free software has gone _beyond_ the point where a simple "hack" will meet the expectations of "ordinary" users - and mostly that is because "ordinary" users expect "windows" and everything that comes with it - and that's a multi-man-decade development effort to "catch up" with.

with all the in-fighting and flame-wars, we (collectively) have a _lot_ to answer for.

15 Jun 2006 (updated 15 Jun 2006 at 22:54 UTC) »
web sites

ajax - the lovely, lovely ajax. _why_ is it that gmail in "standard html" mode is quicker than the already-bloatware "ajax" version??

due to the use of thread-local-storage (tls.py) which my friend found, i can now associate a database connection object with each thread.

the combination of tls.py, mod_python, ajax, CSS style-sheets results in a web site with _insane_ response time. a quarter of a second i now consider to be SLOW.

given that the content is, in a lot of cases, dynamically generated, that's just... staggering.

flame wars and focus

halcy0n, thank you for pointing out what happened with gentoo. i am giving a talk at UKUUG entitled "Linux: state of play" and it is aimed, basically, at a modern-geek-version of the parable of the talents (without the religious bullshit: just the parable).

in essence, and here i agree with the other person who sympathised with you and mentioned "scratching someone else's itch", we are techies: we _can_. other people _cannot_. it is therefore our DUTY to help others.

fuck this "i'm going to do it because i can and because it will help ME" shit.

you _are_ able to help others with your skills - therefore you _should_ help others.

remember that.

oh. and be very very sorry for every time that you deliberately obstructed progress and wasted time which could have been spent helping others.

free software ethos

my basic ethos, when deciding what to do, is "how can i spend as little effort in the shortest amount of time achieving the most for as many people for the longest time?"

that basically means that you have to _think_ about your design, _think_ about where the best compromises are between maintenance and initial development time.

it encapsulates everything that a real free software programmer should aspire to - and there's only _one_ mention of "i" in it - and it's how can *I* help *OTHERS*.


Star Dawn Rising - i recently posted this as a warning to some people who were blatantly cruel for no other reason than that they didn't want to listen.

what surprises the hell out of me is that nearly every person who reads this poem on allpoetry.com writes a comment about it. the expected response rate is about 20%: nearer to 90% is somewhat excessive.

i _got_ to get it translated into chinese.

pesca: your comment about obscure text file editing is too funny.

in a graphical environment, where there already exists an editor which DoesTheJob(tm) and provides exactly the information required - in this case kmenuedit - what the _hell_ is a program doing "bypassing" that - and worse, putting the options into a text file and _not_ providing a graphical editor for it?

that's what bugs me about superkaramba themes that run applications: providing a redundant inappropriate brain-dead version of an existing (accepted!) user interface.

however - if text files _is_ the main "control interface" - i got absolutely no problem with it.

unless it's asterisk of course, which has a config file format "programming language" that is an insane retro throw-back that combines the worst of perl and the worst of BASIC with the worst possible config file format: windows ".ini".

i gave serious consideration to adding python to asterisk: at least it's a decent programming language.

8 Jun 2006 (updated 9 Jun 2006 at 16:37 UTC) »

... good, isn't it? :) try putting superkaramba with "kroller.sez" in front of people (make sure it's KDE 3.5.1 or above that you use).

after significant periods of time using kroller (the original, by caspian) or kroller.sez, people _do_ miss it - like the MAC rollerbar, it's just incredibly in-your-face and obvious as to what you're supposed to do.

click da icon, da program run. very visual. very satisfying. very pretty. very quick.

the advantage of using kroller.sez is that it reads the user's KDE system menu options and presents those at them.

none of this crap obscure text file editing shit.

social networking site framework

well... i am pleased to say that i have a decent application framework that can be used to switch between AJAX and "plain" modes - using the same content, giving exactly the same look irrespective of the "mode". this is crucial to successfully being able to have google adsense work correctly as google adsense cannot work correctly - at all - with AJAX sites. you just can't do it! the stupid javascript that you must insert into the page doesn't even _work_.

i've had to "templatise" - by putting things into python and have a function which constructs AJAX or "plain" on-demand for the following things:

  • all hrefs
  • all form actions and all form submit buttons
  • all "subsections" of web pages (think "child windows" in a GUI app)

the hrefs i handle in AJAX mode by calling a javascript function ajax_dlink('thedivid', 'thehref... &source=thedivid'), and in "plain" mode by adding "&source=thedivid&__mainpage__=index" to the end of the href.

in both cases, the "source=" argument is stripped out _before_ the actual page is constructed; the rest of the URL is then stored in a dictionary, using "thedivid" as the key into the dictionary.

later on, in the case of "plain" mode, whenever a "child window" - or subsection page - named "thedivid" is encountered, this URL is retrieved, the "loading" of that URL is simulated - its python function called - and the content is "substituted" into the parent's web page as the inner HTML of the relevant < div /> tag!

in the AJAX case, what i do is add in an extra AJAX call into the main index page, which will cause the browser to request that "child" page to be substituted into the DOM model at the browser end.

the result is that even when someone causes a "refresh" on the page, the "constructed" page is recreated (because the history of all "subsections" is stored in that dictionary).

i suppose i could do it on the server, in the same way as the "plain" mode - but.... naaaah :)

forms, forms, forms. these didn't turn out to be as tricky as i thought, after i decided that every submit, instead of returning content, should instead do a redirect to the page containing the content.

in this way i have a way to "catch" things and make a decision about where the actual destination should be - and also strip out any further bits which should go into that dictionary of child-window-state-info.

i'm surprised that it all seems to be hanging together as nicely as it does, and i look forward to experimenting to create an automatic "frames" mode at some point, just to see if i can.

8 Jun 2006 (updated 8 Jun 2006 at 12:40 UTC) »

yes it can happen, chicago: people who have Certified you, who are "closer" by degrees to the top-level seeds than you, happen, in your absence, to also Certify other people.

then, the "max flow" algorithm "diverts" flow through those people, leaving not enough to "flow" through your "node".

what is supposed to happen is that as the number of people in the database increases, so is the "maximum capacity" of each "degree" supposed to be increased, in order that more "flow" can reach more people.

that, however, requires a recompile of the c source code mod_virgule.c.

mod_virgule has not been actively maintained since it was written, nearly six years ago.

p.s. i understand the theory, design and implementation behind the mod_virgule code very well, having done a complete rewrite into something called xmlvl (xml virgule language - before zope, dtml and xslt were well-known) and also a port of the trust metric algorithm to python.

p.p.s. lack of disk space on the advogato server(s) has in the past resulted in truncation of people's profiles. i've lost nearly a hundred Certs from other people at least once.


the trust metric algorithm uses a "maximum flow" algorithm to ascertain whether users are "certified" - and the closer you are linked to one of the "top" seeds - raph, alan, miguel and one other - the more likely that you are to receive some "flow".

so it's quite simple: Chicago isn't connected closely enough to one of the top-level seeds.

that's the way the algorithm works.


282 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!