trust metric

Posted 22 Dec 2002 at 23:17 UTC by ishmukler Share This

Does not everyone agree that metric is compromised enough to just be dropped? How about we vote on it?

I think it would make things easier. In the end we all know that RMS came up GPL and Linus hacked initial version of Linux. John Dyson, David Greenman, Alan Cox (not the linux one) and a few other contributed to FreeBSD VM [which mainly account for the OS's popularity among ISPs], Theo started OpenBSD and so forth.

We don't need to dump project pages where we could see who does what. Although credibility there is also not very high. I sometimes see who calls themselves lead developers of projects that I closely follow and it makes me confused. [I usually think that leads should contribute either in form of code or moderation].

Once "trust metric" is no longer reliable, it's can no longer be trusted.

Advogato will still remain an open portal for people interested in Open Source. Not that bad. (BTW, it would allow people who are no longer interested in having advogato accounts to delete themselves)


project affiliations aren't filtered, posted 23 Dec 2002 at 02:44 UTC by rillian » (Master)

I'm not sure what you're trying to say. If your complaint is that the project affiliations aren't guarded by the trust metric, fair enough. I don't think that was the intention. They're protected from spam and vandalism by the account-level ratings. That was the extent of the intent I think.

I think the trust metric has held up pretty well. But aside from it not applying to to your complaint, I'd suggest that to assert as ishmukler has is to misunderstand how the trust metric works. The intended effect is a little more subtle.

Dropped?, posted 23 Dec 2002 at 07:24 UTC by moshez » (Master)

I'm guessing it should be fixed. Here's a way it could be fixed: allow a way of negative certification: I certify that this person is not, say, a Master. Now, have a two stage algorithm: the first stage is the current algorithm. Then, if an X has more than n certifications that he is not X from X or higher than he has certifications that he is X from X or hired (n should be fairly high: maybe 10-15), then his X rating is dropped to X-1. Then the same process goes all over the place with X-1. This has no race condition, but does mean that a "revenge" is possible and effective: if I negatively-certify foo, he can negatively-certify me -- and be effective, because at the stage negative certifications are carried, he is still considered to be certified. However, I doubt this will be a real problem: most people who are deserving of a certain status (and even those who don't :( ) have a lot of certifications.

An alternative would be to drop the capacity of links in the graph faster as their distance from the root set deteriorates: if the whole world has a 7-link property, then the free software world should not have more than 5-links, which means even very fast deterioration in capacity should be effective in stopping abuses. The current problem, as far as I see, is that far away from the root set, the situation deteriorates very quickly in terms of how high-quality the "Masters" are: in particular, the Masters I found wrongly certified (including myself -- I would think I fit only the definition of a Journeyer) are usually pretty far away, along the Master graph, from the root set.

Negative cert?, posted 23 Dec 2002 at 09:56 UTC by yeupou » (Master)

The problem with negative cert, is the fact that it can be easily outdated.

For instance, you can certify someone as not being a master, and if you no longer go regularly on advogato, this cert. remains despite the fact it may be outdated.

A contrario, when you certify someone as a master, normally, there no reason for this thing to change in the future (he's a correctly a maste).

Instead of creating negative stuff, we could propose that certifying someone with a level under his currenty level would lower his level.

Why is it broken, posted 23 Dec 2002 at 12:04 UTC by Chicago » (Journeyer)

Im sorry, perhaps I have misread something, but I dont seem to have found an argument that shows that the trust metric is broken or not. I dont belive that it is, but I do believe that it its slighty irrelevatn. My problem with it is, who is the trust from? Trust surley indicates a personal thing .... so:

I would like to see a system which generates trust as a personal thing - for example when I joined the network I allready knew people like slef, iDunno, thom and co, and therefore trust them more then anyone else

If it could be caculated that, for instancec, iDunno trusts someone called boo to degree x, then it would caculate that I trust boo to degree x modified by some factor that I trust iDunno.

Of course, in terms of programming, that would mean you had to either store the data for each user for each user and update regullary, or have to generate it on the fly (eugh).

This could be offered as an alternative to the current system, as it would use the same links I think?

purpose of trust metrics: issue of negative certs, posted 23 Dec 2002 at 13:29 UTC by lkcl » (Master)

when the "dummy" rating was introduced, it was immediately abused because rms and other recognised masters became spammed with certs.

the point of this site is to be a self-regulating site with a high signal / noise ratio.

if discussions on certs and certifications themselves BECOME part of the noise, then the benefits of the trust metric system is lost.

there are three ways to deal with noise:

- don't make any: your silence speaks louder sometimes than words: don't certify people if they don't deserve it
- dates on certs (expiry): i wish this was used.
- "interest" certification: expressing interest, ambivalence or not interested in, with the default being ambivalence. anyone or any content which has a "not interested" cert just... drops off your radar.

vote with your certs.

Research needed..., posted 23 Dec 2002 at 13:33 UTC by MikeCamel » (Journeyer)

Hi - this is an area in which I hope to be starting a PhD early next year - how trust works. The problem is that there are a several simplistic models out there (I think that Advogato has one of the more sophisticated ones, actually), and very little understanding of the sociological and community aspects of what's going on. I intend to look at this area, and hopefully provide some ideas for people to use to model real systems which are designed for the purpose required, whether that be resilience, fast turnover, security, scaling or whatever. Some of my initial thoughts are available at www.p2ptrust.org, and I certainly welcome thoughts and comments. It's early stages right now, but for a taste of the things that are particularly interesting to me (or were a few months ago!), look at the mini-essays in the trust section.

No voting and no negatives, posted 23 Dec 2002 at 20:03 UTC by Zaitcev » (Master)

I think the author forgets to account for one very important difference between the Advogato and Slapdork and its clones. Advogato belongs to Raph. He bankrolls it and provides gratis service for the purpose of advancing his research. The test subjects voluntarily participate for their reasons. Two consequences of this are: 1) Any sort of voting is out of the question, Raph makes the decision. 2) Trust metrics is [currently] the whole reason of Advogato existence; take it out and the site can be closed down because it would serve no purpose. In contrast, Slapdorks are commercial enterprises, fed by ads.

Secondly, negative certs were tried and proved extremely destabilizing and destructive. Therefore, they were not to be used again. Even the diary threshold is destructive. For instance, when calculated from my root, the Samba Luke gets rating of 2.3, while his diary is by far better than others in that range. Obviously, it's a revenge and spite. Again, if Raph wanted to encourage mud slinging, rivalries, ostrakism and stoning of outcasts, he would be free to do it. But my reading of his intentions excludes that. It seems as if he is trying to set up rules of the game such, that a viable community could emerge. Thus, I do not think any sort of negative ratings are on the cards.

The inflation of all-positive certs is a fact (for example, I am a Master for some reason), but I do not see if it is a problem, as far as the community survival is concerned.

Fixable, posted 23 Dec 2002 at 22:10 UTC by ncm » (Master)

I suspect that the web-o'-trust system could be fixed with just one more knob:

Next to each certification in the list, we might have a checkbox that says "this person appears not to follow the guidelines when certifying others". Arrange for it to weaken their certifications, and maybe even (with enough blackballs) to negate them. At the bottom, add a button to send them off.

If somebody blackballed removes the blackballed certifications, their certification strength would recover. You'd want to keep the blackballs around, though, in case they remove a cert and then add it back again. Of course, blackballs from somebody blackballed ought to count less.

This would help to prune out the network of fakers and wanna-bes who have got themselves over-certified and have run around certifying even more fakers and wanna-bes.

With some of the recent article postings and their (!) defenders, I have found Advogato suddenly a lot less interesting than it once was. This site used to be for and about Free Software and the people who create it, not just People Who Have Opinions about It. I hope Advogato can be fixed.

interest cert - more details, posted 23 Dec 2002 at 23:19 UTC by lkcl » (Master)

think carefully about the implications of the following, whilst excusing the grammatical inconsistencies in the certification names:

- an interest cert, with levels "fascinating", "interesting", "ambivalent" and "no interest".

- a default interest cert level of "ambivalent"

- if a "no interest" cert is applied to a person or an article, that certification is NOT publicly displayed.

the same logic can be applied to any cert type with a... "below-default" level of certification.

it is also useful to be able to make an individual person (one and only one) under user-selection/control the "seed" from which a trust metric is calculated, and to make the results of that calculation available for use (and for personal use only, not as part of the main site) by the user.

the implications of this form of certification usage, to me, are as follows:

- SPAM negative certifications are not encouraged because they don't have a negative social impact: they only appear on the spammer's browser, not anyone else's.

- the "Interest" cert metric-calculation, from a user-selectable viewpoint, becomes a really powerful search engine.

advogato is fine [slightly off-topic post], posted 23 Dec 2002 at 23:29 UTC by lkcl » (Master)

mr ncm,

the purpose of a forum is for people to debate and discuss.

the purpose of advogato is for open source developers to debate and discuss free software, with the sub-text that said people be "advocates" of free software.

if you read between the lines of my articles more carefully than you have, you will find that in fact i am advocating free software, and have chosen to do so in a way that i believe you have totally misunderstood.

if you look carefully and impartially at your own posts, you will find that they should absolutely horrify you: they have been of the calibre "i don't understand", "i hate this person who waltzes in and accuses without justification", "this person is so boring he shouldn't be allowed on this forum".

now i have spent quite a lot of time and effort thinking about those articles and writing them up, for a specific purpose, as preparation, after a significant break from writing on advogato.

if as a result of my articles you find that you are losing interest in advogato, then, to be absolutely honest, and given that the quality of the forum's postings has degraded - by your own hand - then if you chose not to post on the advogato forum, the quality of the postings would improve, yes?

so, why instead, both you and mr ishmukler, instead of advocating a change in the system that allows you to silence people's voices and opinions, simply shut the **** up?

Re: Fixable, posted 24 Dec 2002 at 01:40 UTC by ishmukler » (Journeyer)

Seems that to become fixable entire metric tree must be audited. (by Raph?)

Why aren't you a Master?, posted 24 Dec 2002 at 06:26 UTC by clausen » (Master)

ishmukler: if the trust metric is so bad, why don't you make yourself a Master? Even if you achieve that, then try to make your family all Master as well.

If you can't do these things, then the trust metric has value, IMHO.

Some people think advogato's cert's are inflated. My view is the scale is too poorly defined. Some specific examples of who wouldn't qualify for Master and Journeyer status would be helpful.

No, posted 24 Dec 2002 at 07:13 UTC by deekayen » (Master)

I do sometimes get annoyed that people I don't think should be certed what they are, remain certed from people who haven't even visited the website in more than a year.

But then I think, they were worthy of certing at one time just as students are worthy at one time of receiving a diploma. Just because they take that diploma and do something distasteful with the knowledge doesn't mean they're still not worthy of having the diploma 10 years later.

There are exceptions to that that you could probably work into the metric, but then where do put the limit. Start with negative certs, then there'll be something else until the metric is so complex nobody wants to use it.

The metric is not perfect, but it fills the basic needs of trolling.

Too much dependance on Cert, posted 24 Dec 2002 at 11:02 UTC by Chicago » (Journeyer)

Zaitcev said:

I think the author forgets to account for one very important difference between the Advogato and Slapdork and its clones. Advogato belongs to Raph.
Im not sure that everyone takes the ratings as seriously as people on this fourum do. It seems that there is a tendancy to cert towards journeyer, and that even though most of your votes might be apprentice, once a journeyer, you're stuck as one. Personally I would like my lime-green colours back.

Raph might have formed Advogato to do that, but I think many people just use it as a discussion/blog for Computer related systems. I make a note of what people have to say even though they may not be a master. A master rating dosnt nessicarilly mean they are nice talkative, receptive people, meerly they've been in open source for sometime. It signifies a foginess if anything.

I listen to people other then masters, right down to and including observers, if not because if I ignored them then Im ignoring a point of view which might be the answer to my problems, then because that next year they might be masters, and that I hope people (including masters) listen to me and hear my ideas and opinions.

We're all people, its just some think they're more important then others.

academics, posted 26 Dec 2002 at 16:19 UTC by lkcl » (Master)

I listen to people other then masters, right down to and including observers, if not because if I ignored them then Im ignoring a point of view which might be the answer to my problems, then because that next year they might be masters, and that I hope people (including masters) listen to me and hear my ideas and opinions.

fantastic to hear someone else say this: it brings us full circle. academia is a classic case where there are experts who need both access to people of the same calibre as themselves and also need to be able to listen to "in"expert advice - just on the pure offchance that they may be of assistance.

two ways in which "in"experts are of use to experts: one, by explaining things to them, the expert needs to be clear and divulge themselves of preconceptions. two, the "in"expert may actually have fresh insight that the expert does not.

the whole point of advogato is to act as a filter, reducing the amount of noise, and also to give a VERY rough guide as to the likely value of a contributor's input.

not that anyone should get carried away (rudyard kipling's poem If).

the risks are that people get too big for their boots, or the over-exposure of experts to non-experts turns the experts off even wishing to speak to non-experts. the arrogance thing again...

does it matter?, posted 27 Dec 2002 at 00:46 UTC by Malx » (Journeyer)

There is overall usability of site.
It includes all - engine, skin, hosting, people, cntent, topic (targeting), filtering mechanism etc.

So it really doesn't matter what are filters (certs here). And if this site still exists then it is done well.

Certs should expire, posted 28 Dec 2002 at 01:02 UTC by djm » (Master)

I agree that there are many over-inflated certifications (e.g. mine) and also agree that the s/n has dropped sharply. I think a solution would be to force the expiry of certs, perhaps annually. This would force people to continually reaffirm (and hopefully rethink) their certifications.

I'd also like to see the "interesting" rating apply to the articles as well as journals - so I can killfile articles from known trolls (IMO, of course).

mailing lists, posted 28 Dec 2002 at 12:20 UTC by lkcl » (Master)

http://www.crackmonkey.org/mailman/listinfo/virgule-dev

is the virgule development list.

whether that's to do with the code hosted on cvs.ghostscript.com by raph i don't honestly know.

2 djm, posted 29 Dec 2002 at 01:02 UTC by Malx » (Journeyer)

Certainly you are not deserve to be graded as Master. If certs whould expire there whould be another "MyAdvogato" software, which will automatically renew certs ;)
You just couldn't persuade programmers to do same task forever manually ;)

2 djm, posted 29 Dec 2002 at 01:03 UTC by Malx » (Journeyer)

Certainly you would not deserve to be graded as Master. If certs whould expire there would be another "MyAdvogato" software, which will automatically renew certs ;)
You just couldn't persuade programmers to do same task forever manually ;)

Negative Certification Day, posted 29 Dec 2002 at 09:49 UTC by chalst » (Master)

Raph doesn't want negative certs because they will spoil the good community atmosphere that exists here at advogato. But even the unfriendliest criticism can be healthy once in a while, so why don't we have a day each year where negative certifications can be made and seen?

Values and value systems, posted 31 Dec 2002 at 04:21 UTC by Nygard » (Journeyer)

Jim Coplien made a very emphatic point at OOPSLA 2000. There was a fishbowl talk about whether "Worse is Better" when it comes to software.

Jim's point was that you cannot assign a value to something independent of a value system. In other words, to "evaluate" means to judge something against a particular system of values. When we are talking about software design, one person may value "simplicity", while another may value "elegance", and another may value "cleverness". Each of these three people could look at the same code (good or bad) and assign very different values to it.

A singly-rooted web of trust assumes that every member of the community shares essentially the same value system. That is, they would be likely to evaluate a person based on similar criteria.

As a community grows, however, it becomes increasingly probable that subsets of the community will maintain different value systems. They may even agree on several dimensions of values (major axes), but have different minor axes.

I suspect that we are seeing a demonstration of this effect. There are subsets of users which have different value systems than the original, widely-held community values. (This might even be visible in the trust graph as clusters of certs that are not well-connected back into the core.)

In the limit case, each individual member holds his/her own distinct value system. In such a case, it makes sense to maintain a separate web of trust for each user, with that user as the root. There may be an intermediate case, in which the clusters of users with similar value systems may be represented by an "avatar" or "supernode" who is considered to be the root for that cluster. (This is similar to the collaborative filtering techniques applied by companies like NetPerceptions.)

The trust metric seems to be effective, posted 3 Jan 2003 at 21:41 UTC by Mysidia » (Journeyer)

One can always claim anything is imperfect or could be improved. What you perceive as issues may exist, but they may be relatively small compared to new issues that would arise in trying to resolve them.

Adding things like more filtering, expirations on certs, negative certs, etc, means that your system for preventing abuse, for filtering against noise itself begins to become a burden as much as the noise is.

Because I perceive the metric as being as effective as it is (not being plagued by annoying crap floods), I dare say that making the system yet more aggressive would in all likelihood tend to be more obtrusive and annoying for site users than the residual noise you're trying to block.

different seeds for different purposes., posted 4 Jan 2003 at 21:40 UTC by lkcl » (Master)

In the limit case, each individual member holds his/her own distinct value system. In such a case, it makes sense to maintain a separate web of trust for each user, with that user as the root.

hey, that's very cool to hear someone else talking about this: when i was working full-time on virgule, that's exactly what i did.

by viewing a user's individual page (/person/username) you could see things from the perspective of that person, with them and only them as the root.

additionally, by viewing an individual project page (/proj/projectname), you got exactly the same thing - with the Project itself treated as a user, and therefore the project's developers as the top-level seeds.

the code is still available - http://sf.net/projects/virgule - and has two example sites, xmlvl.org and dcerpc.net.

Why can't the metric just scale?, posted 6 Jan 2003 at 02:36 UTC by Radagast » (Journeyer)

Reading this, it occurred to me that the metric could just scale, that is, the total it requires to become a master increases as the top score increases, so that "Masters" are, say, the top 15% of the Advogato user mass, and so on.

This might be unfair because someone getting lots of certifications would cause people close to the limits to drop in certification level, but the only certification level that means something is observer/apprentice.

But if it was done like this, it would at least nullify the effects of inflation.

Network flow, posted 6 Jan 2003 at 21:36 UTC by lkcl » (Master)

radagast,

it helps to understand that the trust metric system works by doing a "maximum flow" calculation. i.e. like gas pipes.

nodes that are one degree away (i.e directly connected) from the supersink (i.e. raph, miguel, federico and alan) get a flow capacity of (iirc) 800, one degree away from them gets a maximum capacity of 200, three degrees is 50, four is 12 etc.

the capacities _do_ need to be recalculated manually and the source code recompiled [which is a bit silly, but let's ignore that for now] as the population increases.

if any node gets _any_ flow through it, then it is automatically "Certified" at that level.

the second thing that you need to understand is that there are four SEPARATE maximum flow calculations performed, i.e. with four SEPARATE node graphs.

each of the four graphs corresponds to a Certification Level: Master, Journeyer, Apprentice, Novice (Novice? or have i got the other code from xmlvl stuck in my head?).

a user is entered into one of those four graphs and all graphs below the level at which they receive Certifications from other users.

so, a user that receives an Apprentice Cert, a link is entered into the Apprentice Graph _and_ the Novice Graph.

so, with that background in mind, i can now answer the issue you raise: the Trust Metric system _does_ scale automatically.

even if everybody Certified everybody else as Master, the Maximum Flow through the "Master" graph would run out of steam at about the fourth or fifth degree from the supersink, if there were literally several hundred thousand nodes.

there can only be a finite number of Masters, a finite number of Journeyers, a finite number of Apprentices, and it's defined not only by the number of Certifications but also the maximum amount of "flow" capacity that the "network" can hold.

the other really neat thing about using weighted graphs, which was the whole point of the exercise, is that self-certifying hacker/spammer/attacker groups that have no incoming Certs linked close enough to the top-level seeds will get no "flow" and therefore no Certification.

Hierarchical, self-regulating Access Control.

fantastic stuff with far-reaching implications and possibilities - absolutely love it.

... which brings us back full circle: Certifications themselves have absolutely nothing to do with whether people like hearing what the Certified have to say ;)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page