Advogato Has Failed
Posted 24 Apr 2007 at 05:21 UTC by bwtaylor
I would like to call everyone's attention the fact that a certain well
known crank has
achieved and maintained master status on Advogato, despite having posted
several crank articles here: 910, 819, 809, and 769.
I have to ask: Doesn't this prove that advogato's trust metric system is
a failure?
Advogato user mentifex is, in my
opinion, a crank. Others, such as the link above, or this debunk
have documented his crankery better than I can. He deserves no more than
the "observer" certification here, but as of 4/23/2007 he has master
certification. Interestingly, I
tried to certify him as an observer, but after submitting my observer
certification for mentifex, it does not appear on his page. I infer that
either a) Advogato treats explicit observer certification as equivalent
to not having given somebody any certification level or b) advogato is
hacked. If b) then I'm wasting my time. Assuming this is a), then this
is a serious flaw with the advogato trust metric, since someone who
identifies an attacker is effectively silenced, in that their
certification weighs the same as taking no action.
Next lets look at how mentifex got his master certification. According
to his person page:
- badvogato certified mentifex as Master
- bratsche certified mentifex as Master
- wspace certified mentifex as Journeyer
- mirwin certified mentifex as Master
- housel certified mentifex as Apprentice
- garym certified mentifex as Master
- dlc certified mentifex as Journeyer
Four users are responsible for the master certification of Mentifex:
badvogato, bratsche, mirwin, garym. I
encourage anyone who has certified these individuals or those listed
above observer to revoke that certification. Of these, badvogator,
bratsche, and garym are masters themselves, so I focus on these three.
Badvogato has 29 certifications including 14 master certifications. This
is despite the fact that badvogato's two claimed development projects
seem suspect and/or fake. The homepage of one of
his supposed development projects is ColorForth which has
which has the "Forth" connection to Mentifex's "Mind.Forth"
stuff.
Bratsche, aka Cody Russell, has an email address at gnome.org, and a
blog there that may be legit.
Garym, aka Gary Murphy, appears to be the legitimate maintainer of Sportwire, a sourceforge
java project with actual "normal" code.
I suspect that Bratsche and Garym have simply been duped, and that
badvogato is an "attacker" working with mentifex.
Now for the real question and discussion. Given these facts, what can be
done to improve the trust metric so that it isn't so untrustworthy?
First it looks like some people are what I would call "friend whores"
and simply certify lots of people, often unreliably. To counteract this,
I'd suggest weighing certificiations less if you give more of them.
Second, as above, it's clear that there isn't much a "good guy" can do
when he sees a "crank". There should be a way to explicity negatively
certify somebody, so that others can see this and do there own
independent examination.
Remove trust., posted 24 Apr 2007 at 08:21 UTC by quad »
(Journeyer)
Move upstream.
It's hard to rate a metric as having failed when it seems so many users aren't clear on its relevance.
It's very much true in this case that an algorithm is only as good as its inputs.
Cert'ing as observer is the same as no cert, it's how you remove an
existing cert if you've given them one, so the site hasn't been hacked
(relax, take a deep breath :-)
I don't believe those Masters indicate a problem with the trust metric,
only with how it's been used. If I know A and cert them, but A certs a
crank, B, then I should consider A less trustworthy (what are they doing
cert'ing some crackpot they don't even know, let alone trust?)
So I don't think bratshce or garym have been duped, I think they just
cert'ed badvogato for poor reasons (or because it amused them to do so).
At one point in advo's life there seemed to be a habit of people
reciprocating certs, whether they'd come from a complete stranger or not
(so I suppose exploiting that could be considered an attack.)
As quad suggested, remove the trust that mentifex's cert is
based on. I have enough faith in the trust metric to postulate that his
rating could be reduced. If you can't convince badvogato, garym etc. to
un-cert him, then try to convince the people who gave them Master status
to un-cert them (chalst and nconway are regular posters who might agree
to change their certs - or they might not)
P.S., posted 24 Apr 2007 at 09:30 UTC by redi »
(Master)
looking at it, I'd guess bratsche's Master cert comes from mentifex, not the other way around.
Looking at the new front page for the journals/blogs, I see that
Advogato has 13025 users at the time of writing this. Of these, 9864
(75.7%) are rated as observers, 653 (5.0%) are apprentices, 1594 (12.2%)
are journeyers and 910 (7%) are masters.
And only one person has got through? That's not too bad really.
Considering the number of spam attacks we've had, it's a reasonable
figure and may just be an artifact rather than evidence of a failed
system. Given the probabilities, it may show that the trust metric does
work well indeed.
But my opinion is this: People here feel that the work of mentifex has
no place at this site even though it is open source (I think). Even if
most of us agree about the quality of his work, we disagree about his
right to publish it here (you think he doesn't have the right whereas I
do). But I'm so not sure he doesn't belong here, much like a person with
bizarre opinions has the right to voice their nonsense no matter how
crazy it sounds to everyone else.
As long as his work is about Free/open source software, I have to say
that he has the right to post here. After all, we may think that it's
nuts, but his posts and diary entries would be more on topic than those
of many other people here (including mine).
right, posted 24 Apr 2007 at 12:59 UTC by grant »
(Journeyer)
there's no equation for your opinion... it's better than the conceptual framework meant to model it in divergent multiplicity
metrics used for ideas instead of individuals could show us what we believe more clearly than whom it appears other believe
your analysis is correct, yet the assumption that value lies in attempting to improve its reading in this or such cases appears questionable
Doesn't this prove that advogato's trust metric system is a
failure?
That question is answered in the Advogato FAQ. I think I need
to add something about the use of "Observer" as you're not the first
person to misunderstand how it works.
While I agree mentifex is a "crank", I don't agree that he is an
"attacker" in terms of the trust metric. He has made no attempt, that
I'm aware of, to subvert the Advogato
trust metric. He has not created fake user accounts to cross
certify his own. He has not pretended to be someone other than who he
is.
He has written software in Forth, REXX, and in Javascript that
implements his ideas. He has released this software, according to his sourceforge project
page, under the GNU GPL and LGPL. By my understanding of the guidelines for certification,
that's more than enough to qualify him for apprentice level
certification. I personally find the guy annoying, disagree with his
theory of AI, and have not certified him. Other Advogato users have
differing opinions, which they are welcome to express by certifying him.
Badvogato ... the homepage of one of his supposed
development projects is ColorForth which has which has the "Forth"
connection to Mentifex's "Mind.Forth" stuff.
I'm no fan of Forth myself but I think it's a bit far fetched to
conclude that someone using Forth must be in collusion with Authur T.
Murray. badvogato, bratsche, and garym are all real people who are not
connected with Mr. Murray. I don't think it's fair to say they've been
duped. Perhaps they just have a different opinion about the value of Mr.
Murray's software. Maybe they think having a crank or two around is fun.
If you ask them, maybe they'll tell you why they certified him.
In any case, as you point out yourself, the community can get rid of him
if they want by de-certifying those who certified him. He is also a
user on robots.net and that's exactly what happened. Other users got
tired of
him spamming the article replies and eventually bumped him back down to
observer status. You're welcome to run a campaign to encourage other
users to certify or de-certify whoever you want but that sort of thing is
probably better posted in your blog instead of here on the articles page.
I think the trust metric implementation on Advogato does have a couple
of problems that need fixing but this isn't one of them.
I think negative trust certifications should be allowed at some cost
to your own trust level. I also think there should be separate
certifications for what you think of someone's writing versus what you
think of their ability to choose other people, but I'm much less sure
about that than I am about the first one.
By my understanding of the
guidelines for certification, that's more than enough to qualify him for
apprentice level certification.
Well, given that "free software projects" includes things like this
-- which, last I checked, was a Java "Hello World" program which claims to
be the start of the Ultimate Clone of Microsoft Windows -- I must say that
such a guideline's clearly not enough.
This is one thing that Freshmeat explicitly guards against:
Talk more about what your software does than about what it will do.
Submissions come in from time-to-time which say, "Right now, it prints
'Hello, world!', but someday, it will be an air traffic control system."
Cranks are ok, posted 24 Apr 2007 at 18:29 UTC by Zaitcev »
(Master)
Luke is a madman too, Mentifex is an amateur near such a titan. Anyone
wants to campaign to de-certify him? I hope not. Really, where do you
draw the line? Next you'll be coming after BerenD.
I'm more concerned that the certification seems to always pick the
highest available rank. I was an Apprentice for a few months, and this
is how most oldtimers certified me. Then one or two random people bumped
me to Master and voila, I was Master ever since.
So we pretty much evolve into a bipolar set of Observers and Masters, it
seems to me. Although, Salmoni's data seem to contradict this, I wonder
how many of those Journeyers are old and inactive accounts.
Regarding negative certs, we actually had them for a short period of
time at Advogato, but they were accompained with some fundamental math
problem. Ask Raph.
Advogato's trust metrics are not wired so their primary function is to keep out cranks. The goal that they serve is to identify the boundaries of who is and who is not in the community of free software developers, with a reasonable level of accuracy, and a very high degree of openness. If a large number of outsiders wanted to claim membership, the trust metrics would succeed in keeping them out.
You could re-wire the trust metrics in a different way, for example by having an eigenvector rating system for the assertion "user X is a crank." If this rating system were deployed, I'm sure both Mentifex and Luke would score well. Of course, Luke is unequivocably also a member of the free software community, so the trust metric as it is accepts him. Mentifex's status is more questionable, but the trust metrics tend to err on the side of inclusion, so he gets certed too.
Negative certs don't have a fundamental math problem. But the way they were tried didn't work socially, so they were pulled. It might be worth having another go, but that's up to Steve now.
I reviewed some of Mentifex's materials and played with some of his
excercises a bit online. His proposal's and concepts looked at least
as coherent and useful as some of the published works of some of the
pedigreed giants in the field.
Since you are comfortable defining mentifex as a "crank" I assume you
can point at some functioning AI concepts and software which you use as
a comparison basis to demonstrate his incompetence, incorrectness and
all around crankiness sufficient to muzzle him here at Advogato.
Some links and analysis rather than allout ad hominem attacks might be
useful in persuading your audience you are correct in your allegations.
I think I will leave my certification of mentifex as a master of open
sourcery intact while I hold my breath in anticipation of your further
efforts.
As a footnote allow me to point out that since the trust metric locally
only controls who can post or comment on articles on the front page it
is really more about interest than trust.
Do I trust you to be interesting and informative on the front page?
seems to me closer to what the trust metric is actually measuring vs.
"Do I trust your compiled code to function correctly on my home test
system." mirwin
Some links and analysis rather than allout ad hominem attacks might be
useful in persuading your audience you are correct in your
allegations.
mirwin, there are actually some links in the article above. Maybe
you can read them before rushing to the keyboard and accusing everyone of
being closed-minded elitists.
Read especially this
part of Miller's Mentifex FAQ. And this.
Some sort of..., posted 9 May 2007 at 22:21 UTC by nymia »
(Master)
Interesting to note that some statements made here can be used as a variable in a correlation. The word crank and a given username in one sentence, that's a pretty strong relation. And more, it's a valid statement that may tie some UserIDs to other statements made somewhere else outside of Advo. You never know, it might turn out to be some tin-foil hat conspiracy.
Anyway, thanks for posting this article. Thanks for Raph again for his contribution, and another thanks to Steven for his strong support.
Okay, posted 19 Jun 2007 at 03:13 UTC by bratsche »
(Master)
I haven't been using advogato for a long time, and I don't remember who that is or why I ever had added certs to him at all. zbowling on #mono just pointed this article out to me and asked me to remove his certs, so I did. Sorry for the trouble.
A rebuttal to this ad hominem attack article has been posted
at http://cyborg.blogspot.com/2009/10/trustmetric.html and also in response to falsehoods about Mentifex posted in the Wikipedia articles about Advogato and the Trust Metric. (See also the Talk pages for each article.) Please comment here if you
take issue with the Mentifex rebuttal against the slanderous article posted above on Advogato.