"Outcast" Certification patch: Aims and Assessement
Posted 15 Apr 2000 at 07:45 UTC by lkcl
Advogato aims to provide a public forum in which people can be heard --
if it is considered that they have something worth saying. It is
expected that there will be hostile attacks and acts of stupidity. In
amongst this, it is also expected that there will be genuine new voices
wishing
to be heard, and they should not be excluded, or lost amongst the
noise.
This patch aims to
provide a means to cut out the irresponsible
behaviour in a psychologically effective manner.
Outcast
The Outcast
Certification patch provides a means to certify people below
Observer. If my
understanding of the maximal-flow and net-creation algorithms are
correct, certification of someone as Observer is not enough, as that
simply states a "Neutral" stance towards an individual. The addition of
an extra level at the bottom - "Outcast" - makes even the Observer
status worthwhile protecting. The purpose of the Outcast certification
is also a psychological one.
Current code
In the calculations in tmetric.c, the code currently provides the means
to certify people as "Dimwit and above" [could someone please come up
with a better name for this, it's asking for trouble :) - Guest would be
better but not by much]. Anyone who is certified as "Apprentice and
above"is included in the net-flow calculations, and their trust-metric
weight counts towards other trust-metric evaluations. If i understand
the effects of the net_flow.c max-flow correctly, the "default"
weighting is towards the lowest rating, currently "Observer".
Patched code
The aims of this patch:
1) To allow "Observers" to post diary entries, but not articles.
2) To allow "Dimwits" and above to post diary entries and articles.
3) To restrict "Outcasts" to not being able to do anything at all, and
their restricted status to be known to all.
I really need some assistance and reviews of the effects of the patch on
the trust metrics, however i think it works as follows:
1) The "initial" state is "Observer" for anyone without a link (trust
certification) to or from anyone else. However, because the max-flow
algorithms will come out with a default assessement of the lowest rating
(which is now "Outcast"), I had to include "Observers" in the
node-linking. This maintains "Observers" at their own level, even if
they have no links (trust certifications) from anyone else.
2) renaming of CERT_LEVEL_NONE to CERT_LEVEL_OBSERVER because
effectively, CERT_LEVEL_OUTCAST is now CERT_LEVEL_NONE.
Comments
1) I left tmetric_find_node() as-is. This code is responsible for
linking "Apprentice and above" as "certifiable".
2) I went through all the places that test for strcmp(level,
CERT_LEVEL_NONE) - req_ok_to_post(), proj_ok_to_edit() and
acct_person_serve(), and modified them so that CERT_LEVEL_OUTCAST cannot
do any damage, whilst CERT_LEVEL_OBSERVER (what used to be
CERT_LEVEL_NONE) can still do diary entries.
3) I added extra colours to the array for the new certification type.
Being the kind of person whose choices of colours for his kitchen were
light-purple for the walls, dark-green for the door/window frames and
burnt-orange for the shelves, i will leave the colour selection to other
people, even though i have been told that despite sounding disgusting,
those colours work really well.
Assessement
It turns out that, if i think i understand the net-flows properly, that
it is strictly speaking not necessary to have an "Outcast"
certification. It *should* just be possible to certify someone as
"Observer", and they *should* be dropped off the map (the net-flow map
as well as metaphorically). However, that isn't the case: what in fact
happens is that if you certify someone as "Observer", you are taking a
"neutral" stance towards that individual. That person is able to abuse
their trust and there is nothing that can be *actively* done to stop
them.
In the modified code, "Observer" status is actually quite privileged: it
is included in the net-flow system as your default, initial status.
Those people
with "Observer" status therefore have something to lose: they *can* be
actively busted lower than "Observer". The lowest status, "Outcast", is
equivalent to the pre-patched "Observer" status, which is now the one
with no certification, and no rights of involvement in the net-flow
calculations.
I have made a couple of tests, it is fascinating to watch the changes of
certifying someone from "Master" to "Outcast". Not only does that
person drop off the map, but all the people that *they* certified lose
the trust-metric quota allocated to them from the now-demoted
individual, too.
The psychological effect, therefore, of the "Outcast" status is very
obvious, and should not be abused. Assigning someone the "Outcast"
status will take away their rights to make modifications, including
defending their own name. If you assign someone this status, you had
better have a *really* good reason for it, otherwise other people are
likely to assign you the same status.
It is intended that those people certified as Outcast should be given a
chance to be redeemed and rescued by others, or remain crucified and
their name sullied for eternity as an example to others. After all,
it's possible that it was all a big misunderstanding, or a hostile
attack.
Future enhancements to consider
1) Anyone busted to "Outcast" has all their certifications of other
people removed from the database (but not their account).
2) Anyone who rates someone differently from the resultant certification
has the difference between the certification and their assessement
*back-propagated* to affect their ability to make certifications. In
other words, if someone is assessed as "Outcast" by absolutely everyone,
and they rate them as "Master", then automatically their ability to rate
other people is reduced.
3) Leave (2) up to people to decide, don't enshrine it in algorithms.
If someone wants to rate a total Outcast as a Master, it could be the
beginnings of a come-back for the Outcast, or it could be the downfall
of the person making the contrary-assessement.
4) Addition of "reasons" attached to certification status. This allows
other people to read *why* you assigned someone they may not know
anything about a particular certification. Some people are already
starting to use their "notes" section to do this. "reasons", if widely
used, should also help people to identify a whole slew of
hostile-attacks on advogato.
5) Time-weighted metrics. The effect of a certification loses its
effect with time, and must be renewed, or is enhanced when in proximity
to other *related* certifications of the same individual around the same
time. This will require some modifications to potentially allow
multiple certifications (with limits on the number of certifications at
any one time!) by the same individual of another individual, and also
makes it important to have (4) implemented first. This begins to tie in
with neural network research.
raph removed the "Dimwit" rating (i had to update the patch
accordingly).
i presume that this was from what i heard about people causing trouble,
abusing the "Dimwit" rating and certifying respected people,
inappropriately.
how more appropriate to apply a little bit of psychological pressure by
certifying such people as "Outcast".
Oh - something just occurred to me. An Outcast certification should
allow people to only have the following options:
- delete their certifications of other people (allow them to back down).
- commit hari-kiri.
New seed?, posted 15 Apr 2000 at 09:35 UTC by nether »
(Journeyer)
I couldn't help noticing that your patch added yourself to the trust
seed list, so inconspicuously at the end of the patch, without
comments... I do hope it had been left there just for debugging
purposes. :)
-1, posted 15 Apr 2000 at 10:13 UTC by pvg »
(Journeyer)
The meta-related discussion is on everyone's brain and providing a
modified implementation is a very concrete, constructive way to
contribute. I like this approach a lot even though I disagree with the
proposed change. Here's why:
it solves a problem that doesn't exist [yet]
There haven't been any significant 'hostile attacks or acts of
stupidity'. This will undoubtedly change eventually but it seems more
likely that a less intrusive solution can be devised then or a less
intrusive solution might already exist by then as
scalability-related changes are made.
it brings back a problem we already had
The Dimwit rating was supposed to be largely for fun but ended up
being used as a namecalling tool by participants in polarized
discussions - not much use either for content filtering or community
building. The author recognizes these problems and says the function
"should not be abused". Providing a function with a high potential for
abuse is only likely to increase abuse as the community grows and
aquires more 'abuse-inclined' members.
its aims are either met by the current system or are unenforcable
Specifically:
"To allow Observers to post diary entries but not articles"
The current system does this.
"To allow Dimiwits and above to post diary entries and articles"
The current system does this (modulo Dimwits, they're gone)
"To restrict Outcasts to not being able to do anything at all, and
their restricted status to be known to all."
An outcast can then immediately create a new account and resume diary
trolling.
a possible conceptual problem?
[I am no expert in either trust networks or graph algos so the
following may be inaccurate, raph or others with more clue on this can
clarify] If the maxflow calculation on the graph is intended to
compute the flow of, say, 'trust fluid' does it make sense for it to
carry 'negative' trust? This is a conceptual more than a mathematical
question - obviously one can chose the lowest level of trust to mean
anything but if some levels actually represent negative trust I'm not
sure the metric calculation makes as much sense. The pipes that were
laid to represent members' assesment of their open source
contributions get used to find out who receives the sewage.
random comments
I'm not sure including everyone in the net flow calculation gels with
the intent of the system. CERT_NONE is actually a good constant, it
means 'the flow did not reach you in any trust metric calculation'.
This also implies that none of your certifications do anything, even
if you are reachable from a seed. If Observers move a notch up and
actually have a real certification level of their own that means some
observer's outgoing outcast certifications affect the computation of
outcasts.
I see no point in offering certification levels below the default that
individuals get when they first register. If you have such levels,
there is really no reason not to abandon your account once you fall to
that level and start a new one.
At the moment, there is not a problem with new users writing Really
Stupid diary entries. If and when that circumstance arises, it can be
dealt with by restricting diary entries in some manner (perhaps allow
Extremely New People only one diary entry per day with a modest byte
limit so as to control trollage?). Otherwise, I see this patch as
fixing a problem which does not exist.
Allow come-back, posted 15 Apr 2000 at 13:11 UTC by kaig »
(Journeyer)
If outlaws cannot do anything at all (except to revoke their earlier
certifications of others), how can they ever hope to get out of that
status again? As it is, outlaw seems to be a permanent judgment. You
might as well remove them from the system altogether.
I just want to say that if raph decides to apply this patch, and
institute an Outcast rating, I will be the first one to apply it to
myself. Why? Because I don't really wish to belong to a community that
actively seeks to define itself by exclusion, rather than inclusion.
Think about it this way-- what the trust ratings are trying to do is
define a community. A community is a subset of all people, and thus
anyone who is not explicitly "included" is implicitly "excluded" (I hope
that sentence parses). The difference between defining your community by
inclusion and defining your community by exclusion is, to me, a great
and very important difference.
This inclusion/exclusion thing is what tends to separate clubs from hate
groups. Now I certainly wouldn't go so far as to say that an Outcast
rating would make Advogato into a hate group-- far from it. But it gets
the mindset much much closer, and that makes me nervous. Compare: there
are many feminist groups that define themselves as "anyone who considers
her or himself a feminist, and wishes to participate." Whereas most
white power groups define themselves as "people who are not
[black|jewish|etc]". These are the extreme edges of the shift in mindset
that creating an explicit "Outcast" category entails.
I don't want to start a flamewar or attack anyone. It was an earnest
suggestion, and I'm glad people are thinking about this. I just want to
suggest some of the philosophical issues involved in an idea like this,
in case anyone hadn't considered them. If I understand it correctly,
someone with no status within the system is automatically an "Outcast"
under this proposal. Would it not be better to consider everyone a
potential member (or Observer)? If someone needs to be sent a message
about their behavior, then I'd say what they really need is to Observe
the behavior of others for a while, thus re-rating them back to Observer
already takes care of the problem Outcast seeks to solve.
I hope everyone thinks about the semiotic implications of their labeling
before any steps are taken. Thanks. :-)
agreed, posted 15 Apr 2000 at 18:21 UTC by graydon »
(Master)
if the trust metric is viewed as a way to prevent trolls, I think we
already have that. you could have that with just 2 ratings:
article-worthy and non-article-worthy. having 4 or more "levels" of
trust in a system where there's only 1 task you can possibly trust
someone to perform (post a main article) is silly especially given the
fact that the root metrics are from an impersonal source.
Re: New seed, posted 16 Apr 2000 at 06:45 UTC by lkcl »
(Master)
Yikes! [well spotted: i love open source for exactly this reason].
yes, adding my own name was only for debugging: i couldn't work out how
to get started otherwise! :)
Re: -1, posted 16 Apr 2000 at 07:43 UTC by lkcl »
(Master)
hi pvq,
thanks for the encouragement about patches. i'll go over your points
(unashamedly cut/pasting to save typing :)
Re: solving a problem that doesn't exist [yet]
If there are no wars, do you disband your army? That is seen as an
invitation to be attacked, especially on sites that should know better
(e.g internet security companies; banks; government agencies; popular
services). I am working on the assumption that mod_virgule will be put
in a large-scale, hostile environment.
Re: it brings back a problem we already had
The dimwit rating serves a specific purpose, and i particularly dislike
its name. All things considered, i am not sure i like the name
"Outcast", either. If you separate the net-flow diagrams from the
certification levels themselves and the privileges associated with those
levels, and answering regarding the cert levels:
* what was known as the "dimwit" rating had privileges that allowed
article posting, diary entrying, but the certifications by anyone at
this rating were not taken into account. A more appropriate name for
such a certification level might be "novice".
I really do think that there should be categories that allow one more
"Thing", or "Privilege" per category. Regardless of names, they should
be:
* Nothing
* Diary
* Diary and articles
* Diary and articles and certification (small weighting)
* Diary and articles and certification (medium weighting)
* Diary and articles and certification (large weighting)
If you want to avoid possible immature displays of name-calling,
misunderstanding about the certifications and generally keep things
good-natured, then you pick more appropriate names to cover the above
privileges, rather than remove the finer-granularity itself associated
with the category. If you're asking me.
Re: its aims are either met by the current system or are
unenforcable
Specifically:
"To allow Observers to post diary entries but not
articles"
"To allow Dimiwits and above to post diary entries and
articles"
"The current system does this (modulo Dimwits, they're gone)"
Yes, i know. i am marginally disappointed about the removal of a layer
of granularity, here. It means that people have to go for the
"all-or-nothing" approach: removing more privileges that they may feel
comfortable with, or not removing privileges from someone that they
should. Granted, also, the name "Dimwit" also conveys a psychologically
inappropriate stigma.
"To restrict Outcasts to not being able to do anything at all, and
their restricted status to be known to all."
"An outcast can then immediately create a new account and resume diary
trolling."
There is nothing to stop this happening in the current system, either.
*However*, the offender knows damn well that their old name, their
previous, now sullied name, is up there, and is going to stay there.
[assuming that i can guess what diary trolling is]. Diary trolling
isn't going to help their old name one bit.
Re: a possible conceptual problem?
I came up with the idea of "Negative" ratings before i had looked at the
net-flow algorithms. It is not *possible* to convey "Negative" ratings
in the maximum-flow calculations (a bit like the gas-pipe or water-main
problem: how do you calculate the maximum flow of a mains, gas, sewage
system etc). OK, it might be, but it would require a rethink and a
redesign of the algorithms as i suspect they are optimised and limited
to only deal with positive (i.e. one-directional) flow.
It would only beneficial if someone can do a detailed
analysis explaining *why* this would be necessary. I _can_ think of a
reason, i may expand on it at a later date.
So, the idea of "Negative" ratings is simply a psychological one. And
as been demonstrated, here, this forum doesn't *like* psycologically
demeaning certification tags. So pick some new ones! :)
Re: CERT_NONE
"CERT_NONE is actually a good constant, it
means 'the flow did not reach you in any trust metric calculation'.
This also implies that none of your certifications do anything, even
if you are reachable from a seed."
CERT_NONE *is* a good contstant, and this is what i associate the tag of
"Outcast" with, in the patch. However, as i mention above, the cut-off
point for "your certs don't do anything" is actually CERT_APPRENTICE, as
can be seen at line 45 in tmetric.c
So, CERT_NONE (current inappropriate name "Outcast"), CERT_OBSERVER
(current incorrect name "Observer") and CERT_DIMWIT (current, removed,
inappropriate name "Dimwit") *all* do not have any effect.
*whew*. lots to write about :)
hi kuro5in,
"If I understand
it correctly, someone with no status within the system is
automatically an "Outcast" under this proposal. Would it not be better
to
consider everyone a potential member (or Observer)? "
I am glad you brought this up: _yes_ the default in this patch is
"Observer". I had to in fact make "Observer" a level above CERT_NONE.
For exactly the reasons that you specify in the rest of your reply, this
is necessary. Closing people out from any level of participation is
just ... stupid and elitist, and like yourself i do not wish to
participate in *any* community that shuts "unproven" people out, it's
suicidal for that community, long-term.
there *is* a purpose to closed-communities: to keep them occupied and to
keep them from doing harm to other people outside of the
closed-community!!!
So, if you or anyone else can think of more appropriate names for the
"privileges" listed in my previous reply, please say so, it's really
necessary.
hi kaiq,
the purpose of leaving someone with CERT_NONE up on the site is
two-fold:
* it may have all been a horrible mistake or misunderstanding that
someone ended up with CERT_NONE. certifications are not permanent: they
can be revoked. So it is possible to un-CERT_NONE someone, assuming
that they have done something worth un-CERT_NONEing them.
* leaving them up there is a demonstration: this might happen to you if
you do something contrary to the aims of the community. History is all
about forgetting the lessons of the past.
kelly, i answered your first point in other replies. the second point
(limits on CERT_OBSERVER) is a good one. we haven't seen CERT_OBSERVER
abused, yet, in this fashion, fortunately.
[update], posted 17 Apr 2000 at 06:43 UTC by lkcl »
(Master)
retired and replaced "outcast.patch". details in diary for today: lkcl
i thought i'd share some observations with you from some test
scenarios.
Create three accounts. one is a seed name, and two test accounts (test1
and test2).
Have 3 certification levels. Doesn't matter what they are called.
High, Medium and Low.
- test1 certifies test2 as Medium.
- test2 certifies test1 as Medium.
- seed-name certifies test1 as High.
- seed-name certifies test2 as Low.
[hey, i'm learning html, that's so cool!]
The way that the maximum-flow-capacity system works, there are paths
from seed-name to test1 to test2 and another path from seed-name to
test2 to test1. By test1 and test2 cross-linking to each other, they
become self-protecting.
The certification levels come out like this:
- seed at High (it's a seed-name, after all)
- test1 at High
- test2 at Medium
Why didn't seed-name's certification of test2 as Low have any effect?
Because of the cross-linkage between test1 and test2, seed-name's
certification of test1 as High *also* provides a cross-certification of
test2. The thoughts behind this in seed-name's mind are probably
something like this:
"I do not have a good opinion of test2, however I *do* have a good
opinion of test1, so aside from not having any choice in the matter, i
respect test1's opinion of test2. By stating my opinion of test2, i am
making it clear that test1 is responsible for keeping test1's head above
the water. I *could* drop test1 to Low as well, just to make sure that
test2 is *also* dropped to Low, but have too much respect for test1 to
do that."
In other words, the only situation in which someone's certification can
be moved to "Low" is when everyone *else* also moves them to "Low".
I am not entirely certain why i am posting this, other than i found it
fascinating, and i think it's useful. Also, due to a lack of a
mailing-list for Advogato, i'm happy to use the public (articles) and
private (diary) system like this unless otherwise advised.
Instead of adding a divisive solution to a problem that does not yet
exist, make the technology encourage the kind of society you want:
- Encourage personal responsiblity
- Build a feeling of people creating a text together
- Make attacks unrewarding
- Don't offer weapons to attack others