Designing a community driven mechanism to identify the champions of the web


(Jeremie Patonnier) #1

Hi Everybody!

I recently went into many discussion about what could be actually done in solving the issue raise in our web builders research.

The biggest challenge is to found something that works horizontally so that it fits with community building.

At that stage I wish to share with you one of the best ideas I heard so far to get feedback from you.

TL;DR: As it is difficult to identify good resources, we could frame a system of peer review to issue quality badges (worst naming ever but you get the idea) to those resources.

Such a system has many benefits for either resource holders (recognition for their creation and associated skills), resource reviewers (recognition to their ability to judge stuff related to their skills in a meaningful way), and even Mozilla itself (that would get recognition for highlighting what’s best for web builders)

However, such system is valid only if there is some legitimacy over it. In that case, my though is that legitimacy can be achieve through two elements:

  • Our research has indicated that Mozilla is trusted to identify best practices about the web so we already have a legitimacy to initiate it
  • Setting up an open system that transparently define what quality is through peer consensus is, in my opinion, the best way to strengthen that legitimacy.

I see other significant benefits to that idea:

  • As soon as we start issuing quality badges, it becomes easy to highlight resources and people; it also make things easier to build tools to search and filter the resources.
  • Such a mechanism can help creating a virtuous circle of new resources being better and better as resource holders will get an increasing incentive in following consensus driven best practices (the quality criteria used to evaluated the resources)
  • The fact that quality is defined by peers consensus will help to ensure things aren’t frozen or blocked as new type of quality badges can be created (from very generic to very fine grain) and the quality criteria can be improved/changed over time, following industry standards and state of the art best practices.
  • Peers consensus provide potential to go beyond usual web builders (a.k.a: professional web developers and designers), as soon as we provide a clear framework and associated tools, any population can use it for any purpose.

Let’s be clear, at that stage what exactly could and should be that system is unknown. Of course I can see potential issues with such mechanism as the expected benefits are highly tight to the actual designed of the surrounding framework. But, before we get into such details I would love to hear your though about that idea.

So… What do you think?

Thanks for your help and feedback :slight_smile:


(Jeremie Patonnier) #2

Hi!

I spend some times drafting a possible organization for such a system to give you an idea of what it could be.

As usual, comments and feedback are very welcome.


(Janet Swisher) #3

One of the key points I got from Building Web Reputation Systems is that you should only have ratings for things (or actions), not people. As soon as you rate people, it becomes a popularity contest. But if you rate resources that are associated with people, then people gain reputation from their highly-rated resources.

So I agree with your strategy of identifying quality resources, and letting the recognition indirectly accrue to their creators.

It might also be possible to have a collaboratively-generated set of criteria that describe quality in a resource, so that the “consensus driven best practices” can be made explicit, and therefore, easier to strive for.


(Jeremie Patonnier) #4

Yes, absolutely, I haven’t dig deep into that but that’s the idea and IMO the sanest way to handle that question.