Wikipedia:Wikipedia Signpost/2011-02-28/Arbitration statistics
Arbitration Committee hearing fewer cases; longer decision times
Case duration: from open to close
The analysis of case duration counts the number of days it took for a publicly heard case to reach a motion to close, after being opened for evidence submission and workshop input. The graph averages all cases closed during a specific two-month period. This means that a spike in case duration for September and October 2008 probably accounts for cases opened several months earlier.
Starting in January/February 2008, we find an average of 30 days for cases to be resolved. In these two months a total of 12 cases were closed, all of which were resolved in less than two months. The duration of cases in days steadily increased during each two-month interval (climbing from the 30s to the upper 40s) until it spiked in September/October 2008 with an average of 68 days. This can be accounted for by two cases (one which took 131 days and another taking 99). But 2008 saw another long case. The well-known Eastern European case took 115 days. Not only that, its evidence page had submissions from nearly 30 editors, in stark contrast to a case like John Gohde 2 which was resolved in less than two weeks with a third of the number of evidence submissions of the EE case.
Thus, with the first year of data reviewed we can see a clear (and understandable) correlation between large complicated disputes and the length of time to decide a case. An example of this issue is the Scientology case, which took an astounding 169 days to resolve. But the bigger question is not whether these long cases will always happen, but whether this is becoming a trend in regard to all Committee proceedings.
Going back to 2009, we see two cases at the start that already took longer than the longest average for the first half of 2008. For the entire year, only three cases took less than a month to resolve, and the duration of the shortest case approached the higher average of the year before.
While there was a decline in the average duration of cases at the beginning of the year 2009, that average remained consistently higher than for the first half of 2008. With three complex cases resolved in the middle of the year, the average jumped to 72 days. With another election on the horizon, cases were gradually resolved more quickly as the number of days dropped to the 40s by the end of the year.
The year 2010 then becomes intriguing as the number of very large cases began to decline, and the average sank (in March/April 2010) to 33 days, the lowest since the beginning of 2008. But this trend has not continued. By the end of the year, the average jumped back up to 70–80 days, with the longest case at 123 days.
From this data, as the trendline indicates, the duration of cases has significantly increased since 2008.
From 11 to one a month: decline in opened cases
The data on the number of opened cases each month was calculated in a very simple counting of when cases started. The trend here is easier to discern than in the first graph: a steep decline in the cases the Committee hears each year.
By today's standards, a surprising number of 11 cases were opened in January 2008, at the start of the graph. Comparing The Signpost's archives before 2008 also shows this high number of cases resolved by the Committee. This number very quickly dropped to only three accepted in February, and to only one or two a month by the middle of the year.
The numbers then remained in the same range throughout the whole of 2008. The only times the number of accepted cases a month rose to three or more was at the start of each year, when a new slate of Arbitrators came in (shown by a rise in January–March 2009 and the acceptance of three cases in February 2010). After August 2009, the Committee has always accepted one or two cases a month, with only one exception.
In December 2009, the Committee accepted no cases, for the first time in at least 23 months (my record goes back to 2008). Again, for four months between July and October 2010, ArbCom did not accept a single case. A deeper look at records in this long "no-acceptance" period shows that only one arbitrator voted to accept in one of the more than 10 declined cases.
The Signpost's weekly coverage of ArbCom cases is interesting to compare, as it exemplifies a dramatic decline in decisions before the Committee (2006 coverage, 2007, and 2010).
Comments from the Committee
Arbitrator Newyorkbrad says that the declining number of Committee cases is due to a greater reliance on community mechanisms to resolve disputes, such as community sanctions at the ANI board and through the blocking of a user by a single administrator, without a disagreeing admin unblocking. "Today, we get relatively few of those types of cases, and when we do get one, it's typically because there has been disagreement among administrators as to how to handle the situation." He pointed to the complexity of the issues that do make it before the 18-member Committee, which is arguably a cause for the length of decisions. While public perception may be that the Committee is too slow, Newyorkbrad says they always "[try] to keep the case lengths reasonable." In the Signpost's interview with several arbitrators last October, Roger Davies had made the same point about the changing nature of the cases ("these days, it's mostly intense, hard-core disputes that end up with ArbCom—the things the community isn't really set up to handle"), and Kirill Lokshin, while observing that the ArbCom "has traditionally been quite bad at keeping to deadlines", had noted that "an ever-increasing fraction of our workload consists of 'behind the scenes' work" that is not visible in the formal proceedings.
Looking into the future
Whether due to an evolution of the dispute resolution process on Wikipedia, or just a tightening of the requirements for the Committee's acceptance of cases, or (partly) due to a decrease in overall editing activity, the trend is unmistakable. But what does this really mean for Wikipedia and the Dispute Resolution process? In a few years' time, how much work could the Arbitration Committee have? We can only speculate as to the long-term effects.
Discuss this story