Round-balancing proposals
From DPCanadaWiki
As of late March 2007 there was an an awful lot of discussion in DP-INT forum threads suggesting various ways to balance the rounds and solve the problems of our queues. This is a minute-taking page, an attempt to keep track of all the suggestions. Please add to, clarify and reorganise as you wish. But please only minute-take, that is, list from a neutral point of view the elements of the proposal and the comments for and against that are in the forum threads. Actual discussion should take place in the forum threads themselves. Some of these links do not work, as they refer to topics in the DP-INT fgorums.
Second Pass
Proposes allowing books to receive P1 > P1 > P2 and then proceed to F1.
Objective criteria for P3 skipping
Mainly concerns diff-count as a criterion for P3 skipping.
Aim
Reduce P3 Queue
Forum Threads
forumtopic:25583 and related forumtopic:23149
Background
This proposal for round balancing works with the idea that the diff counts of a project between the various P rounds offer an insight into if a project is a potential candidate to skip P3. The thread is an exploration of the criteria that a project would need to satisfy in order to skip P3.
Diff Count Score
This proposal makes use of a 'diffs score'. This number is simply means of expressing the average number of errors corrected in a round.
As per the example in the thread, a diff count of 9.84 in P2 means that on average, proofers in P2 found one error every 9.84 pages.
Details
Proposal 1: A project is allowed to skip P3 if its Pages/P2diffs score is 7 or more.
- In Plain English this means that a project would be allowed to skip P3 if in round P2 proofreaders found no more then one error for every 7 pages on average.
Proposal 2: A project is allowed to skip P3 if all of the following are true: Its Pages/P2diffs score is 4 or more; its Pages/P1 score is 0.5 or more; and its P1diffs/P2diffs ratio is 1.5 or more.
Critiquing
Positive Aspects
- Reduces P3 Queue
- No PP commitment required
Potential Issues
- Does the average page count of proofers need to be considered before allowing a project to skip P3?
- Does the time proofreaders spend on a page impact the quality of their work, and does this need to be considered before allowing a project to skip P3?
- Do the error rates need to be correlated to page sizes to make them more equitable?
- Are there too many ‘click through merchants’ operating in P2 that result in inaccuracuate diff counts?
Further Questions
Some of the following suggestions don't specifically relate to the above idea, but have been included as this section is a report on the thread discussion.
- Should there be testing groups of 50 pages in the same project to gain a better idea if the diff levels are consistent throughout the project before allowing them to skip?
- Do these criteria only get applied to fiction books or all books?
- Could a further criteria for allowing a book to skip P3 be obtained from sending a representative samples of its pages through P3 and then deciding its suitability to skip based on the results?
- Does a method need to be implemented to allow all but some pages (nominated by a PP due to those pages being complex) to skip P3
Actions to date
- User laurawisewell collated original set of data that set proposal in motion
- User rfrank has built a P3 skip tool analyse diffs. (Suitability of this tool questioned further on in thread)
Possible Future Actions
- A tool to judge whether a project has met the criteria to skip
- Willingness of PPers to Post Process these projects
- A way to put a project back to round P3 after F rounds at the request of a PPer
Status
This post by JulietS describes the most recent status of the project, as outlined in the thread.
"Just so folks don't think that nothing has come of this discussion, rfrank has built a P3skip tool that looks at both number of diffs and how fact the proofers went through to arrive at a score. He's skipped a bunch of his projects on the basis of this analysis and will be reporting back what he finds.
I've run most of the projects that I have waiting for P3 through his tool and posted the results in his tools thread. I plan to do a little more analysis on the most promising prospects, then advertise for some PPers who are willing to commit to the projects for skipping P3 and then reporting back what they've found. I know tenaj had volunteered for similar duty, but since she is away for a couple of weeks, I'll see if I can find other people. It will take awhile for these projects to get through to PP and for us to get results."
Adaptive sizing of work in rounds
Background
The belief behind the discussion in this thread is that generally speaking the more projects available in a round, the longer it will take projects to progress through the round. This thread explores DPers thoughts on reducing the amount of material available in the rounds.
Proposal Details
Cut the available work (measured in bytes of text) in a round by 50% for a month. And measure the relative speeds of projects. Do this for P1, P2, and F1.
Reactions in Favor
- tenaj - "If cutting down the amount of work available would help move books through, I would support it. I seriously doubt that proofers would complain about choice so long as there WAS a choice. So it might be a viable thing to do."
- Janet"I'm in favor of reducing the number of available projects in the rounds. I am so overwhelmed by how many books are there!"
Reactions Against
- stygiana- "when you take choices away from volunteers, you will lose some volunteers. Having 200 projects available in a round where over 1000 people have worked (at some time) is not exactly unreasonable."
Further Questions/Considerations
- stygiana- "How would you reduce the available work by 50%? Would you cut genres or languages? No matter what you eliminate, you will offend someone because you are taking away choices."
- spiegel1428- "I'd say genres. LOTE queues don't necessarily behave the same as English."
- tenaj- "One other thing which might help movement of books would be to stop having PM individual queues."
- fvandrog- "As for most things on this site (and live in general) there's an optimal balance. The extremes are of course rather clear: allow one book per round, and it will move fast.... but lots of people won't find the one book available at any given time interesting; allow, say, 2000 books per round and probably most people will find something of their liking to proofread.... but most books will not move with an impressive speed.
The optimum will lie somewhere in between." - crb11- "Two goals I think we should have:
- a proofer should be able to go to one of their genres of interest and find something they want to proof.
By "genre of interest" I mean a genre queue for which they will be happy to proof the bulk of books.
Implication: each genre queue should release at least three books. This allows for one HARD (or other PPD-monster) and one book which is normal but just something the proofer isn't interested in, without the proofer being blocked from the round."
Possible Actions
- JulietS"I can certainly reduce the number of books allowed in a round for each English genre for P2 and P3, which would have the effect of reducing the amount of material available overall. There's no point in changing anything in P1 because almost anything in English releases right away unless it is blocked by same-author constraints.
I sincerely doubt that reducing the number of projects in those rounds will increase the number of pages done each day. It seems unlikely that doing the same number of pages across fewer projects would move more projects through, but intuition can be wrong."
Challenging our assumptions about queues and control
Advertising the forum
Special/birth/other day projects managed by PFs
Request list for P2
Defining goals and measuring progress
Math-verifiable, minimal-code Queue Growth Reduction option
Proposes a system of points that limit how many pages a proofer can do in popular rounds without proofing any in the less popular ones.
Motivating people to work in higher rounds
Eliminate P2 queue and stale diffs
Suggests a mixing of current P1 and P2, in which all would be listed on one page, and pages would be available in both states.
Brainstorming: P1 is for experts
Proposes that a book should not be proofed first by beginners as now, but by Experts first, then Beginners, then Excellent proofers, where Beginner/Expert/Excellent correspond roughly to our current P1/P2/P3 proofers.
Proposal: kudos points
Number of pages in each round and queue
Not a proposal so much as a question, but relevant to the other proposals.
Encouraging P2 and P3 proofers
My adventures discovering the P2 requirements
Proposes (presumably) that the requirements for access to different rounds be made easier to find, and that out-of-date links be removed.
The simplest useful site statistics you can think of
Asks what simple statistics would persuade a P1 proofer to work in P2.
