September 11, 2009
From DPCanadaWiki
Volume 2, Issue #46--September 11, 2009
(Full listing of previous issues is available at DPC Newsletters)
Important Announcements
Simon Says...
First, the not-so-good news--I cannot give you a clear projection on when we will be able to implement "synchronization" with the DP-INT site code. I have spoken too soon several times in the past, only to be frustrated by some obstacle I didn't even see, so any such forecast would be highly suspect.
Instead, I'm going to summarize the process we've been going through, give you a snapshot of where we are currently, and mention the remaining tasks to be completed.
When we started, we quickly discovered that there was no agreement to the methodology we proposed to use--the Squirrels at DP-INT and the various "Powers That Be" imposed conditions on the synchronization program. They were concerned that we would simply "match up" with them, then abdicate any further responsibility for code maintenance to them. That was not our intention. We simply wanted to escape the situation where we constantly had to patch the DPC site code, because it was out-of-date, and no longer supported.
Of course, we also wanted to obtain all the improvements, bug fixes and new features introduced into DP-INT site code since 2006--the date of our site code version.
After extremely long, and sometimes wandering, discussion, we agreed to use an approach called "gettext". This essentially involves inserting a series of markers into the DP-INT site code that are invisible to users, but alert the gettext utilities to consult a set of "look-up tables". Whenever the browser encounters a marker, it looks in the designated table for the string to substitute for the one currently between the markers. This enables administrators to operate multiple versions of the same site code, for what is called "localization" purposes.
This could be merely a series of changes to implement a local version, or it could involve full-blown translation to another language.
In our case, we would substitute "DPC-speak" for "DP-INT speak"--adopting our chosen spelling, grammar, vocabulary, and also the text content that makes DPC unique. But the underlying site code would be exactly the same as DP-INT--we would have all the same routines, methodologies and features that they have, and DPC would remain "in synch" with DP-INT from that time onwards.
In order to make this feasible, we had to change over to operate our own server, and what is called co-location services--so that we can make all the decisions about server configuration, operating systems, and implemented software. This has now been done.
The next step was to go through every page, template, form and include file in the DP-INT site code and mark every text string that could possibly ever need to be be substituted or translated. There are over 500 such pages to be marked up, with literally thousands of text strings. And there was a constant need to carefully distinguish between text strings and variables, which must not be touched.
This process has, at long last, been almost completed--though, until we finally do the synchronization, there is a need to constantly update our tagging whenever anyone makes a change in the DP-INT site (and there are multiple developers working all the time).
BUT, we were also required to implement several dozen types of "related" changes as we proceeded. These essentially involve cleaning up errors, minor glitches, and even poor programming styling that had crept into the site code over the years. Please recognize that the site code is an amalgam of different tools, coding styles, design schema, and time-related points of view that has "happened" over a period of nine years. And, frankly, the documentation of the code is just what you'd expect from such a development stream. We got lost--a lot!!
But, this too is almost complete. All such changes need to be reviewed by a Squirrel, then "committed" by a Site Admin at DP-INT. They are volunteers, with their own priorities--we cannot force the pace.
The next step (which I hope to begin next week some time) is to compare the Test version of the site, thus modified with gettext markers, with the existing "production" site--to make sure that there are absolutely no changes visible to the user, and no "introduced artifacts" or errors--in other words, that the DP-INT site still works exactly as before.
Once that's been done, DP-INT could, in theory, move the code into their production server and operate it daily--but the timing of that is up to them, and no concern of ours. All that matters is that, from that time onwards, every change made to DP-INT will also be made automatically to DPC.
Then we have to create .op tables. These are look-up tables that tell gettext what to substitute for each marked text string. This is a long, but fairly simple, process, and completely under our control. Again, some of this work has been done.
Later, we will create additional .op tables for each language we want to translate the DPC site into. Of course, first among these will be French, and we will be recruiting our Francophone DPC members to carry the ball here. The job will vary from translating short, simple strings to re-working entire paragraphs, to ensure DPC's French version is accurate, modern and comprehensible French (including every text phrase, item name, caption, and form header in the site). The "switch" to French, for example, would be accomplished at the user level, by a setting on the "Member Preferences" tab of the main menu bar.
Next we have to set up the actual tools, programs and configurations on the new server that will operate our new site. These will replicate the operating environment used by DP-INT, with one exception. We are using a comparatively new technology, called virtual machines, that essentially allows us to operate as many separate computing processes as we like, with each one being independent of, or integrated with, the others. This approach gives us the flexibility to operate DPC, a book/scan archive, and the "display site" we plan for the future, each with a complete set of support tools, and multiply redundant archiving and backup procedures.
Finally, the whole shebang needs to be assembled and integrated--this will involve a further period of down-time, with notice--and followed with on-line testing and shakedown, with participation by all of you.
As I said, no prediction of timing, but you need to understand that all this is being done by people who have many other demands on their time, including the operation of active consulting practices.
On a constructive note, perhaps those of you who are very familiar with the DP-INT workflow could put together a list of operational features/procedures we will need to follow once we've "synched", that our newer members may not be familiar with. The obvious example is WordCheck--exactly what do we all need to do to get this going? Yes, a PM has to create a "Good Word" list, and maybe a "Bad Word" list, but what else?
By the way, does anyone know: how many words could a WordCheck check, if a WordCheck could check words?
A quick update on the UberProjects I'm involved in:
Jesuit Relations--I will soon add a fourth volume to P1; the first volume will come to my hands for PPing shortly, so we'll have a lot of styling decisions to discuss in the UberProject forum; the Relations seem to have "stalled out" at DP-INT, due to lack of PM focus, and no-one to commit to PPing them--they have 10 volumes in process, but little happening.
Graham's Magazine--I have just started another Volume of 6 issues from 1852, and am tracking down several other volumes, since everyone seems to like them; 7 issues have been posted to PG-INT so far, with 3 more in PP; you'll note that I have used larger 2-column scans this time, to cut down on OCR prep time--please let me know if they're too hard to proof from--if you get stuck, I can make the original pages available somehow for reference
Scientific American--I sorta ground to a halt on this one--but I may try the multi-column scan approach for proofing, if that works OK on Graham's--it's the prep time that holds me back--again, the ebooks get posted to PG-INT, because PGC isn't too enthusiastic about them.
Editorial
Calling all CPers--De2164 is looking for more projects for P1: Waiting. He has lots of biographies; he needs other types of projects.
DP-Int denizens ... If you can assist Simon with suggestions for workflow/operational procedures--as noted above, all of DPC would be grateful.
If you have any skills in languages other than French or English--even if they are only rudimentary--your help in P1, P2 or F1 would be greatly appreciated on LOTE projects. We are hoping to rescue a few projects from DP-Europe since they are having problems there. I don't know where such projects will start but any help you can give will be gratefully accepted.
Even if you have no skills in other languages and if the text uses Latin characters, doing P1 proofing of LOTE projects is relatively easy since you are matching the text to the scan. P2 and P3 proofers will check spelling and printer errors. And F1 is even easier since you are just concerned with the formatting not the actual words.
PLEASE NOTE: The "new" FAQs and non-English Guidelines, including the Proofreading Guidelines and Formatting Guidelines, that you can access through the new FAQ Central page contain DP-Int content. These are in the process of being revised for DPC.
Proofreading and Formatting Guidelines--Simon and IonaV are working on Version 2 for each of the Guidelines. They will be much closer to what DP-Int's currently are; though we'll still have our basic "match the scan" policy when it comes to things like ellipses.
Server and site problems--We've had our new server for a month now and, naturally, some problems did crop up though several of our old problems did disappear. Our updated site code is coming soon as you will see from Simon's article above. If you do see any major problems, please send a PM to Simon to let him know. Or post in the Code Update Aug 2009 - reports of possible issues thread.
Checklists--Several months ago, Simon was producing basic checklists for DPers--though he has taken a short break while the site code is being updated. He started with PPing basic checklists for simple projects: he has done checklists for startup, text version, HTML version, HTML Tables, "illustrations"--for PPers and CPers and CSS. The final item on Simon's PPing list is "tools for PPers": sounds interesting. He should be back at it once the site upgrade no longer occupies his time.
Simon has created a wikipage (linked to the main DPC Wiki) for the Checklists for New PPers. He has included the checklists that he has done so far. You can see from the Table of Contents what he plans for future checklists.
Any and all comments and/or questions from PPers are requested and welcome. Anyone else can comment as well. Send any questions or comments to Simple Simon.
Once the PPing checklists are done, he'll do some for CPing and PMing.
ÜberProjects--see the updates below. For those that are interested, Simon has decided that he's going to post periodicals to PG-Int as well since the ones we are doing are all pre-1923. It will also give DPC work greater coverage.
Check out the list of "possible future articles" at the end of the newsletter--are you interested in contributing or do you know of someone who could, or would like to, contribute, we'd love to have your article and/or thoughts. And you can do so anonymously if you choose. It doesn't have to be long: a sentence or two, a short paragraph or two, or more if you really like to "talk"; send a PM to Quill.
Revisions to Guidelines, FAQs and the DPC site code
Guideline Differences--Once we get our Proofreading and Formatting Guidelines updated to go along with our upcoming site code upgrade, we intend to produce a wikipage with the differences. We're working on the Guidelines updates at this time and we hope to produce the "differences" page shortly thereafter.
Combined Proofing and Formatting Guidelines-- The eventual "combined guidelines" will be a very summarized version, not a complete document--but it's a huge task, which is the reason for the delay in issuance, among other tasks that are keeping Simon very busy. Even DP-INT has no such animal--so we can't copy theirs. Once it is drafted, Simon will ask for review by experienced proofers and foofers to ensure that it is understandable--that is, not confusing or missing some extremely important rule.
New DPC site code--It's almost here....
ÜberProject #1: Chronicles of Canada
There continue to be lots of spaces for volunteers to PM, PP and even CP. To see the current status of the volumes, check out the ÜberProject wiki page for The Chronicles of Canada here: http://www.pgdpcanada.net/wiki/index.php/The_Chronicles_of_Canada.
- De2164 and IonaV are doing the prep on the rest of the volumes. Sufficient PMs have volunteered--we just need PPers, but the volumes can be picked up from the PP pool when they have completed the rounds unless there is a volume that a PPer really wants to do--in that case let the PM know sooner rather than later.
- We've arranged to get "mint condition" copies of the full set and so will be able to get good scans of all the maps and illustrations as well as picking up missing pages (if any). Thank you to hugger1935 for the use of the books.
- Volumes 4 and 12 have been split into multiple parts and are being used for BEGIN projects. A number of parts of each volume are sitting in P1: Waiting (English BEGIN) until they are needed; several parts are currently in various rounds undergoing proofing or formatting; and several parts of each volume have made it through to PPing though they won't be finalized until all parts are done--that may take awhile depending on how many newbies we get.
- Volume 31 is in P1: Waiting, Volumes 7 and 10 are in P1: Available, Volumes 15 and 32 are in P2: Available, Volume 14 is in P3: Available, Volumes 6 and 9 are in F2: Available; Volumes 11 and 21 are available for Post-Processing; Vols. 3 and 18 have completed PPing. There are several in New Project being prepared for the P1: Waiting queue.
- PP standards have been set and the books do not have anything extremely difficult at all. Check the Chronicles wiki page for the PP standards. Some of the standards for PPing have been updated.
- Volumes 13, 17 and 24 are being done at DP-Int since the authors of these volumes are out of DPC's copyright purview. They will be posted to PG-Int in due time. For those that are interested, the books are all in the P3 waiting queue.
- Seven volumes have been posted to PGC:
- Volume 1: The Dawn of Canadian History, A Chronicle of Aboriginal Canada,
- Volume 2: The Mariner of St Malo: A Chronicle of the Voyages of Jacques Cartier,
- Volume 5: The Seigneurs of old Canada: a Chronicle of New World Feudalism,
- Volume 8: The Great Fortress: A Chronicle of Louisbourg 1720-1760,
- Volume 20: Adventurers of the Far North, A Chronicle of the Arctic Seas,
- Volume 27: The Winning of Popular Government: A chronicle of the Union of 1841, and
- Volume 29: The Day of Sir John MacDonald, A Chronicle of the Early Years of the Dominion.
ÜberProject #2: Jesuit Relations
Vol. 65: Lower Canada, Mississippi Valley, 1696-1702, our first, is currently in F2: Available. Simple Simon is the PM and has decided that he will definitely PP this first volume, though he says that he will undoubtedly seek help from someone fluent in French to "look over his shoulder" on that side of things. He'll probably do the same for other languages as well.
Vol. 66: Illinois, Louisiana, Iroquois, Lower Canada, 1702-1712, our second, is in F2: Available. It is just like Vol. 65: strange characters, old style French spellings and accents, bits and pieces of other languages--lots of fun.
Vol. 67: Lower Canada, Abenakis, Louisiana; 1716-1727 is now in P2: Available.
Simon is working on the prep of another volume.
A set of standards has been determined and added to the Project Comments. An ÜberProject wiki page for Jesuit Relations has been created. It includes the basic instructions for proofing and formatting.
ÜberProject #3: Governor General's Award Books
Warmheart has created an ÜberProject page for this uberproject. The page looks very good and has lots of links. You can see the details of what she has completed to date here: http://www.pgdpcanada.net/wiki/index.php/Governor_General%27s_Literary_Awards Check it out, you might find something interesting there....
There are only a couple of books that are available since many of the authors are still alive--ah, well, we'll get them eventually. :)
ÜberProject #4: Periodicals
Graham's Magazine is the first part of our ongoing periodicals ÜberProject. All the first year's volumes are either in process or completed. There are two issues currently undergoing proofing or formatting with three in Post Processing. Seven issues have been posted to PGC and/or PG-Int. The second year has started now with one in the proofing rounds and five waiting.
Our Young Folks, An illustrated Magazine for Boys and Girls; Issued monthly; Publication Dates: January 1865 thru October 1873 when it merged with St. Nicholas.
Three issues are in PP, four issues are undergoing proofing or formatting. Warmheart also has one in P1: Waiting, and three of the first year are in prep. Vol. 1, issue 1 has been posted to Project Gutenberg Canada.
Warmheart's wikipage for the Periodical can be found through this link: http://www.pgdpcanada.net/wiki/index.php/Our_Young_Folks.
Scientific American--these are very early issues, too--from 1846. The first issue has now been posted. Simon has more being prepped for proofing.
We've started a wiki page for it: http://www.pgdpcanada.net/wiki/index.php/Periodicals/Scientific_American.
The Maple Leaf--This is a Canadian juvenile periodical published between 1852 and 1854. Susan and IonaV are going to share the duties of PMing. Iona is handling the prep, Susan has downloaded the scans and will do the PPing. They are hoping to get it into the rounds within the next week or so.
A wikipage has been created: http://www.pgdpcanada.net/wiki/index.php/The_Maple_Leaf
Remember, these aren't the only periodicals that we can do--there are lots of others out there and some that are very different than "Graham's". Is there one that you'd like to do? Suggestions for favourite OLD periodicals are always welcome, but they must meet 3 criteria:
- published before 1860 to avoid the need for copyright checking
- good quality scans of a reasonable number of issues (preferably in whole volumes=a year)
- interesting content, out of the ordinary
ÜberProject #5: Works by Charles Dickens
Simple Simon and IonaV are Dickens fans. We know that there are many more out there. Although there are some of Dickens' works on the other PG sites, some of it is text only. We've got early editions of those books with illustrations as well as stories that have not yet been posted. As well, there are letters and essays and ...
- Bleak House, the original version illustrated by "Phiz", is in prep.
- The Mystery of Edwin Drood is currently in F2: Available.
- A Christmas Carol is in post processing.
- Two short stories for our anniversary blitz are posted to our wiki (http://www.pgdpcanada.net/wiki/index.php/Dickens_%22First_Anniversary%22_project): The Christmas Tree and What Christmas is as we get older. They will be posted to PG Canada with other Dickens' short stories.
More details and an ÜberProject wiki page to come.
We Grow Every Day
We are now up to 720 members. Welcome to all new members; I hope you find some projects that you will enjoy doing. Many of our members are also members of DP-Int/US and/or DP Europe--we welcome them and thank them for all their continued support and interest in DPC. (Numbers are accurate up to "press" time.)
| New | |
|---|---|
| September 2009 | 3 |
| August 2009 | 4 |
| July 2009 | 16 |
| 2009 Q2 | 55 |
| 2009 Q1 | 168 |
| 2008 total | 235 |
| 2007 total | 239 |
| Total to date | 720 |
Milestones
We are going to use this space to "celebrate" notable milestones achieved by a member, team, group or project. For example, we'll report when a member completes 5,000 pages in any one round; then we'll let you know when they've done 10,000 pages and so on.
The "team" numbers are the cumulative totals for each round for each member while they are members of a particular team. So that's why, the team pages go up whenever a team member's totals increase. Another reason to join a team or two....
Completed 40,000 pages in a round:
- uwe-joachim in F2'
Completed 35,000 pages in a round:
- no one at present...
Completed 30,000 pages in a round:
- rolands in P3
Completed 25,000 pages in a round:
- Team USA in P3
- Foofers' Corner in F2
Completed 20,000 pages in a round:
- Team Canada in P1 NEW!!!
Completed 15,000 pages in a round:
- Team Canada in P2 NEW!!!
- LOTE Flotes my Bote in F2
- Foofers' Corner in F1
Completed 10,000 pages in a round:
- Bibi in P3
- De2164 in F1
- Team German in F2
Completed 5,000 pages in a round:
- Adair in F1
- Daniel G in P1
- rcool in P1 and P2
- snowseraph in P1 and P2
- stygiania in P3 and F2 NEW!!!
- supo88 in P2
- Warmheart in P2 and F1
- yajeulb in F1
- hugger1935 in F1 and F2
- Team USA in P2 and F2
- Team Canada in P3 and F1
- Team German in P3
- Infomaniacs in P3
- Foofers' Corner in P2
- P3 Pros in P3 and F2 NEW!!!
- P2 Expediters in F2 NEW!!!
- DPing with Cats in P2 and F1
And how many pages have been done in each of the rounds? Naturally, F2 will be at the bottom since they are the last round that gets the projects before PPing and P1 will be on top since they get to start all the projects. :grin: Another consideration is that P3 or F2 stats may be slightly smaller if the PM/PPer has skipped his/her/their project through either one or both of those rounds. So let's see where each round was at the end of August:
- P1 completed over 105,000 pages
- P2 completed over 94,000 pages
- P3 completed over 84,000 pages
- F1 completed over 86,000 pages
- F2 completed over 68,000 pages
Great work, people!!!
Proofing & Formatting Stats
All rounds have met and exceeded their August page goals; some by very wide margins.Way to go, people!!!
We can certainly see that Chef Uwe is back ... F2 is already at more than 70% of their September goal. Welcome back, Uwe, you were missed. P2 is following close behind F2, then P1 and P3; F1 is working hard. But we're not yet half way through September--just over 1/3 of the month--so everyone still has 19 full days to work on their monthly page goal.
| Round | P1 | P2 | P3 | F1 | F2 |
| September goal | 3,300 | 3,300 | 3,000 | 3,300 | 3,000 |
| *September to date | 1,743 | 2,257 | 1,211 | 948 | 2,169 |
| September to date % | 52.82% | 68.39% | 40.37% | 28.73% | 72.30% |
| August goal | 3,410 | 3,410 | 3,100 | 3,410 | 3,100 |
| August final | 5,770 | 5,222 | 4.987 | 5,578 | 3,809 |
| August final % | 169.21% | 153.14% | 160.87% | 163.58% | 122.87% |
*Numbers accurate to "press" time.
For more statistics and related links go to: Statistics Central.
Books In Process And/or Posted To PG Canada or PG-International
At press time, we had 216 Gold Star books posted to PGC or PG-Int (with 4 in PPV). Fantastic work, all.
There are currently 174 Silver Star projects in post processing and 148 Bronze Star projects which are currently undergoing proofing or formatting. For more details about these projects, please see the listings on the bottom of the DP Welcome page.
User Team Talk
Have you checked out our user teams? To check out the teams or to join one go to: User Teams. The Team threads are located under Proofing Team Talk. Is there a team that you'd like to see? You can create a team from the link on the User Teams page--you might be surprised by who shows up to join you.
We currently have 24 teams; more are always welcome.
Possible Future Articles
This is YOUR newsletter: yes, there are forum posts and user team posts, but is there something that you want to tell the whole DPC community? You are allowed to brag here. You can also be anonymous if you choose ... we won't tell.
- Tips or Hints: for those people who do/did a lot of proofing and/or formatting on DP-Int/US &/or DP-Eur, do you have any tips or "rules" that you follow to help yourself while you proof or format? Even a re-wording of a guideline that helps you remember how to do something specific.
- Tell us why you joined DPC.
- Quotable Quotes: have you seen a thought-provoking quote that is relevant to the DP Community while you were doing your proofing that you just had to pass on?...
- The most interesting, outrageous, or strangest line you proofed: before it was proofed (if that was the outrageous or strange part) and what it was after proofing (please do not submit obscene language, it will not be published).
- New projects: PMs is there a series of books or books by a very prolific author that you'd like to do?--let us know so that we can help pass the word to the proofers so they'll be there to jump in and work on them.
- Kudos, Bouquets & Compliments: are there any DPers that you would like to give special thanks and/or recognition to: a mentor who gave you lots of guidance, a PPer that made your project into a very wonderful looking book for posting to PG, a CPer or image/text preparer who went the extra little bit to provide you with clean scans, well OCR'd text, or good illustrations for your project, another DPer who made you feel really welcome or provided you with some good advice or....
- "Guest articles" by anyone and everyone who wishes to....
