Web Submission and Review, Reviewer Documentation
Version 0.53 - September 2006
Table of Contents
- Initial access to the site
- Changing your password
- Specifying reviewing preferences
- Individual-review phase
- Working with scorecard files
- Discussion phase
- Discussion boards
- Your watch-list
- Program-committee votes
- Later access to the site
1. Initial access to the site
To access the review site you need to supply your email address and a
password. The chair or administrator will send you an email message with
the initial password that you should use for that purpose. Once you
access the site, you are presented with your review page. At the top
of the page there are a few "general purpose" links, looking more or
less as follows:
(Note that the chair may customize some of these links, so they may not
look exactly as the ones above.) These links will be displayed on all
the review pages that you access. On the page you can also find a button
that takes you to a list of all the submissions (except the ones that
chair explicitly blocked you from seeing), and once the chair assigned
submissions for you to review you will see these submissions listed
directly on your review page.
1a. Changing your password
Clicking the "change password" link takes you to a standard form for
changing password, where you need to provide your email address and
old password and repeat the new password twice. Note that as soon as
you submit this form your password changes immediately, and you will
be prompted to insert the new password before you can continue to
access the site.
1b. Specifying reviewing preferences
If the chair indicated that it want to see your reviewing preferences,
you will see the "Preferences" link that lets you specify them. This
link takes you to a list of all the submissions, and for each
submission you can specify a rank from 0 to 5 (where larger numbers
mean that you want to review these submissions more). The rank 0 is
reserved for submissions for which you feel that you have a conflict
of interests (e.g., ones for which you are a co-author). The chair can
use the preferences that you indicate to decide which submissions will
be assigned for you to review (or which submissions you should be
blocked from seeing).
2. Individual-review phase
Typically, the reviewing process begins with an "individual review
phase" in which the reviewers cannot see each other's reviews and
cannot discuss the submissions. During this phase, all you can do
is download the submission files and upload reports on individual
submissions.
If the chair prepared an archive file containing all the submission
files then you will see a link to "download all submissions in one file".
You can also download individual submissions files using the "download"
buttons in the submission list.
You can upload reports to the site using the "Review" buttons in the
submission list. Once you upload a report on a submission, the "Review"
button changes to a "Revise" button that you can use to revise your
review.
2a. Working with scorecard files
Instead of uploading the reviews one at a time, you have the option of
preparing a file with many reviews (called a scorecard file) and
then upload all the reviews to the server at once. As a starting point,
you can download your current scorecard file and save it into a text
file. (This file contains all the reviews that you already uploaded, and
also empty reviews for the submissions that were assigned to you but
for which you did not yet upload a review.) Then you can fill that text
file with more reviews and upload it back to the server.
3. Discussion phase
You move to the discussion phase once the chair decides to let you
discuss submissions with other reviewers. Your review page changes
to reflect that, adding a "Show reviews" button that lets you see
reports from other reviewers. The "Show reviews" button takes you
to a page that lists some review statistics for every submission
(e.g., grade average etc.) as well as a list of the grades given to
that submission by everyone who reviewed it. The "List submission"
button is still available, with the corresponding page adding for
each submission also its grade average and status. (The status of
a submission could be either "Accept", "Maybe Accept", "Discuss",
"Maybe Reject", "Reject", or "None".)
The submissions assigned to you are no longer listed on your review page,
but you have the option of defining a watch-list of
submissions that you want to be displayed on your review page (see below).
3a. Discussion boards
Next to each submission in the "List submissions" or "Show reviews"
pages you can now find a "Discuss" button that takes you to the
discussion board for tat submission. The presentation of that
button may change, looking like this
if anything on the discussion
board changed since the last time you accessed it or like that
if nothing had changed.
On the discussion board for each submission you can find all the
reports on that submission as well as the discussion on it, and
you can post new comments to the discussion. You can choose if you
want to view the discussion as "threaded" (as in a newsgroup) or
"UNthreaded" (i.e., just a linear list of all the posts). Note that
the choice of threaded/UNthreaded view will affect your view of all
the discussion boards, not only the one that you are currently viewing.
3b. Your watch-list
As mentioned above, you can define a watch-list of submissions for which
you want to follow the discussion, and these submissions will be
displayed on your review page. To add or remove a submission from
you watch-list, just click the eye icon next to that submission in the
submission list.
3c. Program-committee votes
The chair may ask you to vote on various topics during the program-committee
work (e.g., which papers to move from status "Discuss" to "Maybe accept",
or who should be the invited speakers at the conference). When the chair
sets up votes, you will see on your review page links that allow you to
participate in thses votes. Following these links gets you to a voting
page where you can find further instructions and cast your vote.
4. Later access to the site
The chair can keep the review site up also after all the decisions were
made. At this phase, the site is operating in a read-only mode, where
you still have access to all the reports and discussions that you could
see during the discussion phase, but you can no longer revise them.
You have the option of getting a single page with all the reviews
and discussions (in html or ascii) to keep for your records.