Submission and Review Information for AoIR Conferences

How are submissions evaluated?

• Each paper submission (including individual papers and papers within preconstituted panels) will be in the form of an abstract that is a maximum of 1200 words in length (this word count does not include the bibliography). Each paper also has a 250 word abstract, and panels also include an introductory statement of 600-800 words. Each paper abstract should include:
  • • A research question or issue to be discussed
    • A brief discussion of how the research is situated in relation to existing research literature (e.g. citations of relevant work)
    • A few sentences on methodological approach
    • A discussion of sample size, evidence, and/or data and analysis
    • An answer to the “so what?” question (e.g., why is this research important, relevant, and interesting?)
    • Works in progress should be noted as such and should include the above as well as preliminary or expected findings so that it can be evaluated for soundness and promise
    • Clear language so that the proposal’s importance can be understood across disciplinary lines
    • Cross-disciplinary work is encouraged.
  • References to authors and identifying information must be removed so as to undergo the double blind review process.
  • Each submission is evaluated on a scale of 1-10 for:
    • Quality of ideas and content (25%)
    • Originality and contribution to the field (20%)
    • Coherence and development of overall project (20%)
    • Overall recommendation (35%)
  • In general, if a submission does not score an 8 or higher in most or all of these categories, it will NOT be accepted for presentation. Reviewers are encouraged to keep this in mind when assigning scores.
  • Reviewers are also asked to provide qualitative information on what they liked about, and what they believe could improve, the submission.
  • Reviewers are asked to provide a ranking of their own familiarity with the topic under review on a scale of 1-10.
  • In the spirit of community and collegiality for which AoIR and its conferences are well known, reviewers are encouraged to remember that many AoIR presenters are graduate students or early-career researchers who will benefit considerably from a reviewer’s constructive comments even if a proposal is rejected.
  • Similarly, a reviewer’s written feedback will be crucial in making acceptance decisions when two or more reviewers disagree in their assessment of the same paper.
  • Reviewers are encouraged to avoid submitting overly brief reviews that merely restate a scoring decision without explaining reasons for the score.
  • Reviewers are further encouraged to avoid overly lengthy point-by-point refutations of the proposal if they amount to little more than an argument for why your own research approach is superior to that chosen in the proposal.
  • Reviews should provide a statement of the reviewer’s judgment and then provide several reasons for why the reviewer came to that judgment.
• When reviewing paper and panel proposals in particular, reviewers are encouraged to remember that these are made in the form of extended abstracts: they should outline the research and any outcomes to date in sufficient detail, but submitters are not required to present an exhaustive literature review or offer a conclusive analysis of the research outcomes.

How are reviewers matched with submissions?

• Submitters are asked to select up to 2 methods and up to 2 topics that best describe their submission.
• Reviewers are asked to select up to 2 methods and up to 2 topics that best describe their areas of expertise.
• Reviewers and submitters are then matched by ConfTool on the basis of areas of overlap.
• As a reviewer, if you receive a submission that is outside your areas of expertise or for which you are otherwise ill-matched as a reviewer, you will have an opportunity to opt out of reviewing and the submission will be reassigned to a different reviewer. Please do NOT review a submission that is outside your expertise to preserve the integrity of the reviewing process.

What does a good review look like?

For an example of both positive and negative reviews that are useful both for submitters and for conference organizers, please see here: AoIRSampleReviews

Can I submit a work in progress?

Yes, we welcome work in progress. However, please make sure that your submission does provide sufficient detail on your methods, theoretical foundation, and preliminary findings to enable reviewers to clearly judge the quality of your work.

A Note on Intellectual Property

Reviewers are encouraged to remember that the submissions being evaluated are the unpublished work of other authors. Their intellectual property rights and the professional ethics of the reviewer require that the reviewer will not disclose the contents of these submissions or part of them to others and that all submission materials are treated as confidential.