How are submissions evaluated?
- Each paper submission (including individual papers and papers within preconstituted panels) will be in the form of an abstract that is 1200 words in length INCLUDING citations. Each paper also has a 250 word abstract, and panels also include an introductory statement of 600-800 words. Each paper abstract should include:
- A research question or issue to be discussed
- A brief discussion of how the research is situated in relation to existing research literature (e.g. citations of relevant work)
- A few sentences on methodological approach
- A discussion of sample size, evidence, and/or data and analysis
- The relationship to wider conference themes
- An answer to the “so what?” question (e.g., why is this research important, relevant, and interesting?)
- Works in progress should be noted as such and should include the above as well as preliminary or expected findings so that it can be evaluated for soundness and promise
- Clear language so that the proposal’s importance can be understood across disciplinary lines
- Cross-disciplinary work is encouraged.
- References to authors and identifying information must be removed so as to undergo the double blind review process.
- Roundtables, fishbowls, and experimental sessions are designed to encourage interaction among participants, and should include:
- Delegates from differing universities, methodologies, and/or approaches to stimulate interesting conversations
- Roundtables, fishbowls, and experimental sessions are only single-blind reviewed: the quality and expertise of the participants will be recognized in the assessment.
- AoIR values inclusivity and diversity. Those roundtables, fishbowls, and experimental sessions that address issues that cross various geographic, racial/ethnic, and gendered backgrounds and that include a mix of junior and senior scholars will be rated more highly on quality of content.
- Each submission is evaluated on a scale of 1-10 for:
- Quality of content (25%)
- Significance (20%)
- Presentation (20%)
- Overall recommendation (35%)
- In general, if a submission does not score an 8 or higher in most or all of these categories, it will NOT be accepted for presentation. Reviewers are encouraged to keep this in mind when assigning scores.
- Reviewers are also asked to provide qualitative information on what they liked about, and what they believe could improve, the submission.
- Reviewers are asked to provide a ranking of their own familiarity with the topic under review on a scale of 1-10.
- In the spirit of community and collegiality for which AoIR and its conferences are well known, reviewers are encouraged to remember that many AoIR presenters are graduate students or early-career researchers who will benefit considerably from a reviewer’s constructive comments even if a proposal is rejected.
- Similarly, a reviewer’s written feedback will be crucial in making acceptance decisions when two or more reviewers disagree in their assessment of the same paper.
- Reviewers are encouraged to avoid submitting overly brief reviews that merely restate a scoring decision without explaining reasons for the score.
- Reviewers are further encouraged to avoid overly lengthy point-by-point refutations of the proposal if they amount to little more than an argument for why your own research approach is superior to that chosen in the proposal.
- Reviews should provide a statement of the reviewer’s judgment and then provide several reasons for why the reviewer came to that judgment.
- When reviewing paper and panel proposals in particular, reviewers are encouraged to remember that these are made in the form of extended abstracts: they should outline the research and any outcomes to date in sufficient detail, but submitters are not required to present an exhaustive literature review or offer a conclusive analysis of the research outcomes.
- Reviewers for the AoIR conference submissions are drawn from among those who have been a member of AoIR for at least one of the previous three years.
- AoIR has long been a conference that welcomes graduate students, scholars, and practitioners from a variety of professional and disciplinary commitments. People of all backgrounds are encouraged to serve as reviewers.
How are reviewers matched with submissions?
- Submitters are asked to select up to 2 methods and up to 2 topics that best describe their submission.
- Reviewers are asked to select up to 2 methods and up to 2 topics that best describe their areas of expertise.
- Reviewers and submitters are then matched by ConfTool on the basis of areas of overlap.
- As a reviewer, if you receive a submission that is outside your areas of expertise or for which you are otherwise ill-matched as a reviewer, you will have an opportunity to opt out of reviewing and the submission will be reassigned to a different reviewer. Please do NOT review a submission that is outside your expertise to preserve the integrity of the reviewing process.
What does a good review look like?
For an example of both positive and negative reviews that are useful both for submitters and for conference organizers, please see here: http://aoir.org/wp-content/uploads/2017/03/AoIRSampleReviews.pdf.
Can I submit a work in progress?
Yes, we welcome work in progress. However, please make sure that your submission does provide sufficient detail on your methods, theoretical foundation, and preliminary findings to enable reviewers to clearly judge the quality of your work.
A Note on Intellectual Property
Reviewers are encouraged to remember that the submissions being evaluated are the unpublished work of other authors. Their intellectual property rights and the professional ethics of the reviewer require that the reviewer will not disclose the contents of these submissions or part of them to others and that all submission materials are treated as confidential.