Things that strengthen an IR paper

Presentation

  • The paper’s motivation and the potential impact of the addressed problem are discussed.
  • The paper’s original contributions (i.e. the delta over prior art) are clearly stated.
  • The paper’s claims are properly scoped and supported.
  • The paper clearly describes what was done and what was not.
  • The choices made in each step of the research are justified (the why’s).
  • The results are presented effectively in appropriate format.
  • Good discussion accompanies the results.

Experimentation (if applicable)

  • The experimental design and its scale are appropriate.
  • In comparative studies, appropriate baselines are used.
  • The experimental results are reliable and generalizable.
  • The evaluation methods employed are in line with the research questions.
  • Statistical analysis is performed and reported appropriately.
  • Sufficient details (with data and code where appropriate) are provided to help other researchers assess and reproduce the experiments.