  CONTEXT: There has been a rapid growth in the use of data analytics to
underpin evidence-based software engineering. However the combination of
complex techniques, diverse reporting standards and poorly understood
underlying phenomena are causing some concern as to the reliability of studies.
  OBJECTIVE: Our goal is to provide guidance for producers and consumers of
software analytics studies (computational experiments and correlation studies).
  METHOD: We propose using "bad smells", i.e., surface indications of deeper
problems and popular in the agile software community and consider how they may
be manifest in software analytics studies.
  RESULTS: We list 12 "bad smells" in software analytics papers (and show their
impact by examples).
  CONCLUSIONS: We believe the metaphor of bad smell is a useful device.
Therefore we encourage more debate on what contributes to the validty of
software analytics studies (so we expect our list will mature over time).

Software Engineering