I was reading a back issue of ACM Transactions on Computer-Human Interaction (March 2004).  The editor at the time, Jonathan Grudin of Microsoft Research, wrote an interesting article on his views on publications in the field of HCI.  His section on defining quality in a multidisciplinary field was particularly interesting for me.  He said things like:

• The hard sciences generally concur on quality criteria, the humanities agree less, and multi-disciplinary fields least of all...A central challenge [in multidisciplinary fields] is to identify a standard or process or determining the quality of contributions.
• Quality spans importance, originality, and methodological soundness.  These are independent; for example,work can be sound and original but not of much import, or original and interesting but not sound.
• Importance is related to scope. Research that is out of scope is not important to a field...Judgments of importance are where political, institutional, and other biases slip in.
• Originality is also viewed differently...That which seems startlingly original to a fellow specialist can appear to be a minor incremental step to an outside viewer. Again, this creates particular challenges in multi-disciplinary fields.
• Differing views of methodological rigor can be more subtle.  Psychologists brought into HCI the norm or asserting a scientific advance when the probability of a type of error is less than 5% ("p < 0.5")... Social psychologists...might be happy with p < .15.  To a practitioner who must decide between two designs...a lab study that favors one alternative by a 60-40 margin (p < .40) might be welcomed.  A computer scientist building a new form of interactive system may feel that novelty alone merits publication. A proof of concept study might please such researchers even if their unpolished user interface performs somewhat less well than an existing refined alternative (p > .5). And some qualitative researchers consider lab studies to be pointless in principle. 

It was tremendously comforting to read this paper because I had been asking many of these questions to myself.  Although the article does not provide absolute answers, it is clear that lines (no matter how artificial) should be drawn and redrawn depending on the ecology of the field.