[A] restive contingent of our tribe is convinced that it can shed light on traditional philosophical problems by going out and gathering information about what people actually think and say about our thought experiments. [...]
It always irritated me in philosophy discussions when someone would seriously make an argument that something was "intuitively" true or false. There are already some interesting results:
Recently, a team of philosophers led by Machery came up with situations that had the same form as Kripke's and presented them to two groups of undergraduates — one in New Jersey and another in Hong Kong. The Americans, it turned out, were significantly more likely to give the responses that Kripke took to be obvious; the Chinese students had intuitions that were consonant with the older theory of reference.
Does complex, jargon-heavy writing make you seem intelligent? Apparently not. Before rushing to attack Derrida, note that the study's only real result is that fluency is the key. Increased complexity may not necessarily come at the cost of fluency. Long words are only problematic if used inappropriately.
Unfortunately, the chosen samples weren't much good. The complicated samples read as though a thesaurus had been used to inappropriately substitute words. (Which, for the experiment, it had.) The simplified samples read better, but with significant loss of nuance; even if undergraduates didn't notice, a domain expert would have.
By far the most interesting point came from the fifth experiment. Copies of the same document were divided amongst two groups, but the copies given to the second group were printed badly, making the text (optically) difficult to read. The low-toner group rated the intelligence of the author higher.
[W]hen an obvious source for the lack of fluency is present, people discount that lack of fluency when making their judgement. They do so to such an extent that they end up biasing their judgement in the opposite direction!