I am concerned that the casual reader is inclined to interpret the Levitt-List article (forthcoming in Journal of Economic Perspectives) as a general critique of laboratory experimentation in the social sciences. The Levitt-List article argues that it is important to verify to what extent subjects behave in the same way in the field as in the laboratory. Otherwise, it is claimed, the laboratory results lack relevance.
This critique cannot be generalized. It does not apply to at least one class of experiments, experiments with markets, which should be familiar from the work of economists such as Charles Plott, Shyam Sunder, or Vernon Smith. This is also the type of experiments I have been involved in.
In particular, in my own markets experiments I am interested in understanding how risk aversion (or other phenomena of individual human behavior) affects prices in securities markets. I exploit the fact that one observes risk aversion in the laboratory. It does not matter that risk aversion is different in the real world (because, e.g., it is calibrated differently - you don't see as much risk aversion in the real world for the same amount of risk). This is because I am not interested in risk aversion per se, but in the effect of the risk aversion of a large group of humans on market prices, and in the feedback from prices to holdings.
It is useful to make an analogy with physics research. In one class of physics experiments, physicists study the behavior of objects in a vacuum. Since these objects do not behave in the same way in the "real world," and following the logic of the authors, one should dismiss this type of physics experiments as irrelevant. I am sure that no physicist would ever agree with this conclusion.
I think the the critique in Levitt-List applies only to a particular type of experiments, which one could describe as "wind-tunnel" experiments, where the point is to verify whether something is going to work in real-world settings. For such experiments, it matters that subjects behave as in the real world.
The critique does not apply to other experiments that one could label as "pure-science," where the point is to verify some theoretical
statement in a highly controlled setting. The term "controlled" necessarily implies artificiality. The lack of reality that ensues does not make such experiments less relevant.
The distinction between different types of experiments is particularly relevant for the field of asset pricing, because most of the theory is developed in the economic equivalent of a "vacuum": perfect, competitive markets, often without even asymmetric information or heterogeneity in beliefs. These markets do not exist in reality. Does that make laboratory experiments of asset pricing theory irrelevant?
I do not think so. In fact, it makes laboratory experiments mandatory, because one needs control.