Experimental evaluation of wireless simulation assumptions

All analytical and simulation research on ad hoc wireless networks must necessarily model radio propagation using simplifying assumptions. A growing body of research, however, indicates that the behavior of the protocol stack may depend significantly on these underlying assumptions. The standard response to this problem is a call for more realism in designing radio models. But how much realism is enough? This study is the first to approach this question by validating simulator performance (both at the physical and application layers) with the results of real-world data. Referencing an extensive set of measurements from a large outdoor routing experiment, we start by evaluating the relative realism of common assumptions made in radio model design, identifying those which provide a reasonable approximation of reality. Although several such investigations have been made for static sensor networks, radio behavior in mobile network deployments is a much less-studied topic. We then reproduce our experimental setup in our simulator, and generate the same application-layer metrics under progressively smaller sets of these assumptions. By comparing the simulated outcome to the outcome of our experiment, we are able to discern at what point our balance of simplification and realism captures the real behavior of our target environment.