Simulation vs real testbeds: a validation of WSN simulators

In [9] we presented a first effort in assessing the reliability of OMNeT++ in the simulation of Wireless Sensor Networks. We showed that OMNeT++ with MACSimulator framework over-estimates the experimental results obtained by a real testbed. In this paper, we extend our analysis to the well known NS-2 simulator and then to Castalia, a new framework to simulate WSNs on OMNeT++. We first describe the architectures of the two simulators and we analyze how well they model the general structure of a WSN node, mainly in terms of application, MAC, radio and physical layers. We then compare testbed results obtained on several network topologies to the simulation results on the same topologies and we observe that both simulators provide quite reliable results (approximately 10%-30% over/underestimation of experimental results). In our simulations we exploited both the flat disk propagation model and the log-normal shadowing path loss model and, surprisingly, we observed that the simple flat disk model produces more reliable results on dense topologies.