Energy balancing in multi-hop Wireless Sensor Networks: an approach based on reinforcement learning

Wireless Sensor Networks (WSNs) are made of spatially distributed autonomous sensors, which cooperate to monitor a certain physical or environmental condition and pass their data through a network to a central data sink. A promising field of application of WSNs is planet exploration, in which a continuous monitoring of the surface is necessary, to have a clear notion of planet conditions and prepare for a future manned mission. The potentially large size of the region to be monitored and the line-of-sight limitations on remote planets (for instance the Moon, as studied in the SWIPE project [1]), impose constraints on the possibility to have 1-hop sensor-sink communication. Therefore, the sensors must be able to create and maintain a multi-hop ad hoc network, to allow sensed data to reach the sink. This paper extends the Q-Routing algorithm, designed for fixed and mobile networks, in order to be applicable in WSNs. The proposed routing algorithm aims at optimizing the network lifetime, by balancing the routing effort among the sensors, taking into account their current residual batteries, while minimizing the control overhead. Simulation results show an increase of performances, in grid-based networks, which are common topologies for WSNs.