QoS Routing in MANETS with Imprecise Information Using Actor-Critic Reinforcement Learning

This paper proposes a path discovery scheme which supports delay-constrained least cost routing in MANETs. The aim of the scheme is to maximise the probability of success in finding feasible paths while maintaining communication overhead under control in presence of information uncertainty. The problem is viewed as a partially observable Markov decision process (POMDP) and is solved using an actor-critic reinforcement learning (RL) method. The scheme relies on approximate belief states of the environment which captures the network state uncertainty. Numerical results carried out under various scenarios of state uncertainty and stringent QoS requirements show that the proposed RL framework can lead to more efficient control of search messages, i.e., a reduction of up to 63% of average number of search messages with marginal reduction of up to 3 % in success ratio in comparison with a flooding scheme.