A New Reinforcement Learning for Group-Based Marshaling Plan Considering Desired Layout of Containers in Port Terminals

In container yard terminals, containers are brought by trucks in the random order. Containers have to be loaded into the ship in a certain order, since each container has its own shipping destination and it cannot be rearranged after loading. Therefore, containers have to be rearranged from the initial arrangement into the desired arrangement before shipping. In the problem, the number of container-arrangements increases by the exponential rate with increase of total count of containers, and the rearrangement process occupies a large part of the total run time of material handling operation at the terminal. Moreover, conventional methods require enormous time and cost to derive an admissible result for the rearrangement process. In this paper, a Q-learning algorithm considering the desired position of containers for a marshaling in the container yard terminal is proposed. In the proposed method, the learning process consists of two parts: rearrangement plan assuring explicit transfer of container to the desired position, and removal plan for preparing the rearranging operation. Moreover, in the proposed method, each container has several desired positions, so that the learning performance of the method can be improved as compared to the conventional method. In order to show effectiveness of the proposed method, computer simulations for several examples are conducted