Notice of RetractionShaping agent by critical states

Notice of Retraction

After careful and considered review of the content of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE´s Publication Principles.

We hereby retract the content of this paper. Reasonable effort should be made to remove all past references to this paper.

The presenting author of this paper has the option to appeal this decision by contacting TPII@ieee.org.

Shaping is a promising technique for scaling Reinforcement Learning to large and complex problems. But the design and tune of shaping reward are difficult and problem-oriented. We propose an approach to make agent can shape itself by critical states, which are found by agent itself from prior learning. We accumulate the state trajectories that agent experienced in every training episode, and eliminate the state loops existed in the original state trajectories, then the acyclic state trajectories are used to find the critical states. The critical state is a state that has high probability to appear in all these acyclic state trajectories, that means, if agent wants to reach the goal state, then it would have high probability to pass the critical states. So the critical states can be used to shape agent reaching the goal state faster. The Grid-World problem is used to illustrate the applicability and effectiveness of our approach. The more important is our approach makes agent can shape itself by what it learned.