Choosing the cost vector of the linear programming approach to approximate dynamic programming

We consider the linear programming approach to approximate dynamic programming. In the general case of linear combination of features, we prove the existence of a solution which can be used to generate a policy with performance proportional to the strength of the architecture. In the special case of features defined on a partition of the state space, we give a simpler method to find the solution, as well as a stronger performance bound.