An optimal control problem with control variable constraints, such as

, is discussed in this paper. The constraint vector

is also considered as a design parameter vector to be optimized. The gradient functions of the cost function with respect to

, which are both theoretically and practically important for solving this class of optimal control problems, are principally studied. Discussed are the obtaining of the directional derivative and the derivation of both the subgradient and the gradient based on the former directional derivative.