Memory allocation and routing: Some dynamic programming without computation

The telephone age, now nearly over, called for (inter alia) good choices of routes for calls. The new computer age is calling for clever use of memory facilities. In each case, to discern just what \´good choices\´ and \´clever use\´ should mean, one can study dynamic programming problems for controlled Markov processes describing operating networks and memories. It turns out that these two kinds of systems and problems are similar, and that useful parallels can be drawn. The systems share these features: (i) a state space partially ordered by inclusion; (ii) limited and stochastic lifetimes of entered items (calls or data blocks); (iii) the presence of combinatorially "bad" states; (iv) the possibility of restructuring the state (rerouting calls or reshuffling the memory); (v) control laws that indicate where arriving items are to be put. It is not unfair to say that in as much as both the routing and allocation problems are at once combinatorial and stochastic, they have showed themselves unexpectedly resistant to most approaches except number crunching. One can quickly agree with Coffman\´s assessment [1] that the problems are really very hard. Such a situation nevertheless leaves at least the hope that careful examination of even small examples might usefully complement simulation with understanding by describing some principles of "good" allocation or routing, on which the optimal decisions found in the examples seem to be based. Some efforts in the direction appear in [2]-[4]. We have shown that in various small examples a study of the state diagram quickly suggests a natural system of preferred routes and allocations based on combinatorics: the "bad" states are identified and avoided as much as possible; the conjecture that these preferences are optimal for many models and criteria soon gathers overwhelming force. It is justified by a proof that a control (i.e. routing, or allocation) policy based on these preferences achieves the optimal in the dynamic prog- amming equations for suitable models and criteria. In this way one guesses and validates an optimal policy by eye-balling, and without having to do any of the computation that usually accompanies Bellman\´s method of dynamic programming. From the resulting policies one can abstract some of the natural principles of routing and allocation that the policies exemplify. Three such principles, those of parsimony, segregation, and justification, are illustrated in detail in [3]. Larger systems cannot be expected to succumb to the elementary domination methods that served for the small ones, but the principles extracted are broadly applicable.