abstract
the problem of rested and restless multi-armed bandits with constrained availability of arms
is considered. the states of arms evolve in markovian manner and the exact states are hidden from
the decision maker. first, some structural results on value functions are claimed. following these
results, the optimal policy turns out to be a threshold policy. further, indexability of rested bandits
is established and index formula is derived. the performance of index policy is illustrated and
compared with myopic policy using numerical examples.