There are many stochastic resource allocation problems arising in transportation, energy and health that involve high-dimensional state and action variables in the presence of dierent forms of uncertainty. These might involve discrete or continuous resources, and generally involve vectors of random variables that preclude exact computation of expectations. I will also describe our research into the important \exploration vs. exploitation" problem that arises in approximate dynamic programming, where we have the ability to choose the next state we will visit.