Recommended exercises to do on your own:

<br>
<br>

Markov Chains in Discrete Time: 1.5, 1.6, 1.8.

<br>
<br>

Markov Chains in Continuous Time: 1.12, 1.15.

<br>
<br>

Queuing Theory: 2.2, 2.5, 2.6, 2.14.

<br>
<br>

Inventory Theory: 3.1, 3.2, 3.6, 3.8, 3.9

<br>
<br>

Marginal Allocation 4.1, 4.3, 4.4

<br>
<br>

Deterministic Dynamic Programming: 5.1, 5.2, 5.3

<br>
<br>

Stochastic Dynamic Programming:  5.4 (used in one of the exercise classes), 
5.6, 5.8, 5.10-12

<br>
<br>


Markov Decision Processes: 6.2, 6.3, 6.4, 6.9 a,c

<br>
<br>