notes.txt

setting gamma to 0.999999 and epsilon to 0.0000001
gives results same as figure 17.3
python ./aima/mdp.py -i
>>> Fig[17,1] = GridMDP([[-0.04, -0.04, -0.04, +1],
...                       [-0.04, None,  -0.04, -1],
...                       [-0.04, -0.04, -0.04, -0.04]], 
...                       terminals=[(3, 2), (3, 1)], gamma = 0.999999)
>>> value_iteration(Fig[17,1], 0.0000001)
{(0, 1): 0.76155361600262073, (1, 2): 0.86780580763285753, (3, 2): 1.0, (0, 0): 0.70530257573246147, (3, 0): 0.3879184577039676, (3, 1): -1.0, (2, 1): 0.66027205960026847, (2, 0): 0.61140879981566276, (2, 2): 0.91780694239125182, (1, 0): 0.65530170660444154, (0, 2): 0.81155461794564299}


>>> Fig[17,1].T((0,1), direction_to_tuple("right"))
[(0.80000000000000004, (0, 1)), (0.10000000000000001, (0, 0)), (0.10000000000000001, (0, 2))]