Belief update in POMCP #81
Description
Thank you for this amazing rep! I just have some little questions, about pomcp.pyx, update():
...
if agent.tree[real_action][real_observation] is None:
# Never anticipated the real_observation. No reinvigoration can happen.
raise ValueError("Particle deprivation.")
# Update the tree; Reinvigorate the tree's belief and use it
# as the updated belief for the agent.
agent.tree = RootVNodeParticles.from_vnode(agent.tree[real_action][real_observation],
agent.history)
tree_belief = agent.tree.belief
agent.set_belief(particle_reinvigoration(tree_belief,
len(agent.init_belief.particles),
state_transform_func=state_transform_func))
# If observation was never encountered in simulation, then tree will be None;
# particle reinvigoration will occur.
if agent.tree is not None:
agent.tree.belief = copy.deepcopy(agent.belief)
...
- The belief is updated by doing reinvigoration on a chid node's current particles. While in the origin paper, in my understanding, it reuse the child node's particles and do monte carlo sampling to add new particles. In practice I find that it may not be a big issue since when num_sim is big, there are likely enough particles in the child node, and no need for adding new particles. I just wonder why we use this different tech, is there some further reasons?
- I am not sure is this second if condition necessary:
if agent.tree is not None:
agent.tree.belief = copy.deepcopy(agent.belief)
since we already check that
if agent.tree[real_action][real_observation] is None:
# Never anticipated the real_observation. No reinvigoration can happen.
raise ValueError("Particle deprivation.")
and also, I don't think we need to copy the agent.belief to the tree.belief, since we sample the start state in the root node by sample_belief(), which sampling from the agent.belief, not from the agent.tree.belief. Please tell me if I am wrong.