The repo builds on CSRL project (in this article.) of Duke University and apply Q-Learning, Model-Checking, Value Iteration to the grid world case study.

Dependencies

• Python: (>=3.5)
• Rabinizer 4: ltl2ldba must be in PATH (ltl2ldra is optional)
• NumPy: (>=1.15)
• scipy
• dill

The examples in this repository also require the following optional libraries for visualization:

• Matplotlib: (>=3.03)
• JupyterLab: (>=1.0)
• ipywidgets: (>=7.5)
• tqdm

Case_study_Prodcut_MDP.ipynb uses drill to store the product MDP model into workspace product_MDP.pkl. Case_study_2025_Nursery Scenario.ipynb use drill to load the workspace file product_MDP.pkl, then apply Q-Learning, Model-Checking, Value Iteration.