Hybrid mapping #419

Shaobo-Zhou · 2025-07-29T16:12:58Z

Description

This PR introduces significant improvements to the action space and handling of stochastic mapping passes in the reinforcement learning environment:

🚀 Major Changes
Expanded Action Space

Added AIRouting as a routing option and wrapped it in SafeAIRouting for robust integration into the RL pipeline.
Introduced hybrid mapping actions that combine layout and routing stages.

Motivation for Mapping Redesign

The previous separation of layout and routing did not truly reflect the state machine described in the MQT Predictor paper.
If a circuit is mapped (i.e., layout + routing) and then undergoes further optimization passes, it may become "unmapped" again — violating hardware constraints or invalidating the original layout.
In such cases, continuing with the previous layout and only re-running routing leads to suboptimal results, as the old layout is no longer optimal for the changed circuit structure.
The redesigned mapping action ensures layout and routing are always performed together when remapping is needed, preventing mismatches between layout and circuit state and improving both optimality and reliability.

Support for Stochastic Passes

Wrapped stochastic actions (e.g., AIRouting, SabreLayout) in a multi-trial evaluation loop, similar to QiskitO3 pipeline.
Introduced (layout_trials, routing_trials) as parameters to control trial counts, enabling improved predictor performance and stability.
Applied Qiskit O3 default parameters for SabreLayout and VF2Layout.

Fixes and Enhancements

Fixed a bug in OptimizeCliffords by ensuring CollectCliffords runs beforehand.
Prevented Qiskit fallback to default gate basis ['id', 'u1', 'u2', 'u3', 'cx'] (see [https://quantum.cloud.ibm.com/docs/en/api/qiskit/0.24/transpiler]) during:
- decompose
- tk_to_qiskit conversion
- Optimize1qGatesDecomposition
  ensuring correct native gate usage which is essential if individual gate counts are included in the feature space—otherwise RL cannot meaningfully learn from a generic gate basis.
Fixed incorrect usage of GatesInBasis in rl/predictorenv.py
Changed benchmark level to INDEP in test_predictor_rl.py, since the current action space does not guarantee support for high-level gates.

Dependency Update

Added qiskit-ibm-ai-local-transpiler to the dependencies
Pinned networkx==2.8.5 to ensure compatibility with qiskit-ibm-ai-local-transpiler
Upgraded pytket_qiskit>=0.71.0
Upgraded torch>=2.7.1,<2.8.0
Removed support for Python 3.13 due to incompatibility with qiskit-ibm-ai-local-transpiler (requires <=3.12)

Checklist:

The pull request only contains commits that are focused and relevant to this change.
I have added appropriate tests that cover the new/changed functionality.
I have updated the documentation to reflect these changes.
I have added entries to the changelog for any noteworthy additions, changes, fixes, or removals.
I have added migration instructions to the upgrade guide (if needed).
The changes follow the project's style guidelines and introduce no new warnings.
The changes are fully tested and pass the CI checks.
I have reviewed my own code changes.

Update action space and feature space Update actions Update action space

src/mqt/predictor/rl/predictorenv.py

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Remove example_test.py Remove example_test.py

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations

burgholzer · 2025-07-30T16:46:46Z

@Shaobo-Zhou Just fyi: you can also run mypy locally to debug this on your machine without relying on CI by following: https://mqt.readthedocs.io/projects/predictor/en/latest/development_guide.html#code-formatting-and-linting
You can also run the tests locally as described here: https://mqt.readthedocs.io/projects/predictor/en/latest/development_guide.html#running-tests

burgholzer · 2025-07-30T18:09:55Z

@Shaobo-Zhou Is there any reason you keep closing your issues?

Shaobo-Zhou · 2025-07-30T18:31:48Z

@burgholzer Hi, sorry for the confusion. I had been closing PRs because I noticed that each push triggered notifications, and I didn’t want to create unnecessary noise while I was still figuring things out.
I’ll keep this PR open now and run the checks locally first, then push updates once everything passes. Thanks for your clarification!

burgholzer · 2025-07-30T18:36:49Z

Don't worry about the notifications. That's fine and it's exactly what draft PRs are for 😌
You also do not need to force push. You can simply accumulate commits. We squash merge PRs anyway.
If you want to avoid the automatic commits from the pre-commit bot in the CI, you can simply run pre-commit locally before pushing

pre-commit run -a

or

nox -s lint

and commit the resulting changes.

Once the PR is really ready for review, simply give us a ping here.

Fix bugs Fix bugs Fix bugs

burgholzer · 2025-08-05T16:07:06Z

By the way: If you are annoyed that you always have to wait for someone from the team to trigger the CI, the easiest way to get around that is to submit another PR with a small change (could be a typo fix or similar) that we can quickly merge. This makes you a repeating contributor, which automatically runs CI without direct approval.
Just in case that's starting to become annoying.

Shaobo-Zhou · 2025-08-05T16:09:45Z

@burgholzer Thanks for the info!

…e gate check

Fix windows runtime warning problem Fix windows runtime warning issue

codecov · 2025-08-07T20:01:46Z

Codecov Report

❌ Patch coverage is 83.15217% with 31 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/mqt/predictor/rl/helper.py	70.8%	14 Missing ⚠️
src/mqt/predictor/rl/actions.py	81.9%	13 Missing ⚠️
src/mqt/predictor/rl/predictorenv.py	92.8%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

Shaobo-Zhou · 2025-08-08T12:12:42Z

@burgholzer Could you also add @flowerthrower as a reviewer for this PR so we can clarify any potential confusion during the review process?

burgholzer · 2025-08-08T12:14:22Z

@burgholzer Could you also add @flowerthrower as a reviewer for this PR so we can clarify any potential confusion during the review process?

Done ✅

flowerthrower

Hi Shaobo, and thank you for setting up this PR.
Great to see you working with our setup — this should make future PRs easier to manage.

A few general points (in addition to the inline comments):

Why is it no longer possible to work with algorithm-level circuits (but was before)?
This PR should focus as much as possible on adding the new actions.
Other improvements not strictly required for these passes should be left out of this single-purpose PR, perhaps we can make this a bit more lean.

determine_valid_actions_for_state
I agree, better documentation would help, but the function generally works as intended.
No need to merge layout and routing.

If a circuit becomes unmapped, determine_valid_actions_for_state already enforces:

Re-layout if self.layout is unset
Re-routing if unrouted

This depends on self.layout being correctly updated after optimizations.
If that is currently not the case, I would suggest to keep the current structure, but ensure self.layout always matches the circuit state after each optimization.

Next steps:
I haven’t yet reviewed the new passes in detail, but you can already address this feedback (may affect the combined actions) and request another review once done.

Overall, I am happy to see this progressing — this brings us really closer to integrating your work into the MQT Predictor.

flowerthrower · 2025-08-12T14:03:47Z

noxfile.py

@@ -29,7 +30,7 @@

 # TODO(denialhaag): Add 3.14 when all dependencies support it
 #   https://github.com/munich-quantum-toolkit/predictor/issues/420
-PYTHON_ALL_VERSIONS = ["3.10", "3.11", "3.12", "3.13"]


It seems rather unfortunate to lose Python 3.13 support to enable a single pass; it would be preferable to make the pass optional based on the available Python version (ideally with a short comment that mentions the current limitation and can be revisited in the future).

flowerthrower · 2025-08-12T14:06:18Z

noxfile.py

@@ -66,7 +67,9 @@ def _run_tests(
        "test",
        *install_args,
        "pytest",
+        "-v",


No expert on this, but these flags seem like artefacts from debugging and apparently were not necessary before (are they now?).

flowerthrower · 2025-08-12T14:08:57Z

noxfile.py

@@ -82,6 +85,9 @@ def tests(session: nox.Session) -> None:
 @nox.session(reuse_venv=True, venv_backend="uv", python=PYTHON_ALL_VERSIONS)
 def minimums(session: nox.Session) -> None:
    """Test the minimum versions of dependencies."""
+    if platform.system() == "Windows":


Do we know why this is suddenly too slow? Seems unfortunate to miss this test if it was running fine before (note, there have always been longer test durations with the Predictor, so this is potentially fine).

Actually the windows test runs fine and is not skipped...I will delete this part

flowerthrower · 2025-08-12T14:09:36Z

noxfile.py

@@ -133,5 +139,7 @@ def docs(session: nox.Session) -> None:
        "--frozen",
        "sphinx-autobuild" if serve else "sphinx-build",
        *shared_args,
+        "-v",


Same as above.

flowerthrower · 2025-08-12T14:11:01Z

pyproject.toml

@@ -44,8 +44,10 @@ dependencies = [
    "numpy>=1.24; python_version >= '3.11'",
    "numpy>=1.22",
    "numpy>=1.22,<2; sys_platform == 'darwin' and 'x86_64' in platform_machine and python_version < '3.13'",  # Restrict numpy v2 for macOS x86 since it is not supported anymore since torch v2.3.0
-    "torch>=2.7.1,<2.8.0; sys_platform == 'darwin' and 'x86_64' in platform_machine and python_version < '3.13'",  # Restrict torch v2.3.0 for macOS x86 since it is not supported anymore.


What's the reason for deleting this?

flowerthrower · 2025-08-12T14:33:15Z

src/mqt/predictor/rl/helper.py

+        layouted_qc = layout_pm.run(qc)
+        layout_props = dict(layout_pm.property_set)
+    except Exception:
+        return qc, {}


Perhaps an error message as below would be nice.

flowerthrower · 2025-08-12T14:35:36Z

src/mqt/predictor/rl/helper.py

+        qc: The input quantum circuit.
+        max_iteration: A tuple (layout_trials, routing_trials) specifying
+            how many times to try.
+        metric_fn: Optional function to score circuits; defaults to circuit depth.


We could use the already existing figure_of_merits here?

Would be an option, I tried to use the standard SWAP count metric as in Sabre

flowerthrower · 2025-08-12T14:52:06Z

src/mqt/predictor/rl/predictorenv.py

+        if getattr(action, "stochastic", False):
+
+            def metric_fn(circ: QuantumCircuit) -> float:
+                return float(circ.count_ops().get("swap", 0))


In the docstring above, it was mentioned that this defaults to circuit depth?

flowerthrower · 2025-08-12T15:06:41Z

tests/compilation/test_predictor_rl.py

@@ -37,7 +38,7 @@ def test_predictor_env_reset_from_string() -> None:
    device = get_device("ibm_eagle_127")
    predictor = Predictor(figure_of_merit="expected_fidelity", device=device)
    qasm_path = Path("test.qasm")
-    qc = get_benchmark("dj", BenchmarkLevel.ALG, 3)
+    qc = get_benchmark("dj", BenchmarkLevel.INDEP, 3)


If this restriction is really unavoidable (see general comment), then the examples and documentation should be updated accordingly.

flowerthrower · 2025-08-12T15:19:49Z

tests/hellinger_distance/test_estimated_hellinger_distance.py

-            ml_predictor.compile_training_circuits(
-                timeout=600, path_compiled_circuits=target_path, path_uncompiled_circuits=source_path, num_workers=1
-            )
+        ml_predictor.compile_training_circuits(


This should not be an issue as the timeout_watcherstill executes the function (even though without the timeout).

Shaobo-Zhou · 2025-08-12T18:35:28Z

Hi Shaobo, and thank you for setting up this PR. Great to see you working with our setup — this should make future PRs easier to manage.

A few general points (in addition to the inline comments):

Why is it no longer possible to work with algorithm-level circuits (but was before)?

This PR should focus as much as possible on adding the new actions.

Other improvements not strictly required for these passes should be left out of this single-purpose PR, perhaps we can make this a bit more lean.

determine_valid_actions_for_state I agree, better documentation would help, but the function generally works as intended. No need to merge layout and routing.

If a circuit becomes unmapped, determine_valid_actions_for_state already enforces:

Re-layout if self.layout is unset

Re-routing if unrouted

This depends on self.layout being correctly updated after optimizations. If that is currently not the case, I would suggest to keep the current structure, but ensure self.layout always matches the circuit state after each optimization.

Next steps: I haven’t yet reviewed the new passes in detail, but you can already address this feedback (may affect the combined actions) and request another review once done.

Overall, I am happy to see this progressing — this brings us really closer to integrating your work into the MQT Predictor.

Thanks for the feedback! I will work on those points in detail, but maybe first some comments to the general points you mentioned:

Why it is no longer possible to work with algorithm-level circuits (but was before):
From my experience with earlier releases, the RL predictor was specifically designed for indep-level circuits, so I am not sure why the current tests still use algorithm-level circuits in this way. Given that the action space does not include HighLevelSynthesis, it should not be able to resolve special gates such as Oracle. I will try to debug how this worked previously, but my suspicion is that the terminate status might never have been reached and the tests passed due to timeout(see the latest closed PR for reference).
Merging of layout and routing:
As you mentioned, one reason for merging is that the current structure does not perform re-layout after the circuit state changes. Introducing re-layout would require additional checks in the logic to ensure that a re-layout is enforced after every optimization step, in order to match the current circuit state. This can get a bit messy. I can try to do this in the current PR and then include a merged version of layout and routing in a follow-up PR.

flowerthrower · 2025-08-13T06:42:04Z

Merging of layout and routing:
As you mentioned, one reason for merging is that the current structure does not perform re-layout after the circuit state changes. Introducing re-layout would require additional checks in the logic to ensure that a re-layout is enforced after every optimization step, in order to match the current circuit state. This can get a bit messy. I can try to do this in the current PR and then include a merged version of layout and routing in a follow-up PR.

Currently, determine_valid_actions_for_state works as follows:

If not synthesized:
- If laid out → perform synthesis, optimization, or routing
- Else → perform synthesis and optimization only
If synthesized:
- If laid out and mapped → perform optimization or terminate
- If only laid out → perform routing
- Else → perform layout, mapping, or optimization

This structure already ensures re-layout whenever necessary and enforces the flow described in the predictor paper — i.e., start with synthesis before mapping. That is of course, as long as the layout attribute is correctly set after it has changed, e.g., in an optimization pass.

EDIT: I can see how this order is broken in the case of not synthesized + laid out, and a routing pass is selected, but I don't see how that would break the re-layout whenever necessary.

Shaobo-Zhou · 2025-08-14T10:04:23Z

Merging of layout and routing:
As you mentioned, one reason for merging is that the current structure does not perform re-layout after the circuit state changes. Introducing re-layout would require additional checks in the logic to ensure that a re-layout is enforced after every optimization step, in order to match the current circuit state. This can get a bit messy. I can try to do this in the current PR and then include a merged version of layout and routing in a follow-up PR.

Currently, determine_valid_actions_for_state works as follows:

If not synthesized:

If laid out → perform synthesis, optimization, or routing

Else → perform synthesis and optimization only

If synthesized:

If laid out and mapped → perform optimization or terminate

If only laid out → perform routing

Else → perform layout, mapping, or optimization

This structure already ensures re-layout whenever necessary and enforces the flow described in the predictor paper — i.e., start with synthesis before mapping. That is of course, as long as the layout attribute is correctly set after it has changed, e.g., in an optimization pass.

EDIT: I can see how this order is broken in the case of not synthesized + laid out, and a routing pass is selected, but I don't see how that would break the re-layout whenever necessary.

That is of course, as long as the layout attribute is correctly set after it has changed, e.g., in an optimization pass.

I think the real issue is exactly that optimization passes change the circuit structure but don’t reset the layout (only layout actions set the layout), so the layout attribute never changes. Since a layout exists, the only viable option becomes routing, even though a re-layout might actually be needed.

burgholzer · 2025-08-14T10:11:49Z

I think the real issue is exactly that optimization passes change the circuit structure but don’t reset the layout (only layout actions set the layout), so the layout attribute never changes. Since a layout exists, the only viable option becomes routing, even though a re-layout might actually be needed.

Just throwing a random comment in here because I was roughly following the discussion.
Originally, we took special care that the optimizations that are applied after layout and routing are routing-aware in the sense that they preserve the circuit layout.
I think this distinction in the optimizations makes sense. Many of the existing passes can be configured to take a coupling map into account. While that typically limits the optimization potential, it does not require re-routing or re-layouting.
Just my two cents here though.

Shaobo Zhou and others added 5 commits March 29, 2025 19:20

Update predictor(adding callbacks)

129b60f

Update

08889bd

Restore helper.py and predictor.py to match upstream

e2ff3fe

Merge remote-tracking branch 'upstream/main'

1c32d15

Implement new mapping actions

78dc1aa

Update action space and feature space Update actions Update action space

Shaobo-Zhou marked this pull request as ready for review July 29, 2025 16:13

github-advanced-security bot found potential problems Jul 29, 2025

View reviewed changes

src/mqt/predictor/rl/predictorenv.py Fixed Show fixed Hide fixed

src/mqt/predictor/rl/predictorenv.py Fixed Show fixed Hide fixed

src/mqt/predictor/rl/predictorenv.py Fixed Show fixed Hide fixed

Shaobo-Zhou force-pushed the hybrid-mapping branch from 0be7063 to 043be23 Compare July 29, 2025 16:35

Shaobo-Zhou marked this pull request as draft July 29, 2025 16:36

Shaobo-Zhou marked this pull request as ready for review July 29, 2025 16:37

github-advanced-security bot found potential problems Jul 29, 2025

View reviewed changes

src/mqt/predictor/rl/predictorenv.py Fixed Show fixed Hide fixed

Fix: resolve pre-commit issues and add missing annotations

a3ba836

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Remove example_test.py Remove example_test.py

Shaobo-Zhou force-pushed the hybrid-mapping branch 4 times, most recently from fd457a0 to 4b78134 Compare July 29, 2025 19:39

Shaobo-Zhou marked this pull request as draft July 29, 2025 19:53

Fix: resolve pre-commit issues and add missing annotations

5935e6f

Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations Fix: resolve pre-commit issues and add missing annotations

Shaobo-Zhou force-pushed the hybrid-mapping branch from d872e00 to 5935e6f Compare July 30, 2025 16:18

Fix: resolve pre-commit issues and add missing annotations

f71fb29

Shaobo-Zhou force-pushed the hybrid-mapping branch from 5bc0bf7 to f71fb29 Compare July 30, 2025 16:39

Fix: resolve pre-commit issues and add missing annotations

3c7592b

Shaobo-Zhou force-pushed the hybrid-mapping branch from 14b79d4 to 3c7592b Compare July 30, 2025 17:14

Shaobo-Zhou closed this Jul 30, 2025

burgholzer reopened this Jul 30, 2025

Fix mypy errors

6db5c27

burgholzer added enhancement New feature or request minor Part of a minor release refactor PR or issues that refactor code labels Aug 4, 2025

Skip minimums session on Windows due to CI slowness

2692b96

Shaobo-Zhou force-pushed the hybrid-mapping branch from 4c96196 to 2692b96 Compare August 4, 2025 14:53

Fix bugs

f4874e6

Fix bugs Fix bugs Fix bugs

Shaobo-Zhou force-pushed the hybrid-mapping branch from 9a8c77a to f4874e6 Compare August 5, 2025 14:57

Fix bugs

54eec91

Shaobo-Zhou force-pushed the hybrid-mapping branch from 2682c2e to 54eec91 Compare August 7, 2025 10:47

Shaobo-Zhou added 8 commits August 7, 2025 12:52

Use default Qiskit settings for VF2Layout and add assertion for nativ…

845f7de

…e gate check

Debug

3418936

Fix missing argument

ae870cc

Fix warning issues

861bc62

Fix window runtime warning problem

fa989b6

Fix window runtime warning problem

405bd39

Add time limit for VF2PostLayout

7b2f321

Fix windows runtime warning problem

b67d0a6

Fix windows runtime warning problem Fix windows runtime warning issue

Shaobo-Zhou force-pushed the hybrid-mapping branch from b4f563b to b67d0a6 Compare August 7, 2025 18:58

Shaobo-Zhou marked this pull request as ready for review August 8, 2025 12:05

burgholzer requested a review from flowerthrower August 8, 2025 12:14

flowerthrower requested changes Aug 12, 2025

View reviewed changes

Uh oh!

Hybrid mapping #419

Are you sure you want to change the base?

Hybrid mapping #419

Uh oh!

Conversation

Shaobo-Zhou commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

burgholzer commented Jul 30, 2025

Uh oh!

burgholzer commented Jul 30, 2025

Uh oh!

Shaobo-Zhou commented Jul 30, 2025

Uh oh!

burgholzer commented Jul 30, 2025

Uh oh!

burgholzer commented Aug 5, 2025

Uh oh!

Shaobo-Zhou commented Aug 5, 2025

Uh oh!

codecov bot commented Aug 7, 2025

Codecov Report

Uh oh!

Shaobo-Zhou commented Aug 8, 2025

Uh oh!

burgholzer commented Aug 8, 2025

Uh oh!

flowerthrower left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Shaobo-Zhou commented Aug 12, 2025

Uh oh!

flowerthrower commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Shaobo-Zhou commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

burgholzer commented Aug 14, 2025

Uh oh!

Uh oh!

Shaobo-Zhou commented Jul 29, 2025 •

edited

Loading

flowerthrower commented Aug 13, 2025 •

edited

Loading

Shaobo-Zhou commented Aug 14, 2025 •

edited

Loading