SCIP 10

[Joao-Dionisio]

Fix #1006
Fix #955

Naturally the tests won't pass, since we need to update the SCIP version. The IIS stuff still needs more work.

Interesting behaviors:

It seems that an empty relaxator now does not raise an error (or at least pyscipopt is not catching it). Would like to find the explanation before removing the test.

Fixed -> The pricer test is also failing, even with heuristics disabled, it seems that SCIP is finding solutions outside of the pricer.

Not an issue -> Even though implied integers are only gonna be removed in SCIP 11, adding a variable with an M vtype appears to add a continuous variable already. It's probably on purpose, but feels a bit strange to have a variable with a CONTINUOUS vtype and var.isIntegral() returning True.

The remaining issues

• interface SCIP_RATIONAL* with fractions.Fraction
• Cannot convert Python object to SCIP_IIS *

          def iisGreedyMinimize(self, IISfinder iisfinder):
             """
             Perform the greedy deletion algorithm with singleton batches to obtain an irreducible infeasible subsystem (IIS)
             """
      
             PY_SCIP_CALL(SCIPiisGreedyMakeIrreducible(iisfinder._iisfinder))
                                                                ^

[Copilot]

Pull Request Overview

This PR updates support for SCIP version 10, addressing changes in variable types and adding new functionality for statistics reporting and IIS finder callbacks.

• Updated variable type handling and tests (e.g. treating M/IMPLINT as CONTINUOUS in SCIP10)
• Enhanced SCIP interface in scip.pxi and scip.pxd, including new functions for binary/integral checks
• Added JSON-based statistics output and preliminary IIS finder support

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_vars.py	Updated test for variable type, reflecting SCIP10 behavior
tests/test_reader.py	Added test for JSON statistics output
src/pyscipopt/scip.pxi	Updated version definitions and added variable type helper methods; potential issue with non-implied check
src/pyscipopt/scip.pxd	Updated SCIP external declarations accordingly
src/pyscipopt/relax.pxi	Updated docstring for relaxexec method
src/pyscipopt/reader.pxi	Updated function signature for PyReaderWrite
src/pyscipopt/iisfinder.pxi	Added base IISFinder class and implementations for copy, free, and exec callbacks (incomplete)
CHANGELOG.md	Documented new JSON statistics output feature and other changes

[DominikKamp]

It's probably on purpose, but feels a bit strange to have a variable with a CONTINUOUS vtype and var.isIntegral() returning True.

The variable type is just more independent from the integrality property now, so on purpose. Here it means that integrality can be assumed but will not be enforced, maybe worth a small comment. Good that this works!

[Joao-Dionisio]

Good that this works!

Found this comment interesting. It wasn't something that was tested when merging? This gives me an idea. What about having a PySCIPOpt branch always up-to-date with SCIP master, and then people keep adding tests to it? Of course, some things would still need to be tested in SCIP proper, but new/changed API functions could perhaps be tested directly on PySCIPOpt.

[DominikKamp]

Not sure what should have been tested. The implint type just does no longer exist. In principle having a permanent development branch sounds fine, just be aware of that on master API functions can change at any time.

[Joao-Dionisio]

Yeah, I meant mostly as a way to more easily test out new features. At least for me, not having to deal with Criterion, Gdb, and recompiling the tests after every little change is a godsend. Maybe the other devs would also be more willing to create tests this way. And there are cases where it doesn't make a difference where the test is in C-SCIP or PySCIPOpt.

[DominikKamp]

I also believe that if there were a regular development branch, it would be more possible that developers contribute to it, however, some delay makes sense as long as something is under development on the SCIP master branch, which is normally the case until it is released.

[DominikKamp]

Nowadays, SCIPiisGreedyMinimize() is rather called SCIPiisGreedyMakeIrreducible().

[DominikKamp]

For exact SCIP an interface between fractions.Fraction and SCIP_RATIONAL* would be nice, but maybe rather in the long run because the memory handling might be involved.

[Opt-Mucca]

From now on, the binaries will not be uploaded to SCIP's GitHub, so we should get them directly from SCIP's website

@svigerske Created a fully static build????

[svigerske]

From now on, the binaries will not be uploaded to SCIP's GitHub, so we should get them directly from SCIP's website

@svigerske Created a fully static build????

I only made tarballs and zip archives that contain all (shared/dynamic) libs necessary to run on systems with Glibc 2.28 (Linux) / macOS 13 (arm64) / MSVS-redistributables (Windows).

[DominikKamp]

Please revert the last commit, breaks the build with

tomllib.TOMLDecodeError: Cannot overwrite a value (at line 107, column 40).

[Joao-Dionisio]

I'm having some problems with setting up the release, was just trying out the removal of the cached SCIP, in case something outdated got stored. Just reverted.

[Joao-Dionisio]

We can now build the wheels successfully. We just need to check naming conventions for the IIS stuff, and we might want to interface a few more methods. Other than this, the documentation for IIS is also missing, but it's not strictly required for the release, although it would be nice.

[Joao-Dionisio]

I think that is mostly it. Custom IISfinders are still not working properly. I need to find a way to access the IIS object from inside the IISfinder. Problem is that the problem might not even be infeasible, so I will need to think about this later. But, for now, I think we can just do the release.
The presolver plugin will probably warrant a new release, so we can just try to improve it by then.

[Joao-Dionisio]

@Opt-Mucca, can you please take a look at the IIS stuff?
I think the interfacing is now somewhat okay, but maybe you want to have more functionality.
The thing with the test_iisGreddyMakeIrreducible() is just confusing me, though, because it seems that the priorities work as intended, just not the irreducible flag (or something in the interfacing, quite likely).

So, things that happen with the setParam("iis/irreducible", False).

• The custom iisfinder with a higher priority and without the skip option actually finds an IIS, and doesn't get to the greedy one.
• The custom iisfinder with a lower priority gets to the greedy iisfinder first, which actually finds an IIS
• The custom iisfinder with a higher priority and with the skip option also gets to the greedy iisfinder <- this is the behavior we don't expect

We were hoping to isolate the behavior of a custom iis finder by setting the irreducible flag to false.

[DominikKamp]

It really does not make sense that the custom iisfinder with highest priority finds an IIS without stopafterone option but not with stopafterone option. Did the problem or the custom iisfinder change in between? The stopafterone code looks fine as far as I can see (the greedy irreducibilitization is independent as mentioned in the parameter description).

[Joao-Dionisio]

@DominikKamp it's not just the stopafterone that changes, I also change the behavior of the iisfinder. If stopafterone (rather the irreducible flag) the iisfinder doesn't do anything, but it's still called, so I would say it's still expected that the greedy is not called