Refactor: Remove Cython - Reduce tech debt (#346)

* uncythonize, pytest update

* tests passing; fix add_conditions

* Fixup tests

* refactor: UniformFloat

* optim: UniformFloat get neighbors

* refactor: deprecate UniformFloat is_legal_vector

* big overhaul

* fix: depercation wraning of `np.integer` for dtype

* refactor: Big internal rewrite, passing tests

* test passing checkpoint

* pythonize conditionals

* optim: Topological sort and vectorize cond/forbiddens

* optim: Further small optimizations

* optim: Transpose sampling matrix for cache locality

* optim: Add test script for neighbor sampling

* test: Repass tests

* tmpcommmit - lets pray

* refactor: Factor out DAG structure into dedicated class

* refactor: Move things out of `c_util`

* fix: Minimum spanning

* fix: min-cond-span sorted by min(depth of all children)

* refactor: All tests and benchmarks pass

* refactor: Change deprecated test_util

* Revert "refactor: Change deprecated test_util"

This reverts commit da09f7813aaec6d65e510ae10907bf08bdf37da8.

* doc: Remove mention of cython

* build: Remove the use "build" in deps

* ci: Update test workflow

* ci: Remove conda tests since we no longer build w/ cython

* fix(typing): Use typing extension for future TypeAlias

* ci: Use caching for env across tests

* fix(typing): Use `Union` instead of 3.10 syntax

* ci: Remove git porcelain check

* fix(typing): Use typing for inheritance to support <= 3.9

* fix: Remove `strict=True` from `zip` to support <= 3.10

* fix(typing): Using `typing.Mapping` for inheritance

* fix: Move benchmarked functions as main functions to use

* style: Update ruff to use numpy syntax

* fix(functional): `linspace_chunked` acts more like `np.linspace`

* doc: Run `ruff --fix`

* style: Fix ruff checks

* refactor: Convert to `src` layout

* ci: Ignore use of `eval` in test

* style: Fix remaning ruff errors

* refactor: Json encoding

* refactor: Sorry, big dump

* optim: Further speedups to one exchange

* optim: Further speedups

* fix: Hyperparameter attributes are simple types

* fix: Sampling from empty configuration spaces

* fix: Revert changes from different branch

* fix: Test for default_value type

* fix: Deserialization with `"r"`

* fix: Conjunctions match clauses for equality #334

* fix: Import TypeAlias from typing_extensions

* fix: Remove unreliable __hash__ from ConfigurationSpace

* fix: Ruff errors

* typing: Soft deprecate `pcs` and `pcs_new`

* typing: Fix decoder type for json read and write

* ci: Move to ruff for linting

* fix: Better condition cloning in replace hyperparameters

* fix: Use 3.8 compatable isinstance checks

* typing: `configuration_space.py`

* typing: `dictionary.py`

* typing: `_condition_tree.py`

* typing: `forbidden.py`

* ci: Add `scipy.*` to list of mypy ignore missing import

* typing: `conditions.py`

* typing: Fix hyperparameter related issues

* test: Fixups

* refactor: Remove `nx` vendored lib

* test: Use new `add` as old was deprecated

* refactor: Remove deprecations

* fix: Use `isinstance(..., (type1, type2))`

* fix: Better casting for reading from pcs

* debug: Check windows issue of not legal

* ci: Don't use cache for installation

* fix: Change error type to IllegalVectorizedValue

* debug: More debug prints for ci

* ci: Exclude 3.8 3.9 mac

* debug: More ci debugging statements

* fix: UnitScaler promotes to 64bit to prevent overflow on windows

* fix: Remove non assigned type

* fix: Address minor comments

* fix: Minor review comments

* doc: Overhaul to mkdocs, partially complete

* doc: Fix missing comma

* doc: Typo fixes

* doc: typo fix again...

* doc: Major doc overhaul to mkdocs

* fix: Add backwards compatible `check_forbidden`

* fix: Allow backwards compat `"default"` key in encoded spaces

* fix: Backwards compatible private methods

* refactor: Use `check_` instead of `is_` as method raises

* doc: Forbiddens

* test: fix test funcion name

* fix(Forbidden): Uncomment value checking on `ForbiddenEqualsClaus`

* ci: Update doc and test workflows

* test: Remove stray function

* doc: Type fix

* doc: Add note to `_unsafe` code in `hp_components`

* doc: Typo fix

* doc: Improve code doc attribute clarity

* doc: Change `note` to `todo`

* doc: Fix documentation for `continuous_neighborhood`

Author: eddiebergman

.github/workflows/docs.yml

@@ -1,72 +1,27 @@
+# This workflow is just to test that the docs build successfully.
 name: docs
-
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
 on:
-  # Trigger manually
   workflow_dispatch:
-
-  # Trigger on any push to the main
   push:
     branches:
       - main
-      - development
-
-  # Trigger on any push to a PR that targets main
   pull_request:
     branches:
       - main
-      - development
-
 permissions:
   contents: write
-
-env:
-  name: "ConfigSpace"
-
 jobs:
-  build-and-deploy:
+  build:
     runs-on: ubuntu-latest
     steps:
-    - name: Checkout
-      uses: actions/checkout@v3
-
-    - name: Setup Python
-      uses: actions/setup-python@v3
-      with:
-        python-version: "3.8"
-
-    - name: Install dependencies
-      run: |
-        pip install build
-        pip install ".[docs]"
-
-    - name: Make docs
-      run: |
-        make clean
-        make docs
-
-    - name: Pull latest gh-pages
-      if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
-      run: |
-        cd ..
-        git clone https://github.com/${{ github.repository }}.git --branch gh-pages --single-branch gh-pages
-
-    - name: Copy new docs into gh-pages
-      if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
-      run: |
-        branch_name=${GITHUB_REF##*/}
-        cd ../gh-pages
-        rm -rf $branch_name
-        cp -r ../${{ env.name }}/docs/build/html $branch_name
-
-    - name: Push to gh-pages
-      if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
-      run: |
-        last_commit=$(git log --pretty=format:"%an: %s")
-        cd ../gh-pages
-        branch_name=${GITHUB_REF##*/}
-        git add $branch_name/
-        git config --global user.name 'Github Actions'
-        git config --global user.email 'not@mail.com'
-        git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
-        git commit -am "$last_commit"
-        git push
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+      - name: "Install dependancies"
+        run: python -m pip install -e ".[dev]"
+      - name: "Build Docs"
+        run: mkdocs build --clean --strict

.github/workflows/pytest.yml

@@ -1,4 +1,7 @@
 name: Tests
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
 
 on:
   # Allow to manually trigger through github API
@@ -8,13 +11,11 @@ on:
   push:
     branches:
       - main
-      - development
 
   # Triggers with push to a pr aimed at master
   pull_request:
     branches:
       - main
-      - development
 
   schedule:
     # Every day at 7AM UTC
@@ -31,21 +32,9 @@ env:
     --durations=20
     -v
 
-  # Version to run code-cov on
-  # NOTE: These are only acessible inside a jobs **steps** and not in the job setup,
-  #   Hence, some of these varialbes are copied and are just here for reference
-  #
-  code-cov-active: true  # Copied in job setup
-  code-cov-os: ubuntu-latest  # Copied in job setup
-  code-cov-python-version: "3.8"
-  code-cov-args: >-
-    --cov=ConfigSpace
-    --cov-report=xml
-
 jobs:
+  test:
 
-  # General unit tests
-  source-test:
     name: ${{ matrix.python-version }}-${{ matrix.os }}
 
     runs-on: ${{ matrix.os }}
@@ -56,153 +45,24 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
         os: ["ubuntu-latest", "macos-latest", "windows-latest"]
 
     steps:
     - name: Checkout
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
 
     - name: Setup Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
-
-    - name: Install ${{ env.package-name }}
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install -e ".${{ env.extra-requires }}"
-
-    - name: Store git status
-      id: status-before
-      shell: bash
-      run: |
-        echo "::set-output name=BEFORE::$(git status --porcelain -b)"
-
-    - name: Tests
-      timeout-minutes: 45
-      run: |
-        pytest ${{ env.pytest-args }} ${{ env.test-dir }}
-
-    - name: Check for files left behind by test
-      run: |
-        before="${{ steps.status-before.outputs.BEFORE }}"
-        after="$(git status --porcelain -b)"
-        if [[ "$before" != "$after" ]]; then
-            echo "git status from before: $before"
-            echo "git status from after: $after"
-            echo "Not all generated files have been deleted!"
-            exit 1
-        fi
-
-  # Testing with conda
-  conda-tests:
-    name: conda-${{ matrix.python-version }}-${{ matrix.os }}
-    runs-on: ${{ matrix.os }}
-    defaults:
-      run:
-        shell: bash -l {0}  # Default to using bash on all and load (-l) .bashrc which miniconda uses
-
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
-        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
-
-    steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-
-    - name: Conda install
-      uses: conda-incubator/setup-miniconda@v2
+      uses: actions/setup-python@v5
       with:
-        auto-update-conda: true
         python-version: ${{ matrix.python-version }}
+        cache: pip
+        cache-dependency-path: pyproject.toml
 
     - name: Install ${{ env.package-name }}
       run: |
-        python -V
-        python -m pip install --upgrade pip
-        python -m pip install wheel
         python -m pip install -e ".${{ env.extra-requires }}"
 
     - name: Tests
-      timeout-minutes: 45
       run: |
-          pytest ${{ env.pytest-args }} ${{ env.test-dir }}
-          #
-  # Testing a dist install
-  dist-test:
-    name: dist-${{ matrix.python-version }}-${{ matrix.os }}
-
-    runs-on: ${{ matrix.os }}
-    defaults:
-      run:
-        shell: bash
-
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
-        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
-
-    steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-
-    - name: Setup Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
-
-    - name: Create sdist
-      id: sdist
-      run: |
-        python -m pip install --upgrade pip build
-        python -m build --sdist
-        echo "sdist_name=$(ls -t dist/${{ env.package-name }}-*.tar.gz | head -n 1)" >> "$GITHUB_ENV"
-
-    - name: Install ${{ env.package-name }}
-      run: |
-        python -m pip install ${{ env.sdist_name }}${{ env.extra-requires }}
-
-    - name: Tests
-      timeout-minutes: 45
-      run: |
-        pytest ${{ env.pytest-args }} ${{ env.test-dir }}
-
-
-  # Testing with codecov coverage uploaded
-  codecov-test:
-    name: codecov-test
-
-    runs-on: "ubuntu-latest"
-    defaults:
-      run:
-        shell: bash
-
-    steps:
-    - name: Checkout
-      uses: actions/checkout@v2
-
-    - name: Setup Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ env.code-cov-python-version }}
-
-    - name: Install ${{ env.package-name }}
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install wheel
-        python -m pip install -e ".${{ env.extra-requires }}"
-
-    - name: Tests
-      timeout-minutes: 45
-      run: |
-        pytest ${{ env.pytest-args }} ${{ env.code-cov-args }} ${{ env.test-dir }}
-
-    - name: Upload coverage
-      uses: codecov/codecov-action@v2
-      with:
-        fail_ci_if_error: true
-        verbose: true
+        pytest ${{ env.test-dir }}

.gitignore

@@ -9,6 +9,7 @@ docs/examples/*
 # C extensions
 *.so
 *.c
+*.jukit
 
 # Packages
 *.egg
@@ -74,4 +75,5 @@ prof/
 .venv
 
 # Running pre-commit seems to generate these
-.mypy_cache
+.mypy_cache
+*.speedscope

.pre-commit-config.yaml

@@ -38,32 +38,15 @@ repos:
     hooks:
       - id: mypy
         files: '^ConfigSpace/.*\.py$'
+        additional_dependencies:
+          - "types-pyyaml"
         args:
           - "--no-warn-return-any" # Disable this because it doesn't know about 3rd party imports
           - "--ignore-missing-imports"
           - "--show-traceback"
-      - id: mypy
-        files: '^test/.*\.py$'
-        args:
-          - "--no-warn-return-any" # Disable this because it doesn't know about 3rd party imports
-          - "--ignore-missing-imports"
-          - "--show-traceback"
-          - "--disable-error-code"
-          - "no-untyped-def"
-
-  - repo: https://github.com/psf/black
-    rev: 23.3.0
-    hooks:
-      - id: black
-        args: ["--config=pyproject.toml"]
   - repo: https://github.com/charliermarsh/ruff-pre-commit
-    rev: v0.0.263
+    rev: v0.4.0
     hooks:
       - id: ruff
-        args: ["--fix", "ConfigSpace", "test"]
-  - repo: https://github.com/MarcoGorelli/cython-lint
-    rev: v0.15.0
-    hooks:
-      - id: cython-lint
-        args: [--ignore=E501]
-      - id: double-quote-cython-strings
+        args: [--fix, --exit-non-zero-on-fix, --no-cache]
+      - id: ruff-format