feat: v7 mkdocs (#3287)

* Pr/3143 (#3144)

* Start migrating the documentation from .rst sphynx to .md material for mkdocs. Add mkdocs.yml with proper configuration, enable automated generation of the doc API from the docstring, and start converting a few pages (index, getting started, developers) for demo.

* Add automated (opt-in) tests of all python codeblocks in the markdown docs using pytest-markdown-docs. Needed to comment 1 small test that seemingly should fail (AttributeError: DefinedNamespace like object has no attribute '_NS', indeed the DefinedNamespace class expect a _NS, so it makes sense it fails) but for some reason it was not properly failing when ran with regular pytest, but it fails with pytest-markdown-docs

* convert all documentations pages to markdown, convert all docstrings to markdown with google style, updated config for mkdocs (readthedocs, tox, task)
fixed #3128

* delete files, dependencies and mentions related to sphinx

* uncomment test previously commented for experimenting with markdown codeblock testing

* update poetry lock

* blacked again

* ignore mypy errors

---------

Co-authored-by: Vincent Emonet <vincent.emonet@gmail.com>

* build: set mkdocs versions for rdflib v7 compatibility

* style: apply black formatting

* chore: mypy comments

* fix: mypy, ruff, and apply black formatting

* build: only test mkdocs build on python 3.11 and above

* build: update poetry lock

* build: fix python version constraint and add comment explaining why the python constraint for mkdocs deps

* build: update poetry lock

* chore: fix mypy and apply formatting

* build: set setuptools version constraints in tox.ini

* build: set berkeleydb version

* build: loosen berkeleydb constraint

---------

Co-authored-by: Nicholas Car <nick@kurrawong.net>
Co-authored-by: Vincent Emonet <vincent.emonet@gmail.com>

Author: 3 people

.github/dependabot.yml

@@ -9,10 +9,6 @@ updates:
     # see https://github.com/dependabot/dependabot-core/pull/10194
     versioning-strategy: auto
     ignore:
-      - dependency-name: sphinx
-        versions:
-          - 3.4.3
-          - 3.5.2
       # We only use setuptools for a couple of things in the test suite
       # There is no need to keep it bleeding-edge. There are too frequent
       # updates to setuptools, requires too much maintenance to keep it up to date.

.github/workflows/validate.yaml

@@ -34,7 +34,6 @@ jobs:
           - python-version: "3.8"
             os: ubuntu-latest
             extensive-tests: true
-            TOXENV_SUFFIX: "-docs"
           - python-version: "3.8"
             os: ubuntu-latest
             extensive-tests: true
@@ -43,7 +42,6 @@ jobs:
             TOXENV_SUFFIX: "-min"
           - python-version: "3.9"
             os: ubuntu-latest
-            TOXENV_SUFFIX: "-docs"
           - python-version: "3.10"
             os: ubuntu-latest
             TOX_EXTRA_COMMAND: "- black --check --diff ./rdflib"

.gitignore

@@ -1,13 +1,14 @@
 .flakeheaven_cache/
 RDFLib.sublime-project
-/docs/_build/
 RDFLib.sublime-workspace
 coverage/
 cov.xml
 /.hgtags
 /.hgignore
 build/
 /docs/draft/
+/docs/apidocs/
+/docs/_build/
 *~
 test_reports/*latest.ttl
 # PyCharm

.readthedocs.yaml

@@ -9,12 +9,11 @@ formats:
   - htmlzip
   - pdf
 
+# https://docs.readthedocs.com/platform/stable/intro/mkdocs.html
 build:
-  os: ubuntu-20.04
+  os: "ubuntu-24.04"
   tools:
-    # Using 3.9 as earlier versions have trouble generating documentation for
-    # `@typing.overload`` with type aliases.
-    python: "3.9"
+    python: "3"
   jobs:
     post_create_environment:
     # Using requirements-poetry.in as requirements-poetry.txt has conflicts with
@@ -24,8 +23,6 @@ build:
     - poetry export --only=main --only=docs --without-hashes -o requirements.txt
     - pip install --no-cache-dir -r requirements.txt
     - pip install .
-    - python -c "from rdflib import Graph; print(Graph)"
 
-sphinx:
-  configuration: docs/conf.py
-  fail_on_warning: true
+mkdocs:
+  configuration: mkdocs.yml

MANIFEST.in

@@ -9,4 +9,5 @@ recursive-include examples *.py
 graft test
 graft docs
 prune docs/_build
+prune site/
 global-exclude *.pyc *$py.class

README.md

@@ -1,7 +1,7 @@
-![](docs/_static/RDFlib.png)    
+![](docs/_static/RDFlib.png)
+
+# RDFLib
 
-RDFLib
-======
 [![Build Status](https://github.com/RDFLib/rdflib/actions/workflows/validate.yaml/badge.svg?branch=main)](https://github.com/RDFLib/rdflib/actions?query=branch%3Amain)
 [![Documentation Status](https://readthedocs.org/projects/rdflib/badge/?version=latest)](https://rdflib.readthedocs.io/en/latest/?badge=latest)
 [![Coveralls branch](https://img.shields.io/coveralls/RDFLib/rdflib/main.svg)](https://coveralls.io/r/RDFLib/rdflib?branch=main)
@@ -30,8 +30,10 @@ The RDFlib community maintains many RDF-related Python code repositories with di
 
 * [rdflib](https://github.com/RDFLib/rdflib) - the RDFLib core
 * [sparqlwrapper](https://github.com/RDFLib/sparqlwrapper) - a simple Python wrapper around a SPARQL service to remotely execute your queries
-* [pyLODE](https://github.com/RDFLib/pyLODE) - An OWL ontology documentation tool using Python and templating, based on LODE
-* [pySHACL](https://github.com/RDFLib/pySHACL) - A pure Python module which allows for the validation of RDF graphs against SHACL graphs
+* [pyLODE](https://github.com/RDFLib/pyLODE) - An OWL ontology documentation tool using Python and templating, based on LODE.
+* [pyrdfa3](https://github.com/RDFLib/pyrdfa3) - RDFa 1.1 distiller/parser library: can extract RDFa 1.1/1.0 from (X)HTML, SVG, or XML in general.
+* [pymicrodata](https://github.com/RDFLib/pymicrodata) - A module to extract RDF from an HTML5 page annotated with microdata.
+* [pySHACL](https://github.com/RDFLib/pySHACL) - A pure Python module which allows for the validation of RDF graphs against SHACL graphs.
 * [OWL-RL](https://github.com/RDFLib/OWL-RL) - A simple implementation of the OWL2 RL Profile which expands the graph with all possible triples that OWL RL defines.
 
 Please see the list for all packages/repositories here:
@@ -136,18 +138,21 @@ g.add((
     Literal("Nick", datatype=XSD.string)
 ))
 ```
+
 The triple (in n-triples notation) `<http://example.com/person/nick> <http://xmlns.com/foaf/0.1/givenName> "Nick"^^<http://www.w3.org/2001/XMLSchema#string> .`
 is created where the property `FOAF.givenName` is the URI `<http://xmlns.com/foaf/0.1/givenName>` and `XSD.string` is the
 URI `<http://www.w3.org/2001/XMLSchema#string>`.
 
 You can bind namespaces to prefixes to shorten the URIs for RDF/XML, Turtle, N3, TriG, TriX & JSON-LD serializations:
 
- ```python
+```python
 g.bind("foaf", FOAF)
 g.bind("xsd", XSD)
 ```
+
 This will allow the n-triples triple above to be serialised like this:
- ```python
+
+```python
 print(g.serialize(format="turtle"))
 ```
 

Taskfile.yml

@@ -170,19 +170,19 @@ tasks:
     desc: Clean generated documentation
     cmds:
       - task: _rimraf
-        vars: { RIMRAF_TARGET: "docs/_build/" }
+        vars: { RIMRAF_TARGET: "site/" }
 
   docs:
     desc: Build documentation
     cmds:
       - echo "PYTHONPATH=${PYTHONPATH}"
-      - "{{.VENV_PYTHON}} -m sphinx.cmd.build -T -W -b html -d docs/_build/doctree docs docs/_build/html {{.CLI_ARGS}}"
+      - "{{.VENV_PYTHON}} -m mkdocs build {{.CLI_ARGS}}"
 
   docs:live-server:
     desc: Run a live server on generated docs
     cmds:
       - 'echo "NOTE: Docs must be built for this to work"'
-      - npx -p live-server live-server docs/_build/html/ {{.CLI_ARGS}}
+      - npx -p live-server live-server site/ {{.CLI_ARGS}}
 
   default:
     desc: Run validate
@@ -356,7 +356,7 @@ tasks:
         cd var/test-sdist/rdflib-*
         poetry install
         poetry run mypy --show-error-context --show-error-codes -p rdflib
-        poetry run sphinx-build -T -W -b html -d docs/_build/doctree docs docs/_build/html
+        poetry run mkdocs build
         poetry run pytest
 
   test:no_internet:

devtools/diffrtpy.py

@@ -3,18 +3,19 @@
 This is a tool that can be used with git difftool to generate a diff that
 ignores type hints and comments.
 
-The name of this script, ``diffrtpy`` is short for "diff runtime python", as
+The name of this script, `diffrtpy` is short for "diff runtime python", as
 this will only compare the parts of the python code that has a runtime impact.
 
 This is to make it easier to review PRs that contain type hints.
 
 To use this script
 
-.. code-block:: bash
-    task run -- python -m pip install --upgrade strip-hints black python-minifier
-    PYLOGGING_LEVEL=INFO task run -- git difftool -y -x $(readlink -f devtools/diffrtpy.py) upstream/main | tee /var/tmp/compact.diff
+```bash
+task run -- python -m pip install --upgrade strip-hints black python-minifier
+PYLOGGING_LEVEL=INFO task run -- git difftool -y -x $(readlink -f devtools/diffrtpy.py) upstream/main | tee /var/tmp/compact.diff
+```
 
-Then attach ``/var/tmp/compact.diff`` to the PR.
+Then attach `/var/tmp/compact.diff` to the PR.
 """
 
 from __future__ import annotations

docs/CONTRIBUTING.md

@@ -46,7 +46,7 @@ Some ways in which you can contribute to RDFLib are:
 ## Pull Requests
 
 Contributions that involve changes to the RDFLib repository have to be made with
-pull requests and should follow the [RDFLib developers guide](./developers.rst).
+pull requests and should follow the [RDFLib developers guide](./developers.md).
 
 For changes that add features or affect the public API of RDFLib, it is
 recommended to first open an issue to discuss the change before starting to work
@@ -55,5 +55,5 @@ spending time on it.
 
 ## Code of Conduct
 
-All contributions to the project should be consistent with the [code of
-conduct](./CODE_OF_CONDUCT.md) adopted by RDFLib.
+All contributions to the project should be consistent with the
+[code of conduct](./CODE_OF_CONDUCT.md) adopted by RDFLib.