After the change from Cython to ctypes we do not need to compile with numpy/python headers and therefore there should be no problem in removing the pinned configurations