Not working on Windows

[LostInDarkMath]

Hi there!

I'm using your library on Linux and it works fine. Now I tried it on WIndows (WIndows 10) and it doesn't work. Is there a way to make your library work with Windows?

Here is the error I'm getting:

Waiting for container to be ready.

test setup failed
mariadb_container = <pytest_docker_tools.wrappers.container.Container object at 0x0000020293B30460>

    @pytest.fixture(scope="session")
    def mariadb(mariadb_container):
>       mariadb_ip, mariadb_port = mariadb_container.get_addr('3306/tcp')

..\..\..\..\fixtures\mariadb.py:52: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pytest_docker_tools.wrappers.container.Container object at 0x0000020293B30460>
port = '3306/tcp'

    def get_addr(self, port):
        if tests_inside_container():
            return (self.ips.primary, int(port.split("/")[0]))
        else:
>           return ("127.0.0.1", self.ports[port][0])
E           IndexError: list index out of range

C:\Users\Test\AppData\Local\pypoetry\Cache\virtualenvs\maddox-IYXXq9hF-py3.10\lib\site-packages\pytest_docker_tools\wrappers\container.py:248: IndexError

I don't anything special in my code:

mariadb_container = container(
    name='{mariadb_service_name}',
    image='{mariadb_image.id}',
    ports={'3306/tcp': None},
    network='{test_net.name}',
    tmpfs={'/var/lib/mysql': ''},
    environment={
        'MYSQL_ROOT_PASSWORD': MARIA_DB_ROOT_PASSWORD,
    },
    scope="session",
    command="--innodb-ft-min-token-size=1",
    restart_policy={'Name': 'always'}
)

@pytest.fixture(scope="session")
def mariadb(mariadb_container):
    mariadb_ip, mariadb_port = mariadb_container.get_addr('3306/tcp')
    mariadb = MariaDBData(name=mariadb_container.name, host=mariadb_ip, port=mariadb_port, internal_port='3306',
                          username=MARIA_DB_USER, password=MARIA_DB_ROOT_PASSWORD, schema=DATABASE_NAME)
    return mariadb

How can I fix this?

[Jc2k]

Hi! Sorry I only use Linux and macOS myself so I know very little about how Docker on Windows and how it differs. I certainly don't do any testing there.

The self.ports you see above is IIRC a wrapper around ["NetworkSettings"]["Ports"] as returned by the docker api. So someone with a Windows box needs to figure out why that field is different and what we can do instead.

[LostInDarkMath]

Thanks for your fast reply! I'll try to investigate.

[LostInDarkMath]

It's very strange: Sometimes it works fine but most of the time it fails with the error above. It seems to be flaky.

But there are also lots of other issues with your library on windows:

1. reusable containers does not work
2. Your tests aren't executable.

[Jc2k]

I'm not surprised - I mean there are generally lots of failures when you port software to a completely new OS 😆 I would take patches for Windows support if (a) someone was willing to help maintain it and (b) we could get CI running for it.

I did look into (b), but whilst it's quite easy to do Python CI on Windows with GitHub actions, I don't think you could really test Docker for Windows on a Windows GitHub actions.

Unfortunately all my "volunteer" hours are set aside for Home Assistant, so i'm unlikely to be able to help much beyond patch review.

[alpex8]

Other alternatives would be executing tests in a container or using WSL to execute tests. The latter is supported in PyCharm Pro and I´ve found a WSL supporting extension for VSCode.

Maybe this are valid workarounds. This could even be mentioned in the READ.ME in case other users are affected as well.

[LostInDarkMath]

Yes, I completely agee with you. The WSL seems to be a very good solution, I'll try it later in detail. I think it isn't worth the effort to make this package 100 % percent Windows compatible. But it would be very nice if you could add a section in the README.md that explains that this package is not designed to work on Windows and Windows users should use the WSL.

Using WSL with PyCharm Pro: https://www.jetbrains.com/help/pycharm/using-wsl-as-a-remote-interpreter.html

[msukmanowsky]

@Jc2k I also experienced this consistently on MacOS Sequoia 15.3.1 (24D70) running with pytest-xdist[psutil]==3.6.1 and pytest-docker-tools==3.1.3.

Error when trying to access container.ports. I'm sure it'll be tricky to reproduce as it does seem like a race condition as mentioned in #42, but the example looked something like this:

# conftest
import pytest
import pytest_docker_tools
from pytest_docker_tools.wrappers import Container

postgres_image = pytest_docker_tools.fetch(
    repository="ankane/pgvector", tag="latest", scope="session"
)
backend_network = pytest_docker_tools.network(scope="session")
postgres_container = pytest_docker_tools.container(
    image="{postgres_image.id}",
    ports={"5432/tcp": None},
    network="{backend_network.name}",
    environment={
        "POSTGRES_USER": "postgres",
        "POSTGRES_PASSWORD": "postgres",
        "POSTGRES_DB": "postgres",
    },
    scope="session",
)

@pytest.fixture(scope="session")
def db_info(postgres_container: Container):
    pg_user = postgres_container.env["POSTGRES_USER"]
    pg_password = postgres_container.env["POSTGRES_PASSWORD"]
    pg_db = postgres_container.env["POSTGRES_DB"]
    # this would routinely raise IndexError: list index out of range on at least one container
    pg_port = postgres_container.ports["5432/tcp"]

    return pg_user, pg_password, pg_db, pg_port

# test_example.py
import pytest

pytestmark = pytest.mark.anyio

async def test_1(db_info):
    print(db_info)

async def test_2(db_info):
    print(db_info)

As is discussed in #42, the issue is that docker-py's container.attrs attribute is cached and can, even though container.status == "running", be stale.

My workaround was as follows:

@pytest.fixture(scope="session")
def db_info(postgres_container: Container):
    # The postgres_container.ports attribute is a PortMap that uses a pydocker
    # container's attrs['NetworkSettings']['Ports'] attribute as its source.
    # These attrs are cached and it seems, when running under xdist, the cache
    # is not updated in time for the fixture to get the correct port.
    start = time.time()
    while not postgres_container._container.attrs["NetworkSettings"]["Ports"][
        "5432/tcp"
    ]:
        postgres_container._container.reload()
        time.sleep(0.1)
        if time.time() - start > 10:
            raise Exception("Postgres container failed to start.")
    db_port = postgres_container._container.attrs["NetworkSettings"]["Ports"]["5432/tcp"][
        0
    ]["HostPort"]
    # ...other code

It's annoying but a similar approach could be applied to PortMap whereby you either try to do container.reload() before accessing container.attrs or you reload up to a timeout as I did here.

[tworedz]

@msukmanowsky Hi! Can you please check my version of patch? You can find it in #42 related pr. Maybe it would also work on windows?