Added all tests to mfatestcase branch

[mdewhirst]

There are 491 tests in this PR. Four are failing in test_helpers.py due to "TypeError: Object of type bytes is not JSON serializable".

Also, there are a number of exceptions which are swallowed successfully by mfa code and appear in the test console output without failing those tests. To see which tests are revealing those exceptions, run them with verbosity=2 to show the test names.

The base test class inherited by all tests (except test_mfatestcase) is MFATestCase which in turn inherits Django's TestCase or TransactionTestCase depending on the dbms being used. The base class has a number of helper methods and assertion methods to help when writing new tests or changing existing ones. More detail in tests/README.md.

There is a suite of mermaid flow diagrams covering mfa Methods. MFA_Methods_Diagrams.md

There is also a Python script mfatestcase_analysis.py which writes an analysis of which mfatestcase.py functions are called by which tests. Run this again to refresh mfatestcase_usage_analysis.md if tests are added/subtracted.

This is a first stab at these tests and it is expected that errors will be uncovered. The most critical piece of code is test_mfatestcase.py because that guarantees the base class for all other tests. Changes to the base class or that test module should be made with care.

[mdewhirst]

Further to the above, this PR was tested under Django 4.2 with PostgreSQL 14 and Django 5.2 with SQLite3

[mkalioby]

Hello Mike,

I ran the test cases today and it works with Python 3.11 but not with Python 3.10 which is a problem as Python 3.10 shall be supported till Oct-26. I think the issue is in this line

  File "/home/mohamed/django-mfa2/example/mfa/tests/test_u2f.py", line 125
    ) as mock_complete_reg,  # Mock MFA project function to provide reasonable input for bind function
       ^
SyntaxError: invalid syntax

In the same time, with Python 3.11, I get the following tracebacks but the test doesn't fail which is weired

.......Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py", line 194, in authenticate_complete
    data = json.loads(request.body)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

..........Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py", line 194, in authenticate_complete
    data = json.loads(request.body)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

....Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py", line 89, in complete_reg
    data = json.loads(request.body)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

.Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py", line 91, in complete_reg
    auth_data = server.register_complete(
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1124, in __call__
    return self._mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1128, in _mock_call
    return self._execute_mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1183, in _execute_mock_call
    raise effect
ValueError: Invalid credential data

.Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py", line 89, in complete_reg
    data = json.loads(request.body)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

...............................................................................................................................................................................Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py", line 161, in verify
    uk = User_Keys.objects.get(
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
mfa.models.User_Keys.DoesNotExist: User_Keys matching query does not exist.

.
1518 mfa.tests.mfatestcase False Test exception for coverage
.Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py", line 161, in verify
    uk = User_Keys.objects.get(
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
mfa.models.User_Keys.DoesNotExist: User_Keys matching query does not exist.

...................................................................................../home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/fields/__init__.py:1595: RuntimeWarning: DateTimeField User_Keys.expires received a naive datetime (2026-04-24 17:03:09.965747) while time zone support is active.
  warnings.warn(
................Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py", line 161, in verify
    uk = User_Keys.objects.get(
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1124, in __call__
    return self._mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1128, in _mock_call
    return self._execute_mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 1183, in _execute_mock_call
    raise effect
Exception: Database error

..........Traceback (most recent call last):
  File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py", line 161, in verify
    uk = User_Keys.objects.get(
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
mfa.models.User_Keys.DoesNotExist: User_Keys matching query does not exist.

Can you please check what are these issues.

[mdewhirst]

Mohamed, as a general principle, if the fix is not obvious, I think we should simply chop out every problematic test - entirely - then specify any new tests based on coverage required. I must take this opportunity to apologise for such a monster PR. Chopping tests until it is manageable is the only way forward I can see. I will look at this tomorrow and get back with a fix or removal! Cheers, Mike Sent from [Proton Mail](https://proton.me/mail/home) for Android.

…

-------- Original Message --------

On Monday, 10/27/25 at 04:10 Mohamed El-Kalioby ***@***.***> wrote: mkalioby left a comment [(mkalioby/django-mfa2#108)](#108 (comment)) Hello Mike, I ran the test cases today and it works with Python 3.11 but not with Python 3.10 which is a problem as Python 3.10 shall be supported till Oct-26. I think the issue is in this line File "/home/mohamed/django-mfa2/example/mfa/tests/test_u2f.py" , line 125 ) as mock_complete_reg , # Mock MFA project function to provide reasonable input for bind function ^ SyntaxError : invalid syntax In the same time, with Python 3.11, I get the following tracebacks but the test doesn't fail which is weired ....... Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py" , line 194 , in authenticate_complete data = json . loads ( request . body ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/__init__.py" , line 346 , in loads return _default_decoder . decode ( s ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 337 , in decode obj , end = self . raw_decode ( s , idx = _w ( s , 0 ). end ()) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 355 , in raw_decode raise JSONDecodeError ( "Expecting value" , s , err . value ) from None json . decoder . JSONDecodeError : Expecting value : line 1 column 1 ( char 0 ) .......... Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py" , line 194 , in authenticate_complete data = json . loads ( request . body ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/__init__.py" , line 346 , in loads return _default_decoder . decode ( s ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 337 , in decode obj , end = self . raw_decode ( s , idx = _w ( s , 0 ). end ()) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 355 , in raw_decode raise JSONDecodeError ( "Expecting value" , s , err . value ) from None json . decoder . JSONDecodeError : Expecting value : line 1 column 1 ( char 0 ) .... Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py" , line 89 , in complete_reg data = json . loads ( request . body ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/__init__.py" , line 346 , in loads return _default_decoder . decode ( s ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 337 , in decode obj , end = self . raw_decode ( s , idx = _w ( s , 0 ). end ()) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 355 , in raw_decode raise JSONDecodeError ( "Expecting value" , s , err . value ) from None json . decoder . JSONDecodeError : Expecting value : line 1 column 1 ( char 0 ) . Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py" , line 91 , in complete_reg auth_data = server . register_complete ( ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/unittest/mock.py" , line 1124 , in __call__ return self . _mock_call ( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/unittest/mock.py" , line 1128 , in _mock_call return self . _execute_mock_call ( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/unittest/mock.py" , line 1183 , in _execute_mock_call raise effect ValueError : Invalid credential data . Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/FIDO2.py" , line 89 , in complete_reg data = json . loads ( request . body ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/__init__.py" , line 346 , in loads return _default_decoder . decode ( s ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 337 , in decode obj , end = self . raw_decode ( s , idx = _w ( s , 0 ). end ()) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/json/decoder.py" , line 355 , in raw_decode raise JSONDecodeError ( "Expecting value" , s , err . value ) from None json . decoder . JSONDecodeError : Expecting value : line 1 column 1 ( char 0 ) ............................................................................................................................................................................... Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py" , line 161 , in verify uk = User_Keys . objects . get ( ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/manager.py" , line 87 , in manager_method return getattr ( self . get_queryset (), name )( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/query.py" , line 637 , in get raise self . model . DoesNotExist ( mfa . models . User_Keys . DoesNotExist : User_Keys matching query does not exist . . 1518 mfa . tests . mfatestcase False Test exception for coverage . Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py" , line 161 , in verify uk = User_Keys . objects . get ( ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/manager.py" , line 87 , in manager_method return getattr ( self . get_queryset (), name )( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/query.py" , line 637 , in get raise self . model . DoesNotExist ( mfa . models . User_Keys . DoesNotExist : User_Keys matching query does not exist . ..................................................................................... / home / mohamed / django - mfa2 / env3 . 11 / lib / python3 . 11 / site - packages / django / db / models / fields / __init__ . py : 1595 : RuntimeWarning : DateTimeField User_Keys . expires received a naive datetime ( 2026 - 04 - 24 17 : 03 : 09.965747 ) while time zone support is active . warnings . warn ( ................ Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py" , line 161 , in verify uk = User_Keys . objects . get ( ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/unittest/mock.py" , line 1124 , in __call__ return self . _mock_call ( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/unittest/mock.py" , line 1128 , in _mock_call return self . _execute_mock_call ( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/usr/lib/python3.11/unittest/mock.py" , line 1183 , in _execute_mock_call raise effect Exception : Database error .......... Traceback ( most recent call last ): File "/home/mohamed/django-mfa2/example/mfa/TrustedDevice.py" , line 161 , in verify uk = User_Keys . objects . get ( ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/manager.py" , line 87 , in manager_method return getattr ( self . get_queryset (), name )( * args , ** kwargs ) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ File "/home/mohamed/django-mfa2/env3.11/lib/python3.11/site-packages/django/db/models/query.py" , line 637 , in get raise self . model . DoesNotExist ( mfa . models . User_Keys . DoesNotExist : User_Keys matching query does not exist . Can you please check what are these issues. — Reply to this email directly, [view it on GitHub](#108 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AANG4B5LLPTKWN5ZY37YOTD3ZT57TAVCNFSM6AAAAACITOGTGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTINBYG4YDGNRTG4). You are receiving this because you authored the thread.Message ID: ***@***.***>

[mdewhirst]

OK ... there was a Python 10 problem with inline comments in context managers. My next commit will fix that.

The tracebacks in passing tests gave me a headache until I realised the actual MFA code is printing them if an error bubbles up to the last exception handler. The code handles the exception so things don't crash but prints the traceback. I guess it leaves a message in the logs for users. Those 8 tests have scenarios which exercise the final exception handler and test that the print message happens.

What would you like to do about that?

Also, as you can see I have commented out the four failing tests we discussed previously.

If we follow my initial thoughts from above, I should drop those 8 tests until you think they are needed - or perhaps find a way to prevent the traceback being printed in the test output (without touching the MFA code)?