feat: Set up comprehensive Python testing infrastructure with Poetry

[llbbl]

Set up Python Testing Infrastructure

Summary

This PR establishes a comprehensive testing infrastructure for the project using Poetry as the package manager and pytest as the testing framework. The setup provides a solid foundation for writing and running tests with coverage reporting.

Changes Made

Package Management

• Poetry Setup: Created pyproject.toml with Poetry configuration
• Dependency Migration: Migrated all dependencies from requirements.txt to Poetry
• Development Dependencies: Added pytest, pytest-cov, and pytest-mock as development dependencies

Testing Configuration

• pytest Configuration: Configured pytest in pyproject.toml with:

  • Test discovery patterns for test_*.py, *_test.py, and tests.py
  • Coverage reporting with HTML and XML output formats
  • Custom markers: unit, integration, and slow
  • Strict mode options for better error detection

• Coverage Configuration: Set up coverage tracking for all main modules:

  • Tracks: brute, domainsites, lib, reverse, and searchengine modules
  • Excludes test files and __init__.py files from coverage
  • Generates reports in terminal, HTML, and XML formats

Directory Structure

tests/
├── __init__.py
├── conftest.py          # Shared pytest fixtures
├── test_setup_validation.py  # Infrastructure validation tests
├── unit/                # Unit tests directory
│   └── __init__.py
└── integration/         # Integration tests directory
    └── __init__.py

Shared Fixtures (conftest.py)

Created comprehensive test fixtures including:

• temp_dir and temp_file: Temporary file system utilities
• mock_config: Mock configuration object
• mock_requests: Mock HTTP requests
• mock_dns_resolver: Mock DNS resolution
• sample_domain_list and sample_ip_list: Test data
• mock_logger: Mock logging functionality
• mock_file_system: Mock file operations
• environment_vars: Environment variable management
• mock_subprocess: Mock subprocess calls
• capture_output: Stdout/stderr capture utility
• mock_time: Time-related mocking

Additional Updates

• Updated .gitignore: Added entries for:
  • Testing artifacts (.pytest_cache/, .coverage, htmlcov/, etc.)
  • Python artifacts (__pycache__/, *.pyc, etc.)
  • Virtual environments
  • IDE files
  • Claude settings (.claude/*)

How to Use

Installing Dependencies

poetry install

Running Tests

Both of these commands work:

poetry run test
poetry run tests

Running Specific Test Categories

# Run only unit tests
poetry run pytest -m unit

# Run only integration tests
poetry run pytest -m integration

# Run non-slow tests
poetry run pytest -m "not slow"

Coverage Reports

After running tests, coverage reports are available:

• Terminal: Displayed automatically with missing lines
• HTML: Open htmlcov/index.html in a browser
• XML: Available at coverage.xml for CI integration

Notes

• The coverage threshold is currently set to 0% due to Python 2 syntax in existing code preventing proper parsing
• The argparse dependency was removed as it's built into Python 3
• All Poetry commands should be run with poetry run prefix to use the virtual environment
• The infrastructure is ready for developers to start writing actual unit and integration tests

Next Steps

1. Start writing unit tests for individual modules
2. Add integration tests for end-to-end functionality
3. Gradually increase coverage threshold as more tests are added
4. Consider updating existing code to Python 3 syntax for better coverage tracking