Skip to content

Conversation

@npow
Copy link
Contributor

@npow npow commented Oct 17, 2025

BEFORE commit 21f29e2 (no filtering logic):

  1. S3ROOT = "s3://bucket/path/" (with trailing slash)
  2. User does: S3(s3root=f"{S3ROOT}/{test_prefix}/log/")
  3. This creates: "s3://bucket/path//test_prefix/log/"
  4. S3.init does rstrip("/") → "s3://bucket/path//test_prefix/log" (double slash still there!)
  5. When listing, S3 API is queried with prefix path//test_prefix/log
  6. S3 DOES return results because S3 ignores the double slash in the middle! S3 treats path//test_prefix same as path/test_prefix
  7. All keys returned by S3 are included without filtering
  8. Result: Everything worked fine!

AFTER commit 21f29e2 (with filtering logic):

  1. Same setup, S3 API returns results (S3 still treats // as /)
  2. BUT now the filtering logic kicks in:
    - prefix_url.path = "path//test_prefix/log"
    - normalized_prefix = "path//test_prefix/log/" (just adds trailing slash)
    - Checks if key_path (e.g., "path/test_prefix/log/test.txt") starts with "path//test_prefix/log/"
    - NO MATCH because path/ ≠ path//
  3. All results are filtered out!
  4. Result: Tests fail with empty results

@npow npow requested a review from mt-ob October 17, 2025 17:32
mt-ob
mt-ob previously approved these changes Oct 17, 2025
@nflx-mf-bot
Copy link
Collaborator

Netflix internal testing[1390] @ 2280207

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants