Skip to content

Conversation

@ooye-sanket
Copy link
Contributor

When multiple formulas (e.g., python@3.14 and python@3.13) share the same resource file, the download queue deduplicates downloads but only creates a single symlink for the first formula. This causes brew cleanup to remove the shared resource when cleaning the first formula, breaking subsequent formula upgrades.

The fix tracks all formulas requesting each resource and creates a separate symlink for each after the download completes. This ensures reference counting works correctly during cleanup - the file in downloads/ will only be removed when ALL symlinks pointing to it are gone.

Fixes #21327

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) with your changes locally?

@ooye-sanket
Copy link
Contributor Author


Implementation Details

Changes made to Library/Homebrew/download_queue.rb:

  1. Added @symlink_targets hash in initialize to track all downloadables sharing the same cached file
  2. Modified enqueue method to register each downloadable against its cached location
  3. Added call to create_symlinks_for_shared_download after download completes
  4. Implemented new private method create_symlinks_for_shared_download that creates symlinks for all formulas requesting the same resource

How to verify the fix:

# Fetch both formulas that share the same patch
brew fetch --build-from-source python@3.14 python@3.13

# Check downloads directory - patch file exists
ls "$(brew --cache)/downloads"

# Cleanup first formula
brew cleanup python@3.14

# Verify patch file still exists (because python@3.13 still references it)
ls "$(brew --cache)/downloads"

The fix ensures that when python@3.14 is cleaned up, its symlink is removed but the actual patch file remains because python@3.13's symlink still references it.

Note: This PR contains only additions to the codebase - no deletions or modifications to existing code/comments.

Would appreciate a review from the maintainers. Thanks!

@ooye-sanket
Copy link
Contributor Author

ooye-sanket commented Jan 5, 2026

All CI checks are now passing!

The fix has been tested and is ready for review. Please let me know if any changes are needed.
CC @MikeMcQuaid

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great so far, thanks! Where is the original symlink logic? It might be nice to consolidate this logic with that logic so that there's no drift in this logic over time.

@ooye-sanket
Copy link
Contributor Author

ooye-sanket commented Jan 5, 2026

@MikeMcQuaid I've applied your suggestions but running into a type checking error. Here's what I did:

Changes applied:

  • Changed T.must() to .fetch()
  • Changed from [] to Set.new
  • Changed || [] to .fetch(cached_location, Set.new)

Current code (line 32):

@symlink_targets = T.let({}, T::Hash[Pathname, Set[Downloadable]])

Type error:

download_queue.rb:32: Use `T::Set[...]`, not `Set[...]` to declare a typed `Set`

The issue:
Sorbet requires T::Set[Downloadable] in the type annotation on line 32, but I understood your suggestion was to use plain Set.new to avoid the include? check.

I believe the confusion is:

  • Line 32 (type declaration): Needs T::Set[Downloadable] for Sorbet
  • Line 44 (runtime code): Uses plain Set.new as you suggested

Updated line 32:

@symlink_targets = T.let({}, T::Hash[Pathname, T::Set[Downloadable]])

Is this the correct approach - using T::Set in type annotations but Set.new in runtime code? Or did you mean something different?

All other checks pass, just this one type annotation issue. Let me know if I should approach this differently!

All CI checks are now passing!
The fix is ready for review. Please let me know if any changes are needed.
CC @MikeMcQuaid

@MikeMcQuaid
Copy link
Member

Is this the correct approach - using T::Set in type annotations but Set.new in runtime code? Or did you mean something different?

Yes 👍🏻

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the original symlink logic? It might be nice to consolidate this logic with that logic so that there's no drift in this logic over time.

@ooye-sanket thoughts on this?

@ooye-sanket
Copy link
Contributor Author

@MikeMcQuaid Good point! I'll consolidate the symlink logic.

I'll extract it into a helper method in AbstractFileDownloadStrategy so both the original fetch and my new create_symlinks_for_shared_download can call the same code.

Pushing the update now.

When multiple formulas (e.g., python@3.14 and python@3.13) share the same resource file, the download queue deduplicates downloads but only creates a single symlink for the first formula. This causes brew cleanup to remove the shared resource when cleaning the first formula, breaking subsequent formula upgrades.

The fix tracks all formulas requesting each resource and creates a
separate symlink for each after the download completes. This ensures reference counting works correctly during cleanup - the file in downloads/ will only be removed when ALL symlinks pointing to it are gone.
@ooye-sanket
Copy link
Contributor Author

@MikeMcQuaid Thank you for rebasing the branch! All checks are passing now.
Ready for final review when you have time!

@MikeMcQuaid MikeMcQuaid added this pull request to the merge queue Jan 6, 2026
Merged via the queue into Homebrew:main with commit 4dbe056 Jan 6, 2026
36 checks passed
@MikeMcQuaid
Copy link
Member

Thanks @ooye-sanket, great work here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automatic cleanup on install/upgrade removes resouce data used by multiple formula (e.g., 3.13-sysconfig.diff for both python@3.{13,14})

2 participants