-
Notifications
You must be signed in to change notification settings - Fork 43
Added OSV-Scalibr Test Image for winget extractor #162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mzfr
wants to merge
2
commits into
google:main
Choose a base branch
from
mzfr:winget_docker
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
FROM ubuntu:22.04 | ||
|
||
# Install bash and sqlite3 for database verification | ||
RUN apt-get update && apt-get install -y bash sqlite3 && rm -rf /var/lib/apt/lists/* | ||
|
||
# Create Windows-style directory structure for WinGet databases | ||
RUN mkdir -p /Users/test/AppData/Local/Packages/Microsoft.DesktopAppInstaller_8wekyb3d8bbwe/LocalState/Microsoft.Winget.Source_8wekyb3d8bbwe/ | ||
RUN mkdir -p /Users/test/AppData/Local/Packages/Microsoft.DesktopAppInstaller_8wekyb3d8bbwe/LocalState/StoreEdgeFD/ | ||
RUN mkdir -p /ProgramData/Microsoft/Windows/AppRepository/ | ||
|
||
# Create app directory for scalibr binary | ||
RUN mkdir -p /app | ||
|
||
# Copy WinGet test database files into the container | ||
COPY testdata/ / | ||
|
||
# Set working directory | ||
WORKDIR /app | ||
|
||
# Default command: start bash so the container stays alive interactively | ||
CMD ["/bin/bash"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
# OSV-Scalibr: Windows Package Manager (WinGet) Extractor | ||
|
||
This directory contains the test Docker setup for testing OSV-Scalibr's WinGet extractor plugin. Windows Package Manager (WinGet) is Microsoft's official package manager for Windows systems that stores its database files in SQLite format at specific system locations. | ||
|
||
## Overview | ||
|
||
The WinGet extractor analyzes installed Windows packages by reading SQLite database files created by the Windows Package Manager. This testbed simulates the Windows file system structure and provides sample WinGet databases for testing purposes. | ||
|
||
## WinGet Database Locations | ||
|
||
The WinGet extractor looks for databases at these Windows paths: | ||
|
||
1. **User-installed packages**: `%LOCALAPPDATA%/Packages/Microsoft.DesktopAppInstaller_8wekyb3d8bbwe/LocalState/Microsoft.Winget.Source_8wekyb3d8bbwe/installed.db` | ||
2. **Store Edge packages**: `%LOCALAPPDATA%/Packages/Microsoft.DesktopAppInstaller_8wekyb3d8bbwe/LocalState/StoreEdgeFD/installed.db` | ||
3. **System-wide repository**: `%PROGRAMDATA%/Microsoft/Windows/AppRepository/StateRepository-Machine.srd` | ||
|
||
## Test Database Contents | ||
|
||
This testbed includes three sample databases with the following packages: | ||
|
||
### User Installed Database | ||
- **Git.Git** v2.50.1 - Git version control system | ||
- **Microsoft.VisualStudioCode** v1.103.1 - Visual Studio Code editor | ||
- **Google.Chrome** v120.0.6099.109 - Google Chrome browser | ||
|
||
### Store Edge Database | ||
- **Microsoft.PowerShell** v7.4.1 - PowerShell terminal | ||
- **Mozilla.Firefox** v121.0.1 - Mozilla Firefox browser | ||
|
||
### System Repository Database | ||
- **Microsoft.WindowsTerminal** v1.18.3181.0 - Windows Terminal | ||
- **Microsoft.VCRedist.2015+.x64** v14.38.33135.0 - Visual C++ Redistributable | ||
|
||
## Setup Instructions | ||
|
||
### Build the Docker Image | ||
|
||
```bash | ||
cd security-testbeds/microsoft/winget | ||
docker build -t winget-test . | ||
``` | ||
|
||
### Run the Container | ||
|
||
```bash | ||
docker run -it --rm -v $(pwd):/app winget-test | ||
``` | ||
|
||
This will: | ||
- Start an interactive bash session | ||
- Mount the current directory as `/app` inside the container | ||
- Allow you to place the `scalibr` binary in `/app` and run tests | ||
|
||
### Running OSV-Scalibr | ||
|
||
1. Build or copy the `scalibr` binary to the current directory | ||
2. Run the container as shown above | ||
3. Inside the container, run scalibr with the WinGet extractor: | ||
|
||
```bash | ||
# Extract from all WinGet databases | ||
./scalibr --extractors=os/winget --result=winget_output.json | ||
|
||
# Or target specific paths | ||
./scalibr --extractors=os/winget --result=winget_output.json --paths=/Users/test/AppData/Local,/ProgramData/Microsoft | ||
``` | ||
|
||
### Verify Database Contents | ||
|
||
You can inspect the test databases using sqlite3: | ||
|
||
```bash | ||
# Inside the container | ||
sqlite3 /Users/test/AppData/Local/Packages/Microsoft.DesktopAppInstaller_8wekyb3d8bbwe/LocalState/Microsoft.Winget.Source_8wekyb3d8bbwe/installed.db | ||
|
||
# List tables | ||
.tables | ||
|
||
# Query packages | ||
SELECT i.id, n.name, v.version FROM manifest m | ||
JOIN ids i ON m.id = i.rowid | ||
JOIN names n ON m.name = n.rowid | ||
JOIN versions v ON m.version = v.rowid; | ||
``` | ||
|
||
## Regenerating Test Data | ||
|
||
The `generate_testdata.py` script can be used to recreate the test databases with different package sets: | ||
|
||
```bash | ||
python3 generate_testdata.py | ||
``` | ||
|
||
Edit the script to add new packages or modify existing ones before regenerating the databases. | ||
|
||
## Important Note: Windows-Only Extractor | ||
|
||
The WinGet extractor is designed to run **only on Windows systems** due to OS-specific requirements. When running in this Linux Docker container, you will see: | ||
|
||
```bash | ||
./scalibr --plugins=os/winget --result=output.textproto | ||
# Output: plugin os/winget can't be enabled: needs to run on a different OS than that of the scan environment | ||
``` | ||
|
||
This is **expected behavior** and indicates the testbed is set up correctly. | ||
|
||
## Purpose of This Testbed | ||
|
||
This Docker setup serves several purposes: | ||
|
||
1. **Validation**: Provides a standardized environment to test WinGet database parsing logic | ||
2. **Development**: Allows developers to work with realistic WinGet database structures without Windows | ||
3. **CI/CD**: Can be used in automated testing pipelines to verify database schema compatibility | ||
4. **Reference**: Documents the expected WinGet database structure and file locations | ||
|
||
## Expected Output (on Windows) | ||
|
||
When running scalibr successfully on a Windows system, you should see extracted package information for 7 total packages across the three databases, including package names, versions, and metadata like monikers, channels, tags, and commands. | ||
|
||
## Troubleshooting | ||
|
||
- **"needs to run on a different OS"**: This is expected when running on non-Windows systems | ||
- **Database errors**: Verify the database files exist and have proper SQLite schema using the inspection commands above | ||
- **Permission issues**: Make sure the scalibr binary has execute permissions: `chmod +x scalibr` | ||
- **No packages found on Windows**: Ensure the WinGet extractor is enabled with `--plugins=os/winget` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a bit unclear how to get the expected output when running the above SCALIBR command just produces an error message.
Two things I can think of that could fix that:
Option 2. would allow people to run this plugin on Linux as well but it's a bit hacky so I'd prefer if we went with Option 1.