Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,12 @@ are a good place to start.

### Traffic lights

Each item has an (opinionated) traffic light. The meaning of these is:
Each item has an (opinionated) traffic light label. The meaning of these is:

- 🟢 At least one person in ARC uses this. We actively recommend using it above
- <span class="label label-green">Best</span>: At least one person in ARC uses this. We actively recommend using it above
other tools. It is the single recommended tool for a given purpose.
- 🟠 We don't discourage using this, but it may duplicate functionality of a
green tool.
- 🔴 We actively discourage using this. This could be because it's no longer
- <span class="label label-yellow">Good</span>: We don't discourage using this, but it may duplicate functionality of the <span class="label label-green">Best</span> tool.
- <span class="label label-red">Avoid</span>: We actively discourage using this. This could be because it's no longer
maintained, not open source, or difficult to use. Consider moving to
alternatives if you're currently using something that's red. A reason for not
using this is given.
30 changes: 15 additions & 15 deletions docs/pages/benchmarking-profiling.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,29 @@ layout: default

## Benchmarking

| Name | Short description | 🚦 |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-: |
| [asv](https://asv.readthedocs.io/en/stable/) | A tool for benchmarking Python packages over their lifetime. Allows you to write benchmarks and then run them against every commit in the repository, to identify where performance increased or decreased. Comparative benchmarks can also be run, which can be useful for [running them in CI using GitHub runners](https://labs.quansight.org/blog/2021/08/github-actions-benchmarks). | 🟢 |
| Name | Short description | 🚦 |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------------------------------------: |
| [asv](https://asv.readthedocs.io/en/stable/) | A tool for benchmarking Python packages over their lifetime. Allows you to write benchmarks and then run them against every commit in the repository, to identify where performance increased or decreased. Comparative benchmarks can also be run, which can be useful for [running them in CI using GitHub runners](https://labs.quansight.org/blog/2021/08/github-actions-benchmarks). | <span class="label label-green">Best</span> |

## Profiling

### Time

| Name | Short description | 🚦 |
| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | :-: |
| [pyinstrument](https://pyinstrument.readthedocs.io/en/stable) | Python profiler. Tells you how long individual lines of code take to run, so you can focus on the slowest part of your program to speed it up. | 🟢 |
| [line_profiler](https://pypi.org/project/line-profiler/) | A tool for line-by-line profiling of functions. | 🟠 |
| Name | Short description | 🚦 |
| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------: |
| [pyinstrument](https://pyinstrument.readthedocs.io/en/stable) | Python profiler. Tells you how long individual lines of code take to run, so you can focus on the slowest part of your program to speed it up. | <span class="label label-green">Best</span> |
| [line_profiler](https://pypi.org/project/line-profiler/) | A tool for line-by-line profiling of functions. | <span class="label label-yellow">Good</span> |

### Memory

| Name | Short description | 🚦 |
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-: |
| [memray](https://bloomberg.github.io/memray/) | Tracks and reports memory allocations, both in Python code and in compiled extension modules. It also has a [plugin](https://pytest-memray.readthedocs.io/en/latest/) for easy integration with pytest. Only works on Linux and macOS. | 🟠 |
| [memory_profiler](https://pypi.org/project/memory-profiler/) | No longer actively maintained. A Python module for monitoring memory consumption of a process alongside line-by-line analysis of memory consumption. Might be a useful alternative to memray if you need to do memory profiling on Windows. | 🟠 |
| Name | Short description | 🚦 |
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------: |
| [memray](https://bloomberg.github.io/memray/) | Tracks and reports memory allocations, both in Python code and in compiled extension modules. It also has a [plugin](https://pytest-memray.readthedocs.io/en/latest/) for easy integration with pytest. Only works on Linux and macOS. | <span class="label label-yellow">Good</span> |
| [memory_profiler](https://pypi.org/project/memory-profiler/) | No longer actively maintained. A Python module for monitoring memory consumption of a process alongside line-by-line analysis of memory consumption. Might be a useful alternative to memray if you need to do memory profiling on Windows. | <span class="label label-yellow">Good</span> |

### General/other tools

| Name | Short description | 🚦 |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------- | :-: |
| [psutil](https://psutil.readthedocs.io/en/latest/) | System monitoring, profiling, limiting process resources and the management of running processes. | 🟢 |
| [snakeviz](https://jiffyclub.github.io/snakeviz/) | Browser based graphical viewer for the output of Python’s cProfile module. | 🟢 |
| Name | Short description | 🚦 |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------- | :------------------------------------------: |
| [psutil](https://psutil.readthedocs.io/en/latest/) | System monitoring, profiling, limiting process resources and the management of running processes. | <span class="label label-green">Best</span> |
| [snakeviz](https://jiffyclub.github.io/snakeviz/) | Browser based graphical viewer for the output of Python’s cProfile module. | <span class="label label-yellow">Good</span> |
28 changes: 14 additions & 14 deletions docs/pages/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ layout: default

## Continuous integration (CI)

| Name | Short description | 🚦 |
| ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | :-: |
| [GitHub Actions](https://docs.github.com/en/actions) | Continuous integration and continuous delivery platform (integrated with GitHub). | 🟢 |
| [AppVeyor](https://www.appveyor.com/docs/) | Continuous integration and continuous delivery platform. | 🟠 |
| [Bamboo](https://confluence.atlassian.com/bamboo/bamboo-documentation-289276551.html) | Atlassian continuous integration and continuous delivery platform. | 🟠 |
| [Travis CI](https://docs.travis-ci.com/) | Continuous integration and continuous delivery platform. | 🟠 |
| [pre-commit.ci](https://pre-commit.ci/) | A bot that adds a pre-commit job to your GitHub Actions CI, and can automatically fix most trivial linting failures. Free for open-source projects. | 🟢 |
| Name | Short description | 🚦 |
| ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------: |
| [GitHub Actions](https://docs.github.com/en/actions) | Continuous integration and continuous delivery platform (integrated with GitHub). | <span class="label label-green">Best</span> |
| [AppVeyor](https://www.appveyor.com/docs/) | Continuous integration and continuous delivery platform. | <span class="label label-yellow">Good</span> |
| [Bamboo](https://confluence.atlassian.com/bamboo/bamboo-documentation-289276551.html) | Atlassian continuous integration and continuous delivery platform. | <span class="label label-yellow">Good</span> |
| [Travis CI](https://docs.travis-ci.com/) | Continuous integration and continuous delivery platform. | <span class="label label-yellow">Good</span> |
| [pre-commit.ci](https://pre-commit.ci/) | A bot that adds a pre-commit job to your GitHub Actions CI, and can automatically fix most trivial linting failures. Free for open-source projects. | <span class="label label-red">Avoid</span> |

<details><summary> 🟢 explanation</summary><!-- markdownlint-disable-line MD033 -->
<details><summary> <span class="label label-green">Best</span> explanation</summary><!-- markdownlint-disable-line MD033 -->
We have many projects using GitHub CI and, it has good integration with GitHub itself, and is free for public repositories (with limited free monthly minutes for private repositories).
</details>

Expand All @@ -24,11 +24,11 @@ code with highlighting to show which lines are not executed by tests. See
[testing](testing) for our recommendations on packages to generate code coverage
during tests.

| Name | Short description | 🚦 |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-: |
| [Codecov](https://docs.codecov.com/docs) | Hosted service to report code coverage metrics. Occasionally slow to update after a report is updated, can be configured to add extra CI checks. This service is probably more widely used and is [free for both open-source and private projects](https://about.codecov.io/pricing/). | 🟢 |
| [Coveralls](https://docs.coveralls.io/) | Hosted service to report code coverage metrics. Very similar to codecov and we don't strongly recommend one over the other. This service is only [free for open-source projects](https://coveralls.io/pricing). | 🟢 |
| Name | Short description | 🚦 |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-----------------------------------------: |
| [Codecov](https://docs.codecov.com/docs) | Hosted service to report code coverage metrics. Occasionally slow to update after a report is updated, can be configured to add extra CI checks. This service is probably more widely used and is [free for both open-source and private projects](https://about.codecov.io/pricing/). | <span class="label label-green">Best</span> |
| [Coveralls](https://docs.coveralls.io/) | Hosted service to report code coverage metrics. Very similar to codecov and we don't strongly recommend one over the other. This service is only [free for open-source projects](https://coveralls.io/pricing). | <span class="label label-green">Best</span> |

<details><summary> 🟢 explanation</summary> <!-- markdownlint-disable-line MD033 -->
Both services are similar, so both 🟢.
<details><summary> <span class="label label-green">Best</span> explanation</summary> <!-- markdownlint-disable-line MD033 -->
Both services are similar, so both <span class="label label-green">Best</span>.
</details>
Loading