[Testing] Create a new project for each test run (#506)

## Problem

Since all integration tests currently run in a shared project, we have
to make use of randomness to avoid collisions on user-assigned names. I
have a goal to remove this randomness and make tests more deterministic
in order to unblock testing tools such as VCR, but before I can do that
I need to isolate each test run into its own Pinecone project.

## Solution

- **New actions**. Implement new actions: 
- `project-create`. I ended up wrapping this into a reusable workflow
`project-setup.yaml` to encapsulate some of the setup and make it easier
to use in `on-pr.yaml` and `on-merge.yaml` and
`on-pre-dep-changes.yaml`.
- `project-delete`. I ended up wrapping this into a reusable workflow
`project-cleanup.yaml` to make it easier to use in `on-pr.yaml` and
`on-merge.yaml` and `on-pre-dep-changes.yaml`. Deleting is actually a
lot more complicated than creating the project and API key because all
the resources must be cleared out of a project before it can be deleted,
there are several resources types (indexes, collections, backups), and a
lot of different ways that a delete can fail due to precondition failed
/ other pending operations. I implemented a deletion queue with `deque`
that does a lot, but not unlimited, amounts of retrying. Anything that
still fails after retries are exhausted will be tried again during the
nightly cleanup.
  - `secret-encrypt` and `secret-decrypt` (see below)
- **New/updated workflows** 
  - `project-setup.yaml`
- `project-cleanup.yaml`. Cleanup is set to run `always()` so it should
trigger even if upstream jobs are failed or canceled. I learned the hard
way that `always()` jobs should always be used in combination with a
`timeout-minutes` setting because if you make a mistake (such as
implementing a bug that will never succeed with infinite retries), the
job will not respond to normal workflow cancel requests issued from the
Github UI. If you get into this state, the only ways to recover are to
call the [Github REST API to force
cancel](https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2022-11-28#force-cancel-a-workflow-run)
or to wait for the default timeout of 6 hours (FML!!) to elapse.
  - `cleanup-nightly.yaml`
- **Passing API keys**. I needed a way to pass a secret (the API key for
the newly created project) from `project-create` into various other
jobs, but github will not set job outputs that it thinks contain secrets
so I was unable to directly pass the API key. I discovered that I needed
to also implement `secret-encrypt` and `secret-decrypt` so that I could
transmit the api key between different parts of the workflow in an
encrypted form. These were implemented with symmetric encryption using
[Fernet](https://cryptography.io/en/latest/fernet/) from the
`cryptography` package.
- **Repository secrets**. Set new repository secrets for:
- `PINECONE_SERVICE_ACCOUNT_CLIENT_ID` and
`PINECONE_SERVICE_ACCOUNT_CLIENT_SECRET` came from the web console
  - `FERNET_ENCRYPTION_KEY` was generated randomly.
- Adjusting existing workflows
- Adjust `testing-integration.yaml` to use API key passed through inputs
- Adjust `testing-dependency.yaml` to use API key passed through inputs
- Adjust `on-pr.yaml` to create and delete the project, passing the API
key to the integration testing step as a workflow input.
- Adjust `on-merge.yaml` to create and delete the project, passing the
API key to the integration and dependency testing step as a workflow
input.
- Adjust `cleanup-nightly.yaml` to also clean up generated projects and
the resources they contain.

While doing all this surgery on CI stuff I also made the following
changes:
- Deleted several unused actions
- Moved some files from `scripts/` into the directory of the github
action using them to make that relationship more clear. I want
`scripts/` to be for stuff that is mostly manually run in development.
- Renamed several actions and workflows to use a `<noun>-<verb>` format
so that related actions would be grouped by resource when sorted
alphabetically. For example, `create-index` and `delete-index` became
`index-create` and `index-delete`.
- Parameterized unit and integration tests on a python version matrix.
We don't often get version-specific problems, so I think we can just run
the expanded version matrix on merge.
- Temporarily disabled a few asyncio tests that seem to have some sort
of performance issue that needs investigating. I decided to disable
rather than fix because I wanted to keep changes in `tests/` to a
minimum for this diff.


These changes do not affect how tests are run locally. The
`PINECONE_API_KEY` environment variable is still loaded from .env using
`dotenv`.

## Type of Change

- [x] Infrastructure change (CI configs, etc)

Author: jhamon

.github/actions/build-docs/action.yml

@@ -8,17 +8,12 @@ inputs:
 runs:
   using: 'composite'
   steps:
-    - name: Setup Python
-      uses: actions/setup-python@v5
-      with:
-        python-version: ${{ inputs.python-version }}
-
     - name: Setup Poetry
       uses: ./.github/actions/setup-poetry
       with:
         include_grpc: 'true'
         include_dev: 'true'
-
+        python_version: ${{ inputs.python-version }}
     - name: Build html documentation
       shell: bash
       run: |

.github/actions/cleanup-all/action.yml

@@ -1,32 +1,31 @@
 name: 'Cleanup All'
-description: 'Delete all indexes and collections associated with API key'
+description: 'Delete all resources created by tests'
 
 inputs:
   PINECONE_API_KEY:
     description: 'The Pinecone API key'
     required: true
-  DELETE_ALL:
-    description: 'Delete all indexes and collections'
-    required: false
-    default: 'false'
   PINECONE_ADDITIONAL_HEADERS:
     description: 'Additional headers to send with the request'
     required: false
     default: '{"sdk-test-suite": "pinecone-python-client"}'
+  PINECONE_SERVICE_ACCOUNT_CLIENT_ID:
+    description: 'The Pinecone service account client ID'
+    required: true
+  PINECONE_SERVICE_ACCOUNT_CLIENT_SECRET:
+    description: 'The Pinecone service account client secret'
+    required: true
 
 runs:
   using: 'composite'
   steps:
-    - name: Set up Python
-      uses: actions/setup-python@v5
-      with:
-        python-version: 3.9
     - name: Setup Poetry
       uses: ./.github/actions/setup-poetry
     - name: Cleanup all
       shell: bash
-      run: poetry run python3 scripts/cleanup-all.py
+      run: poetry run python3 ./github/actions/cleanup-all/cleanup-test-projects.py
       env:
         PINECONE_API_KEY: ${{ inputs.PINECONE_API_KEY }}
-        DELETE_ALL: ${{ inputs.DELETE_ALL }}
         PINECONE_ADDITIONAL_HEADERS: ${{ inputs.PINECONE_ADDITIONAL_HEADERS }}
+        PINECONE_SERVICE_ACCOUNT_CLIENT_ID: ${{ inputs.PINECONE_SERVICE_ACCOUNT_CLIENT_ID }}
+        PINECONE_SERVICE_ACCOUNT_CLIENT_SECRET: ${{ inputs.PINECONE_SERVICE_ACCOUNT_CLIENT_SECRET }}