chore: Refactor integration tests

Signed-off-by: Steve Hipwell <steve.hipwell@gmail.com>

Author: stevehipwell

CONTRIBUTING.md

@@ -10,11 +10,11 @@ Before submitting an issue or a pull request, please search the repository for e
 
 ## Submitting a pull request
 
-0. Fork and clone the repository.
-0. Create a new branch: `git switch -c my-branch-name`.
-0. Make your change, add tests, and make sure the tests still pass.
-0. Push to your fork and submit a pull request.
-0. Pat yourself on the back and wait for your pull request to be reviewed and merged.
+1. Fork and clone the repository.
+2. Create a new branch: `git switch -c my-branch-name`.
+3. Make your change, add tests, and make sure the tests still pass.
+4. Push to your fork and submit a pull request.
+5. Pat yourself on the back and wait for your pull request to be reviewed and merged.
 
 Here are a few things you can do that will increase the likelihood of your pull request being accepted:
 
@@ -38,21 +38,28 @@ Once you have the repository cloned, there's a couple of additional steps you'll
     - You _must_ make sure that the "Template Repository" item in Settings is checked for this repo.
 - If you haven't already, generate a Personal Access Token (PAT) for authenticating your test runs.
 - Export the necessary configuration for authenticating your provider with GitHub
+
   ```sh
-  export GITHUB_TOKEN=<token of a user with an organization account>
-  export GITHUB_ORGANIZATION=<name of an organization>
+  export GITHUB_TEST_AUTH_MODE="organization"
+  export GITHUB_OWNER="<name of an organization>"
+  export GITHUB_USERNAME="<username of the user who created the token>"
+  export GITHUB_TOKEN="<token of a user with an organization account>"
   ```
+
 - Build the project with `make build`
 - Try an example test run from the default (`main`) branch, like `TF_LOG=DEBUG TF_ACC=1 go test -v ./... -run ^TestAccGithubRepositories`. All those tests should pass.
 
 ### Local Development Iteration
 
 1. Write a test describing what you will fix. See [`github_label`](./github/resource_github_issue_label_test.go) for an example format.
 1. Run your test and observe it fail. Enabling debug output allows for observing the underlying requests and responses made as well as viewing state (search `STATE:`) generated during the acceptance test run.
+
 ```sh
 TF_LOG=DEBUG TF_ACC=1 go test -v ./... -run ^TestAccGithubIssueLabel
 ```
+
 1. Align the resource's implementation to your test case and observe it pass:
+
 ```sh
 TF_ACC=1 go test -v ./... -run ^TestAccGithubIssueLabel
 ```
@@ -67,7 +74,8 @@ Println debugging can easily be used to obtain information about how code change
 
 If a full debugger is desired, VSCode may be used. In order to do so,
 
-0. Create a launch.json file with this configuration:
+1. Create a launch.json file with this configuration:
+
 ```json
 {
 	"name": "Attach to Process",
@@ -77,18 +85,18 @@ If a full debugger is desired, VSCode may be used. In order to do so,
 	"processId": 0,
 }
 ```
-Setting a `processId` of 0 allows a dropdown to select the process of the provider.
 
-0. Add a sleep call (e.g. `time.Sleep(10 * time.Second)`) in the [`func providerConfigure(p *schema.Provider) schema.ConfigureFunc`](https://github.com/integrations/terraform-provider-github/blob/cec7e175c50bb091feecdc96ba117067c35ee351/github/provider.go#L274C1-L274C64) before the immediate `return` call. This will allow time to connect the debugger while the provider is initializing, before any critical logic happens.
+Setting a `processId` of 0 allows a dropdown to select the process of the provider.
 
-0. Build the terraform provider with debug flags enabled and copy it to the appropriate bin folder with a command like `go build -gcflags="all=-N -l" -o ~/go/bin/`.
+1. Add a sleep call (e.g. `time.Sleep(10 * time.Second)`) in the [`func providerConfigure(p *schema.Provider) schema.ConfigureFunc`](https://github.com/integrations/terraform-provider-github/blob/cec7e175c50bb091feecdc96ba117067c35ee351/github/provider.go#L274C1-L274C64) before the immediate `return` call. This will allow time to connect the debugger while the provider is initializing, before any critical logic happens.
 
-0. Create or edit a `dev.tfrc` that points toward the newly-built binary, and export the `TF_CLI_CONFIG_FILE` variable to point to it. Further instructions on this process may be found in the [Building the provider](#using-a-local-version-of-the-provider) section.
+2. Build the terraform provider with debug flags enabled and copy it to the appropriate bin folder with a command like `go build -gcflags="all=-N -l" -o ~/go/bin/`.
 
-0. Run a terraform command (e.g. `terraform apply`). While the provider pauses on initialization, go to VSCode and click "Attach to Process". In the search box that appears, type `terraform-provi` and select the terraform provider process.
+3. Create or edit a `dev.tfrc` that points toward the newly-built binary, and export the `TF_CLI_CONFIG_FILE` variable to point to it. Further instructions on this process may be found in the [Building the provider](#using-a-local-version-of-the-provider) section.
 
-0. The debugger is now connected! During a typical terraform command, the plugin will be invoked multiple times. If the debugger disconnects and the plugin is invoked again later in the run, the developer will have to re-attach each time as the process ID changes.
+4. Run a terraform command (e.g. `terraform apply`). While the provider pauses on initialization, go to VSCode and click "Attach to Process". In the search box that appears, type `terraform-provi` and select the terraform provider process.
 
+5. The debugger is now connected! During a typical terraform command, the plugin will be invoked multiple times. If the debugger disconnects and the plugin is invoked again later in the run, the developer will have to re-attach each time as the process ID changes.
 
 ## Manual Testing
 
@@ -99,7 +107,7 @@ Manual testing should be performed on each PR opened in order to validate the pr
 Build the provider and specify the output directory:
 
 ```sh
-$ go build -gcflags="all=-N -l" -o ~/go/bin/
+go build -gcflags="all=-N -l" -o ~/go/bin/
 ```
 
 This enables verifying your locally built provider using examples available in the `examples/` directory.
@@ -121,7 +129,7 @@ provider_installation {
 }
 ```
 
-See https://www.terraform.io/docs/cli/config/config-file.html for more details.
+See <https://www.terraform.io/docs/cli/config/config-file.html> for more details.
 
 When running examples, you should spot the following warning to confirm you are using a local build:
 
@@ -137,37 +145,50 @@ The following provider development overrides are set in the CLI configuration:
 Commonly required environment variables are listed below:
 
 ```sh
-# enable debug logging
+# Enable debug logging
 export TF_LOG=DEBUG
 
-# enable testing of organization scenarios instead of individual or anonymous
-export GITHUB_ORGANIZATION=
-
-# enable testing of individual scenarios instead of organization or anonymous
-export GITHUB_OWNER=
+# Enables acceptance tests
+export TF_ACC="1"
 
-# enable testing of enterprise appliances
+# Configure the URL override for GHES.
 export GITHUB_BASE_URL=
 
 # enable testing of GitHub Paid features, these normally also require an organization e.g. repository push rulesets
 export GITHUB_PAID_FEATURES=true
 
-# leverage helper accounts for tests requiring them
-# examples include:
-# - https://github.com/github-terraform-test-user
-# - https://github.com/terraformtesting
-export GITHUB_TEST_OWNER=
-export GITHUB_TEST_ORGANIZATION=
-export GITHUB_TEST_USER_TOKEN=
-```
+# Configure acceptance testing mode; one of anonymous, individual, organization, team or enterprise. If not set will default to anonymous
+export GITHUB_TEST_AUTH_MODE=
 
-See [this project](https://github.com/terraformtesting/acceptance-tests) for more information on our old system for automated testing.
+# Configure authentication for testing
+export GITHUB_OWNER=
+export GITHUB_USERNAME=
+export GITHUB_TOKEN=
+
+# Configure user level values
+export GITHUB_TEST_USER_REPOSITORY=
+
+# Configure for the org under test
+export GITHUB_TEST_ORG_USER=
+export GITHUB_TEST_ORG_SECRET_NAME=
+export GITHUB_TEST_ORG_REPOSITORY=
+export GITHUB_TEST_ORG_TEMPLATE_REPOSITORY=
+export GITHUB_TEST_ORG_APP_INSTALLATION_ID=
+
+# Configure external (non-org) users
+export GITHUB_TEST_EXTERNAL_USER=
+export GITHUB_TEST_EXTERNAL_USER_TOKEN=
+export GITHUB_TEST_EXTERNAL_USER2=
+
+# Configure test options
+export GITHUB_TEST_ADVANCED_SECURITY=
+```
 
 There are also a small amount of unit tests in the provider. Due to the nature of the provider, such tests are currently only recommended for exercising functionality completely internal to the provider. These may be executed by running `make test`.
 
 ### GitHub Organization
 
-If you do not have an organization already that you are comfortable running tests against, you will need to [create one](https://help.github.com/en/articles/creating-a-new-organization-from-scratch). The free "Team for Open Source" org type is fine for these tests. The name of the organization must then be exported in your environment as `GITHUB_ORGANIZATION`.
+If you do not have an organization already that you are comfortable running tests against, you will need to [create one](https://help.github.com/en/articles/creating-a-new-organization-from-scratch). The free "Team for Open Source" org type is fine for these tests. The name of the organization must then be exported in your environment as `GITHUB_OWNER`.
 
 Make sure that your organization has a `terraform-template-module` repository ([terraformtesting/terraform-template-module](https://github.com/terraformtesting/terraform-template-module) is an example you can clone) and that its "Template repository" item in Settings is checked.
 

GNUmakefile

@@ -2,6 +2,8 @@ TEST?=$$(go list ./... |grep -v 'vendor')
 WEBSITE_REPO=github.com/hashicorp/terraform-website
 PKG_NAME=github
 
+export TESTARGS=-race -coverprofile=coverage.txt -covermode=atomic
+
 default: build
 
 tools:
@@ -16,18 +18,18 @@ fmt:
 	golangci-lint fmt ./...
 
 fmtcheck:
-	@sh -c "'$(CURDIR)/scripts/gofmtcheck.sh'"
+	@sh -c "'$(CURDIR)/scripts/fmtcheck.sh'"
 
 lint:
 	@echo "==> Checking source code against linters..."
-	golangci-lint run ./...
+	golangci-lint run --fix ./...
 
 test:
 	CGO_ENABLED=0 go test ./...
 	# commenting this out for release tooling, please run testacc instead
 
-testacc: fmtcheck
-	TF_ACC=1 CGO_ENABLED=0 go test $(TEST) -v $(TESTARGS) -timeout 120m
+testacc:
+	TF_ACC=1 CGO_ENABLED=0 go test -run "^TestAcc*" $(TEST) -v $(TESTARGS) -timeout 120m -count=1
 
 test-compile:
 	@if [ "$(TEST)" = "./..." ]; then \