Flakey test in internal/telemetry causing build failures

[Pokom]

I noticed in #1366 that an unrelated test in internal/telemetry caused the build to fail in that PR. A quick review of https://github.com/grafana/synthetic-monitoring-agent/actions?query=is%3Afailure shows that this appears to be a somewhat common occurrence for dependency bot PR's. Here's a stacktrace that indicates that a data race condition exists that leads to a panic:

=== FAIL: internal/telemetry  (0.00s)
PASS
==================
WARNING: DATA RACE
Read at 0x00c000003a83 by goroutine 39:
  testing.(*common).logDepth()
      /usr/local/go/src/testing/testing.go:1059 +0x380
  testing.(*common).log()
      /usr/local/go/src/testing/testing.go:1046 +0x70
  testing.(*common).Log()
      /usr/local/go/src/testing/testing.go:1087 +0x50
  github.com/rs/zerolog.TestWriter.Write()
      /github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/writer.go:178 +0x130
  github.com/rs/zerolog.(*TestWriter).Write()
      <autogenerated>:1 +0x64
  github.com/rs/zerolog.LevelWriterAdapter.WriteLevel()
      /github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/writer.go:27 +0x64
  github.com/rs/zerolog.(*LevelWriterAdapter).WriteLevel()
      <autogenerated>:1 +0x20
  github.com/rs/zerolog.(*Event).write()
      /github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/event.go:80 +0x248
  github.com/rs/zerolog.(*Event).msg()
      /github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/event.go:151 +0x320
  github.com/rs/zerolog.(*Event).Msg()
      /github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/event.go:110 +0x350
  github.com/grafana/synthetic-monitoring-agent/internal/telemetry.(*RegionPusher).push()
      /__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:195 +0x33c
  github.com/grafana/synthetic-monitoring-agent/internal/telemetry.(*RegionPusher).run()
      /__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:107 +0x2f8
  github.com/grafana/synthetic-monitoring-agent/internal/telemetry.NewRegionPusher.gowrap1()
      /__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:81 +0x64

Previous write at 0x00c000003a83 by goroutine 10:
  testing.tRunner.func1()
      /usr/local/go/src/testing/testing.go:1779 +0x5dc
  runtime.deferreturn()
      /usr/local/go/src/runtime/panic.go:610 +0x5c
  testing.(*T).Run.gowrap1()
      /usr/local/go/src/testing/testing.go:1851 +0x40

Goroutine 39 (running) created at:
  github.com/grafana/synthetic-monitoring-agent/internal/telemetry.NewRegionPusher()
      /__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:81 +0x40c
  github.com/grafana/synthetic-monitoring-agent/internal/telemetry.setupTest()
      /__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher_test.go:413 +0x520
  github.com/grafana/synthetic-monitoring-agent/internal/telemetry.TestTenantPusher.func3()
      /__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher_test.go:251 +0x50
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1792 +0x180
  testing.(*T).Run.gowrap1()
      /usr/local/go/src/testing/testing.go:1851 +0x40

Goroutine 10 (finished) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:1851 +0x684
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:2279 +0x7c
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1792 +0x180
  testing.runTests()
      /usr/local/go/src/testing/testing.go:2277 +0x77c
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:2142 +0xb68
  main.main()
      _testmain.go:59 +0x110
==================
panic: Log in goroutine after TestTenantPusher/should_retry_sending_data_once has completed: {"level":"error","error":"unexpected status code","statusCode":4,"statusMessage":"","caller":"/__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:195","time":"2025-07-10T12:22:28Z","message":"error pushing telemetry"}
	

goroutine 67 [running]:
testing.(*common).logDepth(0xc0000c0700, {0xc00039e360, 0x105}, 0x3)
	/usr/local/go/src/testing/testing.go:1064 +0x4e0
testing.(*common).log(...)
	/usr/local/go/src/testing/testing.go:1046
testing.(*common).Log(0xc0000c0700, {0xc00038c130, 0x1, 0x1})
	/usr/local/go/src/testing/testing.go:1087 +0x74
github.com/rs/zerolog.TestWriter.Write({{0xb53310?, 0xc0000c0700?}, 0xc00017baf8?}, {0xc0001a6800, 0x105, 0x1f4})
	/github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/writer.go:178 +0x134
github.com/rs/zerolog.LevelWriterAdapter.WriteLevel(...)
	/github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/writer.go:27
github.com/rs/zerolog.(*Event).write(0xc0001a8230)
	/github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/event.go:80 +0x24c
github.com/rs/zerolog.(*Event).msg(0xc0001a8230, {0xa3a2e7, 0x17})
	/github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/event.go:151 +0x324
github.com/rs/zerolog.(*Event).Msg(...)
	/github/home/go/pkg/mod/github.com/rs/zerolog@v1.34.0/event.go:110
github.com/grafana/synthetic-monitoring-agent/internal/telemetry.(*RegionPusher).push(0xc0001c8200, {{0xa2e428, 0x8}, 0x1, {0xc000382060, 0x3, 0x3}})
	/__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:195 +0x354
github.com/grafana/synthetic-monitoring-agent/internal/telemetry.(*RegionPusher).run(0xc0001c8200, {0xb54f60, 0xc0001c4000}, {0xffca03333af8, 0xc00019e088})
	/__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:107 +0x2fc
created by github.com/grafana/synthetic-monitoring-agent/internal/telemetry.NewRegionPusher in goroutine 16
	/__w/synthetic-monitoring-agent/synthetic-monitoring-agent/internal/telemetry/region_pusher.go:81 +0x410
FAIL	github.com/grafana/synthetic-monitoring-agent/internal/telemetry	0.023s