Accurate rebuilds with helpmakego

[danielrbradley]

Use https://github.com/iwahbe/helpmakego to calculate the file dependencies for a given entry point (including workspaces and embedded files).

Go run is now cached as of go 1.24 so this doesn't need to be downloaded as a standalone binary: golang/go#69290

Calling go build will always update the timestamp of the output binary even if no source files have changed ensuring that any dependants will also be correctly rebuilt.

This approach can be used for the building of all go binaries whether or not they're depended on by subsequent file targets. This is an alternative to making go build targets phonies (see #1386) which would cause anything dependant on the target to always be rebuilt (such as generation or testing steps).

The only remaining complicating factor is that we've generalized the makefile away from building Go project and so we might be dealing non-go builds. The solution here is to simplify what we're modelling for by split the core implementation for the technology we're building.

[thomas11]

I've read the README of helpmakego three times but don't get it. Does it return the list of files that go build would build?

[danielrbradley]

I've read the README of helpmakego three times but don't get it. Does it return the list of files that go build would build?

Does this read better? https://github.com/iwahbe/helpmakego/blob/580d25ee8422f0d3c569ca9e3e88f51c0b9cb482/README.md

[iwahbe]

I'm in favor or using make as a dependency graph, not just a command runner.

Since we are making targets "real" (file based), we could also change .make/schema to represent the actual file: provider/cmd/$(PROVIDER)/schema.json.

[iwahbe]

It might be cleaner to simplify to:

Suggested change

	bin/$(PROVIDER): .make/schema $(shell go run $(HELP_MAKE_GO) provider/cmd/$(PROVIDER))
	bin/$(PROVIDER): .make/schema $(shell $(HELP_MAKE_GO) provider/cmd/$(PROVIDER))

Then we would change HELP_MAKE_GO:

-HELP_MAKE_GO := github.com/iwahbe/helpmakego@v0.1.0
+HELP_MAKE_GO := go run github.com/iwahbe/helpmakego@v0.1.0

[danielrbradley]

I'm in favor or using make as a dependency graph, not just a command runner.
👍
Since we are making targets "real" (file based), we could also change .make/schema to represent the actual file: provider/cmd/$(PROVIDER)/schema.json.

We can't do that because it's a committed file and so doing a checkout causes it to update its timestamp.

We're also blocked on merging this because we currently hack this makefile to also build typescript projects by replacing the provider build command .. and therefore there's nothing accurate we can currently model as the dependencies. Hece I've opened #1392 to break the makefiles apart so we write one makefile per provider technology so we can then fix this better.