Add support for reflection in the produced grpc server

[remyleone]

SDK version

	github.com/hashicorp/terraform-plugin-sdk/v2 v2.37.0

Use-cases

My end goal would be to have a better understanding of the grpc communication happening between terraform and provider. I would like to be able to sniff a socket to see all the communication happening between a terraform client and a provider.

Attempted Solutions

I've tried to launch a provider as a binary and communicate with it using a grpc client (evans, grpcurl, etc...) the problem is that no reflection seems to be enabled which render the communication cumbersome.

Proposal

Could you consider offer as a possibility the feature to enable grpc reflection on the produce provider. It would be disabled by default but could be enabled. It could be very useful in case of migration between the SDKv2 and terraform provider framework to have a better grasp of the difference between the two. I understand that you try to abstract that away with the libraries but in case something goes wrong it would help fill up bug reports.

References

[austinvalle]

Thanks for the request @remyleone 👋🏻 ,

This does look interesting and seems like something we could do from a technical standpoint, as we already have a pattern in place for adding new server options and then applying them to our underlying server configuration:

• https://github.com/hashicorp/terraform-plugin-go/blob/0c07538a954ba67baa844cf275ecc28b2e576709/tfprotov5/tf5server/server.go#L185-L192
• TFECO-9077: feat: Add WithManagedDebugEnvFilePath() option that allows writing TF_REATTACH_PROVIDERS to an environment file terraform-plugin-go#484
• https://github.com/hashicorp/terraform-plugin-go/blob/0c07538a954ba67baa844cf275ecc28b2e576709/tfprotov5/tf5server/server.go#L284-L289

I think in terms of what services would be exposed, I believe there would still need to be some additional understanding/documentation to accompany any usage of reflection for debugging, since our GRPC server is built on top of hashicorp/go-plugin, and requires a handshake to take place prior to executing any RPCs.

On top of that, while you'd be able to inspect the request/response fields with reflection, most of the interesting data is wrapped in msgpack encoding (via DynamicValue in the proto file), which you might need to inspect with a different tool. In the interest of supplying already available options, you can drop all the msgpack data produced during a Terraform run with the TF_LOG_SDK_PROTO_DATA_DIR environment variable today, see:

• https://github.com/hashicorp/terraform-plugin-go/tree/main?tab=readme-ov-file#protocol-data

---

So in general, I think the request is reasonable from a technical standpoint, but I'm not sure how useful it will be for enabling someone unfamiliar with go-plugin's internals to interact with a provider. That would need some more looking into 👀

[remyleone]

I think any introspection/reflection would be helpful because for someone external it is not easy to understand how all the pieces are working together. So any step even incomplete that give a part of the solution would be a step in a direction of more understanding for provider maintainers.
I wonder if it would be possible to have a debug mode for plugin-go that would drop all msgpack and would result in very straight forward protobuf with known schema. It does not need to be the setup that is used with customer but being able to switch to an easy and introspective mode would be helpful to track subtle bugs.