Add support for Proxy Server

[harsha-chamarthi]

Hi Team,

I am running my garm server instance on EC2 behind a proxy server, i have set the proxy settings as environment variables HTTP_PROXY and HTTPS_PROXY pointing to our proxy server. GIthub and the below url is accesible when we do a curl, but when i create a organisation using garm-cli i am getting the below error,

Failure reason | fetching runner tools: Get "https://api.github.com/orgs/*****/actions/runners/downloads": could not refresh installation id **********'s token: could not get access_tokens from GitHub API for installation ID ***********: dial tcp 140.82.116.6:443: connect: connection refused |

We have configured our other self-hosted runners using the environment variables mentioned in the below doc and it works fine.
https://docs.github.com/en/actions/how-tos/hosting-your-own-runners/managing-self-hosted-runners/using-a-proxy-server-with-self-hosted-runners

Please help in resolving this issue or let me know how to pass the proxy settings to our garm server

[gabriel-samfira]

Hi @harsha-chamarthi,

Can you try out the following branch: #439

And see if that fixes your issue? Also, if your runners need the same proxy settings to access http(s), you might also need to override the runner_install_template and set the proxy there as well as an environment variable. This can be done via extra specs:

https://github.com/cloudbase/garm-provider-aws?tab=readme-ov-file#tweaking-the-provider

If your providers also need the proxy to call the IaaS API (aws, gcp, etc), you will also need to set environment_variables = ["HTTP_PROXY", "HTTPS_PROXY"] so GARM can pass them along to the provider binary when calling it. Here is an example:

https://github.com/cloudbase/garm-provider-aws?tab=readme-ov-file#configure

[gabriel-samfira]

The PR is merged in both release/v0.1 and in main. You can build the release/v0.1 branch and get the update. A v0.1.6 release will be cut from release/v0.1 branch soon.

Keeping this issue open until you confirm that it works for you.

[harsha-chamarthi]

@gabriel-samfira thanks for the fix. The proxy settings work now. Can you please add the same proxy support for the AWS provider too?

[gabriel-samfira]

We don't really craft a custom transport for the AWS provider. You might just need to set your provider config to allow passing the HTTP_PROXY environment variable from GARM to the provider.

Try the following:

• Set the HTTP_PROXY and/or HTTPS_PROXY in the garm server environment. If you're using systemd, you will have to modify the unit file. If you're using docker, just include those variables in GARMs env
• Edit your GARM config and locate your provider config
• Update the provider definition to look like the following:

[[provider]]
name = "ec2_external"
description = "external provider for AWS"
provider_type = "external"
disable_jit_config = false
  [provider.external]
  config_file = "/etc/garm/garm-provider-aws.toml"
  provider_executable = "/opt/garm/providers/garm-provider-aws"

  # Allow the proxy variables to pass from GARM to the provider
  environment_variables = ["HTTP_PROXY", "HTTPS_PROXY"]

Notice the last line of the above example.

If the AWS sdk uses the default transport or if it obeys the proxy settings from the env, it should work.

Let me know if it works.

[harsha-chamarthi]

thanks @gabriel-samfira , i will try and let you know

[gabriel-samfira]

the only other place where you might need to think about the proxy settings is in the runner install template. You might have to change it and override the one used by the provider using extra_specs. See the AWS provider README for details.

There is an example of how to do it in the gitea doc, as gitea doesn't yet have proper install template support in the common provider package.

[gabriel-samfira]

yeah, I think it's best to add explicit proxy support. So work items would be:

1. Add a config option for proxy settings in the GARM config
2. Read environment settings and in the absence of explicit config options for HTTPS_PROXY and HTTP_PROXY, use the environment variables.
3. Add option to set proxy settings in runner install metadata/cloud config
4. Modify default templates to take proxy settings into account if they are set

Untill all that is added, you can use environment variables and override the default install templates as detailed above.

[harsha-chamarthi]

We don't really craft a custom transport for the AWS provider. You might just need to set your provider config to allow passing the HTTP_PROXY environment variable from GARM to the provider.

Try the following:

• Set the HTTP_PROXY and/or HTTPS_PROXY in the garm server environment. If you're using systemd, you will have to modify the unit file. If you're using docker, just include those variables in GARMs env
• Edit your GARM config and locate your provider config
• Update the provider definition to look like the following:

[[provider]]
name = "ec2_external"
description = "external provider for AWS"
provider_type = "external"
disable_jit_config = false
[provider.external]
config_file = "/etc/garm/garm-provider-aws.toml"
provider_executable = "/opt/garm/providers/garm-provider-aws"

Allow the proxy variables to pass from GARM to the provider

environment_variables = ["HTTP_PROXY", "HTTPS_PROXY"]
Notice the last line of the above example.

If the AWS sdk uses the default transport or if it obeys the proxy settings from the env, it should work.

Let me know if it works.

@gabriel-samfira ,just fyi I tried passing the proxy through the provider environment variables but it does not work. For now i am passing the proxy settings by overwriting the default install template and it works.

[gabriel-samfira]

@gabriel-samfira ,just fyi I tried passing the proxy through the provider environment variables but it does not work. For now i am passing the proxy settings by overwriting the default install template and it works.

Yes. The install template needs to be overwritten for now. I have not yet added proxy support to the install template. That would be work items 3 and 4 in my previous message.