Skip to content

Conversation

@ohtanim
Copy link
Collaborator

@ohtanim ohtanim commented Oct 31, 2025

Description of Change

Background:
During a recent deployment to the customer's data center, we needed to debug several issues related to environment variable settings in the process where the SPANK plugin is loaded. As a result, it became necessary to improve the current log messages. Additionally, in the customer's environment, access to external data sources requires the use of an HTTP proxy, which meant that environment variables such as http_proxy needed to be set in the process where the SPANK plugin is loaded.

This PR will add:

  1. --env: argument to spank plugin. It allows system adminitrator to add environment variables to the Slurm process where the SPANK plugin is loaded. The format for specifying environment variables is defined as follows.
--env:{variable name}={value}

For example, when interacting with Quantum resources via an HTTP proxy, the environment variables http_proxy, https_proxy, and no_proxy are required. These can be added as shown below.

optional /usr/lib64/slurm/spank_qrmi.so /etc/slurm/qrmi_config.json --env:http_proxy=http://192.168.1.128:3128 --env:https_proxy=http://192.168.1.128:3128
  1. more log messages for debugging environment variable related issues of spank plugin

In addition to log the result (success or failed), we needed to know the detailed error reason to debug. By this PR changes, log message will be enhanced to include the detailed reason information like below.

[2025-10-27T12:53:23.997] [3713617.batch] error: spank_qrmi, ibm_rensselaer is not accessible. Failed to get backend details: Middleware(
    reqwest::Error {
        kind: Request,
        url: "https://iam.cloud.ibm.com/identity/token",
        source: hyper_util::client::legacy::Error(
            Connect,
            ConnectFailed(
                Ssl(
                    Error {
                        code: ErrorCode(
                            1,
                        ),
                        cause: Some(
                            Ssl(
                                ErrorStack(
                                    [
                                        Error {
                                            code: 167772427,
                                            library: "SSL routines",
                                            function: "ssl3_get_record",
                                            reason: "wrong version number",
                                            file: "ssl/record/ssl3_record.c",
                                            line: 355,
                                        },
                                    ],
                                ),
                            ),
                        ),
                    },
                    X509VerifyResult {
                        code: 0,
                        error: "ok",
                    },
                ),
            ),
        ),
    },
)

Checklist ✅

  • Have you included a description of this change?
  • Have you updated the relevant documentation to reflect this change?
  • Have you made sure CI is passing before requesting a review?

Ticket

  • Fixes #
  • Is Part of #

@ohtanim ohtanim self-assigned this Oct 31, 2025
@ohtanim ohtanim added the enhancement New feature or request label Oct 31, 2025
@ohtanim ohtanim changed the title Add more debug messages for environment variable related settings. Add --env: argument to plugin and more log messages for debugging environment variable related issues Oct 31, 2025
@ohtanim ohtanim marked this pull request as ready for review October 31, 2025 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant