Connecting to MySQL with SSL enabled blocks event loop

[daviduebler]

I have found these related issues/pull requests

maybe #4005

Description

Connecting to a MySQL server with SSL enabled blocks significantly longer than connecting without SSL.

Reproduction steps

use {
    tracing_subscriber::{
        fmt,
        filter::{ EnvFilter },
        prelude::*,
    },
    tracing::{ Instrument },
    sqlx::mysql::{ MySqlPoolOptions, MySqlSslMode, MySqlConnectOptions },
};

fn init_logging() {
    let format = fmt::layer()
        .with_span_events(fmt::format::FmtSpan::CLOSE)
        .with_target(true);

    let filter = EnvFilter::from_default_env();

    let registry = tracing_subscriber::registry()
        .with(filter)
        .with(format);

    registry.init();
}

const DB_URI: &str = "mysql://user:pass@host/db";

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    init_logging();

    let options = DB_URI.parse::<MySqlConnectOptions>()?;
    let options_no_ssl = options.clone().ssl_mode(MySqlSslMode::Disabled);

    MySqlPoolOptions::new()
        .connect_with(options)
        .instrument(tracing::info_span!("connect"))
        .await?;

    MySqlPoolOptions::new()
        .connect_with(options_no_ssl)
        .instrument(tracing::info_span!("connect_no_ssl"))
        .await?;

    Ok(())
}

Output on my machine:

2025-09-16T11:36:30.065098Z  INFO connect: sqlx_timing: close time.busy=41.5ms time.idle=4.56ms
2025-09-16T11:36:30.070216Z  INFO connect_no_ssl: sqlx_timing: close time.busy=691µs time.idle=3.7
2ms

Enabling SSL seems to block the event loop for tens of milliseconds.

SQLx version

0.8.6

Enabled SQLx features

mysql, runtime-tokio-native-tls, json, time

Database server and version

MySQL 8.0.31

Operating system

NixOS

Rust version

rustc 1.89.0-nightly (60dabef95 2025-05-19)

[abonander]

One problem is that, compared to #4005, there isn't really a loop that we can just call yield_now() in. Depending on what authentication plugin you're using, it either calculates 2 SHA-1 hashes, 2 SHA-2 hashes, or an RSA public-key encryption: https://github.com/launchbadge/sqlx/blob/c52e129e8341c46b2ea5c9b912e1af665be1ef83/sqlx-mysql/src/connection/auth.rs

Could you verify which plugin is being used?

[abonander]

Also, are these numbers from running this in release mode or debug mode?

[daviduebler]

Thank you for your quick reply 😄

• The Plugin CachingSha2Password is being used
• I ran the example in release mode

New info: I tried running the example with rustls instead of native-tls and the blocking disappeared. The ~40ms are correctly sent to a background thread there:

2025-09-17T06:58:50.415458Z  INFO connect: timing: close time.busy=786µs time.idle=46.4ms
2025-09-17T06:58:50.422427Z  INFO connect_no_ssl: timing: close time.busy=680µs time.idle=6.21ms

This starting to look different from #4005 😉
Seems to me that the native tls implementation (ffi / probably a call to openssl?) is being called synchronously somewhere. That may also be why release mode doesn't really affect the measured time for the connection using tls.

I will try to find the exact place where this happens (if my assumption is correct) but I have no clue about your code base 😄
So if you could point me in the right direction it could speed things up a bit...

[daviduebler]

Ok that was easier than I thought.

It seems to be this call:

sqlx/sqlx-core/src/net/tls/tls_native_tls.rs

Line 64 in c52e129

let connector = builder.build().map_err(Error::tls)?;

To verify, I sent that to a background thread and instrumented it:

let connector = tokio::task::spawn_blocking(move || { builder.build().map_err(Error::tls) })
    .instrument(tracing::info_span!("builder.build()"))
    .await.unwrap()?;

2025-09-17T07:21:03.082653Z  INFO connect:handshake:builder.build(): sqlx_core::net::tls::tls_native_tls: close time.busy=4.53µs time.idle=41.0ms
2025-09-17T07:21:03.084424Z  INFO connect:handshake: sqlx_core::net::tls::tls_native_tls: close time.busy=703µs time.idle=42.1ms
2025-09-17T07:21:03.087232Z  INFO connect: timing: close time.busy=1.05ms time.idle=45.8ms
2025-09-17T07:21:03.092510Z  INFO connect_no_ssl: timing: close time.busy=394µs time.idle=4.24ms

There is still a blocking millisecond left but I could live with that.
Would a change like this be alright with you? If yes, then I could clean that up and open a pull request.
If not, then I guess we will try to switch to rustls 😉

[abonander]

My guess is it's these calls that do filesystem I/O to load the certificate file and system TLS roots: https://docs.rs/native-tls/latest/src/native_tls/imp/openssl.rs.html#276-285

Wrapping this in a spawn_blocking call seems reasonable, although you should use the runtime-agnostic wrapper we have for it (sqlx_core::rt::spawn_blocking()):

sqlx/sqlx-core/src/rt/mod.rs

Line 94 in c52e129

pub fn spawn_blocking<F, R>(f: F) -> JoinHandle<R>

We could also potentially amortize this work by storing the TlsConnector somewhere, like in the ConnectOptions. Once it's constructed it can be used to create arbitrarily many connections so we don't have to build one every time. However, the benefit of constructing it every time is it will pick up any changes to certificate file or TLS roots on-disk when it reconnects, which some people might be relying on.

[daviduebler]

Constructing it every time seems safer to me as well. Sending it to the async runtimes blocking thread pool should not change observable behaviour besides timing.