Panics demangling some symbols from a Linux build of Firefox

[gabrielesvelto]

I've used the current master branch of cpp_demangle on the symbols produced by a local Firefox build on my machine and found 9 that cause cpp_demangle to panic. The list of affected symbols is available here. These were generated by clang 14.0.5.

[gabrielesvelto]

I've fed the symbols into llvm-cxxfilt and the results are here. These all seem to have an odd ::$_<number> component, for example:

std::__invoke_result<mozilla::webgl::ChooseDeviceReplacement(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::$_41>

[khuey]

These fail to demangle here (with "error: an error occurred when formatting an argument") but none of them panic. What is the panic stack?

[khuey]

These seem to trigger a loop when printing the symbol that eventually hits the recursion limit.

[khuey]

I fixed one issue that's definitely a cpp_demangle bug, but after that we still don't agree with LLVM or libiberty (which also don't agree with each other)

cpp_demangle

mozilla::CompactPair<mozilla::net::SimpleChannelCallbacks*, mozilla::DefaultDelete<mozilla::net::SimpleChannelCallbacks> >::CompactPairHelper<mozilla::net::SimpleChannelCallbacksImpl<nsAnnoProtocolHandler::NewFaviconChannel(nsIURI*, nsIURI, nsILoadInfo*, nsIChannel**)::$_15, already_AddRefed<nsILoadInfo*> NS_NewSimpleChannel<nsAnnoProtocolHandler, nsIChannel**>(nsIURI, nsILoadInfo, nsAnnoProtocolHandler*, nsIChannel**&&)::{lambda(bool, nsIChannel, nsIURI)#1}, nsAnnoProtocolHandler>*, mozilla::DefaultDelete<already_AddRefed<nsILoadInfo*> NS_NewSimpleChannel<nsAnnoProtocolHandler, nsIChannel**>(nsIURI, nsILoadInfo, nsAnnoProtocolHandler*, nsIChannel**&&)::{lambda(bool, nsIChannel, nsIURI)#1}> >(mozilla::DefaultDelete<already_AddRefed<nsILoadInfo*> NS_NewSimpleChannel<nsAnnoProtocolHandler, nsIChannel**>(nsIURI, nsILoadInfo, nsAnnoProtocolHandler*, nsIChannel**&&)::{lambda(bool, nsIChannel, nsIURI)#1}>)

lllvm-cxxfilt 16 (llvm/llvm-project@c0d954a)

mozilla::CompactPair<mozilla::net::SimpleChannelCallbacks*, mozilla::DefaultDelete<mozilla::net::SimpleChannelCallbacks>>::CompactPair<mozilla::net::SimpleChannelCallbacksImpl<nsAnnoProtocolHandler::NewFaviconChannel(nsIURI*, nsIURI, nsILoadInfo*, nsIChannel**)::$_15, already_AddRefed<nsILoadInfo*> NS_NewSimpleChannel<nsAnnoProtocolHandler, nsIChannel**>(nsIURI, nsILoadInfo, nsAnnoProtocolHandler*, nsIChannel**&&)::'lambda'(bool, nsIChannel, nsIURI), nsAnnoProtocolHandler>*, mozilla::DefaultDelete<already_AddRefed<nsILoadInfo*> NS_NewSimpleChannel<nsAnnoProtocolHandler, nsIChannel**>(nsIURI, nsILoadInfo, nsAnnoProtocolHandler*, nsIChannel**&&)::'lambda'(bool, nsIChannel, nsIURI)>>(already_AddRefed<nsILoadInfo*>&&, nsIChannel**)

libiberty (binutils 2.38.50)

mozilla::CompactPair<mozilla::net::SimpleChannelCallbacks*, mozilla::DefaultDelete<mozilla::net::SimpleChannelCallbacks> >::CompactPairHelper<mozilla::net::SimpleChannelCallbacksImpl<nsAnnoProtocolHandler::NewFaviconChannel(nsIURI*, nsAnnoProtocolHandler, nsILoadInfo*, nsIChannel**)::$_15, NS_NewSimpleChannel<mozilla::net::SimpleChannelCallbacksImpl, nsIChannel*>(nsAnnoProtocolHandler, nsIURI*, mozilla::net::SimpleChannelCallbacksImpl*, nsIChannel*&&)::{lambda(bool, nsILoadInfo*, nsAnnoProtocolHandler)#1}, mozilla::net::SimpleChannelCallbacksImpl>*, mozilla::DefaultDelete<nsIChannel*&&> >(already_AddRefed&&, mozilla::net::SimpleChannelCallbacksImpl<nsAnnoProtocolHandler::NewFaviconChannel(nsIURI*, nsAnnoProtocolHandler, nsILoadInfo*, nsIChannel**)::$_15, NS_NewSimpleChannel<mozilla::net::SimpleChannelCallbacksImpl, nsIChannel*>(nsAnnoProtocolHandler, nsIURI*, mozilla::net::SimpleChannelCallbacksImpl*, nsIChannel*&&)::{lambda(bool, nsILoadInfo*, nsAnnoProtocolHandler)#1}, mozilla::net::SimpleChannelCallbacksImpl>**)

It would really help to know where these symbols come from in the Mozilla source to determine which (if any) are correct.

[gabrielesvelto]

I believe that's coming from here. But it's just a guess given the types that show up in the demangled versions.

[khuey]

After looking at this with fresh eyes I think the symbol is incorrectly mangled. The above comes from demangling

_ZN7mozilla11CompactPairIPNS_3net22SimpleChannelCallbacksENS_13DefaultDeleteIS2_EEECI2NS_6detail17CompactPairHelperIS3_S5_LNS7_11StorageTypeE1ELS9_0EEEIPNS1_26SimpleChannelCallbacksImplIZN21nsAnnoProtocolHandler17NewFaviconChannelEP6nsIURISD_P11nsILoadInfoPP10nsIChannelE4$_15Z19NS_NewSimpleChannelISC_SJ_E16already_AddRefedISG_ESD_SF_PT_OT0_EUlbSH_SD_E_SC_EENS4_ISS_EEEEOSN_SQ_

CompactPairHelper doesn't appear in the llvm-cxxfilt demangling despite appearing in the symbol. And detail doesn't appear in the demangling in any tool! I think itanium-cxx-abi/cxx-abi#68 is also present here.

[VorpalBlade]

I too ran into this on Firefox libxul.so. Such as _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKlLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_.

c++filt is perfectly happy with it, giving me: std::__invoke_result<opentelemetry::v1::sdk::common::AttributeConverter&, opentelemetry::v1::nostd::span<long const, 18446744073709551615ul> const&>::type std::__invoke[abi:cxx11]<opentelemetry::v1::sdk::common::AttributeConverter&, opentelemetry::v1::nostd::span<long const, 18446744073709551615ul> const&>(opentelemetry::v1::sdk::common::AttributeConverter&, opentelemetry::v1::nostd::span<long const, 18446744073709551615ul> const&)

I would not expect a library that returns a Result to panic, requiring me to wrap this in catch_unwind and using unwinding panic (I prefer aborting panics).

[khuey]

Yeah we should do better here. I'll investigate your symbol shortly.

[khuey]

What version of cpp_demangle are you using? That symbol returns an error but it doesn't panic for me.

[VorpalBlade]

❯ cargo tree | grep cpp_demangle                                                                                                                                 ≡
│   ├── cpp_demangle v0.4.4

I have the following code that first tries to demangle as a Rust symbol, then as a C++ symbol, and if that fails assumes it is a C symbol (and thus mangled and unmangled is the same):

// For reasons I'm doing string interning here, that is what DefaultAtom is. It has a Deref impl to &str.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct MangledSymbolName(pub string_cache::DefaultAtom);

impl MangledSymbolName {
    pub fn as_str(&self) -> &str {
        &self.0
    }

    pub fn new(name: &str) -> Self {
        Self(string_cache::DefaultAtom::from(name))
    }

    // Attempt to demangle name
    pub fn demangle(&self) -> UnmangledSymbolName {
        rustc_demangle::try_demangle(&self.0)
            .map(|d| UnmangledSymbolName(string_cache::DefaultAtom::from(format!("{d:#}"))))
            .ok()
            .or_else(|| {
                let caught_panic = catch_unwind(|| {
                    cpp_demangle::Symbol::new(self.0.as_bytes())
                        .ok()
                        .map(|sym| {
                            UnmangledSymbolName(string_cache::DefaultAtom::from(sym.to_string()))
                        })
                });
                caught_panic
                    .inspect_err(|_err| {
                        tracing::error!(
                            "C++ demangling panic on symbol: {}, using the mangled symbol",
                            self.0
                        );
                    })
                    .ok()
                    .flatten()
            })
            .unwrap_or_else(|| UnmangledSymbolName(self.0.clone()))
    }
}

When I run this on all the symbols in the debug info of libxul I get:

[ERROR] C++ demangling panic on symbol: _ZSt5visitB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKSt7variantIJbiljdPKcSt17basic_string_viewIcSt11char_traitsIcEENS1_5nostd4spanIKbLm18446744073709551615EEENSE_IKiLm18446744073709551615EEENSE_IKlLm18446744073709551615EEENSE_IKjLm18446744073709551615EEENSE_IKdLm18446744073709551615EEENSE_IKSC_Lm18446744073709551615EEEmNSE_IKmLm18446744073709551615EEENSE_IKhLm18446744073709551615EEEEEEENSt13invoke_resultIT_JDpNSt13__conditionalIX21is_lvalue_reference_vIT0_EEE4typeIRNSt19variant_alternativeILm0ENSt16remove_referenceIDTclsr9__variantE4__asclsr3stdE7declvalIS11_EEEEE4typeEE4typeEOS1A_EEEE4typeEOSZ_DpOS11_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKjLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKPKcEENSt15__invoke_resultIT_JDpT0_EE4typeEOSB_DpOSC_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKdLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKmEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKiEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKhLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: RD4, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKlEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKbLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKlLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IZN7mozilla5webglL23ChooseDeviceReplacementERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEE3$_0JEENSt15__invoke_resultIT_JDpT0_EE4typeEOSC_DpOSD_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKSt17basic_string_viewIcSt11char_traitsIcEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKbEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKSt17basic_string_viewIcSt11char_traitsIcEELm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSH_DpOSI_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKjEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKdEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKmLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt8__invokeB5cxx11IRN13opentelemetry2v13sdk6common18AttributeConverterEJRKNS1_5nostd4spanIKiLm18446744073709551615EEEEENSt15__invoke_resultIT_JDpT0_EE4typeEOSD_DpOSE_, using the mangled symbol
[ERROR] C++ demangling panic on symbol: _ZSt6invokeB5cxx11IZN7mozilla5webglL23ChooseDeviceReplacementERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEE3$_0JEENSt13invoke_resultIT_JDpT0_EE4typeEOSC_DpOSD_, using the mangled symbol

As I'm using a tracing hook which hooks and logs panics I can also see this (for each of the above panics):

The application panicked (crashed).
Message:  a Display implementation returned an error unexpectedly: Error
Location: /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/string.rs:2830

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 9 frames hidden ⋮                               
  10: <core[4727a1a5e2f8bf41]::result::Result<(), core[4727a1a5e2f8bf41]::fmt::Error>>::expect<unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1119
  11: <cpp_demangle[6fffd1d6edfb1b57]::Symbol<&[u8]> as alloc[679dddab44c6194d]::string::SpecToString>::spec_to_string<unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/string.rs:2830
  12: <cpp_demangle[6fffd1d6edfb1b57]::Symbol<&[u8]> as alloc[679dddab44c6194d]::string::ToString>::to_string<unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/string.rs:2808
  13: <project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}::{closure#0}::{closure#0}<unknown>
      at /home/user/src/project/crates/project_crate/src/common.rs:50
  14: <core[4727a1a5e2f8bf41]::option::Option<cpp_demangle[6fffd1d6edfb1b57]::Symbol<&[u8]>>>::map::<project_crate[cb21979b4fe2b8de]::common::UnmangledSymbolName, <project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}::{closure#0}::{closure#0}><unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:1146
  15: <project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}::{closure#0}<unknown>
      at /home/user/src/project/crates/project_crate/src/common.rs:49
  16: std[da76efc55aba569b]::panicking::catch_unwind::do_call::<<project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}::{closure#0}, core[4727a1a5e2f8bf41]::option::Option<project_crate[cb21979b4fe2b8de]::common::UnmangledSymbolName>><unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:589
  17: std[da76efc55aba569b]::panicking::catch_unwind::<core[4727a1a5e2f8bf41]::option::Option<project_crate[cb21979b4fe2b8de]::common::UnmangledSymbolName>, <project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}::{closure#0}><unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:552
  18: std[da76efc55aba569b]::panic::catch_unwind::<<project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}::{closure#0}, core[4727a1a5e2f8bf41]::option::Option<project_crate[cb21979b4fe2b8de]::common::UnmangledSymbolName>><unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:359
  19: <project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}<unknown>
      at /home/user/src/project/crates/project_crate/src/common.rs:46
  20: <core[4727a1a5e2f8bf41]::option::Option<project_crate[cb21979b4fe2b8de]::common::UnmangledSymbolName>>::or_else::<<project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle::{closure#1}><unknown>
      at /home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:1609
  21: <project_crate[cb21979b4fe2b8de]::common::MangledSymbolName>::demangle<unknown>
      at /home/user/src/project/crates/project_crate/src/common.rs:45
 [ about 30 stack frames from using rayon ]
  54: std::sys::pal::unix::thread::Thread::new::thread_start::h1ff51d6e85162efd
      at /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/sys/pal/unix/thread.rs:107

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.

[VorpalBlade]

So it looks like it fails to demangle, yet Symbol::new doesn't return an Error but Ok. That seems very wrong to me. And it leads to .to_string panicking.

From https://doc.rust-lang.org/stable/std/fmt/trait.Display.html#tymethod.fmt:

This function should return Err if, and only if, the provided Formatter returns Err. String formatting is considered an infallible operation; this function only returns a Result because writing to the underlying stream might fail and it must provide a way to propagate the fact that an error has occurred back up the stack.

It doesn't seem like to_string here complies with that as it can fail for other reasons. If Symbol can't comply with this, it shouldn't implement Display.

[khuey]

Yeah, see e.g. #233. We need to remove the Display impl.