Yet another REPL freeze issue while typing

[tamasgal]

The symptoms are the same as described here #54131 but the fix does not work in our case. Tried on macOS, Linux with Julia 1.11.0 to 1.11.5, no issues with older versions.

The reproducer in our case requires two packages from our own repository though:

julia> using Pkg; Pkg.Registry.add(); Pkg.Registry.add(RegistrySpec(url = "https://git.km3net.de/common/julia-registry"))

julia> Pkg.add(name="KM3io", version="0.18.8")

julia> Pkg.add("KM3NeTTestData")  # version doesn't matter, it's just sample files

This is the code:

julia> using KM3io, KM3NeTTestData

julia> f = ROOTFile(datapath("offline", "numucc.root"))
ROOTFile{OnlineTree (0 events, 0 summaryslices), OfflineTree (10 events)}

julia> f.offline[1].

The freeze is triggered when typing f.offline[1].. When the dot is pressed, it hangs for several seconds and each keystroke introduces a new freeze (except for the backspace).

Let me know if I can provide anything else!

[Moelf]

what's the type of f.offline[1]? I'm guessing LazyEvent from UnROOT.jl? (which is basically a StructArray.LazyRow)

So I guess what's happening is because the LazyEvent is like a NamedTuple with huge type information inherent from the f.offline::LazyTree, the inference is choking.

This is unfortunate, and I wish I had better foresight when designing it. The reason we went this route is to have absolute peak performance when user writes down for evt in f.offline naively, but in hindsight we may want this to be more explicit, like for evt in IteratedTree(f.offline) and make costly inference/compilation happens at IterateTree() step

[tamasgal]

• Actually the only return value (path) is (yielding) an Evt object (no parametrisation whatsoever). Even if I annotate the getindex with ::Evt, I see no difference. Not sure how to improve but this kind of bad user experience only happens with Julia 1.11, so I wish there was an opt-out of whatever happens 😄

[tamasgal]

Just to make sure: I think the type safety measures we accepted for UnROOT are totally fine. It's the REPL which tries to be too "smart"?

[JeffBezanson]

It seems like this should be fixable in the REPL, because propertynames(f.offline[1]) returns quickly for me, and even after attempting the tab completion (which takes several seconds but eventually happens) several times, every keystroke after the dot is very slow. So it must be doing something unnecessary.

[tamasgal]

A small update: disabling the repl.options.hint_tab_completes gets rid of the instant freeze (I tried that before but probably made a mistake and did not notice any improvement) on typing but unfortunately does not help with the tab-completion, which still hangs for many seconds on each triggering.

atreplinit() do repl
    if VERSION >= v"1.11.0-0"
        repl.options.hint_tab_completes = false
    end
end

[tamasgal]

Are there any news on this? This is really making things unusable for a fairly big part of the astroparticle physics community since the base I/O library is affected which is the foundation for everything else.

I tried to debug this myself but I am fairly lost in the REPL logic. I can try harder of course but it would be much more efficient if the authors of the code could have a quick look.

[Moelf]

we could make the LazyTable not carry all the column names in the type by default (i.e. make it more like DataFrame), and only compile those type information when users explicitly ask for it, to make for-loop fast for example

[IanButterworth]

Is this still a problem on the latest 1.12 release candidate?

[tamasgal]

Yep, on RC1 still present

[IanButterworth]

@aviatesk Just wondering if you see a fix here? It seems to me like we spend a lot of time completing this via type inference because we don't want to execute code that is otherwise fast?