Skip to content

[Bug]: local redirect of stderr does not work in jupyter #562

@ikondov

Description

@ikondov

Code snippet

$ jupyter console
Jupyter console 6.6.3

Python 3.10.12 (main, Nov  4 2025, 08:48:33) [GCC 11.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.25.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from fireworks.core.rocket_launcher import rapidfire

In [2]: from fireworks import LaunchPad

In [3]: from fireworks.fw_config import LAUNCHPAD_LOC

In [4]: from fireworks import PyTask, Firework, Workflow

In [5]: lpad = LaunchPad.from_file(LAUNCHPAD_LOC)

In [6]: lpad.add_wf(Workflow([Firework([PyTask(func='math.log', args=[0.0])])]))
Out[6]: {-1: 1}

In [7]: rapidfire(lpad, local_redirect=True)
Traceback (most recent call last):
  File "/mnt/data/ubuntu/work/python-3.10.12/lib/python3.10/site-packages/fireworks/core/rocket.py", line 260, in run
    m_action = t.run_task(my_spec)
  File "/mnt/data/ubuntu/work/python-3.10.12/lib/python3.10/site-packages/fireworks/user_objects/firetasks/script_task.py", line 187, in run_task
    output = func(*args, **kwargs)
ValueError: math domain error

In [8]: lpad.get_wf_summary_dict(fw_id=1)
Out[8]:
{'state': 'FIZZLED',
 'name': 'unnamed WF',
 'created_on': datetime.datetime(2025, 12, 16, 10, 0, 51, 619000),
 'updated_on': datetime.datetime(2025, 12, 16, 10, 1, 27, 88000),
 'states': {'Unnamed FW--1': 'FIZZLED'},
 'launch_dirs': {'Unnamed FW--1': ['/mnt/data/ubuntu/work/vre-language/examples/launcher_2025-12-16-10-01-27-032894']}}

In [9]: ls /mnt/data/ubuntu/work/vre-language/examples/launcher_2025-12-16-10-01-27-032894
FW.json  FW_job.error  FW_job.out

In [10]: more /mnt/data/ubuntu/work/vre-language/examples/launcher_2025-12-16-10-01-27-032894/FW_job.error

What happened?

  1. I am running a PyTask producing a runtime error and run the firework with the rapidfire with local redirect of output. I use jupyter to interactively steer the computation.
  2. Expected bahavior: The traceback is written to the error file.
  3. Instead, the traceback is not written to the error file but was written to stderr (on the screen)
  4. Proposed workaround: yield error file object in the context manager local_redirect used in rapidfire function. Proposed solution in general: do not use an approach to patch the sys module because this may not work in some contexts.

When I use FireWorks not in Jupyter then the problem does not occur.

Version

upstream/main

Which OS?

  • MacOS
  • Windows
  • Linux

Log output

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions