-
Notifications
You must be signed in to change notification settings - Fork 931
Description
Hello metaflow team!
First of all I want to thank you for this awesome tool!
I'm posting this issue to ask if there is a way to disable metaflow's logging. I see that metaflow will catch all system's message and output them using click.secho. This causes the following issue on our part. We capture all the logs in Splunk which expects a json output format. However metaflow will append a string to the output and "pollutes" the message.
Here's an example piece of code:
from metaflow import FlowSpec, step
import structlog
def test_method(logger):
logger.info("method called", step_method_name="test_method")
structlog.configure(processors=[structlog.processors.JSONRenderer()])
_logger = structlog.get_logger()
class MyFlow(FlowSpec):
@step
def start(self):
_logger.info("flow_started", flow_step_name="start")
test_method(_logger)
self.next(self.end)
@step
def end(self):
_logger.error("flow_ended", status="success")
def run() -> None:
MyFlow()
if __name__ == "__main__":
run()which will output:
2025-05-26 18:51:15.530 Workflow starting (run-id 1748278275528814): 2025-05-26 18:51:15.545 [1748278275528814/start/1 (pid 91978)] Task is starting. 2025-05-26 18:51:15.799 [1748278275528814/start/1 (pid 91978)] {"flow_step_name": "start", "event": "flow_started"} 2025-05-26 18:51:15.832 [1748278275528814/start/1 (pid 91978)] {"step_method_name": "test_method", "event": "method called"} 2025-05-26 18:51:15.833 [1748278275528814/start/1 (pid 91978)] Task finished successfully. 2025-05-26 18:51:15.839 [1748278275528814/end/2 (pid 91981)] Task is starting. 2025-05-26 18:51:16.108 [1748278275528814/end/2 (pid 91981)] {"status": "success", "event": "flow_ended"} 2025-05-26 18:51:16.142 [1748278275528814/end/2 (pid 91981)] Task finished successfully. 2025-05-26 18:51:16.144 Done!
As you can see instead of having {"flow_step_name": "start", "event": "flow_started"}, the output is: 2025-05-26 18:51:15.799 [1748278275528814/start/1 (pid 91978)] {"flow_step_name": "start", "event": "flow_started"}.
Is there a way to output only info from my logger?