Illustration for minrk/wurlitzer#76.
First, set up a logger. This one logs to stdout and includes the timestamp, so we can see how often log lines are produced.
import logging
logger = logging.getLogger("test-log-pipe")
logger.handlers[:] = []
logger.setLevel(logging.INFO)
h = logging.StreamHandler(sys.stdout)
h.setLevel(logging.INFO)
h.setFormatter(logging.Formatter("[{name} {asctime}] {msg}", style="{"))
logger.addHandler(h)
logger.info("test message")
[test-log-pipe 2024-04-23 11:25:12,338] test message
Next, we need to create a Writable (anything with a write
method),
which will take lines and put them onto the logger
import io
class LogPipe(io.BufferedWriter):
"""Writeable that writes lines to a Logger object as they are written"""
def __init__(self, logger):
self.logger = logger
self.buf = ""
def write(self, chunk):
"""Given chunk, split into lines
Log each line as a discrete message
If it ends with a partial line, save it until the next one
"""
if self.buf:
chunk = self.buf + chunk
lines = chunk.splitlines(True)
if not lines[-1].endswith("\n"):
self.buf = lines[-1]
lines = lines[:-1]
else:
self.buf = ""
for line in lines:
self.logger.info(line.rstrip())
pipe = LogPipe(logger)
Test it out: pipe anything written to low-level stderr to our logger (which writes to stdout)
import sys
import time
import wurlitzer
logger.info("start")
with wurlitzer.pipes(stdout=None, stderr=pipe):
for i in range(15):
sys.__stderr__.write(f"step {i}\n")
time.sleep(0.1)
logger.info("end")
[test-log-pipe 2024-04-23 11:27:15,554] start [test-log-pipe 2024-04-23 11:27:15,578] step 0 [test-log-pipe 2024-04-23 11:27:15,679] step 1 [test-log-pipe 2024-04-23 11:27:15,781] step 2 [test-log-pipe 2024-04-23 11:27:15,881] step 3 [test-log-pipe 2024-04-23 11:27:15,982] step 4 [test-log-pipe 2024-04-23 11:27:16,083] step 5 [test-log-pipe 2024-04-23 11:27:16,190] step 6 [test-log-pipe 2024-04-23 11:27:16,291] step 7 [test-log-pipe 2024-04-23 11:27:16,395] step 8 [test-log-pipe 2024-04-23 11:27:16,495] step 9 [test-log-pipe 2024-04-23 11:27:16,600] step 10 [test-log-pipe 2024-04-23 11:27:16,700] step 11 [test-log-pipe 2024-04-23 11:27:16,801] step 12 [test-log-pipe 2024-04-23 11:27:16,906] step 13 [test-log-pipe 2024-04-23 11:27:17,011] step 14 [test-log-pipe 2024-04-23 11:27:17,116] end