|Number of watchers on Github||56|
|Number of open issues||4|
|Average time to merge a PR||2 days|
|Open pull requests||4+|
|Closed pull requests||1+|
|Last commit||over 2 years ago|
|Repo Created||almost 6 years ago|
|Repo Last Updated||over 2 years ago|
|Organization / Author||stripe|
|Do you use unilog? Leave a review!|
|View open issues (4)|
|View unilog activity|
|View on github|
|Fresh, new opensource launches 🚀🚀🚀|
Software engineers: It's time to get promoted. Starting NOW! Subscribe to my mailing list and I will equip you with tools, tips and actionable advice to grow in your career.
unilog is a tool for use with djb's daemontools process-monitor.
The way logging works with daemontools is that all output from a
daemontools-controlled process runs into a pipe, which is passed to a
second program, which can read from the pipe and format log output,
write it to disk or whatever. Daemontools ships with such a tool,
multilog, but it is arcane and weird, so we use
The job of unilog is to read lines of log output from stdin (which will normally be connected to a running daemon by way of a pipe), format them (which normally consists just of adding a timestamp), and write them to a log file provided on the command-line.
If unilog receives a
SIGALRM, it responds by closing and
reopening the output file. This can be used to perform graceful log
rotation without requiring any special support from the running
If unilog is unable to open or write to the output file, it will email about this error, once per hour, until it succeeds in a write, discarding output in the process.
In addition to the pipe buffer, unilog maintains an in-process buffer of log lines that it will fill up if writes to the disk are slow or blocking. This is intended to prevent the kernel pipe buffers from filling up (and thus blocking the daemon from writing) during brief periods of disk overload or hangs (sadly common in virtualized environments).
Unilog can be configured to apply filters to each line and perform arbitrary transformations. (For example, you may want to strip out sensitive information, or strip high-volume logs).
Unilog contains an optional system for managing log volume, using criticality and austerity levels. If this systems is enabled, every log line has a criticality level associated with it. There are four levels of log criticality. In ascending order of importance, they are:
criticalplus. (These names are taken from Site Reliability Engineering, How Google Runs Production Systems.)
During times of high log volume, log lines may be sampled at exponential rates. The criticality level (clevel) of a log line determines its relative priority when sampling. By default, the system austerity level is set to
sheddable, which means that all lines are preserved. If the austerity level is raised to
sheddableplus, then only 10% of lines logged at
sheddable are preserved, and the rest are filtered. If the austerity level is raised to
critical, then 10% of lines logged at
clevel=sheddableplus are preserved, and 1% of lines logged at
clevel=sheddable are preserved, and so forth.
Criticality levels operate using filters, so this system is not just limited to sampling logs to reduce volume - it can be used to apply arbitrary transformations to a random subset of log lines.