Are you happy with your logging solution? Would you help us out by taking a 30-second survey? Click here

py_zipkin

Provides utilities to facilitate the usage of Zipkin in Python

Subscribe to updates I use py_zipkin


Statistics on py_zipkin

Number of watchers on Github 84
Number of open issues 15
Average time to close an issue 13 days
Main language Python
Average time to merge a PR 2 days
Open pull requests 3+
Closed pull requests 0+
Last commit over 1 year ago
Repo Created about 3 years ago
Repo Last Updated over 1 year ago
Size 196 KB
Organization / Authoryelp
Contributors7
Page Updated
Do you use py_zipkin? Leave a review!
View open issues (15)
View on github
Fresh, new opensource launches 🚀🚀🚀
Trendy new open source projects in your inbox! View examples

Subscribe to our mailing list

Evaluating py_zipkin for your project? Score Explanation
Commits Score (?)
Issues & PR Score (?)

Travis Coverage Status PyPi version Supported Python versions

py_zipkin

py_zipkin provides a context manager/decorator along with some utilities to facilitate the usage of Zipkin in Python applications.

Install

pip install py_zipkin

Usage

py_zipkin requires a transport_handler function that handles logging zipkin messages to a central logging service such as kafka or scribe.

py_zipkin.zipkin.zipkin_span is the main tool for starting zipkin traces or logging spans inside an ongoing trace. zipkin_span can be used as a context manager or a decorator.

Usage #1: Start a trace with a given sampling rate

from py_zipkin.zipkin import zipkin_span

def some_function(a, b):
    with zipkin_span(
        service_name='my_service',
        span_name='my_span_name',
        transport_handler=some_handler,
        port=42,
        sample_rate=0.05, # Value between 0.0 and 100.0
    ):
        do_stuff(a, b)

Usage #2: Trace a service call

The difference between this and Usage #1 is that the zipkin_attrs are calculated separately and passed in, thus negating the need of the sample_rate param.

# Define a pyramid tween
def tween(request):
    zipkin_attrs = some_zipkin_attr_creator(request)
    with zipkin_span(
        service_name='my_service',
        span_name='my_span_name',
        zipkin_attrs=zipkin_attrs,
        transport_handler=some_handler,
        port=22,
    ) as zipkin_context:
        response = handler(request)
        zipkin_context.update_binary_annotations(
            some_binary_annotations)
        return response

Usage #3: Log a span inside an ongoing trace

This can be also be used inside itself to produce continuously nested spans.

@zipkin_span(service_name='my_service', span_name='some_function')
def some_function(a, b):
    return do_stuff(a, b)

Other utilities

zipkin_span.update_binary_annotations() can be used inside a zipkin trace to add to the existing set of binary annotations.

def some_function(a, b):
    with zipkin_span(
        service_name='my_service',
        span_name='some_function',
        transport_handler=some_handler,
        port=42,
        sample_rate=0.05,
    ) as zipkin_context:
        result = do_stuff(a, b)
        zipkin_context.update_binary_annotations({'result': result})

zipkin_span.add_sa_binary_annotation() can be used to add a binary annotation to the current span with the key 'sa'. This function allows the user to specify the destination address of the service being called (useful if the destination doesn't support zipkin). See http://zipkin.io/pages/data_model.html for more information on the 'sa' binary annotation.

def some_function():
    with zipkin_span(
        service_name='my_service',
        span_name='some_function',
        transport_handler=some_handler,
        port=42,
        sample_rate=0.05,
    ) as zipkin_context:
        make_call_to_non_instrumented_service()
        zipkin_context.add_sa_binary_annotation(
            port=123,
            service_name='non_instrumented_service',
            host='12.34.56.78',
        )

create_http_headers_for_new_span() creates a set of HTTP headers that can be forwarded in a request to another service.

headers = {}
headers.update(create_http_headers_for_new_span())
http_client.get(
    path='some_url',
    headers=headers,
)

Transport

py_zipkin (for the moment) thrift-encodes spans. The actual transport layer is pluggable, though. The transport_handler is a function that takes a single argument - the thrift-encoded bytes.

The simplest way to get spans to the collector is via HTTP POST. Here's an example of a simple HTTP transport using the requests library. This assumes your Zipkin collector is running at localhost:9411.

import requests

def http_transport(encoded_span):
    # The collector expects a thrift-encoded list of spans.
    requests.post(
        'http://localhost:9411/api/v1/spans',
        data=encoded_span,
        headers={'Content-Type': 'application/x-thrift'},
    )

If you have the ability to send spans over Kafka (more like what you might do in production), you'd do something like the following, using the kafka-python package:

from kafka import SimpleProducer, KafkaClient

def transport_handler(message):
    kafka_client = KafkaClient('{}:{}'.format('localhost', 9092))
    producer = SimpleProducer(kafka_client)
    producer.send_messages('kafka_topic_name', message)

Using in multithreading evironments

If you want to use py_zipkin in a cooperative multithreading environment, e.g. asyncio, you need to explicitly pass an instance of py_zipkin.stack.Stack as parameter context_stack for zipkin_span and create_http_headers_for_new_span. By default, py_zipkin uses a thread local storage for the attributes, which is defined in py_zipkin.stack.ThreadLocalStack.

Firehose mode [EXPERIMENTAL]

Firehose mode records 100% of the spans, regardless of sampling rate. This is useful if you want to treat these spans differently, e.g. send them to a different backend that has limited retention. It works in tandem with normal operation, however there may be additional overhead. In order to use this, you add a firehose_handler just like you add a transport_handler.

This feature should be considered experimental and may be removed at any time without warning. If you do use this, be sure to send asynchronously to avoid excess overhead for every request.

License

Copyright (c) 2018, Yelp, Inc. All Rights reserved. Apache v2

py_zipkin open issues Ask a question     (View All Issues)
  • over 2 years Support 128-bit trace IDs
  • almost 3 years annotation_list_builder hangs if you pass a string instead of an integer
  • almost 3 years py_zipkin doesn't work correctly in threaded applications
  • almost 3 years the zipkin_span decorator may allow too many arguments
  • almost 3 years how do we idiomatically send cs, cr annotations
  • about 3 years Support sending spans in batches
py_zipkin open pull requests (View All Pulls)
  • WIP: Zipkin firehose
  • update zipkinCore.thrift
  • Don't overwrite passed in annotations
py_zipkin list of languages used
Other projects in Python