apicrud.Metrics

class apicrud.Metrics(uid=None, db_session=None, func_send=None)

This implementation supports standard system metrics and two types of usage-billing: granted limits per period, or metered usage.

See this article for a description of how redis makes the implementation straightforward: https://www.infoworld.com/article/3230455/how-to-use-redis-for-real-time-metering-applications.html The data store is in-memory, with snapshot/append persistence to disk.

All metrics are defined in the metrics section of service_config.yaml. These follow a de facto standard described in best-practices documentation section of prometheus.io website. At time of implementation, flask prometheus client is not mature/user-friendly enough or suitable for usage-billing so to keep this consistent with service_config.yaml, there is no intent now or in the future to use it.

Thanks to the expiring keys feature of redis, grant limits can be implemented without any periodic cleanup task. Upon first use, a decrementing key is created with expiration set to the configured period. If it counts down to zero before expiration (for example, 50 video uploads per day), the user is denied access to the resource until the key expires. If a user never returns, there will be no redis keys consuming system resources unless and until the user comes back, at which point a new grant period starts. Scaling to tens of millions of users is practical within a small footprint, and this is why user-tracking metrics are stored in redis rather than a traditional database.

uid

ID of a user, for usage tracking

Type

str

db_session

existing db connection

Type

obj

func_send

function name for sending message via celery

Type

obj

__init__(uid=None, db_session=None, func_send=None)

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__([uid, db_session, func_send])

Initialize self.

check(name)

Check remaining credit against grant-style metric

collect(**kwargs)

Prometheus-compatible metrics exporter.

find(**kwargs)

Look up metrics defined by filter

store(name[, labels, value])

Store a metric: redis keys are in form mtr:<name>:<labels> To avoid possible ambiguity and multiple redis keys for the same label set, this function sorts labels before storing.

check(name)

Check remaining credit against grant-style metric

Params:

name (str): a metric name

Returns

amount of credit remaining, or None

Return type

float

collect(**kwargs)

Prometheus-compatible metrics exporter. Collects current metrics from redis, evaluates metric-type settings in service_config.yaml, and returns a plain-text result in this form:

# TYPE process_resident_memory_bytes gauge process_resident_memory_bytes{instance=”k8s-01.ci.net”} 20576.0 # TYPE process_start_time_seconds gauge process_start_time_seconds{instance=”k8s-01.ci.net”} 1614051156.14 # TYPE api_calls_total counter api_calls_total{resource=”person”} 4

find(**kwargs)

Look up metrics defined by filter

Returns

tuple (results dict, status)

store(name, labels=[], value=None)

Store a metric: redis keys are in form mtr:<name>:<labels> To avoid possible ambiguity and multiple redis keys for the same label set, this function sorts labels before storing.

Params:

name (str): a metric name labels (list): labels, usually in form <label>=<value> value (int or float): value to store

Returns

true in all cases except for grant exceeded

Return type

bool