CacheToolsUtils

CacheTools Utilities. This code is public domain.

Classes

DebugCache

Debug class.

DictCache

Cache class based on dict.

LockedCache

Cache class with a lock.

PrefixedCache

Cache class to add a key prefix.

StatsCache

Cache class to keep stats.

TwoLevelCache

Two-Level Cache class for CacheTools.

EncryptedCache

Encrypted Bytes Key-Value Cache.

BytesCache

Map bytes to strings.

ToBytesCache

Map (JSON-serializable) cache keys and values to bytes.

JsonSerde

JSON serialize/deserialize class for MemCached (pymemcache).

MemCached

MemCached-compatible wrapper class for cachetools with key encoding.

PrefixedMemCached

MemCached-compatible wrapper class for cachetools with a key prefix.

RedisCache

Redis TTL-ed wrapper for cachetools (redis).

PrefixedRedisCache

Prefixed Redis wrapper class for cachetools.

StatsRedisCache

TTL-ed Redis wrapper class for cachetools.

Functions

cached(cache, *args, **kwargs)

Extended decorator with delete and exists.

cacheMethods(cache, obj[, gen, opts])

Cache some object methods.

cacheFunctions(cache, globs[, gen, opts])

Cache some global functions, with a prefix.

json_key(→ str)

JSON serialization of arguments.

full_hash_key(→ str)

Reduce arguments to a single 128-bits hash.

Module Contents

class CacheToolsUtils.DebugCache(cache: MutableMapping, log: logging.Logger, name='cache')

Bases: _StatsMix, MutableMapping

Debug class.

Parameters:
  • cache – actual cache

  • log – logging instance

  • name – name of instance for output

clear()

D.clear() -> None. Remove all items from D.

class CacheToolsUtils.DictCache

Bases: _MutMapMix, MutableMapping

Cache class based on dict.

clear()

D.clear() -> None. Remove all items from D.

class CacheToolsUtils.LockedCache(cache: MutableMapping, lock)

Bases: _MutMapMix, _StatsMix, _RedisMix, MutableMapping

Cache class with a lock.

Parameters:
  • cache – actual cache.

  • lock – lock (context manager) to use

The locked layer should be the last one before the actual cache.

class CacheToolsUtils.PrefixedCache(cache: MutableMapping, prefix: str | bytes = '')

Bases: _KeyMutMapMix, _StatsMix, MutableMapping

Cache class to add a key prefix.

Parameters:
  • cache – actual cache.

  • prefix – prefix to prepend to keys.

class CacheToolsUtils.StatsCache(cache: MutableMapping)

Bases: _MutMapMix, MutableMapping

Cache class to keep stats.

Parameters:

cache – actual cache.

Note that CacheTools cached decorator with info=True provides hits, misses, maxsize and currsize information. However, this only works for its own classes.

hits() float

Return the cache hit ratio.

reset()

Reset internal stats data.

stats() dict[str, Any]

Return available stats data as dict.

clear()

D.clear() -> None. Remove all items from D.

class CacheToolsUtils.TwoLevelCache(cache: MutableMapping, cache2: MutableMapping, resilient=False)

Bases: _MutMapMix, MutableMapping

Two-Level Cache class for CacheTools.

Parameters:
  • cache – first (smaller, shorter TTL) cache

  • cache2 – second (larger, longer TTL) cache

  • resilient – whether to ignore cache2 failures

clear()

D.clear() -> None. Remove all items from D.

class CacheToolsUtils.EncryptedCache(cache: MutableMapping, secret: bytes, hsize: int = 16, csize: int = 0, cipher: str = 'Salsa20')

Bases: _KeyMutMapMix, _StatsMix, MutableMapping

Encrypted Bytes Key-Value Cache.

Parameters:
  • secret – bytes of secret, at least 16 bytes.

  • hsize – size of hashed key, default is 16.

  • csize – value checksum size, default is 0.

  • cipher – chose cipher from “Salsa20”, “AES-128-CBC” or “ChaCha20”.

The key is not encrypted but simply hashed, thus they are fixed size with a very low collision probability.

By design, the clear-text key is needed to recover the value, as each value is encrypted with its own key and nonce.

Algorithms:

  • SHA3: hash/key/nonce derivation and checksum.

  • Salsa20, AES-128-CBC or ChaCha20: value encryption.

class CacheToolsUtils.BytesCache(cache)

Bases: _KeyMutMapMix, _StatsMix, MutableMapping

Map bytes to strings.

class CacheToolsUtils.ToBytesCache(cache)

Bases: _KeyMutMapMix, _StatsMix, MutableMapping

Map (JSON-serializable) cache keys and values to bytes.

CacheToolsUtils.cached(cache, *args, **kwargs)

Extended decorator with delete and exists.

All parameters are forwarded to cachetools.cached.

Parameters:

cache – actual cache.

If f(*args, **kwargs) is the cached function, then:

  • f.cache_in(*args, **kwargs) tells whether the result is cached.

  • f.cache_del(*args, **kwargs) deletes (invalidates) the cached result.

CacheToolsUtils.cacheMethods(cache: MutableMapping, obj: Any, gen: MapGen = PrefixedCache, opts: dict[str, Any] = {}, **funs)

Cache some object methods.

Parameters:
  • cache – cache to use.

  • obj – object instance to be cached.

  • gen – generator of PrefixedCache.

  • opts – additional parameters when calling cached.

  • funs – name of methods and corresponding prefix

cacheMethods(cache, item, PrefixedCache, compute1="c1.", compute2="c2.")
CacheToolsUtils.cacheFunctions(cache: MutableMapping, globs: MutableMapping[str, Any], gen: MapGen = PrefixedCache, opts: dict[str, Any] = {}, **funs)

Cache some global functions, with a prefix.

Parameters:
  • cache – cache to use.

  • globs – global object dictionary.

  • gen – generator of PrefixedCache.

  • opts – additional parameters when calling cached.

  • funs – name of functions and corresponding prefix

cacheFunctions(cache, globals(), PrefixedCache, fun1="f1.", fun2="f2.")
CacheToolsUtils.json_key(*args, **kwargs) str

JSON serialization of arguments.

CacheToolsUtils.full_hash_key(*args, **kwargs) str

Reduce arguments to a single 128-bits hash.

class CacheToolsUtils.JsonSerde

JSON serialize/deserialize class for MemCached (pymemcache).

import pymemcache as pmc
import CacheToolsUtils as ctu
pmc_cache = pmc.Client(server="localhost", serde=ctu.JsonSerde())
class CacheToolsUtils.MemCached(cache)

Bases: _KeyMutMapMix, MutableMapping

MemCached-compatible wrapper class for cachetools with key encoding.

Parameters:

cache – actual memcached cache.

import pymemcache as pmc
import CacheToolsUtils as ctu
cache = ctu.MemCached(pmc.Client(server="localhost", serde=ctu.JsonSerde()))

@ctu.cached(cache=cache)
def whatever(...):
stats() dict[str, Any]

Return MemCached stats.

clear()

Flush MemCached contents.

hits() float

Return overall cache hit ratio.

class CacheToolsUtils.PrefixedMemCached(cache, prefix: str = '')

Bases: MemCached

MemCached-compatible wrapper class for cachetools with a key prefix.

Parameters:
  • cache – actual memcached cache.

  • prefix – post key-encoding prepended prefix.

import pymemcache as pmc
import CacheToolsUtils as ctu
# add a "app." prefix to keys, after serialization
cache = ctu.PrefixedMemCached(pmc.Client(server="localhost", serde=ctu.JsonSerde()), "app.")
class CacheToolsUtils.RedisCache(cache, ttl=600, raw=False)

Bases: MutableMapping

Redis TTL-ed wrapper for cachetools (redis).

Parameters:
  • cache – actual redis cache.

  • ttl – time-to-live in seconds, used as default expiration (ex), default is 600.

  • raw – whether to serialize keys and values, default is False.

Keys and values are serialized in JSON.

import redis
import CacheToolsUtils as ctu
# redis with 1 hour expiration
cache = ctu.RedisCache(redis.Redis(host="localhost"), 3600)
clear()

Flush Redis contents.

info(*args, **kwargs)

Return redis informations.

dbsize(*args, **kwargs)

Return redis database size.

set(index, value, **kwargs)

Set cache contents.

get(index, default=None)

Get cache contents.

delete(index)

Delete cache contents.

hits() float

Return cache hits.

class CacheToolsUtils.PrefixedRedisCache(cache, prefix: str = '', ttl=600)

Bases: RedisCache

Prefixed Redis wrapper class for cachetools.

Parameters:
  • cache – actual redis cache.

  • prefix – post key encoding prefix, default is empty.

  • ttl – time-to-live in seconds, used as default expiration (ex), default is 600.

import redis
import CacheToolsUtils as ctu
# redis with "app." prefix and 1 hour expiration
cache = ctu.PrefixedRedisCache(redis.Redis(host="localhost"), "app.", 3600)
class CacheToolsUtils.StatsRedisCache(cache, prefix: str = '', ttl=600)

Bases: PrefixedRedisCache

TTL-ed Redis wrapper class for cachetools.

This class is nearly empty.