flexmeasures.data.services.job_cache

Index RQ jobs by asset or sensor and cache fetched job lists.

Classes

class flexmeasures.data.services.job_cache.JobCache(connection: Redis)

This class is used for storing jobs and retrieving them from redis cache. This happens in two ways: asset/sensor <> job mapping, and actual job lists. Both are heavily used in the status page, but could be used by other places.

  1. asset/sensor <> job mapping

JobCache creates a lookup layer in Redis, by which each job can be found by its assorted asset or sensor, for as long as the job itself lives (the job itself is not in this layer). Places that create jobs use JobCache so that the status page can make us of it later. The Redis index key includes asset or sensor ID, queue, and entity type:

  • forecasting:sensor:1 (forecasting jobs can be stored by sensor only)

  • scheduling:sensor:2

  • scheduling:asset:3

  1. actual job lists

The status page (and the /asset/ID/jobs endpoint it uses) is paginated. We also want to lower the amount of Redis work when looking up jobs. So we store lists of actual job objects for a minute, to ease browsing the pages.

In particular: - get() reads the current jobs from Redis and removes IDs whose jobs expired. - get_snapshot() reuses fetched job lists in process for one minute. paging does not fetch the same jobs repeatedly.

__init__(connection: Redis)
get(asset_or_sensor_id: int, queue: str, asset_or_sensor_type: str) → list[Job]

Fetch current jobs from the Redis ID index.

get_snapshot(asset_or_sensor_id: int, queue: str, asset_or_sensor_type: str) → list[Job]

Reuse a short-lived in-process job list snapshot.

Exceptions

exception flexmeasures.data.services.job_cache.NoRedisConfigured(message='Redis not configured')