Skip to content

Job Meta Data Overview

moebiusband73 edited this page Dec 18, 2020 · 2 revisions

Below are job related information as used by ClusterCockpit. I find it important to separate data related to job accounting from information required for job performance monitoring. There can be links between job accounting and job performance monitoring, but those two things should not be mixed up in my opinion.

  • job id
  • user id
  • project
  • cluster name
  • number of nodes - (redundant but useful for simplifying queries)
  • job state - (running, aborted, finished successfully, etc.)
  • start time - epoch time in s
  • duration - seconds
  • walltime - (to evaluate used vs requested time)
  • node and CPU list - node and CPU IDs that specify the used compute resources (if nodes are only used exclusively, the CPU list can be omitted)
  • tag list - array of tags, tags are pairs of <tag type>:<tag name>, both can be any string

Optional:

  • job script

Performance footprints in form of metric job averages. Useful to analyse or sort jobs according to HPM metrics. This can be freely configured in ClusterCockpit but one could also settle on a fixed set of metrics.

  • mem_capacity_avg
  • flops_any_avg
  • mem_bw_avg
  • ib_bw_avg
  • lustre_bw_avg

User meta data

  • user id - string
  • user name - string
  • email - string
  • is active - boolean, indicates if user is active or if this is an inactive account

I currently removed additional fields as group and project as I believe that a performance project usually happens together with a user. But this could be discussed. In ClusterCockpit current handling would add a tag to the job with project:<project name> and thereby all jobs of a project could be grouped.

Clone this wiki locally