NAME | C SYNOPSIS | DESCRIPTION | SEMANTIC CHECKS AND RULES | EXPRESSION EVALUATION | CAVEATS | DIAGNOSTICS | SEE ALSO | COLOPHON

PMREGISTERDERIVED(3)      Library Functions Manual      PMREGISTERDERIVED(3)

NAME         top

       pmRegisterDerived, pmRegisterDerivedMetric  - register a derived met‐
       ric name and definition

C SYNOPSIS         top

       #include <pcp/pmapi.h>

       char *pmRegisterDerived(char *name, char *expr);
       int pmRegisterDerivedMetric(char *name, char *expr, char **errmsg);

       cc ... -lpcp

DESCRIPTION         top

       Derived metrics provide a way of extending the Performance Metrics
       Name Space (PMNS) with new metrics defined at the PCP client-side
       using expressions over the existing performance metrics.

       Typical uses would be to aggregate a number of similar metrics to
       provide a higher-level summary metric or to support the ``delta V
       over delta V'' class of metrics that are not possible in the base
       data semantics of PCP.  An example of the latter class would be the
       average I/O size, defined as
                delta(disk.dev.total_bytes) / delta(disk.dev.total)
       where both of the disk.dev metrics are counters, and what is required
       is to to sample both metrics, compute the difference between the
       current and previous values and then calculate the ratio of these
       differences.

       The arguments to pmRegisterDerived are the name of the new derived
       metric and expr is an expression defining how the values of name
       should be computed.

       pmRegisterDerivedMetric is the exact functional equivalent to
       pmRegisterDerived except that it provides a simplified model of error
       handling, where a formatted message is returned via the errmsg
       parameter.

       Syntactic checking is performed at the time pmRegisterDerived is
       called, but semantic checking is deferred until each new PMAPI
       context is created with pmNewContext(3) or re-established with
       pmReconnectContext(3), at which time the PMNS and metadata is
       available to allow semantic checking and the metadata of the derived
       metrics to be determined.  This means pmRegisterDerived does not
       apply retrospectively to any open PMAPI contexts, nor to any PMAPI
       contexts already open at the time pmRegisterDerived is called, so the
       normal use would be to make all calls to pmRegisterDerived (possibly
       via pmLoadDerivedConfig(3)) or pmRegisterDerivedMetric and then call
       pmNewContext(3).

       name should follow the syntactic rules for the names of performance
       metrics, namely one or more components separated with a dot (``.''),
       and each component must begin with an alphabetic followed by zero or
       more characters drawn from the alphabetics, numerics and underscore
       (``_'').  For more details, refer to PCPIntro(1) and pmns(5).

       name must be unique across all derived metrics and should not match
       the name of any regular metric in the PMNS.  It is acceptable for
       name to share some part of its prefix with an existing subtree of the
       PMNS, e.g. the average I/O size metric above could be named
       disk.dev.avgsz which would place it amongst the other disk.dev
       metrics in the PMNS.  Alternatively, derived metrics could populate
       their own subtree of the PMNS, e.g. the average I/O size metric above
       could be named my.summary.disk.avgsz.

       The expression expr follows these syntactic rules:

       * Terminal elements are either names of existing metrics or numeric
         constants.  Recursive definitions are not allowed, so only the
         names of regular metrics (not other derived metrics) may be used.
         Numeric constants are either integers constrained to the precision
         of 32-bit unsigned integers or double precision floating point
         numbers.

       * The usual binary arithmetic operators are supported, namely
         addition (``+''), subtraction (``-''), multiplication (``*'') and
         division (``/'') with the normal precedence rules where
         multiplication and division have higher precedence than addition
         and subtraction, so a+b*c is evaluated as a+(b*c).

       * Unary negation may be used, e.g.  -3*some.metric.

       * C-style relational operators are supported, namely ``<'', ``<='',
         ``=='', ``>='', ``>'' and ``!=''.  Relational expresssions return a
         value as a 32-bit unsigned number being 0 for false and 1 for true.
         The expected operator precedence rules apply, so arithmetic
         operators have higher precedence than relational operators, and a-
         b>c+d is evaluated as (a-b)>(c+d).  All the relational operators
         have equal precedence, so the (slightly odd) expression involving
         consecutive relational operators a>b!=c is evaluated as (a>b)!=c.

       * C-style boolean operators are supported, namely and (``&&'') and or
         (``||'').  Boolean expresssions return a value as a 32-bit unsigned
         number being 0 for false and 1 for true.  The expected operator
         precedence rules apply, so relational operators have higher
         precedence than boolean operators, and a>b*c&&d<=e+f is evaluated
         as (a>(b*c))&&(d<=(e+f)).  Both the boolean operators have equal
         precedence, so the expression involving consecutive boolean
         operators a>=b||b>c&&d!=e||f>g is evaluated as
         (((a>=b)||(b>c))&&(d!=e))||(f>g).

       * Additionally, the ``!'' operator may be used to negate a boolean or
         relational expression, returning a value as a 32-bit unsigned
         number being 0 for false and 1 for true.  The expected operator
         precedence rules apply, so boolean (and relational) operators have
         higher precedence than boolean negation, and !a>b||c<d is evaluated
         as !((a>b)||(c<d)), while !a<b+c is evaluated as !(a<(b+c)).

       * C-style ternary conditional expressions are supported. In general
         terms the expression check ? foo : bar is evaluated as foo (the
         ``true'' operand) if check (the ``guard'') is true, else the
         expression evaluates to bar (the ``false'' operand).  Some special
         semantic rules apply to the ``guard'' expression and the other two
         operand expressions:
         (a) Each expression may involve a singular value or a set of values
             (when the expression involves one or more metrics with an
             instance domain).
         (b) All expressions with a set of values must be defined over the
             same instance domain.
         (c) Both operand expressions must have the same metadata, so the
             same metric type, semantics and units (dimension and scale).
         (d) The ``guard'' expression must have an aritmetic or relational
             or boolean value, so that it can be evaluated as 0 for false,
             else true.
         (e) If the ``guard'' expression has a singular value and one or
             more of the other operand expressions involves an instance
             domain, the ``guard'' applies to all instances.
         (f) If the ``guard'' expression has a set of values and one or more
             of the other operand expressions involves an instance domain,
             the ``guard'' is evaluated once for each instance (there must
             be one instance domain as per rule (b) above).
         (g) If one of the operand expressions has a singular value and the
             other has a set of values, and the singular value is selected
             based on the evaluation of the ``guard'', then the result is a
             set of values (all the same) with instance enumeration being
             taken from the other operand expression. For example in the
             expression: foo ? scalar : set, if foo is true, then the result
             is a set of values (all having the same value, scalar) over the
             instance domain of set.

       * Numeric constants can also be specified using the mkconst()
         constructor which takes a number of arguments: the first is a
         numeric constant (either integer or floating point), then follow
         one or more parameters of the form tag=value or tag= where the
         allowed values of tag and value are as follows:

             ┌──────────┬───────────────────────────────────────────────┐
             │   tagvalue                     │
             ├──────────┼───────────────────────────────────────────────┤
             │type      │ one of the numeric metric types from          │
             │          │ <pcp/pmapi.h>, stripped of the PM_TYPE_       │
             │          │ prefix, so 32, U32, 64, U64, FLOAT or DOUBLE. │
             ├──────────┼───────────────────────────────────────────────┤
             │semantics │ one of the semantic types from <pcp/pmapi.h>, │
             │          │ stripped of the PM_SEM_ prefix, so COUNTER,   │
             │          │ INSTANT or DISCRETE.                          │
             ├──────────┼───────────────────────────────────────────────┤
             │units     │ a specification of dimension and scale        │
             │          │ (together forming the units), in the syntax   │
             │          │ accepted by pmParseUnitsStr(3).               │
             └──────────┴───────────────────────────────────────────────┘
         The value may optionally be enclosed in double quotes, and may
         appear in any mix of upper and/or lower case.  The tag must be in
         lower case as shown in the table above.

         This is most useful when the expression semantics require matching
         type and/or semantics and/or units for operands, e.g.
         idle = mem.util.free > mkconst(10485760, units=Kbyte)
         avg_io_size = delta(disk.dev.total) == 0 ? \
             -mkconst(1.0, semantics=instant, units="kbyte / count") : \
             delta(disk.dev.total_bytes) / delta(disk.dev.total)

       * Expressions may be rescaled using the rescale function that takes
         two arguments.  The first is an arithmetic expression to be
         rescaled, and the second is the desired units after rescaling that
         is a string value in the syntax accepted by pmParseUnitsStr(3).
         For example:
         rescale(network.interface.total.bytes, "Mbytes/hour")

         The expression and the desired units must both have the same
         dimension, e.g Space=1, Time=-1 and Count=0 in the example above.

       * The following unary functions operate on a single performance
         metric and return one or more values.  For all functions (except
         count(), defined() and instant()), the type of the operand metric
         must be arithmetic (integer of various sizes and signedness, float
         or double).

            ┌───────────┬───────────────────────────────────────────────┐
            │ Function  │                     Value                     │
            ├───────────┼───────────────────────────────────────────────┤
            │avg(x)     │ A singular instance being the average value   │
            │           │ across all instances for the metric x.        │
            ├───────────┼───────────────────────────────────────────────┤
            │count(x)   │ A singular instance being the count of the    │
            │           │ number of instances for the metric x.  As a   │
            │           │ special case, if fetching the metric x        │
            │           │ returns an error, then count(x) will be 0.    │
            ├───────────┼───────────────────────────────────────────────┤
            │defined(x) │ A boolean value that is true (``1'') if the   │
            │           │ metric x is defined in the PMNS, else false   │
            │           │ (``0'').  The function is evaluated when a    │
            │           │ new PMAPI context is created with             │
            │           │ pmNewContext(3) or re-established with        │
            │           │ pmReconnectContext(3).  So any subsequent     │
            │           │ changes to the PMNS after the PMAPI context   │
            │           │ has been established will not change the      │
            │           │ value of this function in the expression      │
            │           │ evaluation.                                   │
            ├───────────┼───────────────────────────────────────────────┤
            │delta(x)   │ Returns the difference in values for the      │
            │           │ metric x between one call to pmFetch(3) and   │
            │           │ the next. There is one value in the result    │
            │           │ for each instance that appears in both the    │
            │           │ current and the previous sample.              │
            ├───────────┼───────────────────────────────────────────────┤
            │rate(x)    │ Returns the difference in values for the      │
            │           │ metric x between one call to pmFetch(3) and   │
            │           │ the next divided by the elapsed time between  │
            │           │ the calls to pmFetch(3).  The semantics of    │
            │           │ the derived metric are based on the semantics │
            │           │ of the operand (x) with the dimension in the  │
            │           │ time domain decreased by one and scaling if   │
            │           │ required in the time utilization case where   │
            │           │ the operand is in units of time, and the      │
            │           │ derived metric is unitless.  This mimics the  │
            │           │ rate conversion applied to counter metrics by │
            │           │ tools such as pmval(1), pmie(1) and           │
            │           │ pmchart(1).  There is one value in the result │
            │           │ for each instance that appears in both the    │
            │           │ current and the previous sample.              │
            ├───────────┼───────────────────────────────────────────────┤
            │instant(x) │ Returns the current value of the metric x,    │
            │           │ even it has the semantics of a counter, i.e.  │
            │           │ PM_SEM_COUNTER.  The semantics of the derived │
            │           │ metric are based on the semantics of the      │
            │           │ operand (x); if x has semantics               │
            │           │ PM_SEM_COUNTER, the semantics of instant(x)   │
            │           │ is PM_SEM_INSTANT, otherwise the semantics of │
            │           │ the derived metric is the same as the         │
            │           │ semantics of the metric x.                    │
            ├───────────┼───────────────────────────────────────────────┤
            │max(x)     │ A singular instance being the maximum value   │
            │           │ across all instances for the metric x.        │
            ├───────────┼───────────────────────────────────────────────┤
            │min(x)     │ A singular instance being the minimum value   │
            │           │ across all instances for the metric x.        │
            ├───────────┼───────────────────────────────────────────────┤
            │sum(x)     │ A singular instance being the sum of the      │
            │           │ values across all instances for the metric x. │
            └───────────┴───────────────────────────────────────────────┘
       * Parenthesis may be used for explicit grouping.

       * Lines beginning with ``#'' are treated as comments and ignored.

       * White space is ignored.

SEMANTIC CHECKS AND RULES         top

       There are a number of conversions required to determine the metadata
       for a derived metric and to ensure the semantics of the expressions
       are sound.

       In an arithmetic expression or a relational expression, if the
       semantics of both operands is not a counter (i.e. PM_SEM_INSTANT or
       PM_SEM_DISCRETE) then the result will have semantics PM_SEM_INSTANT
       unless both operands are PM_SEM_DISCRETE in which case the result is
       also PM_SEM_DISCRETE.

       For an arithmetic expression, the dimension of each operand must be
       the same.  For a relational expression, the dimension of each operand
       must be the same, except that numeric constants (with no dimension)
       are allowed, e.g. in the expression network.interface.in.drops > 0 .

       To prevent arbitrary and non-sensical combinations some restrictions
       apply to expressions that combine metrics with counter semantics to
       produce a result with counter semantics.  For an arithmetic
       expression, if both operands have the semantics of a counter, then
       only addition or subraction is allowed, or if the left operand is a
       counter and the right operand is not, then only multiplication or
       division are allowed, or if the left operand is not a counter and the
       right operand is a counter, then only multiplication is allowed.

       Because relational expressions use the current value only and produce
       a result that is not a counter, either or both operands of a
       relational expression may be counters.

       The mapping of the pmUnits of the metadata uses the following rules:

       * If both operands have a dimension of Count and the scales are not
         the same, use the larger scale and convert the values of the
         operand with the smaller scale.

       * If both operands have a dimension of Time and the scales are not
         the same, use the larger scale and convert the values of the
         operand with the smaller scale.

       * If both operands have a dimension of Space and the scales are not
         the same, use the larger scale and convert the values of the
         operand with the smaller scale.

       * For addition and subtraction all dimensions for each of the
         operands and result are identical.

       * For multiplication, the dimensions of the result are the sum of the
         dimensions of the operands.

       * For division, the dimensions of the result are the difference of
         the dimensions of the operands.

       Scale conversion involves division if the dimension is positive else
       multiplication if the dimension is negative. If scale conversion is
       applied to either of the operands, the result is promoted to type
       PM_TYPE_DOUBLE.

       Putting all of this together in an example, consider the derived
       metric defined as follows:
         x = network.interface.speed - delta(network.interface.in.bytes) /
                            delta(sample.milliseconds)
       The type, dimension and scale settings would propagate up the
       expression tree as follows.

       ┌────────────────────────┬────────┬───────────────┬─────────────────┐
       │      Expression        │  Type  │  Dimension &  │ Scale Factor(s) │
       │                        │        │  Scale        │                 │
       ├────────────────────────┼────────┼───────────────┼─────────────────┤
       │sample.milliseconds     │ DOUBLE │ millisec      │                 │
       │delta(...)              │ DOUBLE │ millisec      │                 │
       │network...bytes         │ U64    │ byte          │                 │
       │delta(...)              │ U64    │ byte          │                 │
       │delta(...) / delta(...) │ DOUBLE │ byte/millisec │ /1048576 and    │
       │                        │        │               │ *1000           │
       │network...speed         │ FLOAT  │ Mbyte/sec     │                 │
       │x                       │ DOUBLE │ Mbyte/sec     │                 │
       └────────────────────────┴────────┴───────────────┴─────────────────┘
       Because semantic checking cannot be done at the time
       pmRegisterDerived is called, errors found during semantic checking
       (when any subsequent calls to pmNewContext(3) or
       pmReconnectContext(3) succeed) are reported using pmprintf(3).  These
       include:

       Error: derived metric <name1>: operand: <name2>: <reason>
              There was a problem calling pmLookupName(3) to identify the
              operand metric <name2> used in the definition of the derived
              metric <name1>.

       Error: derived metric <name1>: operand (<name2> [<pmid2>]): <reason>
              There was a problem calling pmLookupDesc(3) to identify the
              operand metric <name2> with PMID <pmid2> used in the
              definition of the derived metric <name1>.

       Semantic error: derived metric <name>: <operand> : <operand>
       Different <metadata> for ternary operands
              For a ternary expression, the ``true'' operand and the
              ``false'' operand must have exactly the same metadata, so
              type, semantics, instance domain, and units (dimension and
              scale).

       Semantic error: derived metric <name>: <operand> <op> <operand>:
       Dimensions are not the same
              Operands must have the same units (dimension and scale) for
              each of addition, subtraction, the relational operators and
              the boolean ``and'' or ``or'' operators.

       Semantic error: derived metric <name>: <operand> <op> <operand>:
       Illegal operator for counter and non-counter
              Only multiplication or division are allowed if the left
              operand has the semantics of a counter and the right operand
              is not a counter.

       Semantic error: derived metric <name>: <operand> <op> <operand>:
       Illegal operator for counters
              If both operands have the semantics of counter, only addition
              or subtraction make sense, so multiplication and division are
              not allowed.

       Semantic error: derived metric <name>: <operand> <op> <operand>:
       Illegal operator for non-counter and counter
              Only multiplication is allowed if the right operand has the
              semantics of a counter and the left operand is not a counter.

       Semantic error: derived metric <metric> <expr> RESCALE <units>:
       Incompatible dimensions
              The parameters <expr> and <units> to the rescale function must
              have the same dimension along the axes of Time, Space and
              Count.

       Semantic error: derived metric <name>: Incorrect time dimension for
       operand
              Rate conversion using the rate() function is only possible for
              operand metrics with a Time dimension of 0 or 1 (see
              pmLookupDesc(3)).  If the operand metric's Time dimension is
              0, then the derived metrics has a value "per second" (Time
              dimension of -1).  If the operand metric's Time dimension is
              1, then the derived metrics has a value of time utilization
              (Time dimension of 0).

       Semantic error: derived metric <name>: <function>(<operand>): Non-
       arithmetic operand for function
              The unary functions are only defined if the operand has
              arithmetic type.  Similarly the first argument to the rescale
              function must be of arithmetic type.

       Semantic error: derived metric <name>: <expr> ? ...: Non-arithmetic
       operand for ternary guard
              The first expression for a ternary operator must have an
              arithmetic type.

       Semantic error: derived metric <name>: ... - ...: Non-arithmetic
       operand for unary negation
              Unary negation only makes sense if the following expression
              has an arithmetic type.

       Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
       arithmetic type for <left-or-right> operand
              The binary arithmetic operators are only allowed with operands
              with an arithmetic type (integer of various sizes and
              signedness, float or double).

       Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
       counter and not dimensionless <left-or-right> operand
              For multiplication or division or any of the relational
              operators, if one of the operands has the semantics of a
              counter and the other has the semantics of a non-counter
              (instantaneous or discrete) then the non-counter operand must
              have no units (dimension and scale).

       Semantic error: derived metric <name>: <expr> ? <expr> : <expr>: Non-
       scalar ternary guard with scalar expressions
              If the ``true'' and ``false'' operands of a ternary expression
              have a scalar value, then the ``guard'' expression must also
              have a scalar value.

       Semantic error: derived metric <name>: <expr> <op> <expr>: Operands
       should have the same instance domain
              For all of the binary operators (arithmetic and relational),
              if both operands have non-scalar values, then they must be
              defined over the same instance domain.

EXPRESSION EVALUATION         top

       For the binary arithmetic operators, if either operand must be scaled
       (e.g. convert bytes to Kbytes) then the result is promoted to
       PM_TYPE_DOUBLE.  Otherwise the type of the result is determined by
       the types of the operands, as per the following table which is
       evaluated from top to bottom until a match is found.

              ┌─────────────────────────┬──────────┬────────────────┐
              │     Operand Types       │ Operator │  Result Type   │
              ├─────────────────────────┼──────────┼────────────────┤
              │either is PM_TYPE_DOUBLE │ any      │ PM_TYPE_DOUBLE │
              ├─────────────────────────┼──────────┼────────────────┤
              │any                      │ division │ PM_TYPE_DOUBLE │
              ├─────────────────────────┼──────────┼────────────────┤
              │either is PM_TYPE_FLOAT  │ any      │ PM_TYPE_FLOAT  │
              ├─────────────────────────┼──────────┼────────────────┤
              │either is PM_TYPE_U64    │ any      │ PM_TYPE_U64    │
              ├─────────────────────────┼──────────┼────────────────┤
              │either is PM_TYPE_64     │ any      │ PM_TYPE_64     │
              ├─────────────────────────┼──────────┼────────────────┤
              │either is PM_TYPE_U32    │ any      │ PM_TYPE_U32    │
              ├─────────────────────────┼──────────┼────────────────┤
              │otherwise (both are      │ any      │ PM_TYPE_32     │
              │PM_TYPE_32)              │          │                │
              └─────────────────────────┴──────────┴────────────────┘

CAVEATS         top

       Derived metrics are not available when using pmFetchArchive(3) as
       this routine does not use a target list of PMIDs that could be
       remapped (as is done for pmFetch(3)).

       There is no pmUnregisterDerived method, so once registered a derived
       metric persists for the life of the application.

DIAGNOSTICS         top

       On success, pmRegisterDerived returns NULL.

       If a syntactic error is found at the time of registration, the value
       returned by pmRegisterDerived is a pointer into expr indicating where
       the error was found.  To identify what the error was, the application
       should call pmDerivedErrStr(3) to retrieve the corresponding parser
       error message.

       pmRegisterDerivedMetric returns 0 and errmsg is undefined if the
       parsing is successful.

       If the given expr does not conform to the required syntax
       pmRegisterDerivedMetric returns -1 and a dynamically allocated error
       message string in errmsg.  The error message is terminated with a
       newline and includes both the input name and expr, along with an
       indicator of the position at which the error was detected.  e.g.
                 Error: pmRegisterDerivedMetric("my.disk.rates", ...) syntax
                 error

       The position indicator line may be followed by an additional
       diagnostic line describing the nature of the error, when available.

       In the case of an error, the caller is responsible for calling
       free(3) to release the space allocated for errmsg.

SEE ALSO         top

       PCPIntro(1), free(3), PMAPI(3), pmDerivedErrStr(3), pmFetch(3),
       pmLoadDerivedConfig(3), pmNewContext(3), pmprintf(3) and
       pmReconnectContext(3).

COLOPHON         top

       This page is part of the PCP (Performance Co-Pilot) project.
       Information about the project can be found at ⟨http://www.pcp.io/⟩.
       If you have a bug report for this manual page, send it to
       pcp@oss.sgi.com.  This page was obtained from the project's upstream
       Git repository ⟨https://github.com/performancecopilot/pcp.git⟩ on
       2017-09-15.  If you discover any rendering problems in this HTML ver‐
       sion of the page, or you believe there is a better or more up-to-date
       source for the page, or you have corrections or improvements to the
       information in this COLOPHON (which is not part of the original man‐
       ual page), send a mail to man-pages@man7.org

Performance Co-Pilot                                    PMREGISTERDERIVED(3)

Pages that refer to this page: pmrep(1)pmderivederrstr(3)pmfetchgroup(3)pmloadderivedconfig(3)pmreconnectcontext(3)pmrep.conf(5)