rfc9951v1.txt   rfc9951.txt 
Internet Engineering Task Force (IETF) T. Graf Internet Engineering Task Force (IETF) T. Graf
Request for Comments: 9951 Swisscom Request for Comments: 9951 Swisscom
Category: Standards Track B. Claise Category: Standards Track B. Claise
ISSN: 2070-1721 Huawei ISSN: 2070-1721 Huawei
A. Huang-Feng A. Huang-Feng
INSA-Lyon INSA-Lyon
March 2026 April 2026
Export of Delay Performance Metrics in IP Flow Information Export Export of Delay Performance Metrics in IP Flow Information Export
(IPFIX) (IPFIX)
Abstract Abstract
This document specifies new IP Flow Information Export (IPFIX) This document specifies new IP Flow Information Export (IPFIX)
Information Elements to export the On-Path delay at each Operations, Information Elements to export the On-Path delay at each Operations,
Administration, and Maintenance (OAM) transit and decapsulating Administration, and Maintenance (OAM) transit and decapsulating
nodes. The On-Path delay is defined as the delay between the OAM nodes. The On-Path delay is defined as the delay between the OAM
skipping to change at line 116 skipping to change at line 116
Network operators usually maintain statistical views of delay across Network operators usually maintain statistical views of delay across
their networks to support diagnostics and performance analysis. their networks to support diagnostics and performance analysis.
These views assist in identifying the location, extent, and potential These views assist in identifying the location, extent, and potential
causes of abnormal delay affecting specific customer traffic or causes of abnormal delay affecting specific customer traffic or
services. To achieve this, delay-related metrics need to be reported services. To achieve this, delay-related metrics need to be reported
from devices covering both data and control planes. Further, in from devices covering both data and control planes. Further, in
order to understand which customers are affected, delay-related order to understand which customers are affected, delay-related
metrics need to be reported in the context of the customer data metrics need to be reported in the context of the customer data
plane. This correlation enables the detection of changes in plane. This correlation enables the detection of changes in
forwarding paths, such as updated intermediate hops or interfaces, forwarding paths, such as updated intermediate hops or interfaces,
and the resulting impact on delay experienced by customer traffic. and of the resulting impact on delay experienced by customer traffic.
Delay measurements in the network are computed using an On-Path Delay measurements in the network are computed using an On-Path
Telemetry protocol, which inserts metadata into the data-plane packet Telemetry protocol, which inserts metadata into the data-plane packet
when entering the monitored domain [RFC9232]. To compute delay when entering the monitored domain [RFC9232]. To compute delay
measurements, the On-Path Telemetry protocol inserts a timestamp measurements, the On-Path Telemetry protocol inserts a timestamp
reference when entering the OAM encapsulating node. Implementation reference when entering the OAM encapsulating node. Implementation
examples are In situ Operations, Administration, and Maintenance examples are In situ Operations, Administration, and Maintenance
(IOAM) [RFC9197] or Enhanced Alternate Marking [ENH-ALT-MARKING]. (IOAM) [RFC9197] or Enhanced Alternate Marking [ENH-ALT-MARKING].
Two modes of On-Path Telemetry are generally recognized: passport Two modes of On-Path Telemetry are generally recognized: passport
skipping to change at line 238 skipping to change at line 238
+-----------------------------+--------------------------------+ +-----------------------------+--------------------------------+
| OWDelay_HybridType1_I | pathDelaySumDeltaMicroseconds | | OWDelay_HybridType1_I | pathDelaySumDeltaMicroseconds |
| P_RFC9951_Seconds_Sum (30) | (533) | | P_RFC9951_Seconds_Sum (30) | (533) |
+-----------------------------+--------------------------------+ +-----------------------------+--------------------------------+
Table 1: Mapping Between IPFIX IEs and Performance Metrics Table 1: Mapping Between IPFIX IEs and Performance Metrics
Assuming time synchronization on devices, the delay is measured by Assuming time synchronization on devices, the delay is measured by
calculating the difference between the timestamp imposed with On-Path calculating the difference between the timestamp imposed with On-Path
Telemetry in the packet at an OAM header encapsulating node and the Telemetry in the packet at an OAM header encapsulating node and the
timestamp exported in the IPFIX flow record from the OAM header timestamp exported in the IPFIX Flow Record from the OAM header
transit and OAM header decapsulating nodes. The lowest, highest, transit and OAM header decapsulating nodes. The lowest, highest,
mean, and the sum of measured path delay can be exported, thanks to mean, and the sum of measured path delay can be exported, thanks to
the different IPFIX IE specifications. the different IPFIX IE specifications.
On-Path Telemetry Domain On-Path Telemetry Domain
......................................... .........................................
. . . .
. D1 . . D1 .
. x-------> . . x-------> .
. . . .
skipping to change at line 306 skipping to change at line 306
All column entries besides the Identifier, Name, URI, Description, All column entries besides the Identifier, Name, URI, Description,
Reference Description (Output only) categories are the same; thus, Reference Description (Output only) categories are the same; thus,
this section defines four closely related performance metrics. As a this section defines four closely related performance metrics. As a
result, IANA has assigned corresponding URIs to each of the four result, IANA has assigned corresponding URIs to each of the four
registered performance metrics. registered performance metrics.
4.1.1. Summary 4.1.1. Summary
This category includes multiple indexes of the registered performance This category includes multiple indexes of the registered performance
metrics: the element Identifier and Metric Name. metrics: the Identifier and Metric Name.
4.1.1.1. ID (Identifier) 4.1.1.1. ID (Identifier)
IANA has allocated the numeric Identifiers 27, 28, 29, and 30 for the IANA has allocated the numeric Identifiers 27, 28, 29, and 30 for the
four Named Metric Entries in the following section. four Named Metric Entries in the following section.
4.1.1.2. Name 4.1.1.2. Name
27: OWDelay_HybridType1_IP_RFC9951_Seconds_Mean 27: OWDelay_HybridType1_IP_RFC9951_Seconds_Mean
skipping to change at line 513 skipping to change at line 513
4.4.1. Type 4.4.1. Type
OWDelay Types are discussed in the subsections below. OWDelay Types are discussed in the subsections below.
4.4.2. Reference Definition 4.4.2. Reference Definition
For all output types: For all output types:
OWDelay_HybridType1_IP: The one-way delay of one IP packet is a OWDelay_HybridType1_IP: The one-way delay of one IP packet is a
Singleton. singleton.
For each <statistic> Singleton, one of the following subsections For each <statistic> singleton, one of the following subsections
applies. applies.
4.4.2.1. OWDelay_HybridType1_IP_RFC9951_Seconds_Mean 4.4.2.1. OWDelay_HybridType1_IP_RFC9951_Seconds_Mean
Similar to Section 7.4.2.2 of [RFC8912], the mean SHALL be calculated Similar to Section 7.4.2.2 of [RFC8912], the mean SHALL be calculated
using the conditional distribution of all packets with a finite value using the conditional distribution of all packets with a finite value
of one-way delay (undefined delays are excluded) -- a single value, of one-way delay (undefined delays are excluded) -- a single value,
as follows: as follows:
See Section 4.1 of [RFC3393] for details on the conditional See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5 distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice. of [RFC6703] for background on this analytic choice.
See Section 4.2.2 of [RFC6049] for details on calculating this See Section 4.2.2 of [RFC6049] for details on calculating this
statistic; see also Section 4.2.3 of [RFC6049]. statistic; see also Section 4.2.3 of [RFC6049].
Mean: The time value of the result is expressed in units of Mean: The time value of the result is expressed in units of
microseconds, as a positive value of type decimal64 with fraction microseconds, as a positive value of type decimal64 with fraction
digits = 9 (similar to decimal64 in YANG, Section 9.3 of digits = 9 (similar to decimal64 in YANG, Section 9.3 of
[RFC6020]) with a resolution of 0.001 microseconds (1.0 ns), and [RFC6020]) with a resolution of 0.001 microseconds (1.0 ns), and
with lossless conversion to/from the 64-bit NTP timestamp as per with lossless conversion to/from the 64-bit NTP timestamp as per
Section 6 of [RFC5905]. Section 6 of [RFC5905].
4.4.2.2. OWDelay_HybridType1_IP_RFC9951_Seconds_Min 4.4.2.2. OWDelay_HybridType1_IP_RFC9951_Seconds_Min
Similar to Section 7.4.2.3 of [RFC8912], the minimum SHALL be Similar to Section 7.4.2.3 of [RFC8912], the minimum SHALL be
calculated using the conditional distribution of all packets with a calculated using the conditional distribution of all packets with a
finite value of one-way delay (undefined delays are excluded) -- a finite value of one-way delay (undefined delays are excluded) -- a
single value, as follows: single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5 distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice. of [RFC6703] for background on this analytic choice.
See Section 4.3.2 of [RFC6049] for details on calculating this See Section 4.3.2 of [RFC6049] for details on calculating this
statistic; see also Section 4.3.3 of [RFC6049]. statistic; see also Section 4.3.3 of [RFC6049].
Min: The time value of the result is expressed in units of Min: The time value of the result is expressed in units of
microseconds, as a positive value of type decimal64 with fraction microseconds, as a positive value of type decimal64 with fraction
digits = 9 (similar to decimal64 in YANG, Section 9.3 of digits = 9 (similar to decimal64 in YANG, Section 9.3 of
[RFC6020]) with a resolution of 0.001 microseconds (1.0 ns), and [RFC6020]) with a resolution of 0.001 microseconds (1.0 ns), and
with lossless conversion to/from the 64-bit NTP timestamp as per with lossless conversion to/from the 64-bit NTP timestamp as per
Section 6 of [RFC5905]. Section 6 of [RFC5905].
4.4.2.3. OWDelay_HybridType1_IP_RFC9951_Seconds_Max 4.4.2.3. OWDelay_HybridType1_IP_RFC9951_Seconds_Max
Similar to Section 7.4.2.4 of [RFC8912], the maximum SHALL be Similar to Section 7.4.2.4 of [RFC8912], the maximum SHALL be
calculated using the conditional distribution of all packets with a calculated using the conditional distribution of all packets with a
finite value of one-way delay (undefined delays are excluded) -- a finite value of one-way delay (undefined delays are excluded) -- a
single value, as follows: single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5 distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice. of [RFC6703] for background on this analytic choice.
See Section 4.3.2 of [RFC6049] for a closely related method for See Section 4.3.2 of [RFC6049] for a closely related method for
calculating this statistic; see also Section 4.3.3 of [RFC6049]. The calculating this statistic; see also Section 4.3.3 of [RFC6049]. The
formula is as follows: formula is as follows:
Max = (FiniteDelay[j]) Max = (FiniteDelay[j])
such that for some index, j, where 1 <= j <= N such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n FiniteDelay[j] >= FiniteDelay[n] for all n
where all packets n = 1 through N have finite singleton delays. where all packets n = 1 through N have finite singleton delays.
skipping to change at line 596 skipping to change at line 596
Section 6 of [RFC5905]. Section 6 of [RFC5905].
4.4.2.4. OWDelay_HybridType1_IP_RFC9951_Seconds_Sum 4.4.2.4. OWDelay_HybridType1_IP_RFC9951_Seconds_Sum
The sum SHALL be calculated using the conditional distribution of all The sum SHALL be calculated using the conditional distribution of all
packets with a finite value of one-way delay (undefined delays are packets with a finite value of one-way delay (undefined delays are
excluded) -- a single value, as follows: excluded) -- a single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5 distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice. of [RFC6703] for background on this analytic choice.
See Section 4.3.5 of [RFC6049] for details on calculating this See Section 4.3.5 of [RFC6049] for details on calculating this
statistic; however, in this case, FiniteDelay or MaxDelay MAY be statistic; however, in this case, FiniteDelay or MaxDelay MAY be
used. used.
Sum: The time value of the result is expressed in units of Sum: The time value of the result is expressed in units of
microseconds, as a positive value of type decimal64 with fraction microseconds, as a positive value of type decimal64 with fraction
digits = 9 (similar to decimal64 in YANG, Section 9.3 of digits = 9 (similar to decimal64 in YANG, Section 9.3 of
[RFC6020]) with a resolution of 0.001 microseconds (1.0 ns), and [RFC6020]) with a resolution of 0.001 microseconds (1.0 ns), and
with lossless conversion to/from the 64-bit NTP timestamp as per with lossless conversion to/from the 64-bit NTP timestamp as per
skipping to change at line 655 skipping to change at line 655
2026-MM-DD 2026-MM-DD
4.4.4. Comments and Remarks 4.4.4. Comments and Remarks
None None
5. Use Cases 5. Use Cases
The measured On-Path delay can be aggregated with Flow Aggregation as The measured On-Path delay can be aggregated with Flow Aggregation as
defined in [RFC7015] to the following device and control-plane defined in [RFC7015] across the following device and control-plane
dimensions [IANA-IPFIX] to determine: dimensions [IANA-IPFIX] to determine:
* With node id and egressInterface(14), on which node which logical * With node id and egressInterface(14), on which node which logical
egress interfaces have been contributing to how much delay. egress interfaces have been contributing to how much delay.
* With node id and egressPhysicalInterface(253), on which node which * With node id and egressPhysicalInterface(253), on which node which
physical egress interfaces have been contributing to how much physical egress interfaces have been contributing to how much
delay. delay.
* With ipNextHopIPv4Address(15) or ipNextHopIPv6Address(62), the * With ipNextHopIPv4Address(15) or ipNextHopIPv6Address(62), the
skipping to change at line 836 skipping to change at line 836
7.1. Time Accuracy 7.1. Time Accuracy
In terms of clock precision, the same recommendation as defined in In terms of clock precision, the same recommendation as defined in
Section 4.5 of [RFC5153] for IPFIX applies to this document as well. Section 4.5 of [RFC5153] for IPFIX applies to this document as well.
7.2. Mean Delay 7.2. Mean Delay
The mean (average) path delay can be calculated by dividing the The mean (average) path delay can be calculated by dividing the
pathDelaySumDeltaMicroseconds(533) by the packetDeltaCount(2) at the pathDelaySumDeltaMicroseconds(533) by the packetDeltaCount(2) at the
IPFIX data collection in order to offload the IPFIX Exporter from IPFIX data collection at the collection time instead of the IPFIX
calculating the mean for every Flow at export time. Exporter at the export time.
7.3. Reduced-Size Encoding 7.3. Reduced-Size Encoding
Unsigned64 has been chosen as the type for Unsigned64 has been chosen as the type for
pathDelaySumDeltaMicroseconds to support cases with large delay pathDelaySumDeltaMicroseconds to support cases with large delay
numbers and where many packets are being accounted. As an example, a numbers and where many packets are being counted. As an example, a
specific Flow Record with path delay of 100 milliseconds cannot specific Flow Record with path delay of 100 milliseconds cannot
observe more than 42949 packets without overflowing the unsigned32 observe more than 42949 packets without overflowing the unsigned32
counter. The procedure described in Section 6.2 of [RFC7011] may be counter. The procedure described in Section 6.2 of [RFC7011] may be
applied to reduce network bandwidth between the IPFIX Exporter and applied to reduce network bandwidth between the IPFIX Exporter and
Collector if unsigned32 would be large enough without wrapping Collector if unsigned32 would be large enough without wrapping
around. around.
7.4. Measurement Interval 7.4. Measurement Interval
The delay metrics are computed for the Flow Record lifetime by The delay metrics are computed for the Flow Record lifetime by
skipping to change at line 888 skipping to change at line 888
use the Extension-Flag defined in [IOAM-DEX] to insert a timestamp in use the Extension-Flag defined in [IOAM-DEX] to insert a timestamp in
the OAM header encapsulating node. The timestamp is encoded as the OAM header encapsulating node. The timestamp is encoded as
defined in Sections 4.4.2.3 and 4.4.2.4 of [RFC9197]. This timestamp defined in Sections 4.4.2.3 and 4.4.2.4 of [RFC9197]. This timestamp
can be used to compute the delay between the inserted timestamp and can be used to compute the delay between the inserted timestamp and
the OAM header transit and decapsulating node. the OAM header transit and decapsulating node.
For the Enhanced Alternate Marking Method, Section 2 of For the Enhanced Alternate Marking Method, Section 2 of
[ENH-ALT-MARKING] and Section 3.2 of [RFC9947] define that, within [ENH-ALT-MARKING] and Section 3.2 of [RFC9947] define that, within
the metaInfo, a nanosecond timestamp can be encoded in an OAM header the metaInfo, a nanosecond timestamp can be encoded in an OAM header
encapsulating node and be read at the OAM header intermediate and encapsulating node and be read at the OAM header intermediate and
decapsulating node to calculate the on-path delay. [RFC9343] defines decapsulating nodes to calculate the On-path delay. [RFC9343]
how this can be applied to the IPv6 extensions header, and [RFC9947] defines how this can be applied to the IPv6 extensions header, and
defines how this can be applied to the SRv6 Segment Routing Header [RFC9947] defines how this can be applied to the SRv6 Segment Routing
[RFC8754]. Header [RFC8754].
Given that the delay measurements are computed with the timestamp Given that the delay measurements are computed with the timestamp
introduced on the OAM header encapsulating node, regardless of the introduced on the OAM header encapsulating node, regardless of the
approach, implementations should document at which point of the approach, implementations should document at which point of the
forwarding plane this timestamp is introduced (e.g., the time at forwarding plane this timestamp is introduced (e.g., the time at
which the packet was received by the node, the time at which the which the packet was received by the node, the time at which the
packet was transmitted by the node, etc.). Based on this packet was transmitted by the node, etc.). Based on this
information, different actions can be taken. information, different actions can be taken.
8. Security Considerations 8. Security Considerations
The IPFIX Information Elements introduced in this document do not The IPFIX Information Elements introduced in this document do not
directly introduce security issues. Rather, they define a set of directly introduce security issues. Rather, they define a set of
performance metrics that may, for privacy or business issues, be performance metrics that may, for privacy or business issues, be
considered sensitive information. considered sensitive information.
For example, exporting delay metrics may make attacks possible for For example, exporting delay metrics may make attacks possible by the
the receiver of this information; otherwise, this would only be receiver of this information; otherwise, this would only be possible
possible for direct observers of the reported Flows along the data for direct observers of the reported Flows along the data path.
path.
IPFIX collectors MUST ensure that IPFIX data originates from trusted IPFIX collectors MUST ensure that IPFIX data originates from trusted
sources. Accepting IPFIX data from unauthenticated sources could sources. Accepting IPFIX data from unauthenticated sources could
lead to data spoofing, policy misapplication, or denial of service. lead to data spoofing, policy misapplication, or denial of service.
Therefore, the underlying protocol used to exchange the information Therefore, the underlying protocol used to exchange the information
described here must apply appropriate procedures to guarantee the described here must apply appropriate procedures to guarantee the
integrity and confidentiality of the exported information. These integrity and confidentiality of the exported information. These
protocols are defined in separate documents; specifically, see the protocols are defined in separate documents; specifically, see the
IPFIX security considerations in Section 11 of [RFC7011]. IPFIX security considerations in Section 11 of [RFC7011].
skipping to change at line 966 skipping to change at line 965
[RFC6049] Morton, A. and E. Stephan, "Spatial Composition of [RFC6049] Morton, A. and E. Stephan, "Spatial Composition of
Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011, Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011,
<https://www.rfc-editor.org/info/rfc6049>. <https://www.rfc-editor.org/info/rfc6049>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX) "Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77, Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013, RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>. <https://www.rfc-editor.org/info/rfc7011>.
[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model
for IP Flow Information Export (IPFIX)", RFC 7012,
DOI 10.17487/RFC7012, September 2013,
<https://www.rfc-editor.org/info/rfc7012>.
[RFC7323] Borman, D., Braden, B., Jacobson, V., and R. [RFC7323] Borman, D., Braden, B., Jacobson, V., and R.
Scheffenegger, Ed., "TCP Extensions for High Performance", Scheffenegger, Ed., "TCP Extensions for High Performance",
RFC 7323, DOI 10.17487/RFC7323, September 2014, RFC 7323, DOI 10.17487/RFC7323, September 2014,
<https://www.rfc-editor.org/info/rfc7323>. <https://www.rfc-editor.org/info/rfc7323>.
[RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Ed., "A One-Way Delay Metric for IP Performance Metrics Ed., "A One-Way Delay Metric for IP Performance Metrics
(IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
2016, <https://www.rfc-editor.org/info/rfc7679>. 2016, <https://www.rfc-editor.org/info/rfc7679>.
 End of changes. 16 change blocks. 
27 lines changed or deleted 21 lines changed or added

This html diff was produced by rfcdiff 1.48.