]> NTT Information Sharing Platform Laboratories

3-9-11 Midori-cho Musashino-shiTokyo 180-8585 Japan +81-422-59-3978 akoba@nttv6.net

NTT Information Sharing Platform Laboratories

3-9-11 Midori-cho Musashino-shiTokyo 180-8585 Japan +81-422-59-3978 nishida.haruhiko@lab.ntt.co.jp

Cisco Systems

De Kleetlaan 6a b1 Diegem 1831 Belgium +32 2 704 5622 bclaise@cisco.com

Operations and Management IPFIX Working Group This document describes a framework for an IPFIX Mediation. This framework details an IPFIX Mediation reference model and the components of the IPFIX Mediation device (IPFIX Mediator).

IPFIX Mediation reroutes, replicates, filters, aggregates, correlates, or modifies Flow Records/Packet Reports or changes a transport protocol. This document describes the framework for IPFIX Mediation. The motivation for the IPFIX Mediation standard comes from the need for flow-based measurement system support for large-scale networks, interdomain networks, and coexistence with traditional Exporters as described in detail in . The standard specification requires a definition of IPFIX Mediation and IPFIX Mediation device (IPFIX Mediator). This document is organized as follows. Section 2 describes terminology related to IPFIX Mediation. Section 3 describes a high level reference model. Section 4 details the components of the IPFIX Mediator.

The terms in this section are in line with those in the IPFIX specification document and the PSAMP specification document . Additional terms required for the IPFIX Mediation are also defined with those in the IPFIX Mediator problem statement . All these terms are capitalized in this document. An Observation Point is a location in the network where IP packets can be observed. Examples include: a line to which a probe is attached, a shared medium, such as an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) of a router. Note that every Observation Point is associated with an Observation Domain (defined below), and that one Observation Point may be a superset of several other Observation Points. For example, one Observation Point can be an entire line card. That would be the superset of the individual Observation Points at the line card's interfaces. An Observation Domain is the largest set of Observation Points for which Flow information can be aggregated by a Metering Process. For example, a router line card may be an Observation Domain if it is composed of several interfaces, each of which is an Observation Point. In the IPFIX Message it generates, the Observation Domain includes its Observation Domain ID, which is unique per Exporting Process. That way, the Collecting Process can identify the specific Observation Domain from the Exporter that sends the IPFIX Messages. Every Observation Point is associated with an Observation Domain. It is RECOMMENDED that Observation Domain IDs also be unique per IPFIX Device. Each of the fields that: 1. belong to the packet header (e.g., destination IP address), 2. are a property of the packet itself (e.g., packet length), 3. are derived from packet treatment (e.g., Autonomous System (AS) number), and that are used to define a Flow are termed Flow Keys. A Flow Record contains information about a specific Flow that was observed at an Observation Point. A Flow Record contains measured properties of the Flow (e.g., the total number of bytes for all the Flow's packets) and usually characteristic properties of the Flow (e.g., source IP address). Packet Reports comprise a configurable subset of a packet's input to the Selection Process, including the Packet Content, information relating to its treatment (for example, the output interface), and its associated selection state (for example, a hash of the Packet Content). The Exporting Process sends Flow Records to one or more Collecting Processes. The Flow Records are generated by one or more Metering Processes. A device that hosts one or more Exporting Processes is termed an Exporter. An IPFIX Device hosts at least one Exporting Process. It may host further Exporting Processes and arbitrary numbers of Observation Points and Metering Processes. A Collecting Process receives Flow Records from one or more Exporting Processes. The Collecting Process might process or store received Flow Records, but such actions are out of the scope of this document. A device that hosts one or more Collecting Processes is termed a Collector. An IPFIX Message is a message originating at the Exporting Process that carries the IPFIX records of this Exporting Process and whose destination is a Collecting Process. An IPFIX Message is encapsulated at the transport layer. An Information Element is a protocol and encoding-independent description of an attribute that may appear in an IPFIX Record. The IPFIX information model defines the base set of Information Elements for IPFIX. The type associated with an Information Element indicates constraints on what it may contain and also determines the valid encoding mechanisms for use in IPFIX. An IPFIX Mediation is a generic term for functions doing something for Flow Records, Packet Reports, and IPFIX Messages. IPFIX Mediation is located in between components: Metering Processes, Exporting Processes, Collecting Processes, and other applications. IPFIX Mediation can be included in any IPFIX Devices. IPFIX Mediation consists of a set of some of the following functions: rerouting input Flow Records/Packet Reports to an appropriate Collecting Process replicating input Flow Records/Packet Reports filtering and selecting input Flow Records/Packet Reports aggregating input Flow Records/Packet Reports based on new Flow Keys correlating a set of Flow Records/Packet Reports for creating new Flow Records/Packet Reports with new metrics modifying input Flow Records/Packet Reports changing transport protocols that carry IPFIX Messages The modification of Flow Records/Packet Reports includes these processes: changing the value of specified Information Elements adding new Information Elements by deriving further Flow or packet properties from existing fields or calculating new metrics deleting specified Information Elements. IPFIX Mediation can be included in any device, such as routers, switches, NMS (Network Management Systems), or stand-alone devices. The Flow-Based Collector Selection evaluates an input Flow Record/Packet Report based on the value of the specified Information Element and then selects a Collector for each input Flow Record/Packet Report. An IPFIX Mediator contains one or more functions defined in IPFIX Mediation. The IPFIX Mediator can be a stand-alone or a virtual device. It also contains one or more Collecting Processes and one or more Exporting Processes. An Original Exporter is an IPFIX Device that hosts Observation Points where IP packets can be directly observed. An IPFIX Proxy is an IPFIX Mediator that receives IPFIX Messages from an Original Exporter and sends IPFIX Messages to one or more Collectors. It may alter part of an IPFIX Message to comply with IPFIX Protocol specifications. It may also change the type of transport protocol, such as UDP, TCP, SCTP, and PR-SCTP, and convert a legacy protocol message to an IPFIX Message, if necessary. An IPFIX Concentrator is an IPFIX Mediator that receives Flow Records/Packet Reports, aggregates them, then exports the aggregated Flow Records. An IPFIX Distributor is an IPFIX Mediator that reroutes input Flow Records/Packet Reports based on the result of Flow-Based Collector Selection. It may filter or replicate input Flow Records/Packet Reports, if necessary. An IPFIX Masquerading Proxy is an IPFIX Mediator that screens out a part of the data of input Flow Records/Packet Reports according to configured policies. It can thus, for example, hide the network topology information or customers' IP addresses. An Intermediate Process in IPFIX Mediators can be considered as a partial Metering Process taken from the Metering Process in Original Exporters as described in . The Intermediate Process generates new sets of Data Records/Packet Reports from input Data Records/Packet Reports. An IPFIX Mediator does not host the Observation Points and Observation Domain. The Observation Domain ID in the IPFIX header sent by the IPFIX Mediator also indicates the largest set of Observation Points from the viewpoint of a Collector. However, this value does not indicate the physical entity of an Original Exporter. The Transport Session is specified in . In SCTP, the Transport Session Information is the SCTP association. In TCP and UDP, the Transport Session Information corresponds to a 5-tuple {Exporter IP address, Collector IP address, Exporter transport port, Collector transport port, and transport protocol}.

The figure below shows the high-level reference model for IPFIX Mediation based on . This figure covers the various possible scenarios that can exist in an IPFIX measurement system.

Figure A: Reference Model for IPFIX Mediation.

The various functional components are indicated within brackets []. The functional components within [*] are not part of . The figure below shows the basic IPFIX Mediator component model. The IPFIX Mediator is formally defined to consist of one or more Collecting Processes, zero or more Intermediate Processes, and one or more Exporting Processes. Basically, IPFIX Mediator devices, i.e., IPFIX Proxy, IPFIX Masquerading Proxy, IPFIX Distributor, and IPFIX Concentrator, described in , are composed of these components.

Figure B: IPFIX Mediator Basic Component Model.

An Original Exporter with a Mediation function is modeled as follows.

Figure C: Component Model for Original Exporter with Mediation.

The section describes the details of each component and examples applicable to that component for IPFIX Mediation and IPFIX Mediator.

The Collecting Processes described in receive Flow Records/Packet Reports with information relating to their treatment in the Metering Process and Exporting Process in the Original Exporter, such as sampling rate, IPFIX header information, and Transport Session Information. The Collecting Processes forward the set of data to multiple components: Intermediate Processes and Exporting Processes. In other words, the processes may duplicate received Flow Records/Packet Reports and forward them to multiple components in sequence or in parallel.

The Exporting Processes described in forward Flow Records/Packet Reports to one or multiple Collectors. The processes manage the reporting Template and make IPFIX Messages.

Intermediate Processes generate new sets of Flow Records/Packet Reports from input Flow Records/Packet Reports with IPFIX header information "Export Time" and "Observation Domain ID". The processes host one of several functions defined below or a combination of them, in any sequence or in any set. In the case of a combination, the output of each function can be the input of other functions. The following subsections show the details of each function.

The Flow Selection function determines which input Flow Records/Packet Reports are selected by matching under a filtering policy and then forwards them to the next processes or functions. The function is similar to the Selection Process described in . The function covers several selection techniques, such as property match filtering and Flow selection, which are described in and , respectively. In property match filtering, if the value of a specified Information Element equals a configured value, the function selects Flow Records/Packet Reports to forward.

The Flow-based Collector Selection function determines to which Collector input Flow Records/Packet Reports are exported. The function may also determine the type of Transport Session. The function evaluates the value of a specified Information Element in input Flow Records/Packet Reports and then selects the Collector. These selection criteria are similar to the property match filtering in Mediator Selection Function. Applicable examples include exporting Flow Records/Packet Reports to a dedicated Collector on the basis of customers or organizations peering. The function classifies Flow Records/Packet Reports on the basis of a peering AS number, as shown in the following figure. The set of classified Flow Records/Packet Reports is exported to a dedicated Collector on the basis of the peering AS number.

Collector #1 | | | Peering AS #20 | | Flow --+---+--+-------------------+-+---> Collector #2 Records | | | Peering AS #30 | | | | +-------------------+-+---> Collector #3 | '----------------------' | '----------------------------' ]]> Figure D: Exporting classified Flow Records to dedicated Collector.

The Aggregation function creates aggregated Flow Records from input Flow Records/Packet Reports. The aggregation method is divided into three types: Choosing a shorter Flow Key than the Flow Key of input Flow Records, such as three, two, or a single Flow Key, can create more aggregated Flow Records. The function gathers Flow Records/Packet Reports within a given interval time and then distinguishes Flow Records/Packet Reports that have common properties. If values of a given key field are the same, that means those Flow Records/Packet Reports have common properties, and the function merges them in accordance with aggregation rules described in . In addition, the function can create statistical data and subsidiary information related to the aggregated Flow Records. Examples include the number of input Flow Records/Packet Reports, the given interval time, and a set of a new Flow Key. Time composition is defined as aggregation with the same Flow Key for long-running Flows. The function may also compute Flow Records statistics, such as average, maximum, and minimum value of each counter. The statistics help to visualize the behavior of traffic volume over a long time period. As another approach, the function collects Flow Records/Packet Reports of a shorter time period from an Original Exporter, and then computes these statistics. Even if output Flow Records of the function indicate a general time period, the accuracy of the minimum, maximum, and average value can be improved. Space composition is defined as aggregation on a larger Observation Domain or on a set of Observation Points. In that case, a Flow key can be applied to other properties, such as Exporter IP address and Observation Domain ID. In addition, a group identifier indicating a spatial Observation Domain can also become a new Flow Key. For example, a group can indicate an area on an ISP network, or a link aggregation interface composd of physical interfaces. The group can also make a relation to a set of values of specified Information Elements in Flow Records by the configuring rule. After converting from the values of specified Information Elements to the group identifier, the function can create aggregated Flow Records by a general aggregation process.

The Correlation function creates new metrics from by evaluating the correlation among sets of Flow Records/Packets Records. These sets can be Flow Records gathered during a certain period, a pair of consecutive Packet Reports, or Packet Reports exported by different Exporters indicating the same packet. After offering new metrics, the function outputs Flow Records with the new metrics field. Applicable examples are as follows. One way delay follows from correlating Packet Reports exported from different Exporters on the path. Packet interval time, or jitter, follows from correlating consecutive Packet Reports exported from the same Exporter. Difference values follow from correlating Flow Records observed at ingress or egress interfaces. The values help to confirm the result of a queueing or rate-limiting function. Average/maximum/minimum values follow from correlating each in a set of Flow Records.

The Modification function modifies input Flow Records/Packet Reports without changing their granularity. The function can add new Information Elements, delete existing Information Elements, or modify the value of specified Information Elements. If the function modifies the data structure of an original Template, it also needs to modify the value of the "flowKeyIndicator". The function obtains the value of a specified Information Element and then adds it into Flow Records/Packet Reports. There are several methods to obtain the value: retrieving the value from a database or calculating the value based on the value of other Information Elements and received traffic data. Applicable examples include adding derived packet property parameters instead of Original Exporters. Doing that can compensate for traditional Exporters or probes unable to add packet property parameters. Therefore, Collectors do not need to recognize the difference among implementations of routers from several vendors or among Exporter types, such as router, switch, or probe. Typical derived packet property parameters include the following. The "bgpNextHop{IPv4|IPv6}Address" described in indicates the egress router of a network domain. That is useful for making a traffic matrix that covers the whole network domain. The BGP Community value indicates the same group of destination or source IP addresses. The "mplsVpnRouteDistinguisher" described in , which cannot be extracted from the core router in MPLS networks, indicates the VPN customer's identification. Network operators can monitor the traffic behavior of each customer by adding "mplsVpnRouteDistinguisher" to Flow Records/Packet Reports. This function deletes existing Information Elements according to instruction rules, which indicate whether an Information Element should be removed. Applicable examples include hiding network topology information and private information. In the case of IPFIX exporting across domains, the function can avoid making a vulnerability by deleting unnecessary Information Elements. Examples of network topology information include "ipNextHopIP{v4|v6}Address", "bgpNextHopIP{v4|v6}Address", and "bgp{Next|Prev}AdjacentAsNumber", described in . In addition, MPLS-related Information Elements, such as "mplsLabelStackSection", are useless for customers in the case of feeding Flow Records/Packet Reports to VPN customers. This function modifies the value of specified Information Elements. Applicable examples include anonymizing customers' private information, such as IP address and port number, according to a privacy protection policy. Several annonymization techniques are described in . The function also reports anonymization methods and part of anonymized data as subsidiary information.

The IPFIX File Writer stores input Flow Records/Packet Reports from any process in a storage system. If received Flow Records/Packet Reports include uninteresting Information Elements, the Modification Function can delete these elements before the IPFIX File Writer handles them. Therefore, IPFIX File Writers can accept input from any process. In either case, input needs to include the IPFIX header information and the Transport Session Information along with Flow Records/Packet Reports. In contrast, the IPFIX File Reader retrieves stored Flow Records/Packet Reports when operators want to retrieve past Flow Records/Packet Reports on the basis of a given time period. If the data structure of output Flow Records/Packet Reports from the IPFIX File Reader is different from what operators want, the Modification function can modify the data structure. Therefore, the output of IPFIX File Readers can be input to any components. The IPFIX File Writer/Reader are described in in detail. The figure shows the IPFIX component model with IPFIX File Writer/Reader. IPFIX File Writer/Reader are located in the same position of Exporting Process/Collecting Process, respectively.

Figure E: IPFIX Mediator Component Model with IPFIX File Writer/Reader.

The Aggregation function needs expiration conditions to export cached Flow Records. These conditions are described in . In the case of IPFIX Mediation, these conditions are as follows: If there are no input/received Flow Records/Packet Reports belonging to a cached Flow for a certain time period, aggregated Flow Records will expire. This time period should be configurable at the Intermediate Process. If the IPFIX Mediator experiences resource constraints, aggregated Flow Records may prematurely expire (e.g., lack of memory to store Flow Records). For long-running Flows, the Intermediate Process should expire the Flow on a regular basis or based on some expiration policy. This periodicity or expiration policy should be configurable at the Intermediate Process. The Correlation function also needs similar expiration conditions. However, when cached Flow Records/Packet Reports prematurely expire and the function can not compute the correlation among them, cached Flow Records/Packet reports may be discarded.

IPFIX Mediation reuse the general information model from and from . The following new Information Elements for IPFIX Mediation are also needed.

As example, in case of Intermediate Processes having different functions, a Collecting Process/IPFIX File Reader replicates Flow Records/Packet Reports, if necessary, and forwards them to a suitable Intermediate Process/Exporting Process. Example figure is shown below.

Figure F: Functional Block Examples for IPFIX Mediator.

IPFIX Mediators use the IPFIX protocol. Security considerations about Flow Records are described in .

This document has no actions for IANA.

|Mediator|---. | | |*1 | | '--------' '--------' | .--------. .---------. '--->|IPFIX | |IPFIX | .--------. .--------. .--->|Mediator|---->|Collector| |IPFIX | |IPFIX | | |*2 | | | |router#2|---->|Mediator|---' '--------' '---------' | | |*1 | '--------' '--------' Figure A.A : Flexible Aggregation with Cascading IPFIX Mediators. ]]>

In case of global ISPs, the distances between PoPs become longer, and the maintenance of a dedicated management network is very expensive. Therefore, the huge number of Flow Records has burdened management networks. An IPFIX Mediator located in each PoP can minimize the number of Flow Records exported to the Collector by aggregating or filtering. If the Collector needs detailed information, it can retrieve Flow Records from IPFIX Mediators that store original Flow Records.

|IPFIX | |IPFIX | |------->|Mediator |----. |router |---'----->|#1 | | |#1 |-' '---------' | '--------' | | POP#North America | .--------. | .--------. | .---------. | .---------. .--------. | |----->|IPFIX | '---->|IPFIX | |IPFIX | |------->|Mediator |--------->|Collector| |router |---'----->|#2 | .---->| | |#4 |-' '---------' | '---------' '--------' | | POP#Europe | .--------. | .--------. | .---------. | .--------. | |----->|IPFIX | | |IPFIX | |------->|Mediator |----' |router |---'----->|#3 | |#7 |-' '---------' '--------' Figure A.B: Traffic Collection Architecture in Global Networks. ]]>

An IPFIX Mediator duplicates Flow Records to achieve redundant storage or utilizes them for several purposes.

Several departments in an ISP want to use the same traffic data for each intended purpose. The network design department measures the traffic matrix to obtain traffic demand. The customer service division uses traffic information for performing accounting services for each customer while network operators use traffic data for trouble shooting analysis. That situation is shown in the following figure. |Collector| | | | | |#1 | '--------' | | '---------' | | Using Accounting .--------. | .---------. | .---------. |IPFIX | '---->|IPFIX |----' |IPFIX | |router#2|--------->|Mediator |--------->|Collector| | | .---->| |----. |#2 | '--------' | '---------' | '---------' | | Using Trouble shooting. .--------. | | .---------. |IPFIX | | | |IPFIX | |router#3|----' '---->|Collector| | | |#3 | '--------' '---------' Figure A.C: Duplication of Flow Records. ]]>

An IPFIX Mediator distribute Flow Records based on the value of specified Information Elements. This function enables load balancing of Collector and sorting Flow Records without extra IPFIX Collector functions. In case of using accounting, IPFIX Mediator can distribute Flow Records to the dedicated IPFIX Collector of each customer.

When network operators disclose traffic information to each customer, security or the privacy policy should be considered. In that case, the IPFIX Mediator hides private information about each customer. In addition, Mediator distributes traffic data based on RD (Route Distinguisher), ingress IF, peering AS number, or BGP next hop, which identify the customer. In the following figure, the IPFIX Mediator distributes Flow Records based on RD. The system securely allows each customer to access only their own records. |Collector|<===> Customer#A | | | | |#1 | '--------' | | '---------' | RD=100:1 | .---------. | .--------. '---->|IPFIX |----' .---------. |IPFIX | |Mediator | RD=100:2 |IPFIX | |router#2|--------->| |--------->|Collector|<===> Customer#B | | | | |#2 | '--------' .---->| |----. '---------' | '---------' | | RD=100:3 .--------. | | .---------. |IPFIX | | | |IPFIX | |router#3|----' '---->|Collector|<===> Customer#C | | |#3 | '--------' '---------' Figure A.E: Distribution of Flow Records. ]]>

An IPFIX Mediator performs filtering based on the value of specified Information Elements. Filter conditions are set depending on suspicious Flows as follows. The IPFIX Collector receives the specified suspicious flow and detects an anomalous Flow by simply monitoring the traffic volume of each suspicious Flow. TCP Flow Records whose "tcpControlBits" value is set to "null" TCP Flow Records whose "tcpControlBits" value is set to the SYN bit only and the packet counter is only 1. ICMP Flow Records whose length is too long.

-->

The authors gratefully acknowledge the contributions of Keisuke Ishibashi, Tsuyoshi Kondoh, and Daisuke Matsubara.