Deep packet inspection (DPI) is a type of data processing that inspects in detail the data (packets) being sent over a computer network, and may take actions such as alerting, blocking, re-routing, or logging them accordingly. Deep packet inspection is often used for baselining application behavior, analyzing network usage, troubleshooting network performance, ensuring that data is in the correct format, checking for malicious code, eavesdropping, and Internet censorship,Duncan Geere, https://www.wired.co.uk/article/how-deep-packet-inspection-works among other purposes. There are multiple headers for IP packets; network equipment only needs to use the first of these (the IP header) for normal operation, but use of the second header (such as TCP or UDP) is normally considered to be shallow packet inspection (usually called stateful packet inspection) despite this definition.
There are multiple ways to acquire packets for deep packet inspection. Using port mirroring (sometimes called Span Port) is a very common way, as well as physically inserting a network tap which duplicates and sends the data stream to an analyzer tool for inspection.
Deep packet inspection (and filtering) enables advanced network management, user service, and security functions as well as Internet data mining, eavesdropping, and censorship. Although DPI has been used for Internet management for many years, some advocates of net neutrality fear that the technique may be used anticompetitively or to reduce the openness of the Internet.
DPI is used in a wide range of applications, at the “enterprise level” (according to corporations and larger institutions), telecommunications service providers, and governments.
Background DPI technology has a long and technologically advanced history, starting in the 1990s, before the technology entered what is seen today as common, mainstream deployments. The technology traces its roots back over 30 years, when many of the pioneers contributed their inventions for use among industry participants, such as through common standards and early innovation, such as the following:
RMON Sniffer Wireshark
Essential DPI functionality includes analysis of packet headers and protocol fields. For example, Wireshark offers essential DPI functionality through its numerous dissectors that display field names and content and, in some cases, offer interpretation of field values.
Some security solutions that offer DPI combine the functionality of an intrusion detection system (IDS) and an intrusion prevention system (IPS) with a traditional stateful firewall. This combination makes it possible to detect certain attacks that neither the IDS/IPS nor the stateful firewall can catch on their own. Stateful firewalls, while able to see the beginning and end of a packet flow, cannot catch events on their own that would be out of bounds for a particular application. While IDSs are able to detect intrusions, they have very little capability in blocking such an attack. DPIs are used to prevent attacks from viruses and worms at wire speeds. More specifically, DPI can be effective against buffer overflow attacks, denial-of-service attacks (DoS), sophisticated intrusions, and a small percentage of worms that fit within a single packet.
DPI-enabled devices have the ability to look at Layer 2 and beyond Layer 3 of the OSI model. In some cases, DPI can be invoked to look through Layer 2-7 of the OSI model. This includes headers and data protocol structures as well as the payload of the message. DPI functionality is invoked when a device looks or takes other action based on information beyond Layer 3 of the OSI model. DPI can identify and classify traffic based on a signature database that includes information extracted from the data part of a packet, allowing finer control than classification based only on header information. End points can utilize encryption and obfuscation techniques to evade DPI actions in many cases.
A classified packet may be redirected, marked/tagged (see quality of service), blocked, rate limited, and of course, reported to a reporting agent in the network. In this way, HTTP errors of different classifications may be identified and forwarded for analysis. Many DPI devices can identify packet flows (rather than packet-by-packet analysis), allowing control actions based on accumulated flow information.Moscola, James, et al. "Implementation of a content-scanning module for an internet firewall." Field-Programmable Custom Computing Machines, 2003. FCCM 2003. 11th Annual IEEE Symposium on. IEEE, 2003.
At the enterprise level Initially security at the enterprise level was just a perimeter discipline, with a dominant philosophy of keeping unauthorized users out, and shielding authorized users from the outside world. The most frequently used tool for accomplishing this has been a stateful firewall. It can permit fine-grained control of access from the outside world to pre-defined destinations on the internal network, as well as permitting access back to other hosts only if a request to the outside world has been made previously.
Vulnerabilities exist at network layers, however, that are not visible to a stateful firewall. Also, an increase in the use of laptops in enterprise makes it more difficult to prevent threats such as viruses, worms, and spyware from penetrating the corporate network, as many users will connect the laptop to less-secure networks such as home broadband connections or wireless networks in public locations. Firewalls also do not distinguish between permitted and forbidden uses of legitimately-accessed applications. DPI enables IT administrators and security officials to set policies and enforce them at all layers, including the application and user layer to help combat those threats.
Deep packet inspection is able to detect a few kinds of buffer overflow attacks.
DPI may be used by enterprise for Data Leak Prevention (DLP). When an e-mail user tries to send a protected file, the user may be given information on how to get the proper clearance to send the file.
At network/Internet service providers In addition to using DPI to secure their internal networks, Internet service providers (ISPs) also apply it on the public networks provided to customers. Common uses of DPI by ISPs are lawful intercept, policy definition and enforcement, targeted advertising, quality of service, offering tiered services, and copyright enforcement.
Lawful interception Service providers are required by almost all governments worldwide to enable lawful intercept capabilities. Decades ago in a legacy telephone environment, this was met by creating a traffic access point (TAP) using an intercepting proxy server that connects to the government's surveillance equipment. The acquisition component of this functionality may be provided in many ways, including DPI, DPI-enabled products that are "LI or CALEA-compliant" can be used – when directed by a court order – to access a user's datastream.
Policy definition and enforcement Service providers obligated by the service-level agreement with their customers to provide a certain level of service and at the same time, enforce an acceptable use policy, may make use of DPI to implement certain policies that cover copyright infringements, illegal materials, and unfair use of bandwidth. In some countries the ISPs are required to perform filtering, depending on the country's laws. DPI allows service providers to "readily know the packets of information you are receiving online—from e-mail, to websites, to sharing of music, video and software downloads".
Targeted advertising Because ISPs route the traffic of all of their customers, they are able to monitor web-browsing habits in a very detailed way allowing them to gain information about their customers' interests, which can be used by companies specializing in targeted advertising. At least 100,000 United States customers are tracked this way, and as many as 10% of U.S. customers have been tracked in this way. Technology providers include NebuAd, Front Porch, and Phorm. U.S. ISPs monitoring their customers include Knology and Wide Open West. In addition, the United Kingdom ISP British Telecom has admitted testing solutions from Phorm without their customers' knowledge or consent.
Quality of service DPI can be used against net neutrality.
Applications such as peer-to-peer (P2P) traffic present increasing problems for broadband service providers. Typically, P2P traffic is used by applications that do file sharing. These may be any kind of files (i.e. documents, music, videos, or applications). Due to the frequently large size of media files being transferred, P2P drives increasing traffic loads, requiring additional network capacity. Service providers say a minority of users generate large quantities of P2P traffic and degrade performance for the majority of broadband subscribers using applications such as e-mail or Web browsing which use less bandwidth. Poor network performance increases customer dissatisfaction and leads to a decline in service revenues.
DPI allows the operators to oversell their available bandwidth while ensuring equitable bandwidth distribution to all users by preventing network congestion. Additionally, a higher priority can be allocated to a VoIP or video conferencing call which requires low latency versus web browsing which does not. This is the approach that service providers use to dynamically allocate bandwidth according to traffic that is passing through their networks.
Tiered services Mobile and broadband service providers use DPI as a means to implement tiered service plans, to differentiate "walled garden" services from "value added", "all-you-can-eat" and "one-size-fits-all" data services.
Copyright enforcement ISPs are sometimes requested by copyright owners or required by courts or official policy to help enforce copyrights. In 2006, one of Denmark's largest ISPs, Tele2, was given a court injunction and told it must block its customers from accessing The Pirate Bay, a launching point for BitTorrent.
Instead of prosecuting file sharers one at a time, the International Federation of the Phonographic Industry (IFPI) and the big four record labels EMI, Sony BMG, Universal Music, and Warner Music have sued ISPs such as Eircom for not doing enough about protecting their copyrights. The IFPI wants ISPs to filter traffic to remove illicitly uploaded and downloaded copyrighted material from their network, despite European directive 2000/31/EC clearly stating that ISPs may not be put under a general obligation to monitor the information they transmit, and directive 2002/58/EC granting European citizens a right to privacy of communications.
The Motion Picture Association of America (MPAA), which aims to enforce movie copyrights, has taken the position with the Federal Communications Commission (FCC) that network neutrality could hurt anti-piracy techniques such as deep packet inspection and other forms of filtering.
By governments
In addition to using DPI for the security of their own networks, governments in North America, Europe, and Asia use DPI for various purposes such as surveillance and censorship. Many of these programs are classified.
China