We start by presenting some basic concepts to facilitate the understanding of the mechanisms that are presented further on.
Packets
A data sender usually splits data to be sent into small units known as packets. Each packet consists of a header and a payload carrying the data to be delivered. The header contains fields defined by the relevant communication protocol. The great majority of packets carried by commercial networks nowadays are so-called IP packets, where “IP” is the Internet Protocol. This ensures that a network of routers can forward any packet from the source to its destination. IP is a connectionless protocol—that means that the header information in each data packet is sufficiently self-contained for routers to deliver it independently of other packets; each packet could even take a different route to reach the destination.
Packet Headers
The Internet Protocol has evolved through a number of versions although only two are currently used: IPv4 and IPv6, the latter currently still having relatively low uptake. The layout and names of the fields of the IPv4 header are shown in FIG. 1, which has been based principally on Internet Engineering Task Force (IETF) “Request for Comments” (RFC) document IETF RFC 791 but also reflects more recent updates (IETF RFCs 2474, 3260 and 3168). IETF RFC 791, entitled “Internet Protocol” was published in September 1981, and is available on the internet at the http world wide website ietf.org/rfc/rfc791.txt. The other IETF RFC documents are also available on the internet via the http world wide website ietf.org/rfc.html or elsewhere.
Referring to FIG. 1, the numbers along the top, above the six rows of field-names, index each bit of the header in rows of 32 bits, numbered “00” to “31”. Three fields which will later be referred to in particular in relation to preferred embodiments of this invention are all on the second row, and relate to packet fragmentation, which will be discussed below. These fields are the ‘Identification’, ‘Flags’ and ‘Fragment Offset’ fields.
FIG. 2 shows the layout and names of the fields of the header of an IPv6 packet. An obvious difference is that IPv6 has larger fields for the source and destination addresses. Also it will be noted that no fields to do with fragmentation are present in the base IPv6 header.
Protocol Evolution
From time to time, a new requirement has to be met, and can be achieved by updating part of a communications protocol, rather than completely replacing it. Most protocols are defined so that some values of various fields are not used, but are reserved for future use. The idea is that new features can be added by bringing values into use from these “reserved” ranges.
When reserved values are brought into use for a new variant of a protocol, there are often problems. Existing equipment built before the protocol was updated is usually meant to take some default action if it encounters a packet carrying one of these reserved values. However, when a reserved value is first used, it is generally discovered that some legacy equipment does not take the specified default action, possibly for one of two reasons:
Bugs:
Many manufacturers cut corners and do not test whether their equipment acts correctly when it encounters packets carrying values not in common usage at the time of implementation;
Precautionary Security:
Many manufacturers deliberately do not set their equipment to implement the standard default action when it encounters a packet carrying a reserved value. Instead, on the basis that the appearance of a previously unused value might be a symptom of some unknown security attack, they set their equipment to discard the packet as a precaution.
Therefore, the first adopters of a new protocol typically discover that arbitrary equipment seems to crash inexplicably, or packets carrying new values never reach their destination because some item of equipment wrongly discards such packets (termed being black-holed). Such equipment will be referred to as a “non-compliant middlebox”.
The outcome is often that new features often fail to be deployed, or their introduction is delayed, because no company wants to bear the cost of all the support calls that will result when their (possibly) perfectly-implemented update is taken to be the cause of numerous problems that it triggers—problems that are actually due to other incorrectly implemented equipment that has already been deployed.
Fragmentation
The three fields referred to earlier as being of particular relevance to packet fragmentation are the ‘Identification’, ‘Flags’ and ‘Fragment Offset’ fields. The IPv4 ‘Flags’ field consists of three bits (shown in FIG. 3), only the second and third of which are currently used, the first being reserved (RES) for future use. The other two are termed ‘Don't Fragment’ (DF) and ‘More Fragments’ (MF). Further information about this is available in IETF RFC 791, referred to earlier.
The sender of every IPv4 packet must label each packet using the 16-bit ‘Identification’ field. A source should cycle through the whole of the possible number space that could be written in the ‘Identification’ field before repeating a previously used identifier. There is no requirement to cycle through the numbers in any particular order, as long as all of the 216 available numbers are used before any are repeated.
Each link in a network is characterised by a maximum size of packet it can transmit—referred to as the link's “maximum transmission unit” (MTU). If an IPv4 router needs to forward an arriving packet down a link that has an MTU smaller than the packet, the router may break the packet down into fragments. Equivalently, a source may break a packet down into fragments before sending it.
The process of fragmentation involves breaking the original payload down into sufficiently small fragments that they will each fit into the link, and placing a copy of the original header on each fragment. Each header is then modified so that, once a set of fragments arrives at its destination, they contain sufficient information to allow reassembly of the original packet:                Because all the fragment headers start as copies of the original packet, they contain the same value in the ‘Identification’ field as the original packet before fragmentation.        The ‘Fragment Offset’ field of each fragment is set to index where in the original payload this fragment was taken from, as a number of octets (bytes) from the start.        The ‘More Fragments’ (MF) flag is set to 1 on all the fragments of a packet except the last, in which it is cleared to zero.        
Thus, each fragment contains sufficient information for the ultimate destination to reassemble them into the original packet.
Note that the sender of an un-fragmented IPv4 packet should initially set the ‘More Fragments’ flag to zero and the ‘Fragment Offset’ to zero.
If the sender (or a router) sets the ‘Don't Fragment’ (DF) flag, then the packet should not be fragmented when it arrives at a subsequent router even if it is too large to fit within the MTU of a link. Instead, the router should discard the packet, and optionally return a control packet to the source containing an appropriate error message.
Modern Approach to Fragmentation
When IPv6 was defined, it was decided that routers would no longer do fragmentation. This allowed routers to be simplified. Instead, it was decided that the sender would become responsible for making sure its packets were small enough to fit through the link with the smallest MTU on the path (the path MTU or PMTU). The sender can do fragmentation in IPv6, but that is not relevant to this discussion.
The IETF is in the process of recommending that IPv4 sources adopt a similar approach to IPv6 in this regard. To achieve this, it is being recommended that all IPv4 sources set the ‘Don't Fragment’ (DF) flag and clear the ‘More Fragments’ (MF) flag and the ‘Fragment Offset’ field (see: “Updated Specification of the IPv4 ID Field” (Joe Touch, 22 Oct. 2010), IETF Internet Draft <draft-ietf-intarea-ipv4-id-update>, available on the internet at ha website tools.ietf.org/html/draft-ietf-intarea-ipv4-id-update-01). Most IPv4 sources already disable fragmentation in this way, so the IETF exercise largely documents what is essentially current best practice.