0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 ===========
0004 Packet MMAP
0005 ===========
0006
0007 Abstract
0008 ========
0009
0010 This file documents the mmap() facility available with the PACKET
0011 socket interface. This type of sockets is used for
0012
0013 i) capture network traffic with utilities like tcpdump,
0014 ii) transmit network traffic, or any other that needs raw
0015 access to network interface.
0016
0017 Howto can be found at:
0018
0019 https://sites.google.com/site/packetmmap/
0020
0021 Please send your comments to
0022 - Ulisses Alonso Camaró <uaca@i.hate.spam.alumni.uv.es>
0023 - Johann Baudy
0024
0025 Why use PACKET_MMAP
0026 ===================
0027
0028 Non PACKET_MMAP capture process (plain AF_PACKET) is very
0029 inefficient. It uses very limited buffers and requires one system call to
0030 capture each packet, it requires two if you want to get packet's timestamp
0031 (like libpcap always does).
0032
0033 On the other hand PACKET_MMAP is very efficient. PACKET_MMAP provides a size
0034 configurable circular buffer mapped in user space that can be used to either
0035 send or receive packets. This way reading packets just needs to wait for them,
0036 most of the time there is no need to issue a single system call. Concerning
0037 transmission, multiple packets can be sent through one system call to get the
0038 highest bandwidth. By using a shared buffer between the kernel and the user
0039 also has the benefit of minimizing packet copies.
0040
0041 It's fine to use PACKET_MMAP to improve the performance of the capture and
0042 transmission process, but it isn't everything. At least, if you are capturing
0043 at high speeds (this is relative to the cpu speed), you should check if the
0044 device driver of your network interface card supports some sort of interrupt
0045 load mitigation or (even better) if it supports NAPI, also make sure it is
0046 enabled. For transmission, check the MTU (Maximum Transmission Unit) used and
0047 supported by devices of your network. CPU IRQ pinning of your network interface
0048 card can also be an advantage.
0049
0050 How to use mmap() to improve capture process
0051 ============================================
0052
0053 From the user standpoint, you should use the higher level libpcap library, which
0054 is a de facto standard, portable across nearly all operating systems
0055 including Win32.
0056
0057 Packet MMAP support was integrated into libpcap around the time of version 1.3.0;
0058 TPACKET_V3 support was added in version 1.5.0
0059
0060 How to use mmap() directly to improve capture process
0061 =====================================================
0062
0063 From the system calls stand point, the use of PACKET_MMAP involves
0064 the following process::
0065
0066
0067 [setup] socket() -------> creation of the capture socket
0068 setsockopt() ---> allocation of the circular buffer (ring)
0069 option: PACKET_RX_RING
0070 mmap() ---------> mapping of the allocated buffer to the
0071 user process
0072
0073 [capture] poll() ---------> to wait for incoming packets
0074
0075 [shutdown] close() --------> destruction of the capture socket and
0076 deallocation of all associated
0077 resources.
0078
0079
0080 socket creation and destruction is straight forward, and is done
0081 the same way with or without PACKET_MMAP::
0082
0083 int fd = socket(PF_PACKET, mode, htons(ETH_P_ALL));
0084
0085 where mode is SOCK_RAW for the raw interface were link level
0086 information can be captured or SOCK_DGRAM for the cooked
0087 interface where link level information capture is not
0088 supported and a link level pseudo-header is provided
0089 by the kernel.
0090
0091 The destruction of the socket and all associated resources
0092 is done by a simple call to close(fd).
0093
0094 Similarly as without PACKET_MMAP, it is possible to use one socket
0095 for capture and transmission. This can be done by mapping the
0096 allocated RX and TX buffer ring with a single mmap() call.
0097 See "Mapping and use of the circular buffer (ring)".
0098
0099 Next I will describe PACKET_MMAP settings and its constraints,
0100 also the mapping of the circular buffer in the user process and
0101 the use of this buffer.
0102
0103 How to use mmap() directly to improve transmission process
0104 ==========================================================
0105 Transmission process is similar to capture as shown below::
0106
0107 [setup] socket() -------> creation of the transmission socket
0108 setsockopt() ---> allocation of the circular buffer (ring)
0109 option: PACKET_TX_RING
0110 bind() ---------> bind transmission socket with a network interface
0111 mmap() ---------> mapping of the allocated buffer to the
0112 user process
0113
0114 [transmission] poll() ---------> wait for free packets (optional)
0115 send() ---------> send all packets that are set as ready in
0116 the ring
0117 The flag MSG_DONTWAIT can be used to return
0118 before end of transfer.
0119
0120 [shutdown] close() --------> destruction of the transmission socket and
0121 deallocation of all associated resources.
0122
0123 Socket creation and destruction is also straight forward, and is done
0124 the same way as in capturing described in the previous paragraph::
0125
0126 int fd = socket(PF_PACKET, mode, 0);
0127
0128 The protocol can optionally be 0 in case we only want to transmit
0129 via this socket, which avoids an expensive call to packet_rcv().
0130 In this case, you also need to bind(2) the TX_RING with sll_protocol = 0
0131 set. Otherwise, htons(ETH_P_ALL) or any other protocol, for example.
0132
0133 Binding the socket to your network interface is mandatory (with zero copy) to
0134 know the header size of frames used in the circular buffer.
0135
0136 As capture, each frame contains two parts::
0137
0138 --------------------
0139 | struct tpacket_hdr | Header. It contains the status of
0140 | | of this frame
0141 |--------------------|
0142 | data buffer |
0143 . . Data that will be sent over the network interface.
0144 . .
0145 --------------------
0146
0147 bind() associates the socket to your network interface thanks to
0148 sll_ifindex parameter of struct sockaddr_ll.
0149
0150 Initialization example::
0151
0152 struct sockaddr_ll my_addr;
0153 struct ifreq s_ifr;
0154 ...
0155
0156 strscpy_pad (s_ifr.ifr_name, "eth0", sizeof(s_ifr.ifr_name));
0157
0158 /* get interface index of eth0 */
0159 ioctl(this->socket, SIOCGIFINDEX, &s_ifr);
0160
0161 /* fill sockaddr_ll struct to prepare binding */
0162 my_addr.sll_family = AF_PACKET;
0163 my_addr.sll_protocol = htons(ETH_P_ALL);
0164 my_addr.sll_ifindex = s_ifr.ifr_ifindex;
0165
0166 /* bind socket to eth0 */
0167 bind(this->socket, (struct sockaddr *)&my_addr, sizeof(struct sockaddr_ll));
0168
0169 A complete tutorial is available at: https://sites.google.com/site/packetmmap/
0170
0171 By default, the user should put data at::
0172
0173 frame base + TPACKET_HDRLEN - sizeof(struct sockaddr_ll)
0174
0175 So, whatever you choose for the socket mode (SOCK_DGRAM or SOCK_RAW),
0176 the beginning of the user data will be at::
0177
0178 frame base + TPACKET_ALIGN(sizeof(struct tpacket_hdr))
0179
0180 If you wish to put user data at a custom offset from the beginning of
0181 the frame (for payload alignment with SOCK_RAW mode for instance) you
0182 can set tp_net (with SOCK_DGRAM) or tp_mac (with SOCK_RAW). In order
0183 to make this work it must be enabled previously with setsockopt()
0184 and the PACKET_TX_HAS_OFF option.
0185
0186 PACKET_MMAP settings
0187 ====================
0188
0189 To setup PACKET_MMAP from user level code is done with a call like
0190
0191 - Capture process::
0192
0193 setsockopt(fd, SOL_PACKET, PACKET_RX_RING, (void *) &req, sizeof(req))
0194
0195 - Transmission process::
0196
0197 setsockopt(fd, SOL_PACKET, PACKET_TX_RING, (void *) &req, sizeof(req))
0198
0199 The most significant argument in the previous call is the req parameter,
0200 this parameter must to have the following structure::
0201
0202 struct tpacket_req
0203 {
0204 unsigned int tp_block_size; /* Minimal size of contiguous block */
0205 unsigned int tp_block_nr; /* Number of blocks */
0206 unsigned int tp_frame_size; /* Size of frame */
0207 unsigned int tp_frame_nr; /* Total number of frames */
0208 };
0209
0210 This structure is defined in /usr/include/linux/if_packet.h and establishes a
0211 circular buffer (ring) of unswappable memory.
0212 Being mapped in the capture process allows reading the captured frames and
0213 related meta-information like timestamps without requiring a system call.
0214
0215 Frames are grouped in blocks. Each block is a physically contiguous
0216 region of memory and holds tp_block_size/tp_frame_size frames. The total number
0217 of blocks is tp_block_nr. Note that tp_frame_nr is a redundant parameter because::
0218
0219 frames_per_block = tp_block_size/tp_frame_size
0220
0221 indeed, packet_set_ring checks that the following condition is true::
0222
0223 frames_per_block * tp_block_nr == tp_frame_nr
0224
0225 Lets see an example, with the following values::
0226
0227 tp_block_size= 4096
0228 tp_frame_size= 2048
0229 tp_block_nr = 4
0230 tp_frame_nr = 8
0231
0232 we will get the following buffer structure::
0233
0234 block #1 block #2
0235 +---------+---------+ +---------+---------+
0236 | frame 1 | frame 2 | | frame 3 | frame 4 |
0237 +---------+---------+ +---------+---------+
0238
0239 block #3 block #4
0240 +---------+---------+ +---------+---------+
0241 | frame 5 | frame 6 | | frame 7 | frame 8 |
0242 +---------+---------+ +---------+---------+
0243
0244 A frame can be of any size with the only condition it can fit in a block. A block
0245 can only hold an integer number of frames, or in other words, a frame cannot
0246 be spawned across two blocks, so there are some details you have to take into
0247 account when choosing the frame_size. See "Mapping and use of the circular
0248 buffer (ring)".
0249
0250 PACKET_MMAP setting constraints
0251 ===============================
0252
0253 In kernel versions prior to 2.4.26 (for the 2.4 branch) and 2.6.5 (2.6 branch),
0254 the PACKET_MMAP buffer could hold only 32768 frames in a 32 bit architecture or
0255 16384 in a 64 bit architecture.
0256
0257 Block size limit
0258 ----------------
0259
0260 As stated earlier, each block is a contiguous physical region of memory. These
0261 memory regions are allocated with calls to the __get_free_pages() function. As
0262 the name indicates, this function allocates pages of memory, and the second
0263 argument is "order" or a power of two number of pages, that is
0264 (for PAGE_SIZE == 4096) order=0 ==> 4096 bytes, order=1 ==> 8192 bytes,
0265 order=2 ==> 16384 bytes, etc. The maximum size of a
0266 region allocated by __get_free_pages is determined by the MAX_ORDER macro. More
0267 precisely the limit can be calculated as::
0268
0269 PAGE_SIZE << MAX_ORDER
0270
0271 In a i386 architecture PAGE_SIZE is 4096 bytes
0272 In a 2.4/i386 kernel MAX_ORDER is 10
0273 In a 2.6/i386 kernel MAX_ORDER is 11
0274
0275 So get_free_pages can allocate as much as 4MB or 8MB in a 2.4/2.6 kernel
0276 respectively, with an i386 architecture.
0277
0278 User space programs can include /usr/include/sys/user.h and
0279 /usr/include/linux/mmzone.h to get PAGE_SIZE MAX_ORDER declarations.
0280
0281 The pagesize can also be determined dynamically with the getpagesize (2)
0282 system call.
0283
0284 Block number limit
0285 ------------------
0286
0287 To understand the constraints of PACKET_MMAP, we have to see the structure
0288 used to hold the pointers to each block.
0289
0290 Currently, this structure is a dynamically allocated vector with kmalloc
0291 called pg_vec, its size limits the number of blocks that can be allocated::
0292
0293 +---+---+---+---+
0294 | x | x | x | x |
0295 +---+---+---+---+
0296 | | | |
0297 | | | v
0298 | | v block #4
0299 | v block #3
0300 v block #2
0301 block #1
0302
0303 kmalloc allocates any number of bytes of physically contiguous memory from
0304 a pool of pre-determined sizes. This pool of memory is maintained by the slab
0305 allocator which is at the end the responsible for doing the allocation and
0306 hence which imposes the maximum memory that kmalloc can allocate.
0307
0308 In a 2.4/2.6 kernel and the i386 architecture, the limit is 131072 bytes. The
0309 predetermined sizes that kmalloc uses can be checked in the "size-<bytes>"
0310 entries of /proc/slabinfo
0311
0312 In a 32 bit architecture, pointers are 4 bytes long, so the total number of
0313 pointers to blocks is::
0314
0315 131072/4 = 32768 blocks
0316
0317 PACKET_MMAP buffer size calculator
0318 ==================================
0319
0320 Definitions:
0321
0322 ============== ================================================================
0323 <size-max> is the maximum size of allocable with kmalloc
0324 (see /proc/slabinfo)
0325 <pointer size> depends on the architecture -- ``sizeof(void *)``
0326 <page size> depends on the architecture -- PAGE_SIZE or getpagesize (2)
0327 <max-order> is the value defined with MAX_ORDER
0328 <frame size> it's an upper bound of frame's capture size (more on this later)
0329 ============== ================================================================
0330
0331 from these definitions we will derive::
0332
0333 <block number> = <size-max>/<pointer size>
0334 <block size> = <pagesize> << <max-order>
0335
0336 so, the max buffer size is::
0337
0338 <block number> * <block size>
0339
0340 and, the number of frames be::
0341
0342 <block number> * <block size> / <frame size>
0343
0344 Suppose the following parameters, which apply for 2.6 kernel and an
0345 i386 architecture::
0346
0347 <size-max> = 131072 bytes
0348 <pointer size> = 4 bytes
0349 <pagesize> = 4096 bytes
0350 <max-order> = 11
0351
0352 and a value for <frame size> of 2048 bytes. These parameters will yield::
0353
0354 <block number> = 131072/4 = 32768 blocks
0355 <block size> = 4096 << 11 = 8 MiB.
0356
0357 and hence the buffer will have a 262144 MiB size. So it can hold
0358 262144 MiB / 2048 bytes = 134217728 frames
0359
0360 Actually, this buffer size is not possible with an i386 architecture.
0361 Remember that the memory is allocated in kernel space, in the case of
0362 an i386 kernel's memory size is limited to 1GiB.
0363
0364 All memory allocations are not freed until the socket is closed. The memory
0365 allocations are done with GFP_KERNEL priority, this basically means that
0366 the allocation can wait and swap other process' memory in order to allocate
0367 the necessary memory, so normally limits can be reached.
0368
0369 Other constraints
0370 -----------------
0371
0372 If you check the source code you will see that what I draw here as a frame
0373 is not only the link level frame. At the beginning of each frame there is a
0374 header called struct tpacket_hdr used in PACKET_MMAP to hold link level's frame
0375 meta information like timestamp. So what we draw here a frame it's really
0376 the following (from include/linux/if_packet.h)::
0377
0378 /*
0379 Frame structure:
0380
0381 - Start. Frame must be aligned to TPACKET_ALIGNMENT=16
0382 - struct tpacket_hdr
0383 - pad to TPACKET_ALIGNMENT=16
0384 - struct sockaddr_ll
0385 - Gap, chosen so that packet data (Start+tp_net) aligns to
0386 TPACKET_ALIGNMENT=16
0387 - Start+tp_mac: [ Optional MAC header ]
0388 - Start+tp_net: Packet data, aligned to TPACKET_ALIGNMENT=16.
0389 - Pad to align to TPACKET_ALIGNMENT=16
0390 */
0391
0392 The following are conditions that are checked in packet_set_ring
0393
0394 - tp_block_size must be a multiple of PAGE_SIZE (1)
0395 - tp_frame_size must be greater than TPACKET_HDRLEN (obvious)
0396 - tp_frame_size must be a multiple of TPACKET_ALIGNMENT
0397 - tp_frame_nr must be exactly frames_per_block*tp_block_nr
0398
0399 Note that tp_block_size should be chosen to be a power of two or there will
0400 be a waste of memory.
0401
0402 Mapping and use of the circular buffer (ring)
0403 ---------------------------------------------
0404
0405 The mapping of the buffer in the user process is done with the conventional
0406 mmap function. Even the circular buffer is compound of several physically
0407 discontiguous blocks of memory, they are contiguous to the user space, hence
0408 just one call to mmap is needed::
0409
0410 mmap(0, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
0411
0412 If tp_frame_size is a divisor of tp_block_size frames will be
0413 contiguously spaced by tp_frame_size bytes. If not, each
0414 tp_block_size/tp_frame_size frames there will be a gap between
0415 the frames. This is because a frame cannot be spawn across two
0416 blocks.
0417
0418 To use one socket for capture and transmission, the mapping of both the
0419 RX and TX buffer ring has to be done with one call to mmap::
0420
0421 ...
0422 setsockopt(fd, SOL_PACKET, PACKET_RX_RING, &foo, sizeof(foo));
0423 setsockopt(fd, SOL_PACKET, PACKET_TX_RING, &bar, sizeof(bar));
0424 ...
0425 rx_ring = mmap(0, size * 2, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
0426 tx_ring = rx_ring + size;
0427
0428 RX must be the first as the kernel maps the TX ring memory right
0429 after the RX one.
0430
0431 At the beginning of each frame there is an status field (see
0432 struct tpacket_hdr). If this field is 0 means that the frame is ready
0433 to be used for the kernel, If not, there is a frame the user can read
0434 and the following flags apply:
0435
0436 Capture process
0437 ^^^^^^^^^^^^^^^
0438
0439 From include/linux/if_packet.h::
0440
0441 #define TP_STATUS_COPY (1 << 1)
0442 #define TP_STATUS_LOSING (1 << 2)
0443 #define TP_STATUS_CSUMNOTREADY (1 << 3)
0444 #define TP_STATUS_CSUM_VALID (1 << 7)
0445
0446 ====================== =======================================================
0447 TP_STATUS_COPY This flag indicates that the frame (and associated
0448 meta information) has been truncated because it's
0449 larger than tp_frame_size. This packet can be
0450 read entirely with recvfrom().
0451
0452 In order to make this work it must to be
0453 enabled previously with setsockopt() and
0454 the PACKET_COPY_THRESH option.
0455
0456 The number of frames that can be buffered to
0457 be read with recvfrom is limited like a normal socket.
0458 See the SO_RCVBUF option in the socket (7) man page.
0459
0460 TP_STATUS_LOSING indicates there were packet drops from last time
0461 statistics where checked with getsockopt() and
0462 the PACKET_STATISTICS option.
0463
0464 TP_STATUS_CSUMNOTREADY currently it's used for outgoing IP packets which
0465 its checksum will be done in hardware. So while
0466 reading the packet we should not try to check the
0467 checksum.
0468
0469 TP_STATUS_CSUM_VALID This flag indicates that at least the transport
0470 header checksum of the packet has been already
0471 validated on the kernel side. If the flag is not set
0472 then we are free to check the checksum by ourselves
0473 provided that TP_STATUS_CSUMNOTREADY is also not set.
0474 ====================== =======================================================
0475
0476 for convenience there are also the following defines::
0477
0478 #define TP_STATUS_KERNEL 0
0479 #define TP_STATUS_USER 1
0480
0481 The kernel initializes all frames to TP_STATUS_KERNEL, when the kernel
0482 receives a packet it puts in the buffer and updates the status with
0483 at least the TP_STATUS_USER flag. Then the user can read the packet,
0484 once the packet is read the user must zero the status field, so the kernel
0485 can use again that frame buffer.
0486
0487 The user can use poll (any other variant should apply too) to check if new
0488 packets are in the ring::
0489
0490 struct pollfd pfd;
0491
0492 pfd.fd = fd;
0493 pfd.revents = 0;
0494 pfd.events = POLLIN|POLLRDNORM|POLLERR;
0495
0496 if (status == TP_STATUS_KERNEL)
0497 retval = poll(&pfd, 1, timeout);
0498
0499 It doesn't incur in a race condition to first check the status value and
0500 then poll for frames.
0501
0502 Transmission process
0503 ^^^^^^^^^^^^^^^^^^^^
0504
0505 Those defines are also used for transmission::
0506
0507 #define TP_STATUS_AVAILABLE 0 // Frame is available
0508 #define TP_STATUS_SEND_REQUEST 1 // Frame will be sent on next send()
0509 #define TP_STATUS_SENDING 2 // Frame is currently in transmission
0510 #define TP_STATUS_WRONG_FORMAT 4 // Frame format is not correct
0511
0512 First, the kernel initializes all frames to TP_STATUS_AVAILABLE. To send a
0513 packet, the user fills a data buffer of an available frame, sets tp_len to
0514 current data buffer size and sets its status field to TP_STATUS_SEND_REQUEST.
0515 This can be done on multiple frames. Once the user is ready to transmit, it
0516 calls send(). Then all buffers with status equal to TP_STATUS_SEND_REQUEST are
0517 forwarded to the network device. The kernel updates each status of sent
0518 frames with TP_STATUS_SENDING until the end of transfer.
0519
0520 At the end of each transfer, buffer status returns to TP_STATUS_AVAILABLE.
0521
0522 ::
0523
0524 header->tp_len = in_i_size;
0525 header->tp_status = TP_STATUS_SEND_REQUEST;
0526 retval = send(this->socket, NULL, 0, 0);
0527
0528 The user can also use poll() to check if a buffer is available:
0529
0530 (status == TP_STATUS_SENDING)
0531
0532 ::
0533
0534 struct pollfd pfd;
0535 pfd.fd = fd;
0536 pfd.revents = 0;
0537 pfd.events = POLLOUT;
0538 retval = poll(&pfd, 1, timeout);
0539
0540 What TPACKET versions are available and when to use them?
0541 =========================================================
0542
0543 ::
0544
0545 int val = tpacket_version;
0546 setsockopt(fd, SOL_PACKET, PACKET_VERSION, &val, sizeof(val));
0547 getsockopt(fd, SOL_PACKET, PACKET_VERSION, &val, sizeof(val));
0548
0549 where 'tpacket_version' can be TPACKET_V1 (default), TPACKET_V2, TPACKET_V3.
0550
0551 TPACKET_V1:
0552 - Default if not otherwise specified by setsockopt(2)
0553 - RX_RING, TX_RING available
0554
0555 TPACKET_V1 --> TPACKET_V2:
0556 - Made 64 bit clean due to unsigned long usage in TPACKET_V1
0557 structures, thus this also works on 64 bit kernel with 32 bit
0558 userspace and the like
0559 - Timestamp resolution in nanoseconds instead of microseconds
0560 - RX_RING, TX_RING available
0561 - VLAN metadata information available for packets
0562 (TP_STATUS_VLAN_VALID, TP_STATUS_VLAN_TPID_VALID),
0563 in the tpacket2_hdr structure:
0564
0565 - TP_STATUS_VLAN_VALID bit being set into the tp_status field indicates
0566 that the tp_vlan_tci field has valid VLAN TCI value
0567 - TP_STATUS_VLAN_TPID_VALID bit being set into the tp_status field
0568 indicates that the tp_vlan_tpid field has valid VLAN TPID value
0569
0570 - How to switch to TPACKET_V2:
0571
0572 1. Replace struct tpacket_hdr by struct tpacket2_hdr
0573 2. Query header len and save
0574 3. Set protocol version to 2, set up ring as usual
0575 4. For getting the sockaddr_ll,
0576 use ``(void *)hdr + TPACKET_ALIGN(hdrlen)`` instead of
0577 ``(void *)hdr + TPACKET_ALIGN(sizeof(struct tpacket_hdr))``
0578
0579 TPACKET_V2 --> TPACKET_V3:
0580 - Flexible buffer implementation for RX_RING:
0581 1. Blocks can be configured with non-static frame-size
0582 2. Read/poll is at a block-level (as opposed to packet-level)
0583 3. Added poll timeout to avoid indefinite user-space wait
0584 on idle links
0585 4. Added user-configurable knobs:
0586
0587 4.1 block::timeout
0588 4.2 tpkt_hdr::sk_rxhash
0589
0590 - RX Hash data available in user space
0591 - TX_RING semantics are conceptually similar to TPACKET_V2;
0592 use tpacket3_hdr instead of tpacket2_hdr, and TPACKET3_HDRLEN
0593 instead of TPACKET2_HDRLEN. In the current implementation,
0594 the tp_next_offset field in the tpacket3_hdr MUST be set to
0595 zero, indicating that the ring does not hold variable sized frames.
0596 Packets with non-zero values of tp_next_offset will be dropped.
0597
0598 AF_PACKET fanout mode
0599 =====================
0600
0601 In the AF_PACKET fanout mode, packet reception can be load balanced among
0602 processes. This also works in combination with mmap(2) on packet sockets.
0603
0604 Currently implemented fanout policies are:
0605
0606 - PACKET_FANOUT_HASH: schedule to socket by skb's packet hash
0607 - PACKET_FANOUT_LB: schedule to socket by round-robin
0608 - PACKET_FANOUT_CPU: schedule to socket by CPU packet arrives on
0609 - PACKET_FANOUT_RND: schedule to socket by random selection
0610 - PACKET_FANOUT_ROLLOVER: if one socket is full, rollover to another
0611 - PACKET_FANOUT_QM: schedule to socket by skbs recorded queue_mapping
0612
0613 Minimal example code by David S. Miller (try things like "./test eth0 hash",
0614 "./test eth0 lb", etc.)::
0615
0616 #include <stddef.h>
0617 #include <stdlib.h>
0618 #include <stdio.h>
0619 #include <string.h>
0620
0621 #include <sys/types.h>
0622 #include <sys/wait.h>
0623 #include <sys/socket.h>
0624 #include <sys/ioctl.h>
0625
0626 #include <unistd.h>
0627
0628 #include <linux/if_ether.h>
0629 #include <linux/if_packet.h>
0630
0631 #include <net/if.h>
0632
0633 static const char *device_name;
0634 static int fanout_type;
0635 static int fanout_id;
0636
0637 #ifndef PACKET_FANOUT
0638 # define PACKET_FANOUT 18
0639 # define PACKET_FANOUT_HASH 0
0640 # define PACKET_FANOUT_LB 1
0641 #endif
0642
0643 static int setup_socket(void)
0644 {
0645 int err, fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP));
0646 struct sockaddr_ll ll;
0647 struct ifreq ifr;
0648 int fanout_arg;
0649
0650 if (fd < 0) {
0651 perror("socket");
0652 return EXIT_FAILURE;
0653 }
0654
0655 memset(&ifr, 0, sizeof(ifr));
0656 strcpy(ifr.ifr_name, device_name);
0657 err = ioctl(fd, SIOCGIFINDEX, &ifr);
0658 if (err < 0) {
0659 perror("SIOCGIFINDEX");
0660 return EXIT_FAILURE;
0661 }
0662
0663 memset(&ll, 0, sizeof(ll));
0664 ll.sll_family = AF_PACKET;
0665 ll.sll_ifindex = ifr.ifr_ifindex;
0666 err = bind(fd, (struct sockaddr *) &ll, sizeof(ll));
0667 if (err < 0) {
0668 perror("bind");
0669 return EXIT_FAILURE;
0670 }
0671
0672 fanout_arg = (fanout_id | (fanout_type << 16));
0673 err = setsockopt(fd, SOL_PACKET, PACKET_FANOUT,
0674 &fanout_arg, sizeof(fanout_arg));
0675 if (err) {
0676 perror("setsockopt");
0677 return EXIT_FAILURE;
0678 }
0679
0680 return fd;
0681 }
0682
0683 static void fanout_thread(void)
0684 {
0685 int fd = setup_socket();
0686 int limit = 10000;
0687
0688 if (fd < 0)
0689 exit(fd);
0690
0691 while (limit-- > 0) {
0692 char buf[1600];
0693 int err;
0694
0695 err = read(fd, buf, sizeof(buf));
0696 if (err < 0) {
0697 perror("read");
0698 exit(EXIT_FAILURE);
0699 }
0700 if ((limit % 10) == 0)
0701 fprintf(stdout, "(%d) \n", getpid());
0702 }
0703
0704 fprintf(stdout, "%d: Received 10000 packets\n", getpid());
0705
0706 close(fd);
0707 exit(0);
0708 }
0709
0710 int main(int argc, char **argp)
0711 {
0712 int fd, err;
0713 int i;
0714
0715 if (argc != 3) {
0716 fprintf(stderr, "Usage: %s INTERFACE {hash|lb}\n", argp[0]);
0717 return EXIT_FAILURE;
0718 }
0719
0720 if (!strcmp(argp[2], "hash"))
0721 fanout_type = PACKET_FANOUT_HASH;
0722 else if (!strcmp(argp[2], "lb"))
0723 fanout_type = PACKET_FANOUT_LB;
0724 else {
0725 fprintf(stderr, "Unknown fanout type [%s]\n", argp[2]);
0726 exit(EXIT_FAILURE);
0727 }
0728
0729 device_name = argp[1];
0730 fanout_id = getpid() & 0xffff;
0731
0732 for (i = 0; i < 4; i++) {
0733 pid_t pid = fork();
0734
0735 switch (pid) {
0736 case 0:
0737 fanout_thread();
0738
0739 case -1:
0740 perror("fork");
0741 exit(EXIT_FAILURE);
0742 }
0743 }
0744
0745 for (i = 0; i < 4; i++) {
0746 int status;
0747
0748 wait(&status);
0749 }
0750
0751 return 0;
0752 }
0753
0754 AF_PACKET TPACKET_V3 example
0755 ============================
0756
0757 AF_PACKET's TPACKET_V3 ring buffer can be configured to use non-static frame
0758 sizes by doing it's own memory management. It is based on blocks where polling
0759 works on a per block basis instead of per ring as in TPACKET_V2 and predecessor.
0760
0761 It is said that TPACKET_V3 brings the following benefits:
0762
0763 * ~15% - 20% reduction in CPU-usage
0764 * ~20% increase in packet capture rate
0765 * ~2x increase in packet density
0766 * Port aggregation analysis
0767 * Non static frame size to capture entire packet payload
0768
0769 So it seems to be a good candidate to be used with packet fanout.
0770
0771 Minimal example code by Daniel Borkmann based on Chetan Loke's lolpcap (compile
0772 it with gcc -Wall -O2 blob.c, and try things like "./a.out eth0", etc.)::
0773
0774 /* Written from scratch, but kernel-to-user space API usage
0775 * dissected from lolpcap:
0776 * Copyright 2011, Chetan Loke <loke.chetan@gmail.com>
0777 * License: GPL, version 2.0
0778 */
0779
0780 #include <stdio.h>
0781 #include <stdlib.h>
0782 #include <stdint.h>
0783 #include <string.h>
0784 #include <assert.h>
0785 #include <net/if.h>
0786 #include <arpa/inet.h>
0787 #include <netdb.h>
0788 #include <poll.h>
0789 #include <unistd.h>
0790 #include <signal.h>
0791 #include <inttypes.h>
0792 #include <sys/socket.h>
0793 #include <sys/mman.h>
0794 #include <linux/if_packet.h>
0795 #include <linux/if_ether.h>
0796 #include <linux/ip.h>
0797
0798 #ifndef likely
0799 # define likely(x) __builtin_expect(!!(x), 1)
0800 #endif
0801 #ifndef unlikely
0802 # define unlikely(x) __builtin_expect(!!(x), 0)
0803 #endif
0804
0805 struct block_desc {
0806 uint32_t version;
0807 uint32_t offset_to_priv;
0808 struct tpacket_hdr_v1 h1;
0809 };
0810
0811 struct ring {
0812 struct iovec *rd;
0813 uint8_t *map;
0814 struct tpacket_req3 req;
0815 };
0816
0817 static unsigned long packets_total = 0, bytes_total = 0;
0818 static sig_atomic_t sigint = 0;
0819
0820 static void sighandler(int num)
0821 {
0822 sigint = 1;
0823 }
0824
0825 static int setup_socket(struct ring *ring, char *netdev)
0826 {
0827 int err, i, fd, v = TPACKET_V3;
0828 struct sockaddr_ll ll;
0829 unsigned int blocksiz = 1 << 22, framesiz = 1 << 11;
0830 unsigned int blocknum = 64;
0831
0832 fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
0833 if (fd < 0) {
0834 perror("socket");
0835 exit(1);
0836 }
0837
0838 err = setsockopt(fd, SOL_PACKET, PACKET_VERSION, &v, sizeof(v));
0839 if (err < 0) {
0840 perror("setsockopt");
0841 exit(1);
0842 }
0843
0844 memset(&ring->req, 0, sizeof(ring->req));
0845 ring->req.tp_block_size = blocksiz;
0846 ring->req.tp_frame_size = framesiz;
0847 ring->req.tp_block_nr = blocknum;
0848 ring->req.tp_frame_nr = (blocksiz * blocknum) / framesiz;
0849 ring->req.tp_retire_blk_tov = 60;
0850 ring->req.tp_feature_req_word = TP_FT_REQ_FILL_RXHASH;
0851
0852 err = setsockopt(fd, SOL_PACKET, PACKET_RX_RING, &ring->req,
0853 sizeof(ring->req));
0854 if (err < 0) {
0855 perror("setsockopt");
0856 exit(1);
0857 }
0858
0859 ring->map = mmap(NULL, ring->req.tp_block_size * ring->req.tp_block_nr,
0860 PROT_READ | PROT_WRITE, MAP_SHARED | MAP_LOCKED, fd, 0);
0861 if (ring->map == MAP_FAILED) {
0862 perror("mmap");
0863 exit(1);
0864 }
0865
0866 ring->rd = malloc(ring->req.tp_block_nr * sizeof(*ring->rd));
0867 assert(ring->rd);
0868 for (i = 0; i < ring->req.tp_block_nr; ++i) {
0869 ring->rd[i].iov_base = ring->map + (i * ring->req.tp_block_size);
0870 ring->rd[i].iov_len = ring->req.tp_block_size;
0871 }
0872
0873 memset(&ll, 0, sizeof(ll));
0874 ll.sll_family = PF_PACKET;
0875 ll.sll_protocol = htons(ETH_P_ALL);
0876 ll.sll_ifindex = if_nametoindex(netdev);
0877 ll.sll_hatype = 0;
0878 ll.sll_pkttype = 0;
0879 ll.sll_halen = 0;
0880
0881 err = bind(fd, (struct sockaddr *) &ll, sizeof(ll));
0882 if (err < 0) {
0883 perror("bind");
0884 exit(1);
0885 }
0886
0887 return fd;
0888 }
0889
0890 static void display(struct tpacket3_hdr *ppd)
0891 {
0892 struct ethhdr *eth = (struct ethhdr *) ((uint8_t *) ppd + ppd->tp_mac);
0893 struct iphdr *ip = (struct iphdr *) ((uint8_t *) eth + ETH_HLEN);
0894
0895 if (eth->h_proto == htons(ETH_P_IP)) {
0896 struct sockaddr_in ss, sd;
0897 char sbuff[NI_MAXHOST], dbuff[NI_MAXHOST];
0898
0899 memset(&ss, 0, sizeof(ss));
0900 ss.sin_family = PF_INET;
0901 ss.sin_addr.s_addr = ip->saddr;
0902 getnameinfo((struct sockaddr *) &ss, sizeof(ss),
0903 sbuff, sizeof(sbuff), NULL, 0, NI_NUMERICHOST);
0904
0905 memset(&sd, 0, sizeof(sd));
0906 sd.sin_family = PF_INET;
0907 sd.sin_addr.s_addr = ip->daddr;
0908 getnameinfo((struct sockaddr *) &sd, sizeof(sd),
0909 dbuff, sizeof(dbuff), NULL, 0, NI_NUMERICHOST);
0910
0911 printf("%s -> %s, ", sbuff, dbuff);
0912 }
0913
0914 printf("rxhash: 0x%x\n", ppd->hv1.tp_rxhash);
0915 }
0916
0917 static void walk_block(struct block_desc *pbd, const int block_num)
0918 {
0919 int num_pkts = pbd->h1.num_pkts, i;
0920 unsigned long bytes = 0;
0921 struct tpacket3_hdr *ppd;
0922
0923 ppd = (struct tpacket3_hdr *) ((uint8_t *) pbd +
0924 pbd->h1.offset_to_first_pkt);
0925 for (i = 0; i < num_pkts; ++i) {
0926 bytes += ppd->tp_snaplen;
0927 display(ppd);
0928
0929 ppd = (struct tpacket3_hdr *) ((uint8_t *) ppd +
0930 ppd->tp_next_offset);
0931 }
0932
0933 packets_total += num_pkts;
0934 bytes_total += bytes;
0935 }
0936
0937 static void flush_block(struct block_desc *pbd)
0938 {
0939 pbd->h1.block_status = TP_STATUS_KERNEL;
0940 }
0941
0942 static void teardown_socket(struct ring *ring, int fd)
0943 {
0944 munmap(ring->map, ring->req.tp_block_size * ring->req.tp_block_nr);
0945 free(ring->rd);
0946 close(fd);
0947 }
0948
0949 int main(int argc, char **argp)
0950 {
0951 int fd, err;
0952 socklen_t len;
0953 struct ring ring;
0954 struct pollfd pfd;
0955 unsigned int block_num = 0, blocks = 64;
0956 struct block_desc *pbd;
0957 struct tpacket_stats_v3 stats;
0958
0959 if (argc != 2) {
0960 fprintf(stderr, "Usage: %s INTERFACE\n", argp[0]);
0961 return EXIT_FAILURE;
0962 }
0963
0964 signal(SIGINT, sighandler);
0965
0966 memset(&ring, 0, sizeof(ring));
0967 fd = setup_socket(&ring, argp[argc - 1]);
0968 assert(fd > 0);
0969
0970 memset(&pfd, 0, sizeof(pfd));
0971 pfd.fd = fd;
0972 pfd.events = POLLIN | POLLERR;
0973 pfd.revents = 0;
0974
0975 while (likely(!sigint)) {
0976 pbd = (struct block_desc *) ring.rd[block_num].iov_base;
0977
0978 if ((pbd->h1.block_status & TP_STATUS_USER) == 0) {
0979 poll(&pfd, 1, -1);
0980 continue;
0981 }
0982
0983 walk_block(pbd, block_num);
0984 flush_block(pbd);
0985 block_num = (block_num + 1) % blocks;
0986 }
0987
0988 len = sizeof(stats);
0989 err = getsockopt(fd, SOL_PACKET, PACKET_STATISTICS, &stats, &len);
0990 if (err < 0) {
0991 perror("getsockopt");
0992 exit(1);
0993 }
0994
0995 fflush(stdout);
0996 printf("\nReceived %u packets, %lu bytes, %u dropped, freeze_q_cnt: %u\n",
0997 stats.tp_packets, bytes_total, stats.tp_drops,
0998 stats.tp_freeze_q_cnt);
0999
1000 teardown_socket(&ring, fd);
1001 return 0;
1002 }
1003
1004 PACKET_QDISC_BYPASS
1005 ===================
1006
1007 If there is a requirement to load the network with many packets in a similar
1008 fashion as pktgen does, you might set the following option after socket
1009 creation::
1010
1011 int one = 1;
1012 setsockopt(fd, SOL_PACKET, PACKET_QDISC_BYPASS, &one, sizeof(one));
1013
1014 This has the side-effect, that packets sent through PF_PACKET will bypass the
1015 kernel's qdisc layer and are forcedly pushed to the driver directly. Meaning,
1016 packet are not buffered, tc disciplines are ignored, increased loss can occur
1017 and such packets are also not visible to other PF_PACKET sockets anymore. So,
1018 you have been warned; generally, this can be useful for stress testing various
1019 components of a system.
1020
1021 On default, PACKET_QDISC_BYPASS is disabled and needs to be explicitly enabled
1022 on PF_PACKET sockets.
1023
1024 PACKET_TIMESTAMP
1025 ================
1026
1027 The PACKET_TIMESTAMP setting determines the source of the timestamp in
1028 the packet meta information for mmap(2)ed RX_RING and TX_RINGs. If your
1029 NIC is capable of timestamping packets in hardware, you can request those
1030 hardware timestamps to be used. Note: you may need to enable the generation
1031 of hardware timestamps with SIOCSHWTSTAMP (see related information from
1032 Documentation/networking/timestamping.rst).
1033
1034 PACKET_TIMESTAMP accepts the same integer bit field as SO_TIMESTAMPING::
1035
1036 int req = SOF_TIMESTAMPING_RAW_HARDWARE;
1037 setsockopt(fd, SOL_PACKET, PACKET_TIMESTAMP, (void *) &req, sizeof(req))
1038
1039 For the mmap(2)ed ring buffers, such timestamps are stored in the
1040 ``tpacket{,2,3}_hdr`` structure's tp_sec and ``tp_{n,u}sec`` members.
1041 To determine what kind of timestamp has been reported, the tp_status field
1042 is binary or'ed with the following possible bits ...
1043
1044 ::
1045
1046 TP_STATUS_TS_RAW_HARDWARE
1047 TP_STATUS_TS_SOFTWARE
1048
1049 ... that are equivalent to its ``SOF_TIMESTAMPING_*`` counterparts. For the
1050 RX_RING, if neither is set (i.e. PACKET_TIMESTAMP is not set), then a
1051 software fallback was invoked *within* PF_PACKET's processing code (less
1052 precise).
1053
1054 Getting timestamps for the TX_RING works as follows: i) fill the ring frames,
1055 ii) call sendto() e.g. in blocking mode, iii) wait for status of relevant
1056 frames to be updated resp. the frame handed over to the application, iv) walk
1057 through the frames to pick up the individual hw/sw timestamps.
1058
1059 Only (!) if transmit timestamping is enabled, then these bits are combined
1060 with binary | with TP_STATUS_AVAILABLE, so you must check for that in your
1061 application (e.g. !(tp_status & (TP_STATUS_SEND_REQUEST | TP_STATUS_SENDING))
1062 in a first step to see if the frame belongs to the application, and then
1063 one can extract the type of timestamp in a second step from tp_status)!
1064
1065 If you don't care about them, thus having it disabled, checking for
1066 TP_STATUS_AVAILABLE resp. TP_STATUS_WRONG_FORMAT is sufficient. If in the
1067 TX_RING part only TP_STATUS_AVAILABLE is set, then the tp_sec and tp_{n,u}sec
1068 members do not contain a valid value. For TX_RINGs, by default no timestamp
1069 is generated!
1070
1071 See include/linux/net_tstamp.h and Documentation/networking/timestamping.rst
1072 for more information on hardware timestamps.
1073
1074 Miscellaneous bits
1075 ==================
1076
1077 - Packet sockets work well together with Linux socket filters, thus you also
1078 might want to have a look at Documentation/networking/filter.rst
1079
1080 THANKS
1081 ======
1082
1083 Jesse Brandeburg, for fixing my grammathical/spelling errors
