0001 .. SPDX-License-Identifier: GPL-2.0+
0002
0003 ==============================================================
0004 Linux kernel driver for Compute Engine Virtual Ethernet (gve):
0005 ==============================================================
0006
0007 Supported Hardware
0008 ===================
0009 The GVE driver binds to a single PCI device id used by the virtual
0010 Ethernet device found in some Compute Engine VMs.
0011
0012 +--------------+----------+---------+
0013 |Field | Value | Comments|
0014 +==============+==========+=========+
0015 |Vendor ID | `0x1AE0` | Google |
0016 +--------------+----------+---------+
0017 |Device ID | `0x0042` | |
0018 +--------------+----------+---------+
0019 |Sub-vendor ID | `0x1AE0` | Google |
0020 +--------------+----------+---------+
0021 |Sub-device ID | `0x0058` | |
0022 +--------------+----------+---------+
0023 |Revision ID | `0x0` | |
0024 +--------------+----------+---------+
0025 |Device Class | `0x200` | Ethernet|
0026 +--------------+----------+---------+
0027
0028 PCI Bars
0029 ========
0030 The gVNIC PCI device exposes three 32-bit memory BARS:
0031 - Bar0 - Device configuration and status registers.
0032 - Bar1 - MSI-X vector table
0033 - Bar2 - IRQ, RX and TX doorbells
0034
0035 Device Interactions
0036 ===================
0037 The driver interacts with the device in the following ways:
0038 - Registers
0039 - A block of MMIO registers
0040 - See gve_register.h for more detail
0041 - Admin Queue
0042 - See description below
0043 - Reset
0044 - At any time the device can be reset
0045 - Interrupts
0046 - See supported interrupts below
0047 - Transmit and Receive Queues
0048 - See description below
0049
0050 Descriptor Formats
0051 ------------------
0052 GVE supports two descriptor formats: GQI and DQO. These two formats have
0053 entirely different descriptors, which will be described below.
0054
0055 Registers
0056 ---------
0057 All registers are MMIO.
0058
0059 The registers are used for initializing and configuring the device as well as
0060 querying device status in response to management interrupts.
0061
0062 Endianness
0063 ----------
0064 - Admin Queue messages and registers are all Big Endian.
0065 - GQI descriptors and datapath registers are Big Endian.
0066 - DQO descriptors and datapath registers are Little Endian.
0067
0068 Admin Queue (AQ)
0069 ----------------
0070 The Admin Queue is a PAGE_SIZE memory block, treated as an array of AQ
0071 commands, used by the driver to issue commands to the device and set up
0072 resources.The driver and the device maintain a count of how many commands
0073 have been submitted and executed. To issue AQ commands, the driver must do
0074 the following (with proper locking):
0075
0076 1) Copy new commands into next available slots in the AQ array
0077 2) Increment its counter by he number of new commands
0078 3) Write the counter into the GVE_ADMIN_QUEUE_DOORBELL register
0079 4) Poll the ADMIN_QUEUE_EVENT_COUNTER register until it equals
0080 the value written to the doorbell, or until a timeout.
0081
0082 The device will update the status field in each AQ command reported as
0083 executed through the ADMIN_QUEUE_EVENT_COUNTER register.
0084
0085 Device Resets
0086 -------------
0087 A device reset is triggered by writing 0x0 to the AQ PFN register.
0088 This causes the device to release all resources allocated by the
0089 driver, including the AQ itself.
0090
0091 Interrupts
0092 ----------
0093 The following interrupts are supported by the driver:
0094
0095 Management Interrupt
0096 ~~~~~~~~~~~~~~~~~~~~
0097 The management interrupt is used by the device to tell the driver to
0098 look at the GVE_DEVICE_STATUS register.
0099
0100 The handler for the management irq simply queues the service task in
0101 the workqueue to check the register and acks the irq.
0102
0103 Notification Block Interrupts
0104 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0105 The notification block interrupts are used to tell the driver to poll
0106 the queues associated with that interrupt.
0107
0108 The handler for these irqs schedule the napi for that block to run
0109 and poll the queues.
0110
0111 GQI Traffic Queues
0112 ------------------
0113 GQI queues are composed of a descriptor ring and a buffer and are assigned to a
0114 notification block.
0115
0116 The descriptor rings are power-of-two-sized ring buffers consisting of
0117 fixed-size descriptors. They advance their head pointer using a __be32
0118 doorbell located in Bar2. The tail pointers are advanced by consuming
0119 descriptors in-order and updating a __be32 counter. Both the doorbell
0120 and the counter overflow to zero.
0121
0122 Each queue's buffers must be registered in advance with the device as a
0123 queue page list, and packet data can only be put in those pages.
0124
0125 Transmit
0126 ~~~~~~~~
0127 gve maps the buffers for transmit rings into a FIFO and copies the packets
0128 into the FIFO before sending them to the NIC.
0129
0130 Receive
0131 ~~~~~~~
0132 The buffers for receive rings are put into a data ring that is the same
0133 length as the descriptor ring and the head and tail pointers advance over
0134 the rings together.
0135
0136 DQO Traffic Queues
0137 ------------------
0138 - Every TX and RX queue is assigned a notification block.
0139
0140 - TX and RX buffers queues, which send descriptors to the device, use MMIO
0141 doorbells to notify the device of new descriptors.
0142
0143 - RX and TX completion queues, which receive descriptors from the device, use a
0144 "generation bit" to know when a descriptor was populated by the device. The
0145 driver initializes all bits with the "current generation". The device will
0146 populate received descriptors with the "next generation" which is inverted
0147 from the current generation. When the ring wraps, the current/next generation
0148 are swapped.
0149
0150 - It's the driver's responsibility to ensure that the RX and TX completion
0151 queues are not overrun. This can be accomplished by limiting the number of
0152 descriptors posted to HW.
0153
0154 - TX packets have a 16 bit completion_tag and RX buffers have a 16 bit
0155 buffer_id. These will be returned on the TX completion and RX queues
0156 respectively to let the driver know which packet/buffer was completed.
0157
0158 Transmit
0159 ~~~~~~~~
0160 A packet's buffers are DMA mapped for the device to access before transmission.
0161 After the packet was successfully transmitted, the buffers are unmapped.
0162
0163 Receive
0164 ~~~~~~~
0165 The driver posts fixed sized buffers to HW on the RX buffer queue. The packet
0166 received on the associated RX queue may span multiple descriptors.
