0001 .. SPDX-License-Identifier: GPL-2.0
0002 .. include:: <isonum.txt>
0003
0004 ====================================
0005 PCI Express I/O Virtualization Howto
0006 ====================================
0007
0008 :Copyright: |copy| 2009 Intel Corporation
0009 :Authors: - Yu Zhao <yu.zhao@intel.com>
0010 - Donald Dutile <ddutile@redhat.com>
0011
0012 Overview
0013 ========
0014
0015 What is SR-IOV
0016 --------------
0017
0018 Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
0019 capability which makes one physical device appear as multiple virtual
0020 devices. The physical device is referred to as Physical Function (PF)
0021 while the virtual devices are referred to as Virtual Functions (VF).
0022 Allocation of the VF can be dynamically controlled by the PF via
0023 registers encapsulated in the capability. By default, this feature is
0024 not enabled and the PF behaves as traditional PCIe device. Once it's
0025 turned on, each VF's PCI configuration space can be accessed by its own
0026 Bus, Device and Function Number (Routing ID). And each VF also has PCI
0027 Memory Space, which is used to map its register set. VF device driver
0028 operates on the register set so it can be functional and appear as a
0029 real existing PCI device.
0030
0031 User Guide
0032 ==========
0033
0034 How can I enable SR-IOV capability
0035 ----------------------------------
0036
0037 Multiple methods are available for SR-IOV enablement.
0038 In the first method, the device driver (PF driver) will control the
0039 enabling and disabling of the capability via API provided by SR-IOV core.
0040 If the hardware has SR-IOV capability, loading its PF driver would
0041 enable it and all VFs associated with the PF. Some PF drivers require
0042 a module parameter to be set to determine the number of VFs to enable.
0043 In the second method, a write to the sysfs file sriov_numvfs will
0044 enable and disable the VFs associated with a PCIe PF. This method
0045 enables per-PF, VF enable/disable values versus the first method,
0046 which applies to all PFs of the same device. Additionally, the
0047 PCI SRIOV core support ensures that enable/disable operations are
0048 valid to reduce duplication in multiple drivers for the same
0049 checks, e.g., check numvfs == 0 if enabling VFs, ensure
0050 numvfs <= totalvfs.
0051 The second method is the recommended method for new/future VF devices.
0052
0053 How can I use the Virtual Functions
0054 -----------------------------------
0055
0056 The VF is treated as hot-plugged PCI devices in the kernel, so they
0057 should be able to work in the same way as real PCI devices. The VF
0058 requires device driver that is same as a normal PCI device's.
0059
0060 Developer Guide
0061 ===============
0062
0063 SR-IOV API
0064 ----------
0065
0066 To enable SR-IOV capability:
0067
0068 (a) For the first method, in the driver::
0069
0070 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
0071
0072 'nr_virtfn' is number of VFs to be enabled.
0073
0074 (b) For the second method, from sysfs::
0075
0076 echo 'nr_virtfn' > \
0077 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
0078
0079 To disable SR-IOV capability:
0080
0081 (a) For the first method, in the driver::
0082
0083 void pci_disable_sriov(struct pci_dev *dev);
0084
0085 (b) For the second method, from sysfs::
0086
0087 echo 0 > \
0088 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
0089
0090 To enable auto probing VFs by a compatible driver on the host, run
0091 command below before enabling SR-IOV capabilities. This is the
0092 default behavior.
0093 ::
0094
0095 echo 1 > \
0096 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
0097
0098 To disable auto probing VFs by a compatible driver on the host, run
0099 command below before enabling SR-IOV capabilities. Updating this
0100 entry will not affect VFs which are already probed.
0101 ::
0102
0103 echo 0 > \
0104 /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe
0105
0106 Usage example
0107 -------------
0108
0109 Following piece of code illustrates the usage of the SR-IOV API.
0110 ::
0111
0112 static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id)
0113 {
0114 pci_enable_sriov(dev, NR_VIRTFN);
0115
0116 ...
0117
0118 return 0;
0119 }
0120
0121 static void dev_remove(struct pci_dev *dev)
0122 {
0123 pci_disable_sriov(dev);
0124
0125 ...
0126 }
0127
0128 static int dev_suspend(struct device *dev)
0129 {
0130 ...
0131
0132 return 0;
0133 }
0134
0135 static int dev_resume(struct device *dev)
0136 {
0137 ...
0138
0139 return 0;
0140 }
0141
0142 static void dev_shutdown(struct pci_dev *dev)
0143 {
0144 ...
0145 }
0146
0147 static int dev_sriov_configure(struct pci_dev *dev, int numvfs)
0148 {
0149 if (numvfs > 0) {
0150 ...
0151 pci_enable_sriov(dev, numvfs);
0152 ...
0153 return numvfs;
0154 }
0155 if (numvfs == 0) {
0156 ....
0157 pci_disable_sriov(dev);
0158 ...
0159 return 0;
0160 }
0161 }
0162
0163 static struct pci_driver dev_driver = {
0164 .name = "SR-IOV Physical Function driver",
0165 .id_table = dev_id_table,
0166 .probe = dev_probe,
0167 .remove = dev_remove,
0168 .driver.pm = &dev_pm_ops,
0169 .shutdown = dev_shutdown,
0170 .sriov_configure = dev_sriov_configure,
0171 };
