Linux PCI subsystem discussion ()

Zhao, Yu | 1 Sep 2008 13:20

[PATCH 0/4 v2] PCI: Linux kernel SR-IOV support

Greetings,

Following patches add SR-IOV capability support to the Linux kernel. With these patches, people can turn a
PCI device with the capability into multiple ones from software perspective.

[PATCH 1/4 v2] PCI: introduce new base functions
[PATCH 2/4 v2] PCI: support ARI capability
[PATCH 3/4 v2] PCI: support SR-IOV capability
[PATCH 4/4 v2] PCI: document the change

Thanks to Randy Dunlap for carefully reviewing the document change.
And also thank Greg KH, Grant Grundler and Jesse Barnes for their comments on the initial version.

---
Single Root I/O Virtualization (SR-IOV) capability defined by PCI-SIG is intended to enable multiple
system software to share PCI hardware resources. PCI device that supports this capability can be
extended to one Physical Functions plus multiple Virtual Functions. Physical Function, which could be
considered as the "real" PCI device, reflects the hardware instance and manages all physical resources.
Virtual Functions are associated with a Physical Function and shares physical resources with the
Physical Function. Software can control allocation of Virtual Functions via registers encapsulated in
the capability structure.

SR-IOV specification can be found at http://www.pcisig.com/members/downloads/specifications/iov/sr-iov1.0_11Sep07.pdf

Devices that support SR-IOV are available from following vendors:
http://download.intel.com/design/network/ProdBrf/320025.pdf
http://www.netxen.com/products/chipsolutions/NX3031.html
http://www.neterion.com/products/x3100.html

headers

Zhao, Yu | 1 Sep 2008 13:20

[PATCH 1/4 v2] PCI: introduce new base functions

Some basic changes to allocation bus range, MMIO resource for SR-IOV device.
And add new sysfs entry to hotplug core to pass parameter to a slot, which will be used by SR-IOV code.

Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>

---
 drivers/pci/bus.c                      |   63 +++++++++++++-------------
 drivers/pci/hotplug/pci_hotplug_core.c |   75 +++++++++++++++++++++++++++++---
 drivers/pci/pci-sysfs.c                |    4 +-
 drivers/pci/pci.c                      |   68 +++++++++++++++++++++--------
 drivers/pci/pci.h                      |    3 +
 drivers/pci/probe.c                    |   37 ++++++++-------
 drivers/pci/proc.c                     |    7 ++-
 drivers/pci/remove.c                   |    3 +-
 drivers/pci/setup-bus.c                |    9 ++--
 drivers/pci/setup-res.c                |   29 ++++++------
 include/linux/pci.h                    |   53 ++++++++++++++++-------
 include/linux/pci_hotplug.h            |   11 ++++-
 12 files changed, 246 insertions(+), 116 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index 529d9d7..15f64c9 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
 <at>  <at>  -105,7 +105,7  <at>  <at>  int pci_bus_add_device(struct pci_dev *dev)
 void pci_bus_add_devices(struct pci_bus *bus)
 {
 	struct pci_dev *dev;
-	struct pci_bus *child_bus;

(Continue reading)

headers

Zhao, Yu | 1 Sep 2008 13:20

[PATCH 2/4 v2] PCI: support ARI capability

Support Alternative Routing-ID Interpretation (ARI), which increases the number of functions that can
be supported by a PCIe endpoint. ARI is required by SR-IOV.

PCI-SIG ARI specification can be found at http://www.pcisig.com/specifications/pciexpress/specifications/ECN-alt-rid-interpretation-070604.pdf

Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>

---
 drivers/pci/Kconfig      |    7 ++++
 drivers/pci/Makefile     |    2 +
 drivers/pci/ari.c        |   71 ++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h        |    8 +++++
 drivers/pci/probe.c      |    3 ++
 include/linux/pci.h      |   25 ++++++++++++++++
 include/linux/pci_regs.h |   14 +++++++++
 7 files changed, 130 insertions(+), 0 deletions(-)
 create mode 100644 drivers/pci/ari.c

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index e1ca425..f43cc46 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
 <at>  <at>  -50,3 +50,10  <at>  <at>  config HT_IRQ
 	   This allows native hypertransport devices to use interrupts.

 	   If unsure say Y.
+
+config PCI_ARI
+	bool "PCI ARI support"

(Continue reading)

headers

Zhao, Yu | 1 Sep 2008 13:21

[PATCH 4/4 v2] PCI: document the change

Complete the hotplug ABI document, and add SR-IOV HOWTO.

Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>

---
 Documentation/ABI/testing/sysfs-bus-pci |   67 ++++++++++++
 Documentation/DocBook/kernel-api.tmpl   |    3 +
 Documentation/PCI/pci-iov-howto.txt     |  177 +++++++++++++++++++++++++++++++
 3 files changed, 247 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/PCI/pci-iov-howto.txt

diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
index ceddcff..374e87b 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci
+++ b/Documentation/ABI/testing/sysfs-bus-pci
 <at>  <at>  -9,3 +9,70  <at>  <at>  Description:
 		that some devices may have malformatted data.  If the
 		underlying VPD has a writable section then the
 		corresponding section of this file will be writable.
+
+What:		/sys/bus/pci/slots/.../power
+Date:		Unknown
+Contact:	linux-pci <at> vger.kernel.org
+Description:
+		This file will appear when PCI hotplug is enabled and
+		the hotplug driver supports this operation.
+		It indicates power status of a slot, and could be written
+		to enable or disable the slot.
+

(Continue reading)

headers

Zhao, Yu | 1 Sep 2008 13:21

[PATCH 3/4 v2] PCI: support SR-IOV capability

Support SR-IOV capability. By default, this feature is not enabled and the SR-IOV device behaves as
traditional PCI device. After it's enabled, each Virtual Function's PCI configuration space can be
accessed using its own Bus, Device and Function Number (Routing ID). Each Virtual Function also has PCI
Memory Space, which is used to map its own register set.

Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>

---
 drivers/pci/Kconfig      |   10 +
 drivers/pci/Makefile     |    2 +
 drivers/pci/iov.c        |  555 ++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c        |   14 +-
 drivers/pci/pci.h        |   44 ++++
 drivers/pci/probe.c      |    5 +
 include/linux/pci.h      |   28 +++
 include/linux/pci_regs.h |   20 ++
 8 files changed, 677 insertions(+), 1 deletions(-)
 create mode 100644 drivers/pci/iov.c

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index f43cc46..0a1fe01 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
 <at>  <at>  -57,3 +57,13  <at>  <at>  config PCI_ARI
 	default n
 	help
 	  This enables PCI Alternative Routing-ID Interpretation.
+
+config PCI_IOV

(Continue reading)

headers

Alex Chiang | 1 Sep 2008 17:27

Re: [PATCH 2/4 v2] PCI: support ARI capability

* Zhao, Yu <yu.zhao <at> intel.com>:
> Support Alternative Routing-ID Interpretation (ARI), which increases the number of functions that can
be supported by a PCIe endpoint. ARI is required by SR-IOV.
> 
> PCI-SIG ARI specification can be found at http://www.pcisig.com/specifications/pciexpress/specifications/ECN-alt-rid-interpretation-070604.pdf
> 
> Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
> Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>
> 
> ---
>  drivers/pci/Kconfig      |    7 ++++
>  drivers/pci/Makefile     |    2 +
>  drivers/pci/ari.c        |   71 ++++++++++++++++++++++++++++++++++++++++++++++
>  drivers/pci/pci.h        |    8 +++++
>  drivers/pci/probe.c      |    3 ++
>  include/linux/pci.h      |   25 ++++++++++++++++
>  include/linux/pci_regs.h |   14 +++++++++
>  7 files changed, 130 insertions(+), 0 deletions(-)
>  create mode 100644 drivers/pci/ari.c
> 
> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
> index e1ca425..f43cc46 100644
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
>  <at>  <at>  -50,3 +50,10  <at>  <at>  config HT_IRQ
>  	   This allows native hypertransport devices to use interrupts.
>  
>  	   If unsure say Y.
> +
> +config PCI_ARI

(Continue reading)

headers

Alex Chiang | 1 Sep 2008 17:30

Re: [PATCH 3/4 v2] PCI: support SR-IOV capability

* Zhao, Yu <yu.zhao <at> intel.com>:
> Support SR-IOV capability. By default, this feature is not enabled and the SR-IOV device behaves as
traditional PCI device. After it's enabled, each Virtual Function's PCI configuration space can be
accessed using its own Bus, Device and Function Number (Routing ID). Each Virtual Function also has PCI
Memory Space, which is used to map its own register set.
> 
> Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
> Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>
> 
> ---
>  drivers/pci/Kconfig      |   10 +
>  drivers/pci/Makefile     |    2 +
>  drivers/pci/iov.c        |  555 ++++++++++++++++++++++++++++++++++++++++++++++
>  drivers/pci/pci.c        |   14 +-
>  drivers/pci/pci.h        |   44 ++++
>  drivers/pci/probe.c      |    5 +
>  include/linux/pci.h      |   28 +++
>  include/linux/pci_regs.h |   20 ++
>  8 files changed, 677 insertions(+), 1 deletions(-)
>  create mode 100644 drivers/pci/iov.c
> 
> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
> index f43cc46..0a1fe01 100644
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
>  <at>  <at>  -57,3 +57,13  <at>  <at>  config PCI_ARI
>  	default n
>  	help
>  	  This enables PCI Alternative Routing-ID Interpretation.
> +

(Continue reading)

headers

Alex Chiang | 1 Sep 2008 17:42

Re: [PATCH 4/4 v2] PCI: document the change

* Zhao, Yu <yu.zhao <at> intel.com>:
> +1. Overview
> +
> +1.1 What is SR-IOV
> +
> +Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
> +capability which makes one physical device appear as multiple virtual
> +devices. The physical device is referred to as Physical Function while
> +the virtual devices are referred to as Virtual Functions. Allocation
> +of Virtual Functions can be dynamically controlled by Physical Function
> +via registers encapsulated in the capability. By default, this feature
> +is not enabled and the Physical Function behaves as traditional PCIe
> +device. Once it's turned on, each Virtual Function's PCI configuration
> +space can be accessed by its own Bus, Device and Function Number (Routing
> +ID). And each Virtual Function also has PCI Memory Space, which is used
> +to map its register set. Virtual Function device driver operates on the
> +register set so it can be functional and appear as a real existing PCI
> +device.
> +
> +1.2 What is ARI
> +
> +Alternative Routing-ID Interpretation (ARI) allows a PCI Express Endpoint
> +to use its device number field as part of function number. Traditionally,
> +an Endpoint can only have 8 functions, and the device number of all
> +Endpoints is zero. With ARI enabled, an Endpoint can have up to 256
> +functions by using device number in conjunction with function number to
> +indicate a function in the device. This is almost transparent to the Linux
> +kernel because the Linux kernel still can use 8-bit bus number field plus
> +8-bit devfn number field to locate a function. ARI is managed via the ARI
> +Forwarding bit in the Device Capabilities 2 register of the PCI Express

(Continue reading)

headers

Roland Dreier | 1 Sep 2008 17:50

Re: [PATCH 2/4 v2] PCI: support ARI capability

 > > +config PCI_ARI
 > > +	bool "PCI ARI support"
 > > +	depends on PCI
 > > +	default n
 > > +	help
 > > +	  This enables PCI Alternative Routing-ID Interpretation.
 > 
 > This Kconfig help text is a little weak. Why not include the text
 > you've already written here:
 > 
 > 	Support Alternative Routing-ID Interpretation (ARI), which
 > 	increases the number of functions that can be supported by a PCIe
 > 	endpoint. ARI is required by SR-IOV.

I agree with this improvement to the help text.  But a further question
is whether ARI even merits its own user-visible config option.  Is it
worth having yet another choice for users?  When would someone want ARI
but not SR-IOV?

 - R.

headers

Alex Chiang | 1 Sep 2008 18:15

Re: [PATCH 1/4 v2] PCI: introduce new base functions

* Zhao, Yu <yu.zhao <at> intel.com>:
> Some basic changes to allocation bus range, MMIO resource for SR-IOV device.

This following comment is a bit confusing:
> And add new sysfs entry to hotplug core to pass parameter to a
> slot, which will be used by SR-IOV code.

I was reading this patch, expecting to see a change to the
hotplug core _API_ taking a param, not just a new sysfs entry.

I would suggest rewording this part of the changelog as:

	Add new sysfs file 'param' to /sys/bus/pci/slots/.../
	which allows the user to pass a parameter to a slot. This
	parameter will be used by the SR-IOV code.

More about this new 'param' file below.

> 
> Signed-off-by: Yu Zhao <yu.zhao <at> intel.com>
> Signed-off-by: Eddie Dong <eddie.dong <at> intel.com>
> 
> ---
>  drivers/pci/bus.c                      |   63 +++++++++++++-------------
>  drivers/pci/hotplug/pci_hotplug_core.c |   75 +++++++++++++++++++++++++++++---
>  drivers/pci/pci-sysfs.c                |    4 +-
>  drivers/pci/pci.c                      |   68 +++++++++++++++++++++--------
>  drivers/pci/pci.h                      |    3 +
>  drivers/pci/probe.c                    |   37 ++++++++-------
>  drivers/pci/proc.c                     |    7 ++-

(Continue reading)

278	May 2013
484	April 2013
467	March 2013
327	February 2013
486	January 2013
200	December 2012
197	November 2012
337	October 2012
397	September 2012
346	August 2012
317	July 2012
382	June 2012
358	May 2012
383	April 2012
303	March 2012
254	February 2012
266	January 2012
114	December 2011
270	November 2011
272	October 2011
237	September 2011
194	August 2011
115	July 2011
186	June 2011
206	May 2011
147	April 2011
119	March 2011
70	February 2011
95	January 2011
84	December 2010
97	November 2010
166	October 2010
112	September 2010
91	August 2010
122	July 2010
250	June 2010
169	May 2010
329	April 2010
242	March 2010
240	February 2010
396	January 2010
264	December 2009
340	November 2009
238	October 2009
336	September 2009
292	August 2009
227	July 2009
544	June 2009
261	May 2009
314	April 2009
458	March 2009
462	February 2009
441	January 2009
345	December 2008
464	November 2008
605	October 2008
383	September 2008
239	August 2008
354	July 2008
111	June 2008
1	January 2002

Mon	Tue	Wed	Thu	Fri	Sat	Sun

1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30