RE: Post-boot PCI bus rescan
Andrea Galbusera <andrea.galbusera <at> teamware.it>
2007-04-02 09:29:22 GMT
Hi Arvid,
sorry for very very late reply... I had to collect a few ideas that I
hope could be useful to you.
First let me state that we didn't reach a completely clean solution to
the PCI rescan issue yet. I have been mainly experimenting on the topic.
Let me share my results...
I had reasonably good result in getting "basic" rescan to work: this can
be done at least with two methods:
1. Exploting the fake-php module already included in vanilla kernels
(see drivers/pci/hotplug/fakephp.c). If you already have a probed device
on the bus (I have the bridge to the CPU in my system) you can fire the
rescan, after configuring your fpga, by "re-powering" the already
available device. This is done by issuing from shell:
echo 1 > /sys/bus/pci/slot/≤PCI-device-ID>/power
I found documentation about fakephp in mailinglist archives by deeply
googlein' on the subject.
NOTE: to enable the choice of this module in kconfig I had to add the
following line to arch/ppc/Kconfig
source "drivers/pci/hotplug/Kconfig"
In my kernel (2.6.16) this is available in arch/powerpc but not in
arch/ppc. Don't know if this is wanted or not. This step should not be
required if you are using ARCH=powerpc instead. After enabling the
choice the module is built ok with my kernel.
After adding the line above you can find "Fake PCI hotplug driver" in
kconfig under Bus Options / PCI Hotplug support.
2. Using the rescan procedure contributed by John Carlson. This is
basically a kernel module that issue a PCI bus rescan when you load it.
John helped me to port this module to 2.6.16 kernel API and I had more
or less the same result using this and fakephp.
I said these two methods would bring only "basic" rescan. The main point
is that, as far as I can figure out, when bus scanning is done at boot,
the ppc platform specific code will start a lot of quirks, irq fixups
and resource remapping functions that will not be executed with the
"manual" rescan done by the methods listed above. As a result, the
config space for the pci device exposed by my fpga is sligtly different
from that I have by rebooting after configuring the fpga. I suppose I
can fix this later i.e. by using the setpci tool, but I don't have deep
enough understanding of all those platform specific fixups and didn't
get the same result of rebooting yet.
Hope this quite confused thoughts may help you implementing your rescan.
Please, keep me informed if you go further and get a cleaner solution.
Regaards,
Andrea
On Wed, 2007-03-21 at 17:32 +0100, Arvid Staub wrote:
> Hi there!
>
> I'm very interested if you found a clean solution to the PCI rescan issue.
> We have a board in development here which should utilize such a mechanism as well - but it's not yet done.
it's a Taihu-like PPC405EP setup with the PCI bus connected to a Virtex-5 LXT FPGA.
>
> can you point me to some information about the rescan in general?
> which PCI bridge and Buses do you use inside your FPGA?
>
> thx & ciao,
> arvid
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: linuxppc-embedded-bounces+arvid.staub=mediornet.com <at> ozlabs.org
[mailto:linuxppc-embedded-bounces+arvid.staub=mediornet.com <at> ozlabs.org] Im Auftrag von Andrea Galbusera
> Gesendet: Mittwoch, 28. Februar 2007 12:41
> An: linuxppc-embedded <at> ozlabs.org
> Betreff: Post-boot PCI bus rescan
>
> Hi all,
> I'm working on a mpc5200 based custom board. This is basically a
> Lite5200 like platform. I have an onboard FPGA that I configure from
> userspace linux with the required "firmware". Only after configuration
> the FPGA shows its PCI device "nature".
> I have the problem to make linux correctly see the PCI device, in order
> to load my device driver and go on running applications.
>
> First solution we implemented was to go through a reboot (without
> powering off the FPGA, of course) and consequent boot-time rescan of the
> bus. Works fine but performance is an issue here.
>
> I then started digging around and found some PCI rescan solution (mainly
> fakephp from the kernel tree and a rescan routine by John Carlson). Both
> these can detect the device and correctly initialize /sys entries.
>
> Unfortunately my PCI device config space appears to be different from
> the one I have with the reboot approach. I then compiled my kernel with
> full PCI debugging enabled and got the log attached below. As far as I
> can understand during boot linux calls some fixups on irqs and resouces
> for my PCI device (0000:00:18.0). Those seem to be called by
> pcibios_init() in arch/ppc/kernel/pci.c.
>
> My question is: am I on the right way to figure out why PCI device
> config space appear different with the two approaches? What is the logic
> behind those fixups? Can I call them again after boot i.e. from my
> rescan module?
>
> Thanks in advance for any kind help
> Andrea
>
>
> ----------------------------------------
> Boot time PCI debug messages if I reboot
> ----------------------------------------
>
> > PCI: Probing PCI hardware
> > DEV: registering device: ID = 'pci0000:00'
> > kobject pci0000:00: registering. parent: <NULL>, set: devices
> > kobject_uevent
> > CLASS: registering class device: ID = '0000:00'
> > kobject 0000:00: registering. parent: pci_bus, set: class_obj
> > kobject_uevent
> > fill_kobj_path: path = '/class/pci_bus/0000:00'
> > class_uevent - name = 0000:00
> > PCI: Scanning bus 0000:00
> > PCI: Found 0000:00:18.0 [1172/abcd] 00ffed 00
> > PCI: Calling quirk c00098bc for 0000:00:18.0
> > PCI: Calling quirk c010d400 for 0000:00:18.0
> > PCI: Found 0000:00:1a.0 [1057/5809] 000680 00
> > PCI: Calling quirk c00098bc for 0000:00:1a.0
> > PCI: Calling quirk c010d400 for 0000:00:1a.0
> > PCI: Fixups for bus 0000:00
> > PCI: Bus scan for 0000:00 returning with max=00
> > DEV: registering device: ID = '0000:00:18.0'
> > kobject 0000:00:18.0: registering. parent: pci0000:00, set: devices
> > kobject_uevent
> > fill_kobj_path: path = '/devices/pci0000:00/0000:00:18.0'
> > bus pci: add device 0000:00:18.0
> > DEV: registering device: ID = '0000:00:1a.0'
> > kobject 0000:00:1a.0: registering. parent: pci0000:00, set: devices
> > kobject_uevent
> > fill_kobj_path: path = '/devices/pci0000:00/0000:00:1a.0'
> > bus pci: add device 0000:00:1a.0
> > PCI: fixup irq: (0000:00:18.0) got 1
> > PCI: fixup irq: (0000:00:1a.0) got 0
> > PCI: bridge rsrc 0..ffffff (100), parent c027a1fc
> > PCI: bridge rsrc 80000000..9fffffff (1200), parent c027a1e0
> > PCI: bridge rsrc a0000000..afffffff (200), parent c027a1e0
> > got res [80000000:80000fff] bus [80000000:80000fff] flags 1208 for
> BAR 0 of 0000:00:18.0
> > PCI: moved device 0000:00:18.0 resource 0 (1208) to 80000000
> > got res [80800000:80ffffff] bus [80800000:80ffffff] flags 1208 for
> BAR 1 of 0000:00:18.0
> > PCI: moved device 0000:00:18.0 resource 1 (1208) to 80800000
> > got res [a0000000:a0003fff] bus [a0000000:a0003fff] flags 200 for
> BAR 2 of 0000:00:18.0
> > PCI: moved device 0000:00:18.0 resource 2 (200) to a0000000
> > got res [a0040000:a007ffff] bus [a0040000:a007ffff] flags 200 for
> BAR 0 of 0000:00:1a.0
> > PCI: moved device 0000:00:1a.0 resource 0 (200) to a0040000
>
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded <at> ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>
.
>
> Best regards. Didier
>
> Konstantin Boyanov said:
>> Hi again,
>>
>> Thank you once again for presenting the VME world to me :)
>>
>> >Looks like you're reading something. Address 0x10100000 is larger than
>> >a 24 bit address. Perhaps you should be using A32 transfers instead of
>> >A24?
>>
>> No, no, I need to do A16 access, thats the only way to read the
>> control registers. Form here comes also my confusion - I thought that
>> when I'm reading CS/CSR register I need to do it in CSR address
>> space. So no algorithms for geographical location of boards are
>> needed in my case I guess.
>> Actually the base address of CSR on the board is 0xA000 (10100000 is
>> the first byte of the address selection in inary, shame on me :( ).
>> But when I try to configure it with outWinCfgADC.zlatedAddrL =
>> 0xA000, when I run the programm I get an errno 29 (Invalid argument)
>> from ioctl(). In fact if I set the xlatedAddrL to some value smaller
>> that 0x10000 I get this error. Here's my most recent configuration:
>>
>> outWinCfgADC.windowNbr = 0;
>> outWinCfgADC.windowEnable = 1;
>> outWinCfgADC.wrPostEnable = 0;
>> outWinCfgADC.userAccessType = VME_SUPER;
>> outWinCfgADC.dataAccessType = VME_DATA;
>> outWinCfgADC.windowSizeL = 0x10000;
>> outWinCfgADC.xferProtocol = VME_SCT;
>> outWinCfgADC.addrSpace = VME_A16;
>> outWinCfgADC.maxDataWidth = VME_D16;
>> outWinCfgADC.xlatedAddrL = 0x10000000;
>>
>> I think that 0x10000 for window size is big enough to encompass the
>> whole A16 addressable space. With the 0x10000000 xlatedAddr I get no
>> errors and read the desired registers when I increment this address
>> to 0x1000A000. But nevertheless it seems strange to me that the
>> device does not accept xlatedAddr lower that 0x10000. Maybe its due
>> to the configuration of the VME controller?
>>
>
RSS Feed