headers
Grahame Kelly | 1 Aug 05:35
Gravatar

Problem getting PCI devices hidden from Dom0

Hi All.

I have tried hiding two PCI cards from Dom0 on boot up in accordance  
with the documentation.

Xen 3.3.1_18546_16-0.1 upon openSuSE-11.1 release.

Using the following /boot/grub/menu.1st directives:

title Xen -- openSUSE 11.1 - 2.6.27.25-0.1
root (hd0,0)
kernel /boot/xen.gz
module /boot/vmlinuz-2.6.27.25-0.1-xen
root=/dev/disk/by-id/scsi-1AMCC_U15626923F6A26004196-part1 resume=/dev/ 
disk/by-id/scsi-1AMCC_U15626923F6A26004196-part2
splash=silent showopts vga=0x317
pciback.permissive pciback.hide-(04:00.0) (06:04.0) (08:0e.0)
module /boot/initrd-2.6.27.25-0.1-xen

after booting - if I run "lspci",  Dom0 still sees the PCI devices I  
am trying to hide exclusively for the use of DomU's.

Can anyone advise what may be the issue why this is not working?

Thanks. Grahame

(appended is my vm-dmesg if required).

 __  __
(Continue reading)

Permalink | Reply | 2 comments |
headers
James Harper | 1 Aug 06:32
Picon

RE: Problem getting PCI devices hidden from Dom0

> 
> Hi All.
> 
> I have tried hiding two PCI cards from Dom0 on boot up in accordance
> with the documentation.
> 
> Xen 3.3.1_18546_16-0.1 upon openSuSE-11.1 release.
> 
> Using the following /boot/grub/menu.1st directives:
> 
> title Xen -- openSUSE 11.1 - 2.6.27.25-0.1
> root (hd0,0)
> kernel /boot/xen.gz
> module /boot/vmlinuz-2.6.27.25-0.1-xen
> root=/dev/disk/by-id/scsi-1AMCC_U15626923F6A26004196-part1
resume=/dev/
> disk/by-id/scsi-1AMCC_U15626923F6A26004196-part2
> splash=silent showopts vga=0x317
> pciback.permissive pciback.hide-(04:00.0) (06:04.0) (08:0e.0)
> module /boot/initrd-2.6.27.25-0.1-xen
> 
> after booting - if I run "lspci",  Dom0 still sees the PCI devices I
> am trying to hide exclusively for the use of DomU's.
> 
> Can anyone advise what may be the issue why this is not working?
> 

I can't answer any of your questions directly, but have you tried using
the 'unbind' virtual file in /sys ?
(Continue reading)

Permalink | Reply | 1 comment |
headers
Grahame Kelly | 1 Aug 06:56
Gravatar

Binding hidden PCI devices to a transferred DomU

Hi All.

***
I am not sure if this should be addressed to this mailing-list rather  
then the developer mailing-list but I hope
that any developer is watching what go on in this list.  I know cross  
posting is frond upon.
***

Using "S" for source machine and "D" for destination machine  
classification. We can talk about S:DomU:2 (that is
machine "S" DomU number 2) - just to assist you understanding my  
questions.

Say I have 3 x DomU's running on source machine "S".  Additionally, I  
have a several x DomU's running on machine "D".
Assuming all Xen machines on the intranet are correctly configured for  
fall-over condition handling.
Again assuming that the hardware on both the "S" and "D" machines  
contain exactly the same plug-able h/w components,
that have been hidden correctly from Dom0 at boot time.

Let us assume for this discussion sake, that the PCI resources held on  
the "S" machine are labels namely
04:0.0, 05:0.1 and 06:0.0 (as found by "lspci" command).

Lets look at two cases:

Case 1 -
Where the PCI resource ID's on both machines match. i.e.: resource PCI
(Continue reading)

Permalink | Reply | 0 comments |
headers
Olivier B. | 1 Aug 12:48
Picon
Gravatar

dom0 unable to launch domU

Hi,

I have a problem with a dom0 which is unable to launch my domU : all my
dom0 are Debian Lenny amd64 with Debian kernel 2.6.26-2-xen-amd64 and
xen-hypervisor-3.2-1-amd64. Then all domU (PV) are Debian Lenny amd64
too, with vanilla kernel 2.6.29.6.

So, this new dom0 can't launch any domU which are working on an other
dom0. Software is same on both dom0, and hardware is near the same.

When I start the domU, I have this :
> root! nyrki:~# xm create /etc/xen/thundercat.cfg -c
> Using config file "/etc/xen/thundercat.cfg".
> Started domain thundercat

but the domU seem "freezed" : xm report a "r" status (it always stay in
running status), and there is no error in logs. But console didn't show
anything.
I repeat, this domU works like a charm on an other dom0. I try with LVM
and DRBD storage, without success.

 From xentop, I saw that :
  thundercat -----r        914  100.0     262144    3.1     262144       3.1     1    1        0        0    0        0        0        0 2149627
072

Have you an idea where can I found informations about why it is blocking
? Logs don't seem really usefull here. I put the content of xend.log
about the last "xm create" :

[2009-08-01 12:32:21 3310] DEBUG (XendDomainInfo:84)
(Continue reading)

Permalink | Reply | 4 comments |
headers
Olivier B. | 1 Aug 13:00
Picon
Gravatar

Re: dom0 unable to launch domU

well, this xend.log was not the good one, drbd devices was not present 
in the conf file.

so the conf file :
kernel      = '/boot/vmlinuz-2.6.29.6-dae-xen'
memory      = '256'
root        = '/dev/xvda2 ro'
disk        = [
                  'drbd:thundercat-swap,xvda1,w',
                  'drbd:thundercat-disk,xvda2,w',
              ]
name        = 'thundercat'
vif         = [ 'ip=91.121.*hidden*,mac=00:16:3E:3B:15:97' ]
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

and the logs :
[2009-08-01 12:55:25 3310] DEBUG (XendDomainInfo:84) 
XendDomainInfo.create(['vm', ['name', 'thundercat'], ['memory', '256'], 
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], ['on_crash', 
'restart'], ['vcpus', 1], ['on_xend_start', 'ignore'], ['on_xend_stop', 
'ignore'], ['image', ['linux', ['kernel', 
'/boot/vmlinuz-2.6.29.6-dae-xen'], ['root', '/dev/xvda2 ro']]], 
['device', ['vbd', ['uname', 'drbd:thundercat-swap'], ['dev', 'xvda1'], 
['mode', 'w']]], ['device', ['vbd', ['uname', 'drbd:thundercat-disk'], 
['dev', 'xvda2'], ['mode', 'w']]], ['device', ['vif', ['ip', 
'91.121.*hidden*'], ['mac', '00:16:3E:3B:15:97']]]])
[2009-08-01 12:55:25 3310] DEBUG (XendDomainInfo:1618) 
XendDomainInfo.constructDomain
(Continue reading)

Permalink | Reply | 3 comments |
headers
Olivier B. | 1 Aug 18:06
Picon
Gravatar

Re: dom0 unable to launch domU

So, the problem seem to come from my vanilla kernel : with the Debian 
kernel domU boot on this dom0.

I will try to find where is the problem. The RAID card is different on 
this dom0, but I was thinking this hadn't impact for domU.

Olivier

Olivier B. a écrit :
> well, this xend.log was not the good one, drbd devices was not present 
> in the conf file.
>
> so the conf file :
> kernel      = '/boot/vmlinuz-2.6.29.6-dae-xen'
> memory      = '256'
> root        = '/dev/xvda2 ro'
> disk        = [
>                  'drbd:thundercat-swap,xvda1,w',
>                  'drbd:thundercat-disk,xvda2,w',
>              ]
> name        = 'thundercat'
> vif         = [ 'ip=91.121.*hidden*,mac=00:16:3E:3B:15:97' ]
> on_poweroff = 'destroy'
> on_reboot   = 'restart'
> on_crash    = 'restart'
>
Permalink | Reply | 2 comments |
headers
Jeroen Groenewegen van der Weyden | 1 Aug 18:15
Gravatar

Re: Problem getting PCI devices hidden from Dom0

Hi, in my opinion the problem starts here (XEN) I/O virtualisation disabled 1) check the bios of vt-d is enabled 2) the grub boot sequence 3) pciback compiled into the kernel. mfg, jeroen

James Harper wrote:
Hi All. I have tried hiding two PCI cards from Dom0 on boot up in accordance with the documentation. Xen 3.3.1_18546_16-0.1 upon openSuSE-11.1 release. Using the following /boot/grub/menu.1st directives: title Xen -- openSUSE 11.1 - 2.6.27.25-0.1 root (hd0,0) kernel /boot/xen.gz module /boot/vmlinuz-2.6.27.25-0.1-xen root=/dev/disk/by-id/scsi-1AMCC_U15626923F6A26004196-part1
resume=/dev/
disk/by-id/scsi-1AMCC_U15626923F6A26004196-part2 splash=silent showopts vga=0x317 pciback.permissive pciback.hide-(04:00.0) (06:04.0) (08:0e.0) module /boot/initrd-2.6.27.25-0.1-xen after booting - if I run "lspci", Dom0 still sees the PCI devices I am trying to hide exclusively for the use of DomU's. Can anyone advise what may be the issue why this is not working?
I can't answer any of your questions directly, but have you tried using the 'unbind' virtual file in /sys ? In theory you should be able to unbind a module from the device (therefore making it available to pass through) even if it was bound on boot. It works great with usb but I've never tried it with pci so it may not work... usb drivers expect to have devices taken away from them without warning while most pci drivers don't so there may well be bugs that prevent it from working. Also, I've never used pci passthrough so it may be that the device needs to be hidden at a much lower level or something. The USB syntax is described in http://lwn.net/Articles/143397/, the PCI syntax shouldn't be that much different - I expect that the device name for a pci device would be something like '0000:00:0e.0' If you do try it and it works, please post back to the list as doing it that way might suck a lot less than having to fiddle around with boot scripts... James _______________________________________________ Xen-users mailing list Xen-users <at> lists.xensource.com http://lists.xensource.com/xen-users No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.392 / Virus Database: 270.13.38/2274 - Release Date: 07/31/09 05:58:00 

_______________________________________________
Xen-users mailing list
Xen-users <at> lists.xensource.com
http://lists.xensource.com/xen-users
Permalink | Reply | 0 comments |
headers
xensource | 1 Aug 20:10

Re: Network Interface Problems for DomU Firewall

Hi,

I ran with such a config for about 3 years on my home network without problem :
- Linux with shorewall in a domU
- PCI pass through for the ethernet card connected to internet.
- Two bridges : br-dmz and br-loc configured at the OS level on dom0. (disabled the network-bridge script).
- As all my dmz host were domU, there was no physical interface linked to the br-dmz bridge.
- All guests paravirtualized. (no virtualization support in my CPU at that time).

Nothing to say, this just worked. AFAIR, I had some problems with the pci passthrough that I solved by using a different brand for the ethernet card connected to internet. This is probably fixed now.

Some 5 months ago, I had to migrate to KVM/libvirt because of lack of support for ivtv and nvidia in a xen dom0. I had to use a bridge for the connection to internet interface, this works too.

François.


----- Original Message -----
From: "Christian Fischer" <Christian.Fischer <at> fischundfischer.com>
To: xen-users <at> lists.xensource.com
Sent: Friday, 31 July, 2009 21:46:04 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: Re: [Xen-users] Network Interface Problems for DomU Firewall

On Friday 31 July 2009, Tom Jensen wrote:
>
[snip]
>
> As I mentioned before, my ultimate goal is to configure a standard three
> interface firewall within the DomU.  Most of the information I have found
> on the subject suggests the most secure way to accomplish this is to
> dedicate the interface connected to the Internet to the DomU using PCI
> passthrough.  The other two interfaces (DMZ & LAN) would be virtual
> interfaces bridged to the Dom0.  I am open to other concepts for creating
> a firewall DomU if anyone cares to share their configurations.

How about to have the firewall inside dom0? If it hasn't more to do than
routing/firewalling i think a separate domU is a bit blown.

You could replace /etc/xen/scripts/network-bridge with a dummy script (always
exit 0, no interface renaming), create simple bridges eg. brnet (bridge
interfaces eth0), brlan/brdmz (no bridge interfaces, no ip) and add the domU
vifs to these bridges.

You could now firewall inside the bridges.

Have a look at http://www.shorewall.net/manpages/shorewall-hosts.html if you
use it. Works fine.

Christian

>
> > --
> > Fajar
>
> _______________________________________________
> Xen-users mailing list
> Xen-users <at> lists.xensource.com
> http://lists.xensource.com/xen-users



--
"Without music to decorate it, time is just a bunch of boring production
 deadlines or dates by which bills must be paid."
        --- Frank Vincent Zappa

_______________________________________________
Xen-users mailing list
Xen-users <at> lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users <at> lists.xensource.com
http://lists.xensource.com/xen-users
Permalink | Reply | 0 comments |
headers
Olivier B. | 2 Aug 02:17
Picon
Gravatar

Re: dom0 unable to launch domU

Hi,

I added "earlyprintk=xen" in cmdline options to can view kernel messages.

Here is the output :
Started domain thundercat
(early) last_map_addr: 100000000 end: 100000000
(early) No NUMA configuration found
(early) Faking a node at 0000000000000000-0000000100000000
(early) Bootmem setup node 0 0000000000000000-0000000100000000
(early)   NODE_DATA [0000000000001000 - 0000000000004fff]
(early)   bootmap [0000000000008000 -  0000000000027fff] pages 20
(early) (5 early reservations) ==> bootmem [0000000000 - 0100000000]
(early)   #0 [0000000000 - 0000001000]   BIOS data page(early)  ==> 
[0000000000 - 0000001000]
(early)   #1 [0001123000 - 0001130000]   XEN PAGETABLES(early)  ==> 
[0001123000 - 0001130000]
(early)   #2 [0000006000 - 0000008000]       TRAMPOLINE(early)  ==> 
[0000006000 - 0000008000]
(early)   #3 [0000200000 - 000091f838]    TEXT DATA BSS(early)  ==> 
[0000200000 - 000091f838]
(early)   #4 [0001130000 - 0001925000]          PGTABLE(early)  ==> 
[0001130000 - 0001925000]
(early) Zone PFN ranges:
(early)   DMA      0x00000000 -> 0x00001000
(early)   DMA32    0x00001000 -> 0x00100000
(early)   Normal   0x00100000 -> 0x00100000
(early) Movable zone start PFN for each node
(early) early_node_map[3] active PFN ranges
(early)     0: 0x00000000 -> 0x000000a0
(early)     0: 0x00000100 -> 0x00000920
(early)     0: 0x00001123 -> 0x00100000
(early) SMP: Allowing 1 CPUs, 0 hotplug CPUs
(early) No local APIC present
(early) PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
(early) PM: Registered nosave memory: 0000000000920000 - 0000000001123000
(early) PCI: Warning: Cannot find a gap in the 32bit address range
(early) PCI: Unassigned devices with 32bit resource registers may break!
(early) Allocating PCI resources starting at 100200000 (gap: 
100100000:400000)
(early) NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1
(early) PERCPU: Allocating 61440 bytes of per cpu data
(early) Built 1 zonelists in Node order, mobility grouping on.  Total 
pages: 1030267
(early) Policy zone: DMA32
(early) Kernel command line: root=/dev/xvda2 ro earlyprintk=xen
(early) Initializing CPU#0
(early) invalid opcode: 0000 [#1] (early) SMP (early)
(early) last sysfs file:
(early) CPU 0 (early)
(early) Modules linked in:(early)
(early) Pid: 0, comm: swapper Not tainted 2.6.29.6-dae-xen #1
(early) RIP: e030:[<ffffffff805c34b4>] (early)  [<ffffffff805c34b4>] 
xsave_cntxt_init+0xb5/0x194
(early) RSP: e02b:ffffffff80819e38  EFLAGS: 00010046
(early) RAX: 0000000000000003 RBX: ffffffff80819e44 RCX: 0000000000000000
(early) RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000042620
(early) RBP: ffffffff80819e68 R08: ffffffff80819e38 R09: ffffffff80819e3c
(early) R10: 00000000ffffffff R11: 000000000000e710 R12: ffffffff80819e40
(early) R13: ffffffff80819e3c R14: ffffffff80819e38 R15: ffff880001949000
(early) FS:  0000000000000000(0000) GS:ffffffff808cf000(0000) 
knlGS:0000000000000000
(early) CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
(early) CR2: 0000000000000000 CR3: 0000000000201000 CR4: 0000000000002620
(early) DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
(early) DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
(early) Process swapper (pid: 0, threadinfo ffffffff80818000, task 
ffffffff80796340)
(early) Stack:
(early)  0000024000000000(early)  0000000300000240(early)  
0000000080050033(early)  0000000000000000(early)
(early)  ffffffff808cd000(early)  0000000000000005(early)  
ffffffff80819e78(early)  ffffffff805dab4f(early)
(early)  ffffffff80819e98(early)  ffffffff805dabcb(early)  
ffff88008109e000(early)  00000000ffff8800(early)
(early) Call Trace:
(early)  [<ffffffff805dab4f>] init_thread_xstate+0x12/0x20
(early)  [<ffffffff805dabcb>] fpu_init+0x6e/0xbe
(early)  [<ffffffff805dc728>] cpu_init+0x310/0x33c
(early)  [<ffffffff808339c5>] trap_init+0x5b0/0x5b7
(early)  [<ffffffff8082cb24>] start_kernel+0x1cb/0x365
(early)  [<ffffffff8082c2c1>] x86_64_start_reservations+0xac/0xb0
(early)  [<ffffffff80832dfd>] xen_start_kernel+0x814/0x823
(early) Code: (early) e9 (early) 30 (early) 00 (early) 00 (early) 00 
(early) 04 (early) 00 (early) ff (early) 15 (early) ba (early) be 
(early) 1d (early) 00 (early) 89 (early) c7 (early) 81 (early) cf 
(early) 00 (early) 00 (early) 04 (early) 00 (early) ff (early) 15 
(early) b4 (early) be (early) 1d (early) 00 (early) 31 (early) c9 
(early) 48 (early) 8b (early) 05 (early) c3 (early) 07 (early) 31 
(early) 00 (early) 48 (early) 89 (early) c2 (early) 48 (early) c1 
(early) ea (early) 20 (early) <0f> (early) 01 (early) d1 (early) c7 
(early) 45 (early) dc (early) 0d (early) 00 (early) 00 (early) 00 
(early) c7 (early) 45 (early) d4 (early) 00 (early) 00 (early) 00 
(early) 00 (early) 48 (early) 89 (early) df (early) 4c (early)
(early) RIP (early)  [<ffffffff805c34b4>] xsave_cntxt_init+0xb5/0x194
(early)  RSP <ffffffff80819e38>
(early) ---[ end trace 4eaa2a86a8e2da22 ]---
(early) Kernel panic - not syncing: Attempted to kill the idle task!
(early) ------------[ cut here ]------------
(early) WARNING: at kernel/smp.c:329 smp_call_function_many+0x40/0x1f7()
(early) Modules linked in:(early)
(early) Pid: 0, comm: swapper Tainted: G      D    2.6.29.6-dae-xen #1
(early) Call Trace:
(early)  [<ffffffff802407b7>] warn_slowpath+0xaf/0xd6
(early)  [<ffffffff805e3c3d>] ? _spin_unlock_irqrestore+0x18/0x1c
(early)  [<ffffffff805e3c3d>] ? _spin_unlock_irqrestore+0x18/0x1c
(early)  [<ffffffff80240e11>] ? release_console_sem+0x18f/0x1c4
(early)  [<ffffffff80241322>] ? vprintk+0x2b9/0x2e6
(early)  [<ffffffff802606ff>] smp_call_function_many+0x40/0x1f7
(early)  [<ffffffff802608d3>] smp_call_function+0x1d/0x21
(early)  [<ffffffff8020e2a2>] xen_smp_send_stop+0x14/0x16
(early)  [<ffffffff805e14d5>] panic+0x7d/0x11e
(early)  [<ffffffff80243a07>] do_exit+0x71/0x67f
(early)  [<ffffffff802142cd>] oops_end+0x97/0x9c
(early)  [<ffffffff80214484>] die+0x55/0x5e
(early)  [<ffffffff80212192>] do_trap+0x115/0x124
(early)  [<ffffffff80212551>] do_invalid_op+0x98/0xa1
(early)  [<ffffffff805c34b4>] ? xsave_cntxt_init+0xb5/0x194
(early)  [<ffffffff805e3c3d>] ? _spin_unlock_irqrestore+0x18/0x1c
(early)  [<ffffffff805e3c3d>] ? _spin_unlock_irqrestore+0x18/0x1c
(early)  [<ffffffff80240e11>] ? release_console_sem+0x18f/0x1c4
(early)  [<ffffffff8021151b>] invalid_op+0x1b/0x20
(early)  [<ffffffff805c34b4>] ? xsave_cntxt_init+0xb5/0x194
(early)  [<ffffffff805c34a4>] ? xsave_cntxt_init+0xa5/0x194
(early)  [<ffffffff805dab4f>] init_thread_xstate+0x12/0x20
(early)  [<ffffffff805dabcb>] fpu_init+0x6e/0xbe
(early)  [<ffffffff805dc728>] cpu_init+0x310/0x33c
(early)  [<ffffffff808339c5>] trap_init+0x5b0/0x5b7
(early)  [<ffffffff8082cb24>] start_kernel+0x1cb/0x365
(early)  [<ffffffff8082c2c1>] x86_64_start_reservations+0xac/0xb0
(early)  [<ffffffff80832dfd>] xen_start_kernel+0x814/0x823
(early) ---[ end trace 4eaa2a86a8e2da23 ]---

So I have :
 > (early) Initializing CPU#0
 > (early) invalid opcode: 0000 [#1] (early) SMP (early)

Should I consider it has a "CPU bug" ?
It's a Xeon E5405 stepping 10. I don't think it's so special, but my 
other dom0 seems to be Xeon E5405 stepping 6.

Olivier

PS : is the "xen-devel" list more appropriate ?

Olivier B. a écrit :
> So, the problem seem to come from my vanilla kernel : with the Debian 
> kernel domU boot on this dom0.
>
> I will try to find where is the problem. The RAID card is different on 
> this dom0, but I was thinking this hadn't impact for domU.
>
> Olivier
Permalink | Reply | 1 comment |
headers
Olivier B. | 2 Aug 03:45
Picon
Gravatar

Re: dom0 unable to launch domU [RESOLVED]

Sorry for noise, it is a known bug with 2.6.28 and 2.6.29 vanilla kernels :
    http://lists.xensource.com/archives/html/xen-bugs/2009-04/msg00005.html

So, I have to upgrade to 2.6.30.

Olivier
Permalink | Reply | 0 comments |

Gmane