Question about pf, NAT and routing. (attempt #2)

For some reason my other message never made it through, so here we are 
again!

A while ago I needed to give a jail access to two networks/IPs (one 
external for a web site and one internal for a DB connection). I ended 
up using a localhost IP (127.0.0.2) for the jail and nat/binat in pf to 
control where the traffic went, depending on destination. I'm trying to 
set up a jail now that's similar. My host has multiple interfaces on 
multiple networks, and the jail is on lo0, and I would like to NAT 
traffic to internal networks from one IP out one interface, and 
everything else out another IP through the external interface.

I found an email on here from jpaetzel (o/) explaining how to use 
route-to, and that works; it fixed default route problem (thanks!). 
Unfortunately that only seems to work if the jail is using an IP on one 
of the interfaces in question. I suppose there is some sort of problem 
between the NATing and routing.

Here is my crazy config:

ra# ifconfig
bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 
1500 
options=1bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4>
         ether 00:1e:0b:ed:f9:ec
         media: Ethernet autoselect (1000baseTX <full-duplex>)
         status: active
         lagg: laggdev lagg0
bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

Re: FreeBSD 7 and multiple IP (mijail-patch in 6.x)

On Mon, 31 Mar 2008, Johan Ström wrote:

Hi,

> I got a machine running 6.2 right now, which is being replaced. And since SMP 
> performance is much better on 7.x I'd like to go with 7.0 (and many ppl have 
> indeed verified that it works good on this box, HP DL360 G5)...
> But, now when I start to setup the machine, I recalled that i've patched the 
> 6.2 box with the freebsd mijail patch 
> (http://www.digitaldaemon.com/FreeBSD/FreeBSD/FreeBSD_6.2-STABLE-mijail.patch).
> However, I cannot find anywhere about FreeBSD 7 and a similar patch. A quick 
> look at the patch vs the 7.x source tells me it won't apply cleanly, but from 
> what I've seen quickly, it could maybe be done. The differences I've seen 
> doesn't look too advanced, but then again, I'm  not a kernel developer...
>
> So, I'd like to know if anyone considered this on 7.x, or if anyone can tell 
> me immediately that this wont work or will be LOTS of work, or just some 
> patch line adjusting? Ie, how big are the changes from 6.x to 7.x in these 
> sections?

I had planned to have a patch for multiv4/v6 jails last month but it's not
yet publicly available. I have sent it off to some people for review.

In case the above is a successor of pjd's multi-ip v4 jail patch I can
give you a plain forward port to a FreeBSD 7 system (which might have
possible locking issues I have never experienced).

All depends on how quickly you need it.

/bz

Re: FreeBSD 7 and multiple IP (mijail-patch in 6.x)

On Apr 3, 2008, at 8:39 PM, Bjoern A. Zeeb wrote:

> On Mon, 31 Mar 2008, Johan Ström wrote:
>
> Hi,
>
>> I got a machine running 6.2 right now, which is being replaced. And  
>> since SMP performance is much better on 7.x I'd like to go with 7.0  
>> (and many ppl have indeed verified that it works good on this box,  
>> HP DL360 G5)...
>> But, now when I start to setup the machine, I recalled that i've  
>> patched the 6.2 box with the freebsd mijail patch
(http://www.digitaldaemon.com/FreeBSD/FreeBSD/FreeBSD_6.2-STABLE-mijail.patch 
>> ).
>> However, I cannot find anywhere about FreeBSD 7 and a similar  
>> patch. A quick look at the patch vs the 7.x source tells me it  
>> won't apply cleanly, but from what I've seen quickly, it could  
>> maybe be done. The differences I've seen doesn't look too advanced,  
>> but then again, I'm  not a kernel developer...
>>
>> So, I'd like to know if anyone considered this on 7.x, or if anyone  
>> can tell me immediately that this wont work or will be LOTS of  
>> work, or just some patch line adjusting? Ie, how big are the  
>> changes from 6.x to 7.x in these sections?
>
> I had planned to have a patch for multiv4/v6 jails last month but  
> it's not
> yet publicly available. I have sent it off to some people for review.
>
> In case the above is a successor of pjd's multi-ip v4 jail patch I can

Jail patch submission

This is my first submission to freebsd so I hope this
is the right way to do it. I have attached a patch
that I would like to submit to expand the capability
of the /etc/rc.d/jail

This patch allows for taking a netmask variable from
the rc.conf to build the jail with. It also has the
ability to write in a default netmask if no variable
is specified. This has been tested with the defaults
and custom jail settings. 

I hope this works for everyone else. Feedback is
always appreciated.

      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com

This is my first submission to freebsd so I hope this
is the right way to do it. I have attached a patch
that I would like to submit to expand the capability
of the /etc/rc.d/jail

This patch allows for taking a netmask variable from
the rc.conf to build the jail with. It also has the
ability to write in a default netmask if no variable
is specified. This has been tested with the defaults
and custom jail settings. 

freebsd-jail@...

Current FreeBSD problem reports
Critical problems
Serious problems

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
s kern/89528   jail       [jail] [patch] impossible to kill a jail
o kern/119842  jail       [smbfs] [jail] "Bad address" with smbfs inside a jail

2 problems total.

Non-critical problems

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o bin/32828    jail       [jail] w(1) incorrectly handles stale utmp slots with 
o kern/68192   jail       [quotas] [jail] Cannot use quotas on jailed systems
o kern/72498   jail       [libc] [jail] timestamp code on jailed SMP machine gen
o kern/74314   jail       [resolver] [jail] DNS resolver broken under certain ja
o kern/84215   jail       [jail] [patch] wildcard ip (INADDR_ANY) should not bin
o kern/89989   jail       [jail] [patch] Add option -I (ASCII 73) PID  to specif
o kern/97071   jail       [jail] [patch] add security.jail.jid sysctl
o bin/99566    jail       [jail] [patch] fstat(1) according to specified jid
o bin/119305   jail       [jail] [patch] jexec(8): jexec -n prisonname: selectio
o kern/120753  jail       [jail] Zombie jails (jailed child process exits while 
o kern/122270  jail       [jail] [patch] jail numbers keep incrementing

11 problems total.

Re: Jail patch submission

Quoting William Bentley <gwydion_1@...> (from Sun, 6 Apr 2008  
12:37:40 -0700 (PDT)):

> This is my first submission to freebsd so I hope this
> is the right way to do it. I have attached a patch
> that I would like to submit to expand the capability
> of the /etc/rc.d/jail
>
> This patch allows for taking a netmask variable from
> the rc.conf to build the jail with. It also has the
> ability to write in a default netmask if no variable
> is specified. This has been tested with the defaults
> and custom jail settings.
>
> I hope this works for everyone else. Feedback is
> always appreciated.

To have it in the base system, it needs to be documented at the place  
where the other jail variables are documented. It's also better to  
send-pr it (or by using gtk-send-pr or the webinterface at our  
homepage).

Bye,
Alexander.

--

-- 
BOFH excuse #357:

I'd love to help you -- it's just that the Boss won't let me near the  
computer.

Re: kern/122270: [jail] [patch] jail numbers keep incrementing

Synopsis: [jail] [patch] jail numbers keep incrementing

State-Changed-From-To: open->patched
State-Changed-By: delphij
State-Changed-When: Fri Apr 11 21:32:08 UTC 2008
State-Changed-Why: 
Committed against -HEAD, MFC reminder.

Responsible-Changed-From-To: freebsd-jail-≥delphij
Responsible-Changed-By: delphij
Responsible-Changed-When: Fri Apr 11 21:32:08 UTC 2008
Responsible-Changed-Why: 
Take.

http://www.freebsd.org/cgi/query-pr.cgi?pr=122270

freebsd-jail@...

Current FreeBSD problem reports
Critical problems
Serious problems

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
s kern/89528   jail       [jail] [patch] impossible to kill a jail
o kern/119842  jail       [smbfs] [jail] "Bad address" with smbfs inside a jail

2 problems total.

Non-critical problems

S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o bin/32828    jail       [jail] w(1) incorrectly handles stale utmp slots with 
o kern/68192   jail       [quotas] [jail] Cannot use quotas on jailed systems
o kern/72498   jail       [libc] [jail] timestamp code on jailed SMP machine gen
o kern/74314   jail       [resolver] [jail] DNS resolver broken under certain ja
o kern/84215   jail       [jail] [patch] wildcard ip (INADDR_ANY) should not bin
o kern/89989   jail       [jail] [patch] Add option -I (ASCII 73) PID  to specif
o kern/97071   jail       [jail] [patch] add security.jail.jid sysctl
o bin/99566    jail       [jail] [patch] fstat(1) according to specified jid
o bin/119305   jail       [jail] [patch] jexec(8): jexec -n prisonname: selectio
o kern/120753  jail       [jail] Zombie jails (jailed child process exits while 

10 problems total.

ypserv in a jail?

Anybody ever set up a NIS server in a jail?  I'm running a jail on a 6.2
system.  When I run ypserv on the parent things work great.  When I shut down
ypserv on the parent and bring it up in the jail, it comes up fine but never
answers any of the broadcasts.  I have ruled out any firewalling.  I have also
tried forcing ypbind to look directly at the jailed ypserv with the -S option.
That was ineffective.  I have toyed with sysctl variables and have gone as far
as:

security.jail.allow_raw_sockets=1
security.jail.sysvipc_allowed=1
security.jail.set_hostname_allowed=1
security.jail.socket_unixiproute_only=0

Doign a ps on the parent shows the jailed ypserv, sockstat shows ypserv
listening on the ports.

Thoughts on what I'm missing?

--
  Randy    (schulra@...)      765.983.1283         <*>

Love with your heart, think with your head;  not the other way around.

freebsd-update on jails

  I previously posted a howto to use zfs to manage jails.  The first
update through freebsd-update has been released.  Testing this I get

(in jail)
ldap1#freebsd-update install
Installing updates...chflags: ///usr/lib/libssh.a: Operation not
supported

After this error I enabled chflags in sysctl on the host system
#sysctl security.jail.chflags_allowed=1

This did not not work in fixing the issue
after reading the freebsd-update man page I thought this should be
possible

#freebsd-update -b /jails/ldap1/ -d /jails/ldap1/var/db/freebsd-update/
\ install
Installing updates...chflags: /jails/ldap1///usr/lib/libssh.a: Operation
not supported

But I still get that same error.  Does anyone have any idea what would
keep this from working?  If there is a way to update the host and all
subsequent jails vi the host that would be great, as i would prefer not
to allow chflags from within the jails.

thanks in advance