headers
Randy Schultz | 1 Oct 2007 21:18

djbdns on 1270.0.1 in a jail problem

Heya,

Playing around with jails and have run across something weird, I was wondering 
if somebody could explain.

I'm trying to get djbdns to run inside the jail, with tinydns running on
127.0.0.1.  The thing I cannot figure out is why tinydns always comes up on
the jail's IP address, and not lo0, as reported by sockstat: 
Root Dude ? sockstat -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS 
root     sshd       863   3  tcp4   159.28.1.59:22        *:*
tinydns  tinydns    862   3  udp4   159.28.1.59:53        *:*
root     syslogd    800   4  dgram  /var/run/log
root     syslogd    800   5  dgram  /var/run/logpriv
root     syslogd    800   6  udp4   159.28.1.59:514       *:*
root     sshd       638   3  tcp4   159.28.1.66:22        *:*
root     syslogd    530   4  dgram  /var/run/log
root     syslogd    530   5  dgram  /var/run/logpriv
root     syslogd    530   6  udp6   *:514                 *:*
root     syslogd    530   7  udp4   *:514                 *:*
root     devd       464   4  stream /var/run/devd.pipe

My setup(really just a standard install) runs fine on a non-jailed system,
tinydns comes up on 127.0.0.1.  The jail does have the correct env setting:
[root <at> opal /]# cat /service/tinydns/env/IP
127.0.0.1

At first I thought it was because lo0 was not in /dev in the jail.  I've gone
as far as unhiding *everything* in /dev via:
Root Dude ? cat /etc/devfs.rules
(Continue reading)

Permalink | Reply | 2 comments |
headers
Alain Wolf | 2 Oct 2007 02:03
Picon
Favicon

Re: djbdns on 1270.0.1 in a jail problem

Randy Schultz wrote:
> Heya,
>
> Playing around with jails and have run across something weird, I was
wondering if somebody could explain.
>
> I'm trying to get djbdns to run inside the jail, with tinydns running on
127.0.0.1.  The thing I cannot figure out is why tinydns always comes up
on
> the jail's IP address, and not lo0, as reported by sockstat: Root Dude ?

Hi Randy,

I fell in the same hole on my first setup.
There is no such thing as 127.0.0.1 in a FreeBSD Jail.
There is just the IP, which the Jail is configured for.
I am not a developer, but as far as I understand, a Jail and its IP, is
some kind of virtualization, which can not contain any virtualized
environment inside itself again. At least not in 6.x

So it looks that 127.0.0.1 would be an additional IP like any other one,
which is NOT possible in FreeBSD Jails.

I read promising things about a fully virtualized IP environment in
FreeBSD 7.x, where we can do a lot more than this, but we have to wait for
that.

After I realized that, I redesigned my plans and I liked them even better.
My DJB-DNS setup is now as follows, and works flawless.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Randy Schultz | 2 Oct 2007 05:09

Re: djbdns on 1270.0.1 in a jail problem

On Tue, 2 Oct 2007, Alain Wolf spaketh thusly:

-}Hi Randy,
-}
-}I fell in the same hole on my first setup.
-}There is no such thing as 127.0.0.1 in a FreeBSD Jail.
-}There is just the IP, which the Jail is configured for.
-}I am not a developer, but as far as I understand, a Jail and its IP, is
-}some kind of virtualization, which can not contain any virtualized
-}environment inside itself again. At least not in 6.x
-}
-}So it looks that 127.0.0.1 would be an additional IP like any other one,
-}which is NOT possible in FreeBSD Jails.
-}
-}I read promising things about a fully virtualized IP environment in
-}FreeBSD 7.x, where we can do a lot more than this, but we have to wait for
-}that.
-}
-}After I realized that, I redesigned my plans and I liked them even better.
-}My DJB-DNS setup is now as follows, and works flawless.
-}
-}dnscache runs in its own Jail in every physical machine, caching DNS
-}queries for all other Jails on the same machine.
-}
-}Two copies of TinyDNS run each in its own Jail too. Providing a (rather
-}expensive) Split-Horizon DNS Solution.
-}
-}Hope this helps

Indeed it does.  Tnx heaps and loads Alain.  Now I can stop focusing on
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tom Evans | 3 Oct 2007 18:25

Cannot ssh from jail

Hi stable <at> , jail <at>  [jail <at>  plz cc me as I'm not subscribed]

I'm having some problems setting up some jails for semi-isolated
development (ie, so we can isolate the developers into a jail, give them
all the root access they want, and not worry about them blowing up more
than their own jail) on 6.2-RELEASE-p5.

I have set up a jail, using ezjail, which appeared to work fine. I can
start the jail, and use jexec to spawn a shell inside the jail. However,
if I then try to ssh from the jail to another box, ssh fails with the
error message (with -v):

 debug1: read_passphrase: can't open /dev/tty: Device busy
 Host key verification failed.

The only ezjail.conf option I changed/added from default was to set
ezjail_jaildir. I left ezjail_devfs_enable="YES",
ezjail_devfs_ruleset="devfsrules_jail", the defaults.

From outside the jail, devfs appears to be mounted:

 /data2/ezjails/basejail on /data2/ezjails/monotest/basejail (nullfs,
local, read-only)
 devfs on /data2/ezjails/monotest/dev (devfs, local)
 fdescfs on /data2/ezjails/monotest/dev/fd (fdescfs)
 procfs on /data2/ezjails/monotest/proc (procfs, local)

From inside the jail, there doesn't appear to be a /dev/tty, unless you
look for it:
 # ls /dev
(Continue reading)

Permalink | Reply | 0 comments |
headers
LI Xin | 3 Oct 2007 18:57

Re: Cannot ssh from jail

Tom Evans wrote:
> Hi stable <at> , jail <at>  [jail <at>  plz cc me as I'm not subscribed]
> 
> I'm having some problems setting up some jails for semi-isolated
> development (ie, so we can isolate the developers into a jail, give them
> all the root access they want, and not worry about them blowing up more
> than their own jail) on 6.2-RELEASE-p5.
> 
> I have set up a jail, using ezjail, which appeared to work fine. I can
> start the jail, and use jexec to spawn a shell inside the jail. However,
> if I then try to ssh from the jail to another box, ssh fails with the
> error message (with -v):

I think the problem is that if you jexec into a jail then you don't have
a TTY at hand, so bad things would happen.  If you login into the jail
by some ways (e.g. by ssh or telnet or whatever that spawns a TTY for
you) then it would work I bet.

Cheers,
--

-- 
Xin LI <delphij <at> delphij.net>	http://www.delphij.net/
FreeBSD - The Power to Serve!
Permalink | Reply | 0 comments |
headers
Tom Evans | 4 Oct 2007 10:20

Re: Cannot ssh from jail

On Thu, 2007-10-04 at 10:17 +0200, Kim Attree wrote:
> LI Xin wrote:
> > Tom Evans wrote:
> >   
> >> Hi stable <at> , jail <at>  [jail <at>  plz cc me as I'm not subscribed]
> >>
> >> I'm having some problems setting up some jails for semi-isolated
> >> development (ie, so we can isolate the developers into a jail, give them
> >> all the root access they want, and not worry about them blowing up more
> >> than their own jail) on 6.2-RELEASE-p5.
> >>
> >> I have set up a jail, using ezjail, which appeared to work fine. I can
> >> start the jail, and use jexec to spawn a shell inside the jail. However,
> >> if I then try to ssh from the jail to another box, ssh fails with the
> >> error message (with -v):
> >>     
> >
> > I think the problem is that if you jexec into a jail then you don't have
> > a TTY at hand, so bad things would happen.  If you login into the jail
> > by some ways (e.g. by ssh or telnet or whatever that spawns a TTY for
> > you) then it would work I bet.
> >
> > Cheers,
> >   
> I had the same problem, setup SSHD in the jail, ssh'ed into that, and
> then from there got a TTY and could ssh to anywhere.
> 
> Li is right, with jexec you don't get allocated a TTY.
> 
> Laters,
(Continue reading)

Permalink | Reply | 0 comments |
headers
Kim Attree | 4 Oct 2007 10:17

Re: Cannot ssh from jail

LI Xin wrote:
> Tom Evans wrote:
>   
>> Hi stable <at> , jail <at>  [jail <at>  plz cc me as I'm not subscribed]
>>
>> I'm having some problems setting up some jails for semi-isolated
>> development (ie, so we can isolate the developers into a jail, give them
>> all the root access they want, and not worry about them blowing up more
>> than their own jail) on 6.2-RELEASE-p5.
>>
>> I have set up a jail, using ezjail, which appeared to work fine. I can
>> start the jail, and use jexec to spawn a shell inside the jail. However,
>> if I then try to ssh from the jail to another box, ssh fails with the
>> error message (with -v):
>>     
>
> I think the problem is that if you jexec into a jail then you don't have
> a TTY at hand, so bad things would happen.  If you login into the jail
> by some ways (e.g. by ssh or telnet or whatever that spawns a TTY for
> you) then it would work I bet.
>
> Cheers,
>   
I had the same problem, setup SSHD in the jail, ssh'ed into that, and
then from there got a TTY and could ssh to anywhere.

Li is right, with jexec you don't get allocated a TTY.

Laters,
(Continue reading)

Permalink | Reply | 1 comment |
headers
Bill Moran | 9 Oct 2007 16:16
Favicon

Mysterious jail lockups


Has anyone else seen this?

The symptoms are a jail that has no processes in it, and thus can not
be stopped/killed/whatever.  Only solution is to reboot the host system.
Trying to jexec into the jail results in an error, so new processes can't
be started therein.

It doesn't happen very often, and I've been unable to reproduce it on
demand.  What I'm looking for at this point are whether or not anyone
else has seen this, and advice on how to track it down/reproduce it, with
the eventual goal of fixing the problem.

It would be nice if there were a command, let's say "jkill" that killed
the _jail_.  There is a port called jkill that (allegedly) does this, but
looking at the perl code, all it does it loop through a ps listing
killing off processes.  In the event of a jail with no processes, this
doesn't help any.

Theoretically, this would be some sort of kernel bug, whereby the
reference counter to the jail is not properly decremented when processes
die and thus the jail never shuts down.  Given the infrequency of the
occurrence and my inability to produce a reproducible case, I expect
it to be challenging to track down.

Any advice?

--

-- 
Bill Moran
Collaborative Fusion Inc.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Bill Moran | 9 Oct 2007 21:49
Favicon

Re: Mysterious jail lockups

In response to Kurt Jaeger <pi@...>:

> Hi!
> 
> > Has anyone else seen this?
> > 
> > The symptoms are a jail that has no processes in it, and thus can not
> > be stopped/killed/whatever.  Only solution is to reboot the host system.
> > Trying to jexec into the jail results in an error, so new processes can't
> > be started therein.
> 
> Have a look at this:
> 
> http://g-rave.nl/junk/freebsd-jail-nodevdep.diff
> wg. http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/89528

Thanks, Kurt.  I'm following up to see if I can get someone interested
in fixing this who knows that part of the code.  Any suggestions?

--

-- 
Bill Moran
Collaborative Fusion Inc.
http://people.collaborativefusion.com/~wmoran/

wmoran@...
Phone: 412-422-3463x4023
Permalink | Reply | 0 comments |
headers
D Hill | 9 Oct 2007 21:39

Re: Mysterious jail lockups

On Tue, 9 Oct 2007 at 10:16 -0400, wmoran@... confabulated:

> Has anyone else seen this?
>
> The symptoms are a jail that has no processes in it, and thus can not
> be stopped/killed/whatever.  Only solution is to reboot the host system.
> Trying to jexec into the jail results in an error, so new processes can't
> be started therein.
>
> It doesn't happen very often, and I've been unable to reproduce it on
> demand.  What I'm looking for at this point are whether or not anyone
> else has seen this, and advice on how to track it down/reproduce it, with
> the eventual goal of fixing the problem.
>
> It would be nice if there were a command, let's say "jkill" that killed
> the _jail_.  There is a port called jkill that (allegedly) does this, but
> looking at the perl code, all it does it loop through a ps listing
> killing off processes.  In the event of a jail with no processes, this
> doesn't help any.
>
> Theoretically, this would be some sort of kernel bug, whereby the
> reference counter to the jail is not properly decremented when processes
> die and thus the jail never shuts down.  Given the infrequency of the
> occurrence and my inability to produce a reproducible case, I expect
> it to be challenging to track down.
>
> Any advice?

Same thing seen here running:
(Continue reading)

Permalink | Reply | 0 comments |

Gmane