headers
Stephen Bosch | 2 Jun 2004 00:56
Picon

peculiar dnscache hangs

Hello everyone...

I've done my very best to find the source of this particular problem,
but I am stumped.

I have dnscache configured as an external cache on a machine that is
connected to a bridging firewall but which has only a public IP address.

First thing to note -- I suspect this is important -- I can't
get dnscache to reply to queries sent to the loopback address. It only
works if I talk to the external address.

Here's what's in /etc/dnscache/root/ip:

>  292198 -rw-------    1 root     root            0 May 10 11:34 127.0.0.1
>  292237 -rw-r--r--    1 root     root            0 May 10 11:35 192.168
>  292241 -rw-r--r--    1 root     root            0 May 10 12:03 209.115.249.136
>  292240 -rw-r--r--    1 root     root            0 May 10 12:02 209.115.249.138
>  292242 -rw-r--r--    1 root     root            0 May 10 12:03 209.115.249.161

But here's the main reason I need help:

Every once in a while (I'm not certain of the exact interval) dnscache
will simply hang. The hang is usually brief -- I've never seen it last
more than a minute -- and it's not fatal, but it's annoying.

If I look at the logs while this is happening (tail -f 
/etc/dnscache/log/main/current | tai64nlocal) while my clients make 
resolution attempts ("resolving www.blah.com"), I see no evidence of the 
query attempt in the logs. There's no evidence that clients are
(Continue reading)

Permalink | Reply | 542 comments |
headers
Stephen Bosch | 2 Jun 2004 05:14
Picon

Re: peculiar dnscache hangs

Charlie Brady wrote:
> On Tue, 1 Jun 2004, Stephen Bosch wrote:
> 
> 
>>First thing to note -- I suspect this is important -- I can't
>>get dnscache to reply to queries sent to the loopback address. It only
>>works if I talk to the external address.
> 
> 
> dnscache only binds to a single IP address. It'll only be accessible via 
> the IP address that you configure it to run on. [Unless you do something 
> fancy with packet forwarding, such as iptables DNAT.] 
> 
> If you want dnscache to be accessible via the loopback address and the
> external address, you might be able to do so by specifying IP=0.0.0.0 (I
> don't know, I haven't tried). Or you could run two instances, one on the
> loopback and one on the external address. Or you could use DNAT. Or you
> could not bother - I can't think why you'd need the resolver to be
> accessible via multiple IP addresses.

Well, I personally don't care. I thought it might be related to the 
problem I am experiencing. If not -- great!

>>If I look at the logs while this is happening (tail -f 
>>/etc/dnscache/log/main/current | tai64nlocal) while my clients make 
>>resolution attempts ("resolving www.blah.com"), I see no evidence of the 
>>query attempt in the logs. There's no evidence that clients are 
>>connecting at all.
> 
>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Charlie Brady | 2 Jun 2004 04:57

Re: peculiar dnscache hangs


On Tue, 1 Jun 2004, Stephen Bosch wrote:

> First thing to note -- I suspect this is important -- I can't
> get dnscache to reply to queries sent to the loopback address. It only
> works if I talk to the external address.

dnscache only binds to a single IP address. It'll only be accessible via 
the IP address that you configure it to run on. [Unless you do something 
fancy with packet forwarding, such as iptables DNAT.] 

If you want dnscache to be accessible via the loopback address and the
external address, you might be able to do so by specifying IP=0.0.0.0 (I
don't know, I haven't tried). Or you could run two instances, one on the
loopback and one on the external address. Or you could use DNAT. Or you
could not bother - I can't think why you'd need the resolver to be
accessible via multiple IP addresses.

> Every once in a while (I'm not certain of the exact interval) dnscache
> will simply hang. The hang is usually brief -- I've never seen it last
> more than a minute -- and it's not fatal, but it's annoying.
> 
> If I look at the logs while this is happening (tail -f 
> /etc/dnscache/log/main/current | tai64nlocal) while my clients make 
> resolution attempts ("resolving www.blah.com"), I see no evidence of the 
> query attempt in the logs. There's no evidence that clients are 
> connecting at all.

What does strace say is happening with the dnstrace process during 
the apparent pauses?
(Continue reading)

Permalink | Reply | 4 comments |
headers
John Peacock | 2 Jun 2004 17:33

Re: peculiar dnscache hangs

Stephen Bosch wrote:
> Every once in a while (I'm not certain of the exact interval) dnscache
> will simply hang. The hang is usually brief -- I've never seen it last
> more than a minute -- and it's not fatal, but it's annoying.
> 
> Does this sound familiar to anybody?

Yes.  Check your disk quotas.  I had this happen on a box and it made me 
crazy.  It turned out that when dnscache was in the process of rotating 
the logs, it would go just over the quota that was set for that user. 
That would cause dnscache to hang.  I don't remember it ever fixing 
itself; I usually wound up rebooting at that point, since restarting 
wouldn't help.  Once I turned off quotas, the problem disappeared.

One of the basic design features of multilog is that is assumes you are 
smart enough to allow it enough storage to support the number of logs 
which you have preset.  If multilog cannot write to the logfile, it will 
hang, which will cause dnscache (in this case) to hang as well.  It is 
possible that there is some other problem with your system which is 
preventing the log files from being written, but this is exactly the 
behavior I have seen before.

John

--

-- 
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
(Continue reading)

Permalink | Reply | 541 comments |
headers
akdns | 2 Jun 2004 21:54
Picon

RE: peculiar dnscache hangs


> > First thing to note -- I suspect this is important -- I can't
> > get dnscache to reply to queries sent to the loopback address. It only
> > works if I talk to the external address.
>
> dnscache only binds to a single IP address. It'll only be accessible via
> the IP address that you configure it to run on. [Unless you do something
> fancy with packet forwarding, such as iptables DNAT.]
>
> If you want dnscache to be accessible via the loopback address and the
> external address, you might be able to do so by specifying IP=0.0.0.0 (I
> don't know, I haven't tried). Or you could run two instances, one on the
> loopback and one on the external address. Or you could use DNAT. Or you
> could not bother - I can't think why you'd need the resolver to be
> accessible via multiple IP addresses.

There's a patch on tinydns.org that makes it possible to bind to multiple IP addresses. Beware - there are
separate patches for
tinydns and dnscache.

Regards,
Andrzej Kukula
Permalink | Reply | 0 comments |
headers
travis@norris.net | 3 Jun 2004 01:40

dns data file help (I believe)

Hello - any help would be greatly appeciated.

I have the following setup on two separate servers.

dnscache pointed to an internal address 10.7.7.2 and tinydns to 
127.0.0.1 on a.ns.example.com &
dnscache pointed to an internal address 10.7.7.4 and tinydns to 
127.0.0.1 on b.ns.example.com

dnscache and tinydns appear to be working fine.  I can dig anything 
externally:

-bash-2.05b$ dig  <at> 10.7.7.4 bored.com

; <<>> DiG 8.3 <<>>  <at> 10.7.7.4 bored.com
; (1 server found)
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUERY SECTION:
;;      bored.com, type = A, class = IN

;; ANSWER SECTION:
bored.com.              7h46m40s IN A   207.44.240.79

;; Total query time: 185 msec
;; FROM: b.ns.example.com to SERVER: 10.7.7.4  10.7.7.4
;; WHEN: Wed Jun  2 23:27:15 2004
;; MSG SIZE  sent: 27  rcvd: 43
(Continue reading)

Permalink | Reply | 1 comment |
headers
Paul Jarc | 3 Jun 2004 06:46
Picon

Re: dns data file help (I believe)

"travis <at> norris.net" <travis <at> norris.net> wrote:
> dnscache pointed to an internal address 10.7.7.2 and tinydns to
> 127.0.0.1 on a.ns.example.com &
> dnscache pointed to an internal address 10.7.7.4 and tinydns to
> 127.0.0.1 on b.ns.example.com

<URL:http://homepages.tesco.net./~J.deBoynePollard/FGA/dont-obscure-your-dns-data.html>

Are your domains supposed to be reachable by the rest of the Internet,
or just you?  If they're supposed to be publicly accessible, then
tinydns must be on publicly reachable addresses.  Hiding it behind
dnscache won't work.
<URL:http://homepages.tesco.net./~J.deBoynePollard/FGA/dns-server-roles.html>

paul
Permalink | Reply | 0 comments |
headers
Stephen Bosch | 3 Jun 2004 07:02
Picon

Re: peculiar dnscache hangs

Charlie Brady wrote:
> What does strace say is happening with the dnstrace process during 
> the apparent pauses?

How should I invoke dnscache with strace?

-Stephen-
Permalink | Reply | 1 comment |
headers
Stephen Bosch | 3 Jun 2004 07:04
Picon

Re: peculiar dnscache hangs

John Peacock wrote:
> Stephen Bosch wrote:
> 
>> Every once in a while (I'm not certain of the exact interval) dnscache
>> will simply hang. The hang is usually brief -- I've never seen it last
>> more than a minute -- and it's not fatal, but it's annoying.
>>
>> Does this sound familiar to anybody?
> 
> 
> Yes.  Check your disk quotas.  I had this happen on a box and it made me 
> crazy.  It turned out that when dnscache was in the process of rotating 
> the logs, it would go just over the quota that was set for that user. 
> That would cause dnscache to hang.  I don't remember it ever fixing 
> itself; I usually wound up rebooting at that point, since restarting 
> wouldn't help.  Once I turned off quotas, the problem disappeared.

Quota is installed but there are no quotas set:

$ quota
Disk quotas for user sfbosch (uid 501): none

> One of the basic design features of multilog is that is assumes you are 
> smart enough to allow it enough storage to support the number of logs 
> which you have preset.  If multilog cannot write to the logfile, it will 
> hang, which will cause dnscache (in this case) to hang as well.  It is 
> possible that there is some other problem with your system which is 
> preventing the log files from being written, but this is exactly the 
> behavior I have seen before.
(Continue reading)

Permalink | Reply | 10 comments |
headers
Jeff King | 3 Jun 2004 07:19

Re: peculiar dnscache hangs

On Wed, 2 Jun 2004, Stephen Bosch wrote:

> > What does strace say is happening with the dnstrace process during
> > the apparent pauses?
> How should I invoke dnscache with strace?

You can put it in the actual /etc/dnscache/run script (the final line
should be similar to this):
...
exec envuidgid dnscache softlimit -o250 -d "$DATALIMIT" strace -o /tmp/dnscache.trace /usr/local/bin/dnscache

Then just "svc -t /service/dnscache". Don't leave the strace on a
production machine; it will drastically slow things down.

Does that answer your question, or was there something else about strace
you didn't understand?

-Peff
Permalink | Reply | 0 comments |

Gmane