headers
Frank Ch. Eigler | 1 Sep 2005 03:03
Picon
Favicon
Gravatar

Re: lots of systemtap language questions


hunt wrote:

> [...]
> If we are going to have builtin variables, how to do it?  For example
> "pid" which would be simply $current->pid (when not in interrupts)? Can
> we make this a variable or do we make it a function, which is easy?

For now, let's skip "builtin variables" as such, and live with
functions that return the values.  Even these functions are not
"builtins", but ones just pulled in from the script library.  (The
current files named src/tapset/builtin_* may get just renamed at some
point.)

(By the way, "$current" does not exist as such: on 386, it's a macro
that expands to an inline function.)

> Whichever we pick, should the basic builtins have some common prefix?
> Or we just have pid(), caller(), pid(), ppid(), gid(), stack(), etc?

I suggest going for simple names for the obvious ones ("caller" and
"stack" are not quite as obvious as the others).

> For things like printing registers or stack, how should it work?
> I have this working:
> 
> probe kernel.function("uptime_read_proc") {
> 	log("Now in uptime")
> 	print_regs()
> 	print_stack()
(Continue reading)

Permalink | Reply | 2 comments |
headers
Martin Hunt | 1 Sep 2005 05:57
Picon
Favicon

Re: lots of systemtap language questions

On Wed, 2005-08-31 at 21:03 -0400, Frank Ch. Eigler wrote:
> hunt wrote:
> 
> > [...]
> > If we are going to have builtin variables, how to do it?  For example
> > "pid" which would be simply $current->pid (when not in interrupts)? Can
> > we make this a variable or do we make it a function, which is easy?
> 
> For now, let's skip "builtin variables" as such, and live with
> functions that return the values.  Even these functions are not
> "builtins", but ones just pulled in from the script library.  (The
> current files named src/tapset/builtin_* may get just renamed at some
> point.)

I called them builtins because are core functions and we have the
builtin_ tapsets. Maybe three of us care how we implement them.

> > Whichever we pick, should the basic builtins have some common prefix?
> > Or we just have pid(), caller(), pid(), ppid(), gid(), stack(), etc?
>
> I suggest going for simple names for the obvious ones ("caller" and
> "stack" are not quite as obvious as the others).

Hmm.  How about calling_function(), calling_address() and backtrace()?

So far I have done 
pid()
ppid()
execname() - name of the executable, current->comm
pexecname() - name of the parent executable, current->parent->comm
(Continue reading)

Permalink | Reply | 1 comment |
headers
Frank Ch. Eigler | 1 Sep 2005 17:00
Picon
Favicon
Gravatar

Re: lots of systemtap language questions

Hi -

hunt wrote:

> [...]
> > > Whichever we pick, should the basic builtins have some common prefix?
> > > Or we just have pid(), caller(), pid(), ppid(), gid(), stack(), etc?
> >
> > I suggest going for simple names for the obvious ones ("caller" and
> > "stack" are not quite as obvious as the others).
> 
> Hmm.  How about calling_function(), calling_address() and backtrace()?

Good.

> [...]
> stack() - should be backtrace()?

Yeah.

> print_regs() - register dump

> Maybe I should call everything that prints directly print_xxx()?  The
> rest of the functions return ints or strings.

Yes, good idea.

> I've also done print(), which is like log() except uses _stp_print() [...]

OK.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Martin Hunt | 1 Sep 2005 20:25
Picon
Favicon

a couple simple scripts

Would it be helpful to post some sample scripts that do useful work?

Here's a one-liner:
>stap -e 'probe kernel.function("sys_open") {print(execname()."[".string(pid())."]"." opened ".$filename)}'

And here's a version of shellsnoop:

-------------------------------------
global pids

probe kernel.function("do_execve") {
        if (execname() == "bash" || execname() == "sh" || execname == "tcsh") {
                print("user= ".string(uid())."\tpid= ".string(pid())."\tppid= ".string(ppid())."\texec ".$filename)
                pids[pid()] = 1
        }
}

probe kernel.function("sys_open") {
        if (pids[pid()])
                print(execname()."[".string(pid())."]"." opened ".$filename)
}

probe kernel.function("sys_read") {
        if (pids[pid()])
                print(execname()."[".string(pid())."]"." read fd ".string($fd))
}

probe kernel.function("sys_write") {
        if (pids[pid()])
                print(execname()."[".string(pid())."]"." write fd ".string($fd)." ".string($count)." bytes")
(Continue reading)

Permalink | Reply | 502 comments |
headers
Keshavamurthy Anil S | 1 Sep 2005 22:49
Picon
Favicon

[PATCH]kprobes fix bug when probed on task and isr functions

	This patch fixes a race condition where in system used to hang
or sometime crash within minutes when kprobes are inserted on 
ISR routine and a task routine.

The fix has been stress tested on i386, ia64, pp64 and on x86_64.
To reproduce the problem insert kprobes on schedule() and do_IRQ() functions and
you should see hang or system crash.

This patch should apply cleanly on 2.6.13-mm1 kernel.

Please apply.

Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy <at> intel.com>
Signed-off-by: Ananth N Mavinakayanahalli <ananth <at> in.ibm.com>
Acked-by: Prasanna S Panchamukhi <prasanna <at> in.ibm.com>

=====================================================
 arch/i386/kernel/kprobes.c   |    3 ++-
 arch/ia64/kernel/kprobes.c   |   22 +++++++++++++++++++---
 arch/ppc64/kernel/kprobes.c  |   11 ++++++-----
 arch/x86_64/kernel/kprobes.c |    3 ++-
 include/asm-ia64/kprobes.h   |    1 +
 include/asm-ppc64/kprobes.h  |    3 +++
 kernel/kprobes.c             |    8 ++++++++
 7 files changed, 41 insertions(+), 10 deletions(-)

Index: linux-2.6.13-mm1/arch/i386/kernel/kprobes.c
===================================================================
--- linux-2.6.13-mm1.orig/arch/i386/kernel/kprobes.c
+++ linux-2.6.13-mm1/arch/i386/kernel/kprobes.c
(Continue reading)

Permalink | Reply | 0 comments |
headers
Andrew Morton | 1 Sep 2005 23:09

Re: [PATCH]kprobes fix bug when probed on task and isr functions

Keshavamurthy Anil S <anil.s.keshavamurthy <at> intel.com> wrote:
>
> 	This patch fixes a race condition where in system used to hang
> or sometime crash within minutes when kprobes are inserted on 
> ISR routine and a task routine.

It's desirable that the patch descriptions tell us _how_ a bug was fixed,
as well as what the bug was.  It means that people don't have to ask
questions like:

>  void __kprobes lock_kprobes(void)
>  {
> +	unsigned long flags = 0;
> +
> +	local_irq_save(flags);
>  	spin_lock(&kprobe_lock);
>  	kprobe_cpu = smp_processor_id();
> + 	local_irq_restore(flags);
>  }

what is this change trying to do?  If a lock is taken from both process and
irq contexts then local IRQs must be disabled for the entire period when the
lock is held, not just for a little blip like this.  If IRQ-context code is
running this function then the code is deadlockable.

Now, probably there's deep magic happening here and I'm wrong.  If so then
please explain the code's magic via a comment patch so the question doesn't
arise again, thanks.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Kevin Stafford | 1 Sep 2005 23:11
Picon
Favicon

Re: a couple simple scripts

Martin Hunt wrote:

>Would it be helpful to post some sample scripts that do useful work?
>
>Here's a one-liner:
>  
>
>>stap -e 'probe kernel.function("sys_open") {print(execname()."[".string(pid())."]"." opened ".$filename)}'
>>    
>>
>
>And here's a version of shellsnoop:
>
>-------------------------------------
>global pids
>
>probe kernel.function("do_execve") {
>        if (execname() == "bash" || execname() == "sh" || execname == "tcsh") {
>                print("user= ".string(uid())."\tpid= ".string(pid())."\tppid= ".string(ppid())."\texec ".$filename)
>                pids[pid()] = 1
>        }
>}
>
>  
>
This one breaks on my system.

user= 501       pid= 11925      ppid= 8176      exec /bin/ls
ERROR: pointer dereference fault near identifier '$filename' at 
/home/krstaffo/mytests/shell_snoop.stp:13:67
(Continue reading)

Permalink | Reply | 502 comments |
headers
Martin Hunt | 1 Sep 2005 23:20
Picon
Favicon

Re: a couple simple scripts

On Thu, 2005-09-01 at 14:11 -0700, Kevin Stafford wrote:

> This one breaks on my system.
> 
> user= 501       pid= 11925      ppid= 8176      exec /bin/ls
> ERROR: pointer dereference fault near identifier '$filename' at 
> /home/krstaffo/mytests/shell_snoop.stp:13:67
> 
> I thought this was because of $filename looking into userspace. (see 
> BZ#1243). Does this script work on your machine?

Sometime yesterday it got fixed. It was broken on 3 different systems
and they all now work for me. Are you running the latest from CVS?

Martin
Permalink | Reply | 5 comments |
headers
William Cohen | 1 Sep 2005 23:21
Picon
Favicon

attempt to code LKST sysv ipc events in systemtap

As a part of the exercise that everyone is doing to code instrumentation 
using the systemtap translator I coded up the ones in the LKST for sysv 
ipc events. They are instrument the function entry so should be fairly 
straight forward for system tap. I was just trying to log the same set 
of arguments that LKST was using.

I was trying to do this on a FC4 machine running the 2.6.12-1.1447_FC4 
kernel. I built a local 0.3.1-1 SystemTap RPM created from a cvs 
snapshot earlier today.

Is there a problem with the debugging information in this particular FC4 
kernel, e.g. missing information about arguments?  The arguments are 
definitely in the source code. When trying to install the ipc_lkst.stp 
instrumentation I got the following errors.

$ stap ipc_lkst.stp
semantic error: unsupported type tag 23: identifier '$arg' at 
ipc_lkst.stp:25:17semantic error: no match for probe point
          while: resolving probe point kernel.function("sys_semctl")
semantic error: unable to find local 'key' near pc 0xc01e3bc8: 
identifier '$key' at ipc_lkst.stp:46:11
semantic error: no match for probe point
          while: resolving probe point kernel.function("sys_msgget")
semantic error: unable to find local 'raddr' near pc 0xc01e78fe: 
identifier '$raddr' at ipc_lkst.stp:63:20
semantic error: no match for probe point
          while: resolving probe point kernel.function("sys_shmat")
semantic error: unable to find local 'shmaddr' near pc 0xc01e791d: 
identifier '$shmaddr' at ipc_lkst.stp:70:15
semantic error: no match for probe point
(Continue reading)

Permalink | Reply | 0 comments |
headers
Keshavamurthy Anil S | 1 Sep 2005 23:27
Picon
Favicon

Re: [PATCH]kprobes fix bug when probed on task and isr functions

On Thu, Sep 01, 2005 at 02:09:38PM -0700, Andrew Morton wrote:
> Keshavamurthy Anil S <anil.s.keshavamurthy <at> intel.com> wrote:
> >
> > 	This patch fixes a race condition where in system used to hang
> > or sometime crash within minutes when kprobes are inserted on 
> > ISR routine and a task routine.
> 
> It's desirable that the patch descriptions tell us _how_ a bug was fixed,
> as well as what the bug was.  It means that people don't have to ask
> questions like:
Sure, our current kprobes model is very serialized where in we serve only
one kprobe in the exception handler by holding this lock_kprobes() and
release this lock i.e unlock_kprobes() when we are done with single stepping

> 
> >  void __kprobes lock_kprobes(void)
> >  {
> > +	unsigned long flags = 0;
> > +
> > +	local_irq_save(flags);
> >  	spin_lock(&kprobe_lock);
> >  	kprobe_cpu = smp_processor_id();
> > + 	local_irq_restore(flags);
> >  }
> 
> what is this change trying to do?  If a lock is taken from both process and
> irq contexts then local IRQs must be disabled for the entire period when the
> lock is held, not just for a little blip like this.  If IRQ-context code is
> running this function then the code is deadlockable.
(Continue reading)

Permalink | Reply | 0 comments |

Gmane