Maynard Johnson | 26 Aug 18:58 2014
Picon

[PATCH] Fix Java profiling regression bug from Aug 13 Coverity fixes

Fix Java profiling regression bug from Aug 13 Coverity fixes

One of the changes made in the Aug 13 commit to fix issues
identified by Coverity caused a regression in oprofile's JIT
support. The libopagent.so (used by libjvm[t|p]i.so) may incorrectly
return an error from the op_write_native_code function. This patch
fixes that issue.

Signed-off-by: Maynard Johnson <maynardj <at> us.ibm.com>
---
 libopagent/opagent.c |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/libopagent/opagent.c b/libopagent/opagent.c
index f567f9e..431dfae 100644
--- a/libopagent/opagent.c
+++ b/libopagent/opagent.c
 <at>  <at>  -374,17 +374,22  <at>  <at>  again:
 	 */
 	if (fwrite_unlocked(&rec, sizeof(rec), 1, dumpfile) &&
 	    fwrite_unlocked(symbol_name, sz_symb_name, 1, dumpfile)) {
-		size_t sz = 0;
-		if (code)
+		size_t expected_sz, sz;
+		expected_sz = sz = 0;
+		if (code) {
 			sz = fwrite_unlocked(code, size, 1, dumpfile);
-		if (padding_count)
+			expected_sz++;
+		}
(Continue reading)

C K Kashyap | 26 Aug 17:32 2014
Picon

Help with oprofile on android

Hi,

I have a asus nexus 7 where I need to profile my c++ app. Is there a place where I can see a step by step instruction for enabling/collecting oprofile data on my app. I found this link http://brownydev.blogspot.in/2012/06/android-oprofile.html from the mailing list but it seems its old - I dont see any "CONFIG_OPROFILE_ARMV7" in the kernel source - I am using - https://android.googlesource.com/kernel/msm.git

I'd appreciate any help with this very much. For some reason, googling it does not seem to yield any current links to me. 

Regards,
Kashyap
------------------------------------------------------------------------------
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
oprofile-list mailing list
oprofile-list <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
Brian Hall | 25 Aug 20:01 2014
Picon

missing Java method information in operf profile

I'm running the operf from AT 7.0-5 on a POWER8 system:
  # /opt/at7.0/bin/operf -v
  /opt/at7.0/bin/operf: oprofile 1.0.0git compiled on Jul 24 2014 13:49:08
using the command (as root):
  /opt/at7.0/bin/operf --events PM_RUN_CYC:10290000 --separate-thread --separate-cpu --system-wide

There are 14 active JVM processes running on the system that I am trying to profile.  For some reason, the profiles only contain full JIT compiler method information for 2 or 3 of the processes (it varies across different runs).  I have checked that the agentlib option is being correctly passed to the JVMs, and that the JIT dump files for all 14 processes are available in /tmp/.oprofile/jitdump at the time that operf is run.

A casual look at verbose output from operf didn't indicate any problems in accessing the JIT dump files, it just didn't try to process all of them.  I tried adding --lazy-conversion to the operf options, and that run did show an issue at the end of the verbose output:
  add sym: name=Ljava/utilchild received signal 9

Brian

------------------------------------------------------------------------------
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
oprofile-list mailing list
oprofile-list <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
Grant Edwards | 19 Aug 18:07 2014
Picon

Using oprofile w/o loadable module support?

I'm trying to get oprofile 0.9.9 running on a system where the kernel
does not support runtime loadable kernel modules. I have oprofile
support enabled in the kernel (2.6.33), but oprofile keeps wanting to
"modprobe" something (I haven't been able to figure out what):

 # opcontrol --init
 modprobe: chdir(/lib/modules): No such file or directory
 modprobe: chdir(/lib/modules): No such file or directory
 Kernel doesn't support oprofile

But I _do_ have oprofile support enabled in the kernel.  What module
is oprofile looking for?

--

-- 
Grant Edwards               grant.b.edwards        Yow! I invented skydiving
                                  at               in 1989!
                              gmail.com            

------------------------------------------------------------------------------
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/
William Cohen | 19 Aug 18:41 2014
Picon

Multiple definitions of INST_RETIRED for processors using i386/arch_perfmon

Hi Andi,

When testing oprofile Michael Petlan noticed there were a number of Intel architectures that include
i386/arch_perfmon.  The Intel architected events set up INST_RETIRED with a unit mask of zero:

event:0xc0 counters:cpuid um:zero minimum:6000 filter:1 name:INST_RETIRED : number of instructions retired

The arch_perfmon/events is included in the following files:

$ grep include:i386/arch_perfmon */events
broadwell/events:include:i386/arch_perfmon
haswell/events:include:i386/arch_perfmon
ivybridge/events:include:i386/arch_perfmon
sandybridge/events:include:i386/arch_perfmon
silvermont/events:include:i386/arch_perfmon
westmere/events:include:i386/arch_perfmon

However, a number of Intel processor events files have have other unit masks being specific (1 or a bit mask
or 1,2, and 4) for INST_RETIRED (0xc0):

$ grep 0xc0 */events |grep -v zero
atom/events:event:0xc0 counters:0,1 um:one minimum:6000 name:INST_RETIRED : number of instructions retired
broadwell/events:event:0xc0 counters:1 um:inst_retired minimum:2000003 name:inst_retired :
core_2/events:event:0xc0 counters:0,1 um:inst_retired minimum:6000 name:INST_RETIRED : number of
instructions retired
haswell/events:event:0xc0 counters:1 um:one minimum:2000003 name:inst_retired_prec_dist :
nehalem/events:event:0xc0 counters:0,1,2,3 um:inst_retired minimum:6000 name:INST_RETIRED :
number of instructions retired
sandybridge/events:event:0xc0 counters:1 um:one minimum:2000000 name:inst_retired : Instructions retired
westmere/events:event:0xc0 counters:0,1,2,3 um:inst_retired minimum:2000000 name:INST_RETIRED :
Instructions retired (Programmable counter and Precise Event)

Notice that the broadwell, haswell, sandybridge, and westmere have multiple definitions of
INST_RETIRED, one from arch_perfmon which has a zero unit mask and another one that has a non-zero unit
masks.  How is the INST_RETIRED event suppose to work for processors that support the Intel architected
performance events?  Always allow a unit mask of 0?  Any suggestions n how the testsuite lib/op_events.exp
should properly exercise INST_RETIRED event to test that things are working?

-Will

------------------------------------------------------------------------------
Grant Edwards | 19 Aug 18:17 2014
Picon

Using oprofile w/o loadable module support?

I'm trying to get oprofile 0.9.9 running on a system where the kernel
does not support runtime loadable kernel modules. I have oprofile
support enabled in the kernel (2.6.33), but oprofile keeps wanting to
"modprobe" something (I haven't been able to figure out what):

 # opcontrol --init
 modprobe: chdir(/lib/modules): No such file or directory
 modprobe: chdir(/lib/modules): No such file or directory
 Kernel doesn't support oprofile

But I _do_ have oprofile support enabled in the kernel.  What module
is oprofile looking for?

--

-- 
Grant Edwards               grant.b.edwards        Yow! Are we THERE yet?
                                  at               
                              gmail.com            

------------------------------------------------------------------------------
Maynard Johnson | 15 Aug 18:09 2014
Picon

Away for the week of Aug 16 - Aug 23

I'll be out of the office, literally in the wilderness, and will not have internet access.  I look forward to
seeing lots of replies from the community regarding test results for the OProfile 1.0.0 RC1 when I return.  :-)

-Maynard

------------------------------------------------------------------------------
Maynard Johnson | 15 Aug 17:54 2014
Picon

Announcement: Release Candidate 1 for OProfile 1.0.0

We are pleased to announce OProfile 1.0.0 Release Candidate 1.  You can download this release at:
	http://sourceforge.net/projects/oprofile/files/oprofile/oprofile-1.0.0-rc1/

Please download and test this release candidate, and send your feedback by replying to this message. 
Please include your hardware platform and Linux distribution information in your reply.

Thanks.
-Maynard Johnson

-----------------------------------------------------------------

Release Notes
===============
OProfile 1.0.0 Release Candidate 1 has been released. A major change in
this release is the removal of the legacy opcontrol-based profiler.
This legacy profiling tool has been deprecated since release 0.9.8 when
operf was first introduced. The following components and processor
types that were dependent on opcontrol have also been removed:

   - GUI component (i.e., oprof_start)
   - IBS events removed from AMD processors
   - All Alpha processors, except for EV67 (which *is* supported by operf/ocount)
   - Architecture avr32
   - Architecture ia64
   - Processor model IBM Cell
   - Processor model P.A. Semi PA6T
   - RTC (real time clock mode)

OProfile users still running on any of these affected systems or
needing any of the removed components listed above should not upgrade
to OProfile release 1.0. Alternatively, you can obtain all of the new
features, enhancements, and bug fixes described below and still have
access to opcontrol, by doing the following:

	git clone git://git.code.sf.net/p/oprofile/oprofile oprofile
	cd oprofile
	git checkout PRE_RELEASE_1_0

and then build/install as usual.

More information about OProfile can be seen at
    http://oprofile.sf.net

Incompatibilities with previous release
---------------------------------------

- Sample data collected with previous releases of OProfile are incompatible
  with release 1.0.
- ophelp schema: Major version changed for removal of unit mask 'extra'
  attribute and addition of unit mask 'name'.

New features
------------

- Enhance ocount to support millisecond time intervals
- Obtain kernel symbols from /proc/kallsyms if no vmlinux file specified

- New/updated Processor Support
    * (New) Freescale e6500 
    * (New) Freescale e500mc
    * (New) Intel Silvermont
    * (New) ARM ARMv7 Krait
    * (New) ARM ARMv8 (AArch64)
    * (New) Intel Broadwell
    * (New) ARM Cortex A57
    * (New) ARM Cortex A53
    * Added little endian support for IBM POWER8
    * Update events for IBM POWER8
    * Added edge-detect events for IBM POWER7
    * Update events for Intel Haswell

Bug fixes
---------

Filed bug reports:
-------------------------------------------------------------------------
|  BUG ID   |  Summary 
|-----------|------------------------------------------------------------
|   236     | opreport schema: Fix count field maxOccurs (changed to
|           | 'unbounded')
|   245     | Fix compile error on ppc/uClibc platform: 'AT_BASE_PLATFORM'
|           | undeclared'
|   248     | Duplicate event specs passed to ocount show up twice in
|           | output
|   252     | Fix operf/ocount default unit mask selection
|   253     | ocount: print the unit mask, kernel and user modes if
|           | specified for the event
|   254     | ophelp schema is not included in installed files
|   255     | Remove unused 'extra' attribute from ophelp schema
|   256     | opreport from 'operf --callgraph' profile shows false
|           | recursive calls
|   257     | Fix handling of default named unit masks longer than 11 chars
|   259     | Print unit mask name where applicable in ophelp XML output
|   260     | Fix profiling of multi-threaded apps when using "--pid"
|           | option
|   262     | Fix operf/opreport kernel throttling detection
|   263     | Fix sample attribution problem when using multiple events
|   266     | exclude/include files option doesn't work for opannotate -a
-------------------------------------------------------------------------

Other bug fixes and improvements without a filed report (e.g., posted to the list):
---------------
   - Update Alpha EV67 CPU support and remove all other Alpha CPU support
   - operf main process improperly killing conversion process
   - Fix up S390 support to work with operf/ocount
   - Link ocount with librt for clock_gettime only when needed
   - Fix 'Invalid argument' running 'opcontrol --start --callgraph=<n>' in
     Timer mode
   - Make sure hypervisor is excluded from ocount and operf
   - operf log may over-report "sample address not in expected range for
     domain"
   - Allow root to remove old jitdump files from /tmp/.oprofile/jitdump
   - Remove opreport warnings for /no-vmlinux, [vdso], [hypervisor_bucket]
     not found
   - Fix event codes for marked architected events (IBM ppc64)
   - Make operf/ocount detect invalid timer mode from opcontrol
   - Reduce overhead of operf waiting for profiled app to end
   - Fix "Unable to open cpu_type file for reading" for IBM POWER7+
   - Allow all native events for IBM POWER8 in POWER7 compat mode
   - Fix spurious "backtraces skipped due to no file mapping" log entries
   - Fix the units for the reported CPU frequency

Known problems and limitations
-------------------------
- When using operf to profile multiple events, the absolute number of
  events recorded may be substantially fewer than expected. This can be
  due to knwon bug in the Linux kernel's Performance Events Subsystem that
  was fixed sometime between Linux kernel version 3.1 and 3.5.

------------------------------------------------------------------------------
William Cohen | 15 Aug 15:43 2014
Picon

Re: [oprof-cvs] Oprofile on mapreduce task

On 08/14/2014 07:06 PM, Tommy wrote:
> Hi,
> 
> I am trying to profile mapreduce task using operf.
> I have some troubles with getting correct symbols of java threads.
> 
> What I did was to operf the tasktracker which spawns map and reduce tasks (threads).
> I did like the following to start the tasktracker.
> *operf /usr/lib/jvm/java-oracle/bin/java -agentlib:jvmti_oprofile ......*
> 
> However, only the tasktracker thread gets the correct symbols for all addresses. I found that only one
".jo" file is created for the tasktracker, not the child threads that inherit the tasktracker, such as map
and reduce tasks. All the child threads get "anon".
> 
> Please help me to resolve the issue.
> Thank you very much.
> 
> 
> ------------------------------------------------------------------------------

Hi Tommy,

This email should be on oprofile-list <at> lists.sf.net .  The oprofile-commits <at> lists.sf.net is an mailing
list that tracks source code checkins for oprofile and most people are not going to be reading it.

There are a couple patches that might address the problem:

http://sourceforge.net/p/oprofile/oprofile/ci/2d60e8a980bbbaa62aeb1346827592092bc0743e/
http://sourceforge.net/p/oprofile/oprofile/ci/eb75586eed1d105a22b12d879a31ec32d6a94913/

-Will

------------------------------------------------------------------------------
Michael Cree | 15 Aug 12:39 2014
Picon

Adding Alpha EV67 support

Maynard,

I follow with the patch to add Alpha EV67 support to operf and remove
all other alpha support.

It compiles fine but does not appear to work properly.  ophelp prints
out:

oprofile: available events for CPU type "Alpha EV67"

See Alpha Architecture Reference Manual
http://download.majix.org/dec/alpha_arch_ref.pdf
For architectures using unit masks, you may be able to specify
unit masks by name.  See 'operf' or 'ocount' man page for more details.

CYCLES: (counter: 0, 1)
	Total cycles (min count: 500)
INSTRUCTIONS: (counter: 0)
	Retired instructions (min count: 500)
BCACHE_MISS: (counter: 1)
	Bcache misses/long probe latency (min count: 500)
MBOX_REPLAY: (counter: 1)
	Mbox replay traps (min count: 500)

which is as expected.  However trying to run operf results in:

$ operf ls
perf_event_open failed with Operation not permitted
Caught runtime_error: Internal Error.  Perf event setup failed.
Error running profiler

I haven't been able to work out why that is occurring.  My own code
to use the kernel performance events works fine, so I would hazard a
guess that operf is setting some flag/parameter when calling the kernel
performance event syscall that I have never tested with.

I am now out of time today to examine this any further.

Cheers
Michael.

------------------------------------------------------------------------------
Tommy | 15 Aug 01:09 2014
Picon

oprofile on mapreduce tasks (java)

Hi,

I am trying to profile mapreduce task using operf.
I have some troubles with getting correct symbols of java threads.

What I did was to operf the tasktracker which spawns map and reduce tasks (threads).
I did like the following to start the tasktracker.
operf /usr/lib/jvm/java-oracle/bin/java -agentlib:jvmti_oprofile ......

However, only the tasktracker thread gets the correct symbols for all addresses. I found that only one ".jo" file is created for the tasktracker, not the child threads that inherit the tasktracker, such as map and reduce tasks. All the child threads get "anon".

Please help me to resolve the issue.
Thank you very much.

Tommy
------------------------------------------------------------------------------
_______________________________________________
oprofile-list mailing list
oprofile-list <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list

Gmane