William Cohen | 18 Dec 18:20 2014
Picon

Questionable array size check in operf_counter.cpp

I ran the oprofile currently in the git resository through coverity to see if there were any issues in the
code.  One thing that did look questionable was:

	Error: BAD_SIZEOF (CWE-467): [#def3]
oprofile-1.1.0git/libperf_events/operf_counter.cpp:772: bad_sizeof: The expression "sizeof
(this->poll_data) / sizeof (this->poll_data[0])" is suspicious. Note that "this->poll_data" is a
pointer and therefore the division will not return the number of array elements which may have been the intent.

The method making a larger size array and copying the data over. It is attempting to get the size of the array with:

	num_mmaps = sizeof(poll_data)/sizeof(poll_data[0]);

It appears this was added in commit eb75586eed1d105a22b12d879a31ec32d6a94913

It looks like the class should have the information about the size of poll_data in there. Should the
operf_record class include a num_mmaps field so the size of poll_data is known? Or could the some other
field (such as poll_count) be used to track that information?

-Will

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Maynard Johnson | 15 Dec 23:55 2014
Picon

Transition of oprofile maintainership to Will Cohen

To all oprofile community members,
Will Cohen has kindly offered to take over the maintainership role for oprofile as
I ride off into the sunset.  Will accepted the "caretaker" role that I proposed on
Nov 18, with duties as follows:

  1) Ensure architecture-specific patches are reviewed by an appropriate community member
  2) Review architecture-independent patches or enlist someone else to do the review
  3) Push approved patches upstream
  4) Once a year (or so), put out a new release (or enlist someone else to do it)

We ask that the rest of the oprofile community help out with answering questions on
the mailing list to keep this caretaker role from becoming overly time-consuming.

I want to give one more note of thanks to John Levon, the oprofile creator, for entrusting
me with the lead maintainership role.  It was a great learning experience.  I bid you all
farewell and good fortune. It's been a pleasure working with the all members of the oprofile
community these past 6 years.

-Maynard

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Carl Love | 15 Dec 23:35 2014
Picon

User/kernel mode support in the various platforms


If you would let me know if the architecture you work on supports or 
does not support user and kernel mode profiling, it would be greatly 
appreciated.  Please run "uname -m" and let me know what the string 
says so I can get the correct string into the patch.  

I need the information for a patch that I am working on OProfile 
testsuite to skip the user/kernel scaling tests if the architecture
doesn't support data in user mode only and kernel mode only.

Thanks.

              Carl Love

------------------------------------------------------------------

Oprofile-test, skip mode selection tests if not supported.

Event counts may occur when the system is running in user mode, kernel
mode or hypervisor modes.  Some architectures have settings that allow
the user to select which modes the events should be collected.  The
testsuite contains tests to make sure sum of counting in each of the modes
individually is the same as counting in each mode simultaneously.  This
patch skips these tests for architectures that do not support mode selection

Signed-off-by: Carl Love <carll <at> us.ibm.com>
---
 testsuite/lib/ocount_util.exp            |   21 +++++++++++++++++++++
 testsuite/oprofile-ocount/ocount-run.exp |   14 ++++++++++++++
 2 files changed, 35 insertions(+), 0 deletions(-)
(Continue reading)

Henry May | 11 Dec 22:32 2014
Picon

ocount fails with "too many open files"

I'm running ocount on a P8 and getting this:

/usr/local/bin/ocount -b -e PM_RUN_CYC,PM_RUN_INST_CMPL,PM_DATA_FROM_L2MISS,PM_DATA_FROM_L2,PM_DATA_FROM_L3,PM_RC_LIFETIME_EXC_32,PM_RC_LIFETIME_EXC_256,PM_RC_LIFETIME_EXC_1024,PM_RC_LIFETIME_EXC_2048 -c -s -i 1000 -f /tmp/pmu_counts.csv
Starting ocount Thu Dec 11 16:26:46 EST 2014
Process ID: 154011
perf_event_open failed with Too many open files
Caught runtime error while setting up counters
Internal Error.  Perf event setup failed.
Error running ocount

If I reduce the number of counters, it works fine.  Also, I know I was running this same command successfully on a different P8 (now gone)


Henry May
IBM InfoSphere Streams Performance
hjmay <at> us.ibm.com
720-342-8873
Tie: 963-8873
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
oprofile-list mailing list
oprofile-list <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
Maynard Johnson | 11 Dec 00:21 2014
Picon

[PATCH] Passing a bogus value to 'configure --with-java' should cause error

Passing a bogus value to 'configure --with-java' should cause error

The "--with-java" configure option expects either "no", "yes", or a
path to a directory which has an 'include' directory containing either
jvmpi.h or jvmti.h or both. If the user who is running 'configure'
specifies an incorrect pathname, the configure script continues on to
the end and exits normally with no error message. If the user looks
carefully at the output of the configure script, they will see messages
indicating jvmpi.h and jvmti.h were not found, but that's not sufficient
warning to the user that something went wrong.

This patch will detect an invalid Java home pathname and exit immediately
with an error.

Signed-off-by: Maynard Johnson <maynardj <at> us.ibm.com>
---
 configure.ac |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/configure.ac b/configure.ac
index 65b43f2..015d7be 100644
--- a/configure.ac
+++ b/configure.ac
 <at>  <at>  -224,6 +224,8  <at>  <at>  if test "x$with_java" = "xyes"; then
 	JAVA_HOMEDIR="/usr"
 else if test "x$with_java" = "xno"; then
 	JAVA_HOMEDIR=
+	else
+		JAVA_HOME_SPECIFIED="yes"
 	fi
 fi
 AC_SUBST(JAVA_HOMEDIR)
 <at>  <at>  -232,6 +234,15  <at>  <at>  if test -n "$JAVA_HOMEDIR"; then
 	AC_CHECK_FILE("$JAVA_HOMEDIR/include/jvmti.h",JVMTI_H_EXISTS="yes",)
 	AC_CHECK_FILE("$JAVA_HOMEDIR/include/jvmpi.h",JVMPI_H_EXISTS="yes",)
 fi
+if test -n "$JAVA_HOME_SPECIFIED"  && test -z "$JVMTI_H_EXISTS" && test -z "$JVMPI_H_EXISTS"; then
+	if test -d "$JAVA_HOMEDIR"; then
+		echo "You specified the "--with-java" option, but neither jvmti.h nor jvmpi.h were found."
+	else
+		echo "You specified the "--with-java" option, but $JAVA_HOMEDIR does not exist."
+	fi
+	exit
+fi
+

 AM_CONDITIONAL(BUILD_JVMTI_AGENT, test -n "$JVMTI_H_EXISTS")
 AM_CONDITIONAL(BUILD_JVMPI_AGENT, test -n "$JVMPI_H_EXISTS")
--

-- 
1.7.1

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Maynard Johnson | 3 Dec 20:16 2014
Picon

Patch review needed for AMD Athlon-XP CPU

Hi, Suravee,
Can you please review the patch posted to bug https://sourceforge.net/p/oprofile/bugs/275/ to add
support for AMD Athlon-XP CPU.  Thanks!

-Maynard

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Maynard Johnson | 21 Nov 23:38 2014
Picon

[PATCH] Add a check to ensure event codes in events files fit in an integer

This patch has already been pushed upstream.

-----------------------------------------------------------------

Add a check to ensure event codes in events files fit in an integer

The read_events() function in libop/op_events.c is enhanced to
validate that the hex codes found in events files are not too big
to fit in an integer, since the rest of oprofile assumes hex codes
are integers.  This check will execute whenever ocount or operf is
run, as well as when doing 'make check' or 'make distcheck'.

Signed-off-by: Maynard Johnson <maynardj <at> us.ibm.com>
---
 libop/op_events.c |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/libop/op_events.c b/libop/op_events.c
index 29dc2f3..99266c6 100644
--- a/libop/op_events.c
+++ b/libop/op_events.c
 <at>  <at>  -498,6 +498,7  <at>  <at>  static void read_events(char const * file)
 	int seen_event, seen_counters, seen_um, seen_minimum, seen_name, seen_ext;
 	FILE * fp = fopen(file, "r");
 	int tags;
+	int fail = 0;

 	if (!fp) {
 		fprintf(stderr, "oprofile: could not open event description file %s\n", file);
 <at>  <at>  -510,6 +511,8  <at>  <at>  static void read_events(char const * file)
 	line = op_get_line(fp);

 	while (line) {
+		int bad_val = 0;
+		u64 tmp_val = 0ULL;
 		if (empty_line(line) || comment_line(line))
 			goto next;

 <at>  <at>  -535,11 +538,21  <at>  <at>  static void read_events(char const * file)
 				if (strchr(value, '.') != NULL)
 					parse_error("invalid event name");
 				event->name = value;
+				if (bad_val) {
+					fprintf(stderr, "Event %s event code (0x%llx) is too big to fit in an int\n",
+					        event->name, tmp_val);
+					fail = 1;
+					bad_val = 0;
+				}
 			} else if (strcmp(name, "event") == 0) {
 				if (seen_event)
 					parse_error("duplicate event: tag");
 				seen_event = 1;
-				event->val = parse_hex(value);
+				tmp_val = parse_long_hex(value);
+				if (tmp_val > 0xffffffff)
+					bad_val = 1;
+				else
+					event->val = (u32)tmp_val;
 				free(value);
 			} else if (strcmp(name, "counters") == 0) {
 				if (seen_counters)
 <at>  <at>  -599,6 +612,8  <at>  <at>  next:
 	}

 	fclose(fp);
+	if (fail)
+		exit(EXIT_FAILURE);
 }

 
--

-- 
1.6.2.rc2

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
Maynard Johnson | 21 Nov 20:56 2014
Picon

[PATCH v2] Add support for IBM Power event codes longer than sizeof int

Add support for IBM Power event codes longer than sizeof int

A small number of events on newer IBM Power processors have event codes
that are larger than sizeof(int). Rather than change the width of the
event code everywhere to be a long int (which would include having to
change the sample file format), we have defined some internal-use-only
unit masks for those events. These unit masks are not shown in the ophelp
output, and IBM Power users should never use them in event specifications;
instead, they should use the usual 'null' unit mask value of '0x0' in event
specifications -- e.g.,
       PM_L1MISS_LAT_EXC_256:0x0:0:1

See libpe_utils/op_pe_utils.cpp:_get_event_code for how these unit masks are
used.

Signed-off-by: Maynard Johnson <maynardj <at> us.ibm.com>
---
 events/ppc64/power8/events     |   16 ++++++++--------
 events/ppc64/power8/unit_masks |    8 ++++++++
 libop/op_events.c              |   39 +++++++++++++++++++++++++++++++++++++--
 libop/op_events.h              |    2 +-
 libpe_utils/op_pe_utils.cpp    |   15 +++++++++++----
 utils/ophelp.c                 |    2 +-
 6 files changed, 66 insertions(+), 16 deletions(-)

diff --git a/events/ppc64/power8/events b/events/ppc64/power8/events
index cc1163a..012ca89 100644
--- a/events/ppc64/power8/events
+++ b/events/ppc64/power8/events
 <at>  <at>  -451,10 +451,10  <at>  <at>  event:0x30a8 counters:0,1,2,3 um:zero minimum:10000 name:PM_ISU_REJ_VS0 : VS0 IS
 event:0x30aa counters:0,1,2,3 um:zero minimum:10000 name:PM_ISU_REJ_VS1 : VS1 ISU reject
 event:0x38a8 counters:0,1,2,3 um:zero minimum:10000 name:PM_ISU_REJ_VSU : ISU
 event:0x30b8 counters:0,1,2,3 um:zero minimum:10000 name:PM_ISYNC : Isync count per thread
-event:0x200301ea counters:2 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_1024 : Reload latency
exceeded 1024 cyc
-event:0x200401ec counters:3 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_2048 : Reload latency
exceeded 2048 cyc
-event:0x200101e8 counters:0 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_256 : Reload latency
exceeded 256 cyc
-event:0x200201e6 counters:1 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_32 : Reload latency
exceeded 32 cyc
+event:0x200301ea counters:2 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_1024 : Reload
latency exceeded 1024 cyc
+event:0x200401ec counters:3 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_2048 : Reload
latency exceeded 2048 cyc
+event:0x200101e8 counters:0 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_256 : Reload
latency exceeded 256 cyc
+event:0x200201e6 counters:1 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_32 : Reload
latency exceeded 32 cyc
 event:0x26086 counters:1 um:zero minimum:10000 name:PM_L1PF_L2MEMACC : valid when first beat of data
comes in for an L1pref where data came from mem(or L4)
 event:0x1002c counters:0 um:zero minimum:10000 name:PM_L1_DCACHE_RELOADED_ALL : L1 data cache
reloaded for demand or prefetch .
 event:0x408c counters:0,1,2,3 um:zero minimum:10000 name:PM_L1_DEMAND_WRITE : Instruction Demand
sectors wriittent into IL1
 <at>  <at>  -879,10 +879,10  <at>  <at>  event:0x10054 counters:0 um:zero minimum:10000 name:PM_PUMP_CPRED : Pump predict
 event:0x40052 counters:3 um:zero minimum:10000 name:PM_PUMP_MPRED : Pump Mis prediction Counts
across all types of pumpsfor all data types excluding data prefetch (demand load,inst prefetch,inst fetch,xlate).
 event:0x16081 counters:0 um:zero minimum:10000 name:PM_RC0_ALLOC : 0.0
 event:0x16080 counters:0 um:zero minimum:10000 name:PM_RC0_BUSY : RC mach 0 Busy. Used by PMU to sample
ave RC livetime(mach0 used as sample point)
-event:0x200301ea counters:2 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_1024 : Reload latency
exceeded 1024 cyc
-event:0x200401ec counters:3 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_2048 : Threshold
counter exceeded a value of 2048
-event:0x200101e8 counters:0 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_256 : Threshold
counter exceed a count of 256
-event:0x200201e6 counters:1 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_32 : Reload latency
exceeded 32 cyc
+event:0x200301ea counters:2 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_1024 : Reload
latency exceeded 1024 cyc
+event:0x200401ec counters:3 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_2048 :
Threshold counter exceeded a value of 2048
+event:0x200101e8 counters:0 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_256 : Threshold
counter exceed a count of 256
+event:0x200201e6 counters:1 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_32 : Reload
latency exceeded 32 cyc
 event:0x36088 counters:2 um:zero minimum:10000 name:PM_RC_USAGE : Continuous 16 cycle(2to1) window
where this signals rotates thru sampling each L2 RC machine busy. PMU uses this wave to then do 16 cyc count
to sample total number of machs running
 event:0x34808e counters:3 um:zero minimum:10000 name:PM_RD_CLEARING_SC : rd clearing sc
 event:0x34808c counters:3 um:zero minimum:10000 name:PM_RD_FORMING_SC : rd forming sc
diff --git a/events/ppc64/power8/unit_masks b/events/ppc64/power8/unit_masks
index 988dd41..203af97 100644
--- a/events/ppc64/power8/unit_masks
+++ b/events/ppc64/power8/unit_masks
 <at>  <at>  -5,5 +5,13  <at>  <at> 
 #
 # ppc64 POWER8 possible unit masks
 #
+# NOTE: The 'rc_machine' and 'L1_latency' unit masks are for internal use only,
+#       to workaround oprofile's 32-bit limitation for event codes.
+#       See libpe_utils/op_pe_utils.cpp:_get_event_code for how these codes are
+#       used.
 name:zero type:mandatory default:0x0
 	0x0 No unit mask
+name:rc_machine type:mandatory default:0xde
+	0xde Thresholdable start/stop for rc machine for sampled instruction
+name:L1_latency type:mandatory default:0x67
+	0x67 Thresholdable start/stop for L1 sampled instruction load miss/reload
diff --git a/libop/op_events.c b/libop/op_events.c
index 8bfd3d2..29dc2f3 100644
--- a/libop/op_events.c
+++ b/libop/op_events.c
 <at>  <at>  -1081,12 +1081,21  <at>  <at>  static int _is_um_valid_bitmask(struct op_event * event, u32 passed_um)
 	return retval;
 }

-int op_check_events(int ctr, u32 nr, u32 um, op_cpu cpu_type)
+static int _is_ppc64_cpu_type(op_cpu cpu_type) {
+	char const * cpu_name = op_get_cpu_name(cpu_type);
+	if (strncmp(cpu_name, "ppc64/power", strlen("ppc64/power")) == 0)
+		return 1;
+	else
+		return 0;
+}
+
+int op_check_events(char * evt_name, int ctr, u32 nr, u32 um, op_cpu cpu_type)
 {
 	int ret = OP_INVALID_EVENT;
 	size_t i;
 	u32 ctr_mask = 1 << ctr;
 	struct list_head * pos;
+	int ibm_power_proc = _is_ppc64_cpu_type(cpu_type);

 	load_events(cpu_type);

 <at>  <at>  -1095,6 +1104,11  <at>  <at>  int op_check_events(int ctr, u32 nr, u32 um, op_cpu cpu_type)
 		if (event->val != nr)
 			continue;

+		// Why do we have to do this, since event codes are supposed to be unique?
+		// See the big comment below.
+		if (ibm_power_proc && strcmp(evt_name, event->name))
+			continue;
+
 		ret = OP_OK_EVENT;

 		if ((event->counter_mask & ctr_mask) == 0)
 <at>  <at>  -1108,7 +1122,28  <at>  <at>  int op_check_events(int ctr, u32 nr, u32 um, op_cpu cpu_type)
 				if (event->unit->um[i].value == um)
 					break;
 			}
-			if (i == event->unit->num)
+			/* A small number of events on the IBM Power8 processor have real event
+			 * codes that are larger than sizeof(int). Rather than change the width of
+			 * the event code everywhere to be a long int (which would include having to
+			 * change the sample file format), we have defined some internal-use-only
+			 * unit masks for those events. In oprofile's power8 events file, we have
+			 * truncated those event codes to integer size, and the truncated bits are
+			 * used as a unit mask value which is ORed into the event code by
+			 * libpe_utils/op_pe_utils.cpp:_get_event_code(). This technique allowed
+			 * us to handle this situation with minimal code perturbation.  The one
+			 * downside is that the truncated event codes are not unique.  So in this
+			 * function, where we're searching for events by 'nr' (i.e., the event code),
+			 * we have to also make sure the name matches.
+			 *
+			 * If the user gives us an event specification such as:
+			 *      PM_L1MISS_LAT_EXC_256:0x0:1:1
+			 * the above code will actually find a non-zero unit mask for this event and
+			 * we'd normally fail at this point since the user passed '0x0' for a unit mask.
+			 * But we don't expose these internal-use-only UMs to the user, so there's
+			 * no way for them to know about it or to try to use it in their event spec;
+			 * thus, we handle it below.
+			 */
+			if ((i == event->unit->num) && !((um == 0) && ibm_power_proc))
 				ret |= OP_INVALID_UM;
 		}

diff --git a/libop/op_events.h b/libop/op_events.h
index be609f7..ec345e5 100644
--- a/libop/op_events.h
+++ b/libop/op_events.h
 <at>  <at>  -113,7 +113,7  <at>  <at>  enum op_event_check {
  *
  * \sa op_cpu, OP_EVENTS_OK
  */
-int op_check_events(int ctr, u32 event, u32 um, op_cpu cpu_type);
+int op_check_events(char * name, int ctr, u32 event, u32 um, op_cpu cpu_type);

 /**
  * free memory used by any call to above function. Need to be called only once
diff --git a/libpe_utils/op_pe_utils.cpp b/libpe_utils/op_pe_utils.cpp
index 8c69894..c5b6ee7 100644
--- a/libpe_utils/op_pe_utils.cpp
+++ b/libpe_utils/op_pe_utils.cpp
 <at>  <at>  -379,7 +379,7  <at>  <at>  out:
 static void _get_event_code(operf_event_t * event, op_cpu cpu_type)
 {
 	FILE * fp;
-	char oprof_event_code[9];
+	char oprof_event_code[11];
 	string command;
 	u64 base_code, config;
 	char buf[20];
 <at>  <at>  -412,7 +412,6  <at>  <at>  static void _get_event_code(operf_event_t * event, op_cpu cpu_type)

 
 #if defined(__i386__) || defined(__x86_64__)
-	char mask[OP_MAX_UM_NAME_LEN];
 	// Setup EventSelct[11:8] field for AMD
 	const char * vendor_AMD = "AuthenticAMD";
 	if (op_is_cpu_vendor((char *)vendor_AMD)) {
 <at>  <at>  -422,8 +421,10  <at>  <at>  static void _get_event_code(operf_event_t * event, op_cpu cpu_type)

 	// Setup EventSelct[7:0] field
 	config |= base_code & 0xFFULL;
-
-	// Setup unitmask field
+#endif
+#if defined(__i386__) || defined(__x86_64__) || defined(__powerpc64__)
+	char mask[OP_MAX_UM_NAME_LEN];
+// Setup unitmask field
 handle_named_um:
 	if (event->um_name[0]) {
 		command = OP_BINDIR;
 <at>  <at>  -489,7 +490,12  <at>  <at>  handle_named_um:
 			strncpy(event->um_name, mask, OP_MAX_UM_NAME_LEN - 1);
 			goto handle_named_um;
 		}
+#if defined(__powerpc64__)
+		config = base_code;
+		config |= ((event->evt_um & 0xFFULL) << 32);
+#else
 		config |= ((event->evt_um & 0xFFULL) << 8);
+#endif
 	} else {
 		config |= ((event->evt_um & 0xFFULL) << 8);
 	}
 <at>  <at>  -505,6 +511,7  <at>  <at>  handle_named_um:
 		}
 	}
 	event->evt_code = config;
+	cverb << vdebug << "Final event code is " << hex << event->evt_code << endl;
 }

 #if PPC64_ARCH
diff --git a/utils/ophelp.c b/utils/ophelp.c
index e38e417..a80fec8 100644
--- a/utils/ophelp.c
+++ b/utils/ophelp.c
 <at>  <at>  -180,7 +180,7  <at>  <at>  static void check_event(struct parsed_event * pev,
 	if (pev->unit_mask_name)
 		ret = 0;
 	else
-		ret = op_check_events(0, event->val, pev->unit_mask, cpu_type);
+		ret = op_check_events(pev->name, 0, event->val, pev->unit_mask, cpu_type);

 	if (ret & OP_INVALID_UM) {
 		fprintf(stderr, "Invalid unit mask 0x%x for event %s\n",
--

-- 
1.7.1

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
Maynard Johnson | 19 Nov 23:51 2014
Picon

[PATCH] Add support for IBM Power event codes longer than sizeof int

Hi, Henry,
The patch below addresses the issue you raised yesterday on the oprofile
list (subject "PM_L1MISS_LAT_EXC_xxx events").  At first, I looked into
what it would take to add general support for long event codes, but I soon
discovered that it would entail a change to the sample file format, and
I was loathe to do that just for eight events from IBM POWER8.  I verified
that there are no other events for any architecture other than these eight
that have event codes longer than an integer. So I implemented a bit of a
hack, using the concept of a unit mask as an event qualifier. It's an OK
fit, conceptually, it's just a bit hacky because no other ppc64 events
use unit masks.  But the fix was fairly small and simple, so I think it's
fine.

Please give it a test and let me know how it works for you.

Thanks!
-Maynard

------------------------------------------------------------------------

Add support for IBM Power event codes longer than sizeof int

A small number of events on newer IBM Power processors have event codes
that are larger than sizeof(int). Rather than change the width of the
event code everywhere to be a long int (which would include having to
change the sample file format), we have defined some internal-use-only
unit masks for those events. These unit masks are not shown in the ophelp
output, and IBM Power users should never use them in event specifications;
instead, they should use the usual 'null' unit mask value of '0x0' in event
specifications -- e.g.,
       PM_L1MISS_LAT_EXC_256:0x0:0:1

See libpe_utils/op_pe_utils.cpp:_get_event_code for how these unit masks are
used.

Signed-off-by: Maynard Johnson <maynardj <at> us.ibm.com>
---
 events/ppc64/power8/events     |   16 ++++++++--------
 events/ppc64/power8/unit_masks |    8 ++++++++
 libop/op_events.c              |   25 ++++++++++++++++++++++++-
 libpe_utils/op_pe_utils.cpp    |   14 ++++++++++----
 4 files changed, 50 insertions(+), 13 deletions(-)

diff --git a/events/ppc64/power8/events b/events/ppc64/power8/events
index cc1163a..012ca89 100644
--- a/events/ppc64/power8/events
+++ b/events/ppc64/power8/events
 <at>  <at>  -451,10 +451,10  <at>  <at>  event:0x30a8 counters:0,1,2,3 um:zero minimum:10000 name:PM_ISU_REJ_VS0 : VS0 IS
 event:0x30aa counters:0,1,2,3 um:zero minimum:10000 name:PM_ISU_REJ_VS1 : VS1 ISU reject
 event:0x38a8 counters:0,1,2,3 um:zero minimum:10000 name:PM_ISU_REJ_VSU : ISU
 event:0x30b8 counters:0,1,2,3 um:zero minimum:10000 name:PM_ISYNC : Isync count per thread
-event:0x200301ea counters:2 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_1024 : Reload latency
exceeded 1024 cyc
-event:0x200401ec counters:3 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_2048 : Reload latency
exceeded 2048 cyc
-event:0x200101e8 counters:0 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_256 : Reload latency
exceeded 256 cyc
-event:0x200201e6 counters:1 um:zero minimum:10000 name:PM_L1MISS_LAT_EXC_32 : Reload latency
exceeded 32 cyc
+event:0x200301ea counters:2 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_1024 : Reload
latency exceeded 1024 cyc
+event:0x200401ec counters:3 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_2048 : Reload
latency exceeded 2048 cyc
+event:0x200101e8 counters:0 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_256 : Reload
latency exceeded 256 cyc
+event:0x200201e6 counters:1 um:L1_latency minimum:10000 name:PM_L1MISS_LAT_EXC_32 : Reload
latency exceeded 32 cyc
 event:0x26086 counters:1 um:zero minimum:10000 name:PM_L1PF_L2MEMACC : valid when first beat of data
comes in for an L1pref where data came from mem(or L4)
 event:0x1002c counters:0 um:zero minimum:10000 name:PM_L1_DCACHE_RELOADED_ALL : L1 data cache
reloaded for demand or prefetch .
 event:0x408c counters:0,1,2,3 um:zero minimum:10000 name:PM_L1_DEMAND_WRITE : Instruction Demand
sectors wriittent into IL1
 <at>  <at>  -879,10 +879,10  <at>  <at>  event:0x10054 counters:0 um:zero minimum:10000 name:PM_PUMP_CPRED : Pump predict
 event:0x40052 counters:3 um:zero minimum:10000 name:PM_PUMP_MPRED : Pump Mis prediction Counts
across all types of pumpsfor all data types excluding data prefetch (demand load,inst prefetch,inst fetch,xlate).
 event:0x16081 counters:0 um:zero minimum:10000 name:PM_RC0_ALLOC : 0.0
 event:0x16080 counters:0 um:zero minimum:10000 name:PM_RC0_BUSY : RC mach 0 Busy. Used by PMU to sample
ave RC livetime(mach0 used as sample point)
-event:0x200301ea counters:2 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_1024 : Reload latency
exceeded 1024 cyc
-event:0x200401ec counters:3 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_2048 : Threshold
counter exceeded a value of 2048
-event:0x200101e8 counters:0 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_256 : Threshold
counter exceed a count of 256
-event:0x200201e6 counters:1 um:zero minimum:10000 name:PM_RC_LIFETIME_EXC_32 : Reload latency
exceeded 32 cyc
+event:0x200301ea counters:2 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_1024 : Reload
latency exceeded 1024 cyc
+event:0x200401ec counters:3 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_2048 :
Threshold counter exceeded a value of 2048
+event:0x200101e8 counters:0 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_256 : Threshold
counter exceed a count of 256
+event:0x200201e6 counters:1 um:rc_machine minimum:10000 name:PM_RC_LIFETIME_EXC_32 : Reload
latency exceeded 32 cyc
 event:0x36088 counters:2 um:zero minimum:10000 name:PM_RC_USAGE : Continuous 16 cycle(2to1) window
where this signals rotates thru sampling each L2 RC machine busy. PMU uses this wave to then do 16 cyc count
to sample total number of machs running
 event:0x34808e counters:3 um:zero minimum:10000 name:PM_RD_CLEARING_SC : rd clearing sc
 event:0x34808c counters:3 um:zero minimum:10000 name:PM_RD_FORMING_SC : rd forming sc
diff --git a/events/ppc64/power8/unit_masks b/events/ppc64/power8/unit_masks
index 988dd41..203af97 100644
--- a/events/ppc64/power8/unit_masks
+++ b/events/ppc64/power8/unit_masks
 <at>  <at>  -5,5 +5,13  <at>  <at> 
 #
 # ppc64 POWER8 possible unit masks
 #
+# NOTE: The 'rc_machine' and 'L1_latency' unit masks are for internal use only,
+#       to workaround oprofile's 32-bit limitation for event codes.
+#       See libpe_utils/op_pe_utils.cpp:_get_event_code for how these codes are
+#       used.
 name:zero type:mandatory default:0x0
 	0x0 No unit mask
+name:rc_machine type:mandatory default:0xde
+	0xde Thresholdable start/stop for rc machine for sampled instruction
+name:L1_latency type:mandatory default:0x67
+	0x67 Thresholdable start/stop for L1 sampled instruction load miss/reload
diff --git a/libop/op_events.c b/libop/op_events.c
index 8bfd3d2..434d7c8 100644
--- a/libop/op_events.c
+++ b/libop/op_events.c
 <at>  <at>  -1081,12 +1081,21  <at>  <at>  static int _is_um_valid_bitmask(struct op_event * event, u32 passed_um)
 	return retval;
 }

+static int _is_ppc64_cpu_type(op_cpu cpu_type) {
+	char const * cpu_name = op_get_cpu_name(cpu_type);
+	if (strncmp(cpu_name, "ppc64/power", strlen("ppc64/power")) == 0)
+		return 1;
+	else
+		return 0;
+}
+
 int op_check_events(int ctr, u32 nr, u32 um, op_cpu cpu_type)
 {
 	int ret = OP_INVALID_EVENT;
 	size_t i;
 	u32 ctr_mask = 1 << ctr;
 	struct list_head * pos;
+	int ibm_power_proc = _is_ppc64_cpu_type(cpu_type);

 	load_events(cpu_type);

 <at>  <at>  -1108,7 +1117,21  <at>  <at>  int op_check_events(int ctr, u32 nr, u32 um, op_cpu cpu_type)
 				if (event->unit->um[i].value == um)
 					break;
 			}
-			if (i == event->unit->num)
+			/* A small number of events on newer IBM Power processors have event codes
+			 * that are larger than sizeof(int). Rather than change the width of the
+			 * event code everywhere to be a long int (which would include having to
+			 * change the sample file format), we have defined some internal-use-only
+			 * unit masks for those events, which are to be ORed into the event code
+			 * by libpe_utils/op_pe_utils.cpp:_get_event_code. If the user gives us an
+			 * event specification such as:
+			 *      PM_L1MISS_LAT_EXC_256:0x0:1:1
+			 * we will actually find a non-zero unit mask for this event and would
+			 * normally fail at this point since the user passed '0x0' for a unit mask.
+			 * But we don't expose these internal-use-only UMs to the user, so there's
+			 * no way for them to know about it or to try to use it in their event spec;
+			 * thus, we handle it below.
+			 */
+			if ((i == event->unit->num) && !((um == 0) && ibm_power_proc))
 				ret |= OP_INVALID_UM;
 		}

diff --git a/libpe_utils/op_pe_utils.cpp b/libpe_utils/op_pe_utils.cpp
index 8c69894..9c5aeb4 100644
--- a/libpe_utils/op_pe_utils.cpp
+++ b/libpe_utils/op_pe_utils.cpp
 <at>  <at>  -379,7 +379,7  <at>  <at>  out:
 static void _get_event_code(operf_event_t * event, op_cpu cpu_type)
 {
 	FILE * fp;
-	char oprof_event_code[9];
+	char oprof_event_code[11];
 	string command;
 	u64 base_code, config;
 	char buf[20];
 <at>  <at>  -412,7 +412,6  <at>  <at>  static void _get_event_code(operf_event_t * event, op_cpu cpu_type)

 
 #if defined(__i386__) || defined(__x86_64__)
-	char mask[OP_MAX_UM_NAME_LEN];
 	// Setup EventSelct[11:8] field for AMD
 	const char * vendor_AMD = "AuthenticAMD";
 	if (op_is_cpu_vendor((char *)vendor_AMD)) {
 <at>  <at>  -422,8 +421,10  <at>  <at>  static void _get_event_code(operf_event_t * event, op_cpu cpu_type)

 	// Setup EventSelct[7:0] field
 	config |= base_code & 0xFFULL;
-
-	// Setup unitmask field
+#endif
+#if defined(__i386__) || defined(__x86_64__) || defined(__powerpc64__)
+	char mask[OP_MAX_UM_NAME_LEN];
+// Setup unitmask field
 handle_named_um:
 	if (event->um_name[0]) {
 		command = OP_BINDIR;
 <at>  <at>  -489,7 +490,12  <at>  <at>  handle_named_um:
 			strncpy(event->um_name, mask, OP_MAX_UM_NAME_LEN - 1);
 			goto handle_named_um;
 		}
+#if defined(__powerpc64__)
+		config = base_code;
+		config |= ((event->evt_um & 0xFFULL) << 32);
+#else
 		config |= ((event->evt_um & 0xFFULL) << 8);
+#endif
 	} else {
 		config |= ((event->evt_um & 0xFFULL) << 8);
 	}
--

-- 
1.7.1

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
Henry May | 19 Nov 02:47 2014
Picon

PM_L1MISS_LAT_EXC_xxx events

I have RHEL7 on a P8 and am unable to collect any of the PM_L1MISS_LAT_EXC_xxx events with ocount.  Any suggestions?

[root <at> pftul3 ~]# /opt/at7.0/bin/ocount -s -c -b -e PM_L1MISS_LAT_EXC_32
perf_event_open failed with Invalid argument
Caught runtime error while setting up counters
Internal Error.  Perf event setup failed.
Error running ocount

Henry May
IBM InfoSphere Streams Performance
hjmay <at> us.ibm.com
720-342-8873
Tie: 963-8873
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
oprofile-list mailing list
oprofile-list <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
Richard Purdie | 7 Nov 13:32 2014

[PATCH] opjitconv: Fix incorrect open() flag

When using compile fortification options on a mips build we saw:

| In file included from /media/build1/poky/build/tmp/sysroots/qemumips/usr/include/fcntl.h:302:0,
|                  from opjitconv.c:25:
| In function 'open',
|     inlined from 'copy_dumpfile' at opjitconv.c:219:6:
| /media/build1/poky/build/tmp/sysroots/qemumips/usr/include/bits/fcntl2.h:50:4: error: call
to '__open_missing_mode' declared with attribute error: open with O_CREAT in second argument needs 3 arguments
|     __open_missing_mode ();
|     ^
| Makefile:440: recipe for target 'opjitconv.o' failed

Why does this only happen on mips? mips has:

O_CREAT = 0x100 and S_IRUSR = 0400 and these (in hex and otcal) are equivalent. 
Most other platforms have O_CREAT = 0100.

Obviously the call is simply wrong so fix it...

Signed-off-by: Richard Purdie <richard.purdie <at> linuxfoundation.org>

Index: oprofile-1.0.0/opjitconv/opjitconv.c
===================================================================
--- oprofile-1.0.0.orig/opjitconv/opjitconv.c	2014-09-12 14:39:47.000000000 +0000
+++ oprofile-1.0.0/opjitconv/opjitconv.c	2014-11-06 13:14:25.941639003 +0000
 <at>  <at>  -216,7 +216,7  <at>  <at> 
 	int file_locked = 0;
 	unsigned int usecs_waited = 0;
 	int rc = OP_JIT_CONV_OK;
-	int fd = open(dumpfile, S_IRUSR);
+	int fd = open(dumpfile, O_CREAT, S_IRUSR|S_IWUSR);
 	if (fd < 0) {
 		perror("opjitconv failed to open JIT dumpfile");
 		return OP_JIT_CONV_FAIL;

------------------------------------------------------------------------------

Gmane