[PATCH 2/2] S/390: Enhance the user space tools for System z hardware sampling
Andreas Krebbel <krebbel <at> linux.vnet.ibm.com>
2011-11-02 15:10:58 GMT
This patch makes use of the event mechanism to allow for dynamic
enabling and disabling of the System z hardware sampling facility with
the OProfile user space tools.
A single virtual counter is created which can be used to
enable/disable hardware sampling dynamically from user space. The
counter can be used with the two events 0 and 1. Using event 0
enables use of timer based sampling while event 1 turns on hardware
sampling. These values have to stay like this since the
/dev/oprofile/0/event always has to match
/dev/oprofile/hwsampling/hwsampler content in order to support the
existing interface in parallel. Apart from 'event' only the 'kernel'
and 'user' flags are evaluated by the kernel code.
This adds two new opcontrol options --s390hwsampbufsize and
--s390hwsamprate which can be used to set the two hardware sampling
specific values at opcontrol cmdline.
Signed-off-by: Andreas Krebbel <krebbel <at> linux.vnet.ibm.com>
---
doc/oprofile.xml | 62 ++++++++++++++++++
events/Makefile.am | 4 -
events/s390x/basic_mode_sampling_v1/events | 8 ++
events/s390x/basic_mode_sampling_v1/unit_masks | 8 ++
libop/op_cpu_type.c | 1
libop/op_cpu_type.h | 1
libop/op_events.c | 3
libpp/op_header.cpp | 10 +-
utils/opcontrol | 84 +++++++++++++++++++++++++
utils/ophelp.c | 4 +
10 files changed, 179 insertions(+), 6 deletions(-)
--- a/doc/oprofile.xml
+++ b/doc/oprofile.xml
<at> <at> -1328,6 +1328,68 <at> <at> Note: * All IBS fetch event must have th
</sect2>
+<sect2 id="systemz">
+<title>IBM System z hardware sampling support</title>
+<para>
+IBM System z provides a facility which does instruction sampling as
+part of the CPU. This has great advantages over the timer based
+sampling approach like better sampling resolution with less overhead
+and the possibility to get samples within code sections where
+interrupts are disabled (useful especially for Linux kernel code).
+</para>
+<para>
+A public description of the System z CPU-Measurement Facilities can be
+found here:
+<ulink
url="http://www-01.ibm.com/support/docview.wss?uid=isg26fcd1cc32246f4c8852574ce0044734a">The
Load-Program-Parameter and CPU-Measurement Facilities</ulink>
+</para>
+<para>
+System z hardware sampling can be used for Linux instances in LPAR
+mode. The hardware sampling support used by OProfile was introduced
+for System z10 in October 2008.
+</para>
+<para>
+To enable hardware sampling for an LPAR you must activate the LPAR
+with authorization for basic sampling control. See the "Support
+Element Operations Guide" for your mainframe system for more
+information.
+</para>
+<para>
+The hardware sampling facility can be enabled and disabled using the
+event interface. A `virtual' counter 0 has been defined that supports
+two events, TIMER and HWSAMPLING. By default the HWSAMPLING event is
+used on machines providing the facility. For both events only the
+`kernel' and `user' flags are evaluated by the kernel module.
+</para>
+<para>
+Altough <command>opcontrol</command> requires to specify a value for
+`count' this value is ignored.
+</para>
+<para>
+The unit mask `um' is required to be zero.
+</para>
+<para>
+Since the counter is needed for both available sampling modes it is
+always enabled and cannot be disabled by writing into
+the <filename>events/0/enabled</filename> file.
+</para>
+<para>
+The opcontrol tool provides a few options specific to System z
+hardware sampling. The hardware sampling rate and the sampling buffer
+size is stored into the OProfile configuration file.
+</para>
+
+<itemizedlist>
+<listitem>--s390hwsamprate="cycles"
+Number of machine cycles between taking samples.</listitem>
+<listitem>--s390hwsampbufsize="buffers"
+Number of 2MB areas used per CPU for storing sample data. The best
+size for the sample memory depends on the particular system and the
+workload to be measured. Providing the sampler with too little memory
+results in lost samples. Reserving too much system memory for the
+sampler impacts the overall performance and, hence, also the workload
+to be measures</listitem>
+</itemizedlist>
+</sect2>
<sect2 id="misuse">
<title>Dangerous counter settings</title>
--- a/events/Makefile.am
+++ b/events/Makefile.am
<at> <at> -68,8 +68,8 <at> <at> event_files = \
ppc/7450/events ppc/7450/unit_masks \
ppc/e500/events ppc/e500/unit_masks \
ppc/e500v2/events ppc/e500v2/unit_masks \
- ppc/e300/events ppc/e300/unit_masks
-
+ ppc/e300/events ppc/e300/unit_masks \
+ s390x/basic_mode_sampling_v1/events s390x/basic_mode_sampling_v1/unit_masks
install-data-local:
for i in ${event_files} ; do \
dir=`dirname $$i` ; \
--- /dev/null
+++ b/events/s390x/basic_mode_sampling_v1/events
<at> <at> -0,0 +1,8 <at> <at>
+# Copyright OProfile authors
+# Copyright (c) International Business Machines, 2011.
+# Contributed by Andreas Krebbel <krebbel <at> linux.vnet.ibm.com>.
+#
+# S/390 Basic Mode Sampling events
+#
+event:0x00 counters:0 um:zero minimum:100 name:TIMER : Sampling using timer interrupt
+event:0x01 counters:0 um:zero minimum:0 name:HWSAMPLING : Sampling using Basic Mode Hardware Sampling
--- /dev/null
+++ b/events/s390x/basic_mode_sampling_v1/unit_masks
<at> <at> -0,0 +1,8 <at> <at>
+# Copyright OProfile authors#
+# Copyright (c) International Business Machines, 2011.
+# Contributed by Andreas Krebbel <krebbel <at> linux.vnet.ibm.com>.
+#
+# S/390 Basic Mode Hardware Sampling unit masks
+#
+name:zero type:mandatory default:0x0
+ 0x0 No unit mask
--- a/libop/op_cpu_type.c
+++ b/libop/op_cpu_type.c
<at> <at> -94,6 +94,7 <at> <at> static struct cpu_descr const cpu_descrs
{ "ARMv7 Scorpion", "arm/armv7-scorpion", CPU_ARM_SCORPION, 5 },
{ "ARMv7 ScorpionMP", "arm/armv7-scorpionmp", CPU_ARM_SCORPIONMP, 5 },
{ "Intel Sandy Bridge microarchitecture", "i386/sandybridge", CPU_SANDYBRIDGE, 8 },
+ { "IBM System z with basic mode hardware sampling support", "s390x/basic_mode_sampling_v1",
CPU_S390_HWSAMPV1, 1 },
};
static size_t const nr_cpu_descrs = sizeof(cpu_descrs) / sizeof(struct cpu_descr);
--- a/libop/op_cpu_type.h
+++ b/libop/op_cpu_type.h
<at> <at> -91,6 +91,7 <at> <at> typedef enum {
CPU_ARM_SCORPION, /**≤ ARM SCORPION */
CPU_ARM_SCORPIONMP, /**≤ ARM SCORPIONMP */
CPU_SANDYBRIDGE, /* Intel Sandy-Bridge microarchitecture */
+ CPU_S390_HWSAMPV1, /* IBM System z with Basic Mode Sampling support */
MAX_CPU_TYPE
} op_cpu;
--- a/libop/op_events.c
+++ b/libop/op_events.c
<at> <at> -1125,6 +1125,9 <at> <at> void op_default_event(op_cpu cpu_type, s
case CPU_PPC_E300:
descr->name = "CPU_CLK";
break;
+ case CPU_S390_HWSAMPV1:
+ descr->name = "HWSAMPLING";
+ break;
// don't use default, if someone add a cpu he wants a compiler
// warning if he forgets to handle it here.
--- a/libpp/op_header.cpp
+++ b/libpp/op_header.cpp
<at> <at> -254,11 +254,13 <at> <at> string const describe_cpu(opd_header con
str = xml_utils::get_profile_header(cpu_name, header.cpu_speed);
} else {
str += string("CPU: ") + op_get_cpu_type_str(cpu);
- str += ", speed ";
+ if (header.cpu_speed > 0) {
+ ostringstream ss;
- ostringstream ss;
- ss << header.cpu_speed;
- str += ss.str() + " MHz (estimated)";
+ str += ", speed ";
+ ss << header.cpu_speed;
+ str += ss.str() + " MHz (estimated)";
+ }
}
return str;
}
--- a/utils/opcontrol
+++ b/utils/opcontrol
<at> <at> -239,6 +239,11 <at> <at> opcontrol: usage:
--xen Xen image (for Xen only)
--active-domains=<list> List of domains in profiling session (for Xen)
(list contains domain ids separated by commas)
+
+ System z specific options
+
+ --s390hwsamprate=cycles Number of machine cycles between taking samples.
+ --s390hwsampbufsize=buffers Number of 2MB areas used per CPU for storing sample data.
EOF
}
<at> <at> -381,6 +386,11 <at> <at> do_init()
IBS_OP_COUNT=0
IBS_OP_UNITMASK=0
+ # System z specific values
+ S390_HW_SAMPLER=0
+ S390_HW_SAMPLER_RATE=0
+ S390_HW_SAMPLER_BUFSIZE=0
+
OPROFILED="$OPDIR/oprofiled"
# location for daemon setup information
<at> <at> -405,6 +415,9 <at> <at> do_init()
IS_TIMER=1
else
case "$CPUTYPE" in
+ s390x/basic_mode_sampling_v1)
+ S390_HW_SAMPLER=1
+ ;;
ia64/*)
IS_PERFMON=$KERNEL_SUPPORT
;;
<at> <at> -496,6 +509,14 <at> <at> do_save_setup()
if test "$XEN_RANGE"; then
echo "XEN_RANGE=$XEN_RANGE" >> $SETUP_FILE
fi
+ if test "$S390_HW_SAMPLER" == "1"; then
+ if test "$S390_HWSAMPLER_RATE" != "0"; then
+ echo "S390_HW_SAMPLER_RATE=$S390_HW_SAMPLER_RATE" >> $SETUP_FILE
+ fi
+ if test "$S390_HWSAMPLER_BUFSIZE" != "0"; then
+ echo "S390_HW_SAMPLER_BUFSIZE=$S390_HW_SAMPLER_BUFSIZE" >> $SETUP_FILE
+ fi
+ fi
SETUP_FILE="$SAVE_SETUP_FILE"
}
<at> <at> -538,6 +559,32 <at> <at> do_load_setup()
done < $SETUP_FILE
}
+# Various checks related to Hardware sampler option
+check_platform_args()
+{
+ case "$CPUTYPE" in
+ s390x/basic_mode_sampling_v1)
+ if test "$S390_HW_SAMPLER_RATE" == "0"; then
+ return
+ fi
+ if test -r $MOUNT/hwsampling/hw_min_interval; then
+ min_interval=`cat $MOUNT/hwsampling/hw_min_interval`
+ fi
+ if test -r $MOUNT/hwsampling/hw_max_interval; then
+ max_interval=`cat $MOUNT/hwsampling/hw_max_interval`
+ fi
+
+ if test "$S390_HW_SAMPLER_RATE" -lt "$min_interval" -o \
+ "$S390_HW_SAMPLER_RATE" -gt "$max_interval"; then
+ echo "Invalid value for hardware sampling rate" >&2
+ echo "should be between $min_interval and $max_interval" >&2
+ exit 1
+ fi
+ ;;
+ *)
+ ;;
+ esac
+}
check_valid_args()
{
<at> <at> -971,6 +1018,19 <at> <at> do_options()
exec $OPHELP
;;
+ --s390hwsamprate)
+ error_if_not_number "$arg" "$val"
+ S390_HW_SAMPLER_RATE=$val
+ DO_SETUP=yes
+ ;;
+
+ --s390hwsampbufsize)
+ error_if_not_number "$arg" "$val"
+ S390_HW_SAMPLER_BUFSIZE=$val
+ DO_SETUP=yes
+ ;;
+
+
*)
echo "Unknown option \"$arg\". See opcontrol --help" >&2
exit 1
<at> <at> -1418,6 +1478,15 <at> <at> do_param_setup()
return
fi
+ if test "$S390_HW_SAMPLER" == "1"; then
+ if test "$S390_HW_SAMPLER_RATE" != "0"; then
+ echo $S390_HW_SAMPLER_RATE >$MOUNT/hwsampling/hw_interval
+ fi
+ if test "$S390_HW_SAMPLER_BUFSIZE" != "0"; then
+ echo $S390_HW_SAMPLER_BUFSIZE >$MOUNT/hwsampling/hw_sdbt_blocks
+ fi
+ fi
+
# use the default setup if none set
if test "$NR_CHOSEN" = 0; then
set_event 0 $DEFAULT_EVENT
<at> <at> -1701,6 +1770,20 <at> <at> do_status()
echo "CPU buffer size: $CPU_BUF_SIZE"
fi
fi
+ if test "$S390_HW_SAMPLER" == "1"; then
+ echo -n "System z hardware sampling rate in cycles: "
+ if test "$S390_HW_SAMPLER_RATE" == "0"; then
+ cat $MOUNT/hwsampling/hw_interval
+ else
+ echo "$S390_HW_SAMPLER_RATE"
+ fi
+ echo -n "System z hardware sampling buffer size (in 2MB areas): "
+ if test "$S390_HW_SAMPLER_BUFSIZE" == "0"; then
+ cat $MOUNT/hwsampling/hw_sdbt_blocks
+ else
+ echo "$S390_HW_SAMPLER_BUFSIZE"
+ fi
+ fi
exit 0
}
<at> <at> -1855,6 +1938,7 <at> <at> do_operations()
fi
if test "$SETUP" = "yes"; then
+ check_platform_args
check_valid_args
do_save_setup
fi
--- a/utils/ophelp.c
+++ b/utils/ophelp.c
<at> <at> -731,6 +731,10 <at> <at> int main(int argc, char const * argv[])
"Chapter 6: Performance Counters\n"
"http://www.atmel.com/dyn/resources/prod_documents/doc32000.pdf\n";
+ case CPU_S390_HWSAMPV1:
+ event_doc = "IBM System z Basic Mode Sampling\n";
+ break;
+
case CPU_RTC:
break;
------------------------------------------------------------------------------
RSA® Conference 2012
Save $700 by Nov 18
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev1
RSS Feed