headers
Stacey Son | 23 May 2013 00:17
Picon
Favicon
Subject: binmiscctl(8) (and imgact_binmisc kernel module)

binmiscctl(8) (and imgact_binmisc kernel module)

Hi all:

I added a command-line utility called 'binmiscctl' for the imgact_binmisc kernel module that I
previously proposed on this list.  As you may recall, imgact_binmisc is an image activator for
miscellaneous binary file types that are executed with the help of a user-level interpreter or emulator. 
It has been proposed that imgact_binmisc be added to the kernel as a module.  The main reason I created this
is to support cross building packages using qemu user mode (see my dev summit slides at
http://people.freebsd.org/~sson/imgact_binmisc/20130515-bsdcan-xbuild-ports.pdf) but there
are a lot of other applications for this module as well.  For example, Nathan Whitehorn previously
proposed on this list a similar code change (but much less general) to support transparently execute  LLVM
bitcode using the 'lli' JIT compiler.  This kernel module if flexible enough that it supports that as well.

Baptiste has also added support in poudrière for cross-building mips64 packages in a "cross jail" using
qemu user mode.  See his slides from BSDCan 2013 (pg. 7):

	http://people.freebsd.org/~bapt/modern-package-management.pdf

Bapt mentioned that he built over 10,000 mips64 packages in about 30 hours.  Of course, this is before adding
imgact_binmisc which greatly improves the cross build speed by allowing both native (amd64) cross build
tools to be used along with emulated mips64 binaries in a hybrid fashion.  With my limited testing of cross
building a handful of ports the overhead compared to building the port natively on a commodity amd64 host
is 2x to 4x using this kernel module.  Without the module the overhead is 10x or much more.

The recently added 'binmiscctl' command-line utility allows for easy configuration and management of
the image activators in this imgact_binmisc kernel module.   For example, to add an image activator for
qemu-mips64 (the qemu user mode emulator for mips64):

	# binmiscctl add mips64elf --interpreter "/usr/local/bin/qemu-mips64" --magic \
"\x7f\x45\x4c\x46\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08" \
--mask "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff"
(Continue reading)

Permalink | Reply | 0 comments |
headers
Orit Moskovich | 21 May 2013 06:56
Favicon

compatibility layer - workqueues

Hi,

I'm working on understanding the difference between Linux and FreeBSD interrupt handling.
I looked at the compatibility layer and noticed this:

*         Linux workqueues are implemented using FreeBSD taskqueues (under sys/ofed/include/linux/workqueue.h)

*         In linux, the function schedule_work() puts a job in the kernel global workqueue 'events'. This
workqueue consists of worker threads - one per processor

*         The compatibility layer wraps this function to a macro, that implements the functionality using
taskqueue_enqueue() and set it to work on taskqueue_thread, that executes its tasks in the context of a
kernel thread

*         BUT,  taskqueue_thread is initialized in:

o   sys/kern/subr_taskqueue.c  line 536:
TASKQUEUE_DEFINE_THREAD(thread);

o   which is defined in sys/taskqueue.h line 133
and run taskqueue_start_threads() with only 1 thread, and not MAXCPU

I'll appreciate your help understanding this issue,

Thanks,

Orit Moskovich

_______________________________________________
freebsd-arch <at> freebsd.org mailing list
(Continue reading)

Permalink | Reply | 2 comments |
headers
Orit Moskovich | 14 May 2013 12:04
Favicon

FreeBSD spinlock - compatibility layer

Hi,

I read about the FreeBSD mutex implementation for spinlock in the compatibility layer.
I might be wrong, but I noticed a code section that might be problematic:

Taken from http://svn.freebsd.org/base/release/9.1.0/sys/ofed/include/linux/spinlock.h:

static inline void
spin_lock_init(spinlock_t *lock)
{

        memset(&lock->m, 0, sizeof(lock->m));
        mtx_init(&lock->m, "lnxspin", NULL, MTX_DEF | MTX_NOWITNESS);
}

But MTX_DEF initializes mutex as a sleep mutex:

By default, MTX_DEF mutexes will context switch when they are already

     held.

There is a flag MTX_SPIN Which I think is the right one in this case .

I'd appreciate your take on this issue.

Thanks,

Orit Moskovich

_______________________________________________
(Continue reading)

Permalink | Reply | 20 comments |
headers
Justin Hibbits | 13 May 2013 09:20
Picon
Favicon

late suspend/early resume

I'd like to solicit opinions on adding new kobj device API calls,
device_late_suspend and device_early_resume.

I've been working on PowerPC suspend/resume, and certain devices must be
suspended last and resumed first on Apple hardware, namely the chipsets.
It happens that one device (uninorth) appears first in the devinfo chain,
while the other (mac-io) appears off a later PCI bus.  So, rather than
special casing this to suspend the mac-io and its children, as well as the
uninorth, last with specific calls, I'd like to add a device_suspend_late
and device_resume_early, that would simply be identical to
device_suspend/device_resume, but could be called after and before,
respectively, to do last minute order suspend and first pass
initialization, to initialize things like clocks required for other devices.

It's not difficult to explicitly call suspend and resume functions on the
chipsets, but I think it's cleaner to do it through the newbus/kobj
interface, and it might be useful for other architectures as well.

Thoughts?

- Justin
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 8 comments |
headers
Simon J. Gerraty | 11 May 2013 07:45
Favicon

Making bmake default? (was: Re: ports/173299: [exp-run] test bmake conversion)


On Mon, 6 May 2013 00:11:51 +0200, Baptiste Daroussin writes:
>The ports tree on current is still in very bad shape but I don't see
>anymore errors due to bmake specifically.
>
>You have my approval as portmgr to switch base make to bmake.

As an interim step, I would propose the change below.

Index: share/mk/bsd.own.mk
===================================================================
--- share/mk/bsd.own.mk	(revision 249461)
+++ share/mk/bsd.own.mk	(working copy)
 <at>  <at>  -254,6 +254,7  <at>  <at>  __DEFAULT_YES_OPTIONS = \
     BIND_UTILS \
     BINUTILS \
     BLUETOOTH \
+    BMAKE \
     BOOT \
     BSD_CPIO \
     BSNMP \
 <at>  <at>  -361,7 +362,6  <at>  <at>  __DEFAULT_NO_OPTIONS = \
     BIND_LIBS \
     BIND_SIGCHASE \
     BIND_XML \
-    BMAKE \
     BSDCONFIG \
     BSD_GREP \
     CLANG_EXTRAS \
_______________________________________________
(Continue reading)

Permalink | Reply | 0 comments |
headers
Editor IJTEMT | 10 May 2013 05:37

Call for Papers IJTEMT. Kindly impart in your University/Organization/College/Colleagues/Academia/Circle.

    INTERNATIONAL JOURNAL OF TRENDS IN ECONOMICS MANAGEMENT & TECHNOLOGY
     IJTEMT invites you to submit your research paper for publishing in
                      Volume II, Issue II ( June 2013).
                     CALL FOR PAPERS VOLUME II, ISSUE II

                               www.ijtemt.org

   About IJTEMT

   International Journal of Trends in Economics Management and Technology
   (IJTEMT) in an International Academic Journal e-published bimonthly in
   India and open to the world.
   In this present interdisciplinary era, here at IJTEMT, a group of
   intellectual came together to find a common platform for three major
   components of any economy i.e., Economics, Management and Technology.
   Here we provide a forum to bridge the gap between the brushed-up
   professional in their respective fields and the new researcher which
   will results in better understanding and fruitful outcomes.
   The focus of this journal is to publish paper on economics management
   and technology. Submitted papers are reviewed by a full double blind
   manner by the technical committee of the journal. The audience for the
   journal is professionals from related fields, academicians and new
   students & research scholars.
   All submitted articles should report original, previously unpublished
   research results, experimental or theoretical, and will be
   peer-reviewed. Articles submitted to the journal should meet these
   criteria and must not be under consideration for publication elsewhere.
   Manuscripts should follow the style of the journal and are subject to
   both review and editing.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Garrett Cooper | 7 May 2013 22:05
Picon
Gravatar

[RFC] adding a variable to .mk and Makefile.inc1 to point to top of the FreeBSD source tree

Hi,
    A common pattern that I've seen at Isilon and something else that I've
wanted to have for a while is the ability to designate where the top of a
source tree was. This is important and helpful when dealing with source
files that build upon each other or depend on sources located in other
sections of the tree; contrib stuff needs to set .PATH appropriately to
point to sources at the top of the tree, sys stuff is riddled with S= in
order to point to where /sys, etc lives, we build upon FreeBSD within an
expected directory structure as well.
    I haven't come up with a name, but was wondering if this was a good
idea, and if so does anyone have any outstanding patches for this that can
be pushed into FreeBSD?
Thanks!
-Garrett
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 15 comments |
headers
John Baldwin | 7 May 2013 20:33
Picon
Favicon

Extending MADV_PROTECT

One of the issues I have with our current MADV_PROTECT is that it isn't very 
administrative-friendly.  That is, as a sysadmin I can't easily protect 
arbitrary processes from the OOM killer.  Instead, the binary has to be 
changed to invoke madvise().  Furthermore, once the protection is granted it 
can't be revoked.  Also, any binaries that want this have to be run as root.  
Instead, I would like to be able to both set and revoke this for existing 
processes and possibly even allow it to be inherited (so I can tag a top-level 
daemon that forks and have all its future children be protected for example).  
To that end I've whipped up a simple patch (against 8, but should port to HEAD 
easily if folks think it is a good idea) to add a new pprotect() system call 
and userland program (protect) that can be used similar to ktrace(1) either as 
a modifier when running a new program or as a tool for setting or clearing 
protection for existing processes.

The inherit feature isn't implemented yet, but it should be simple to do.  One 
would simply need a new flag that PPROT_INHERIT sets that is checked on fork 
and propagates P_PROTECTED if it is set.  Also, one other thought I had is 
that at some point we might want to make P_PROTECTED more fine-grained, e.g. 
by allowing for OOM "priorities".  To that end, it may make sense to add a new 
argument to protect, though you could also reserve part of the 'op' parameter 
to encode a priority.

The manpage for the proposed protect command is below, then the source of the 
command, then the patch to add pprotect():

PROTECT(1)              FreeBSD General Commands Manual             PROTECT(1)

NAME
     protect -- protect processes from being killed when swap space is
     exhausted
(Continue reading)

Permalink | Reply | 19 comments |
headers
Pawel Jakub Dawidek | 5 May 2013 22:14
Picon
Favicon

Building library that depends on another library.

Hi.

I'm trying to connect two libraries to the build without hacks and it
doesn't work...

My two libraries are libcapsicum and libnv. libcapsicum depends on libnv.
I want to specify this dependency explicitely in libcapsicum's Makefile:

	DPADD=	${LIBNV}
	LDADD=	-lnv

(LIBNV was added to bsd.libnames.mk, in case you wonder.)

From conversation with kan <at>  (Alexander Kabaev) declaring dependency
directly in library's Makefile is required for symbol versioning to
work. It just sounds right, too.

If this is done, libcapsicum doesn't compile:

	===> lib/libcapsicum (all)
	cc [...]
	cc [...]
	building static capsicum library
	ranlib libcapsicum.a
	cc [...]
	cc [...]
	make: don't know how to make
	/usr/home/pjd/obj/usr/home/pjd/p4/capsicum/tmp/usr/lib/libnv.a. Stop
	*** [all] Error code 2
(Continue reading)

Permalink | Reply | 11 comments |
headers
The Urban Shopper | 4 May 2013 18:35
Favicon

Negro League Baller Recalls the Era, You Gotta Have Balls, Entrepreneurs, Recipes, Blogs...

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>The Urban Shopper | May 2013</TITLE>
<META content="text/html; charset=iso-8859-1" http-equiv=Content-Type>
<STYLE type=text/css>
<!--
style1 {
	font-size: 10px;
	color: #666666;
}
a:link {
	text-decoration: none;
}
a:visited {
	text-decoration: none;
}
a:hover {
	text-decoration: underline;
}
a:active {
	text-decoration: none;
}
-->
</STYLE>

<META name=keywords 
content="african-american woman, african american, recipes, negro league
baseball, jackie robinson, derek jeter, magic johnson, tasers, cremation,
overdosing help, blackberry z10, urban blogs, fashion, beauty">
<META name=description 
content="May 2013 issue of TheUrbanShopper.com features watercolor artist
(Continue reading)

Permalink | Reply | 0 comments |
headers
Stacey Son | 2 May 2013 01:02
Picon
Favicon

A General or Miscellaneous Binary Image Activator

Hi All,

I have written a miscellaneous (or general) binary interpreter image activator to support cross building
mips64 ports using user mode qemu.   The source code can be found at:

	http://people.freebsd.org/~sson/imgact_binmisc/imgact_binmisc.h
	http://people.freebsd.org/~sson/imgact_binmisc/imgact_binmisc.c

I just saw Nathan Whitehorn's posting 'LLVM Image Activator' on this same list and hope we are not
duplicating work.  Like Nathan's this code invokes a user-level interpreter or emulator when it
recognizes the magic bytes in a header of a binary.  Unlike Nathan's it is much more general purpose.   It does
this by using a magic field that may be up to 'IBE_MAGIC_MAX'  (or 256) bytes long , an offset value in the
binary, and, optionally, a bit mask field the same size as the magic field that can mask out and ignore any of
the bits in the magic. Therefore, it is very flexible in supporting binary file types which I needed to
figure out the arch for ELF files.

Of course, image activators have been a source for security issues.  If this or something like this is
include in the kernel then I would recommend it be optional.  It has proven to be very useful for creating a
mixed arch cross build environment.  This allows running things like the cross compiler/toolchain and
/bin/sh natively on the host system which significantly helps performance.  Using this, and the user mode
qemu port I have been working on, I have successfully cross built well over 9000 mips64 packages from ports
using an old amd64 athlon (FYI, see https://wiki.freebsd.org/QemuUserModeHowTo for more information
about user mode qemu).

It is configured from user space using sysctl(3).  For example, the following is for adding an activator
that runs mips64 executables on an amd64 host using "user mode" qemu:

#include <sys/imgact_binmisc.h>

#define MIPS64_ELF_MAGIC                			"\x7f\x45\x4c\x46\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08"
(Continue reading)

Permalink | Reply | 2 comments |

Gmane