headers
Doug Barton | 1 Jun 2012 03:40
Picon
Favicon

Re: Allow small amount of memory be mlock()'ed by unprivileged process?

On 5/31/2012 5:23 AM, Andriy Gapon wrote:
> In fact, FreeBSD also has this rlimit and there seems to be full support for it on
> both user and kernel sides.
> OTOH, PRIV_VM_MLOCK privilege seems to be granted only to the super-user in the
> default configuration.  And this privilege kind of defeats the limit.
> 
> Perhaps, we should/could kill the privilege and set the limit to a sufficiently
> small/safe value for ordinary users?

I like this idea, but someone else in the thread (sorry, don't have it
handy) brought up the point that we don't want the aggregate of per-user
limits to be able to bring down the system either. So the right solution
would seem to be a reasonable per-user limit, and a cap on the maximum
total amount of locked pages for all unprivileged users, probably based
on some percentage of total available memory?

Doug

--

-- 

    This .signature sanitized for your protection
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 11 comments |
headers
Bryan Drewery | 1 Jun 2012 17:41
Gravatar

Re: Allow small amount of memory be mlock()'ed by unprivileged process?

On 5/31/2012 8:40 PM, Doug Barton wrote:
> On 5/31/2012 5:23 AM, Andriy Gapon wrote:
>> In fact, FreeBSD also has this rlimit and there seems to be full support for it on
>> both user and kernel sides.
>> OTOH, PRIV_VM_MLOCK privilege seems to be granted only to the super-user in the
>> default configuration.  And this privilege kind of defeats the limit.
>>
>> Perhaps, we should/could kill the privilege and set the limit to a sufficiently
>> small/safe value for ordinary users?
> 
> I like this idea, but someone else in the thread (sorry, don't have it
> handy) brought up the point that we don't want the aggregate of per-user
> limits to be able to bring down the system either. So the right solution
> would seem to be a reasonable per-user limit, and a cap on the maximum
> total amount of locked pages for all unprivileged users, probably based
> on some percentage of total available memory?
> 
> Doug
> 

I like this approach. A per-user ulimit, and a global max sysctl that
can be overridden, but by default based on a percentage of available memory.

--

-- 
Regards,
Bryan Drewery
Permalink | Reply | 9 comments |
headers
Giovanni Trematerra | 1 Jun 2012 19:53
Picon
Gravatar

[RFC] Kernel shared variables

Hello,
I'd like to discuss a way to provide a mechanism to share some read-only
data between kernel and user space programs avoiding syscall overhead,
implementing some them, such as gettimeofday(3) and time(3) as ordinary
user space routine.

The patch at
http://www.trematerra.net/patches/ksvar_experimental.patch

is in a very experimental stage. It's just a proof-of-concept.
Only works for an AMD64 kernel and only for 64-bit applications.
The idea is to have all the variables that we want to share between kernel
and user space into one or more consecutive pages of memory that will be
mapped read-only into every running process. At the start of the first
shared page
there'll be a table with as many entries as the number of the shared variables.
Each entry is a 32-bit value that is the offset between the start of the shared
page and the start of the variable in the page. The user space processes need
to find out the map address of shared page and use the table to access to the
shared variables.
Kernel will export a variable to user space as an index, so user space code
must refer to a specific index to access a kernel shared variable.
Let's take a quick look to the KPI/API for exporting/importing kernel
shared variables.
Say we want implement a routine to export an int from the kernel.
To define the variable to be exported inside the kernel you would use

KSVAR_DEFINE(0, int, test_value);

You have just defined an int variable named "test_value" at index 0.
(Continue reading)

Permalink | Reply | 9 comments |
headers
Lev Serebryakov | 1 Jun 2012 21:22
Picon

Re: [RFC] Kernel shared variables

Hello, Giovanni.
You wrote 1 июня 2012 г., 21:53:15:

GT> I'm looking forward to hear about your comments and suggestions.
   It is great, that you start this work!

   This approach was discussed several times according to my memory, as
 way to make cheap sysclass like gettimeofday() (because some
 Linux-orientet programs like to call them very often and have them
 cheap is very good idea), and every time conclusion was, that it is
 very good approach, but without any resulting code.

--

-- 
// Black Lion AKA Lev Serebryakov <lev <at> serebryakov.spb.ru>

_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-unsubscribe <at> freebsd.org"
Permalink | Reply | 0 comments |
headers
Konstantin Belousov | 1 Jun 2012 21:35
Picon

Re: [RFC] Kernel shared variables

On Fri, Jun 01, 2012 at 07:53:15PM +0200, Giovanni Trematerra wrote:
> Hello,
> I'd like to discuss a way to provide a mechanism to share some read-only
> data between kernel and user space programs avoiding syscall overhead,
> implementing some them, such as gettimeofday(3) and time(3) as ordinary
> user space routine.
> 
> The patch at
> http://www.trematerra.net/patches/ksvar_experimental.patch
> 
> is in a very experimental stage. It's just a proof-of-concept.
> Only works for an AMD64 kernel and only for 64-bit applications.
> The idea is to have all the variables that we want to share between kernel
> and user space into one or more consecutive pages of memory that will be
> mapped read-only into every running process. At the start of the first
> shared page
> there'll be a table with as many entries as the number of the shared variables.
> Each entry is a 32-bit value that is the offset between the start of the shared
> page and the start of the variable in the page. The user space processes need
> to find out the map address of shared page and use the table to access to the
> shared variables.
> Kernel will export a variable to user space as an index, so user space code
> must refer to a specific index to access a kernel shared variable.
> Let's take a quick look to the KPI/API for exporting/importing kernel
> shared variables.
> Say we want implement a routine to export an int from the kernel.
> To define the variable to be exported inside the kernel you would use
> 
> KSVAR_DEFINE(0, int, test_value);
>
(Continue reading)

Permalink | Reply | 4 comments |
headers
Bruce Evans | 1 Jun 2012 23:21
Picon

Re: [RFC] Kernel shared variables

On Fri, 1 Jun 2012, Giovanni Trematerra wrote:

> I'd like to discuss a way to provide a mechanism to share some read-only
> data between kernel and user space programs avoiding syscall overhead,
> implementing some them, such as gettimeofday(3) and time(3) as ordinary
> user space routine.

This is particularly unsuitable for implementing gettimeofday(), since for
it to work you would need to use approximately 1 CPU spinning in the
kernel to update the time every microsecond.  For time(3), it only needs
a relatively slow update.  For clock_gettime() with nansoeconds precision,
it is even more unsuitable.  For clock_gettime() with precisions between
1 second and 1 microseconds, it is intermediately unsuitable.

It also requires some complications for locking/atomicity and coherency
(much the same as in the kernel.  Not just for times.  For times, the
kernel handles locking/atomicity fairly well, and coherency fairly badly.

> The patch at
> http://www.trematerra.net/patches/ksvar_experimental.patch
>
> is in a very experimental stage. It's just a proof-of-concept.
> Only works for an AMD64 kernel and only for 64-bit applications.
> The idea is to have all the variables that we want to share between kernel
> and user space into one or more consecutive pages of memory that will be
> mapped read-only into every running process. At the start of the first
> shared page
> there'll be a table with as many entries as the number of the shared variables.
> Each entry is a 32-bit value that is the offset between the start of the shared
> page and the start of the variable in the page. The user space processes need
(Continue reading)

Permalink | Reply | 1 comment |
headers
Giovanni Trematerra | 1 Jun 2012 23:42
Picon
Gravatar

Re: [RFC] Kernel shared variables

On Fri, Jun 1, 2012 at 9:35 PM, Konstantin Belousov <kostikbel <at> gmail.com> wrote:
> On Fri, Jun 01, 2012 at 07:53:15PM +0200, Giovanni Trematerra wrote:
>> Hello,
>> I'd like to discuss a way to provide a mechanism to share some read-only
>> data between kernel and user space programs avoiding syscall overhead,
>> implementing some them, such as gettimeofday(3) and time(3) as ordinary
>> user space routine.
>>
>> The patch at
>> http://www.trematerra.net/patches/ksvar_experimental.patch
>>
>
> I very much dislike what you described, it makes ABI maintanence
> a nightmare.

While I respect your decision to dislike my work, could you please give me
some concrete examples of ABI maintenance nightmare? I mean not based
on speculation.

> Below is some mail I wrote around Spring 2009, making some notes about
> desired proposal.

I wonder why your proposal isn't on the Ideas Page wiki.
By the way, is this proposal valid?
http://wiki.freebsd.org/IdeasPage#Avoiding_syscall_overhead_.28GSoC.29

> This is what called vdso in Linux land.

I know what vdso is in Linux land but while implementing vdso will give us some
additional features in any case it needs a mechanism like the one
(Continue reading)

Permalink | Reply | 0 comments |
headers
Giovanni Trematerra | 2 Jun 2012 00:23
Picon
Gravatar

Re: [RFC] Kernel shared variables

On Fri, Jun 1, 2012 at 11:21 PM, Bruce Evans <brde <at> optusnet.com.au> wrote:
> On Fri, 1 Jun 2012, Giovanni Trematerra wrote:
>
>> I'd like to discuss a way to provide a mechanism to share some read-only
>> data between kernel and user space programs avoiding syscall overhead,
>> implementing some them, such as gettimeofday(3) and time(3) as ordinary
>> user space routine.
>
>
> This is particularly unsuitable for implementing gettimeofday(), since for
> it to work you would need to use approximately 1 CPU spinning in the
> kernel to update the time every microsecond.  For time(3), it only needs
> a relatively slow update.  For clock_gettime() with nansoeconds precision,
> it is even more unsuitable.  For clock_gettime() with precisions between
> 1 second and 1 microseconds, it is intermediately unsuitable.
>
> It also requires some complications for locking/atomicity and coherency
> (much the same as in the kernel.  Not just for times.  For times, the
> kernel handles locking/atomicity fairly well, and coherency fairly badly.
>

Well, the primary intend of the patch is to provide a mechanism to share data
between kernel and user land without switching in kernel mode. Not to provide
a complete re-implementation in user mode of all time stuff.

>
>> The patch at
>> http://www.trematerra.net/patches/ksvar_experimental.patch
>>
>> is in a very experimental stage. It's just a proof-of-concept.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Julian Elischer | 2 Jun 2012 02:10
Picon
Favicon

Re: Allow small amount of memory be mlock()'ed by unprivileged process?

On 6/1/12 8:41 AM, Bryan Drewery wrote:
> On 5/31/2012 8:40 PM, Doug Barton wrote:
>> On 5/31/2012 5:23 AM, Andriy Gapon wrote:
>>> In fact, FreeBSD also has this rlimit and there seems to be full support for it on
>>> both user and kernel sides.
>>> OTOH, PRIV_VM_MLOCK privilege seems to be granted only to the super-user in the
>>> default configuration.  And this privilege kind of defeats the limit.
>>>
>>> Perhaps, we should/could kill the privilege and set the limit to a sufficiently
>>> small/safe value for ordinary users?
>> I like this idea, but someone else in the thread (sorry, don't have it
>> handy) brought up the point that we don't want the aggregate of per-user
>> limits to be able to bring down the system either. So the right solution
>> would seem to be a reasonable per-user limit, and a cap on the maximum
>> total amount of locked pages for all unprivileged users, probably based
>> on some percentage of total available memory?
>>
>> Doug
>>
> I like this approach. A per-user ulimit, and a global max sysctl that
> can be overridden, but by default based on a percentage of available memory.

I'd go a different route.
I'd have it inherited, and I'd have the value be 0 by default, but 
settable to
some different value at login.conf, or by an ancestor with root privs.

_______________________________________________
freebsd-arch <at> freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
(Continue reading)

Permalink | Reply | 8 comments |
headers
Andriy Gapon | 2 Jun 2012 13:03
Picon
Favicon

Re: Allow small amount of memory be mlock()'ed by unprivileged process?

on 01/06/2012 04:40 Doug Barton said the following:
> On 5/31/2012 5:23 AM, Andriy Gapon wrote:
>> In fact, FreeBSD also has this rlimit and there seems to be full support for it on
>> both user and kernel sides.
>> OTOH, PRIV_VM_MLOCK privilege seems to be granted only to the super-user in the
>> default configuration.  And this privilege kind of defeats the limit.
>>
>> Perhaps, we should/could kill the privilege and set the limit to a sufficiently
>> small/safe value for ordinary users?
> 
> I like this idea, but someone else in the thread (sorry, don't have it
> handy) brought up the point that we don't want the aggregate of per-user
> limits to be able to bring down the system either.

The unprivileged users can not spawn any new users on their own and there is a
limit on number of processes per user, so a system administrator should be able
to plan resource limits based on system capacity and utilization.

> So the right solution
> would seem to be a reasonable per-user limit, and a cap on the maximum
> total amount of locked pages for all unprivileged users, probably based
> on some percentage of total available memory?

I would agree for a default limit of zero even.  As long as I (as a system
administrator) am able to increase it for selected users and groups.

--

-- 
Andriy Gapon
_______________________________________________
freebsd-arch <at> freebsd.org mailing list
(Continue reading)

Permalink | Reply | 0 comments |

Gmane