Matthew Dillon | 2 May 04:21 2012

Master branch and 3.0 release branch performance & disk use fixes

    The following fixes have gone into master and also been MFC'd to the
    release branch (commitids for master):

	b642a6c1f5bbb295e29522d99c65038f459288ac
	kernel - Fix degenerate cluster_write() cases

	66030e2b4635359f2d84f23298c9d8ce1e6af5da
	HAMMER VFS - Only set B_CLUSTEROK on 64K buffers

    This fixes a serious performance issue which we introduced when we
    turned on the cluster buf code in HAMMER.  The code inadvertently
    removed a file EOF check that caused fragmentory writes to the
    end of a file to immediately flush to disk, greatly reducing filesystem
    performance by causing excessive writes to disk as well as temporary
    blocking conditions due to the I/O in progress.

    Since HAMMER does allocate-on-flush this also caused bloated
    filesystem space usage.  The space would be recovered automatically
    by the nightly hammer cleanup code but in the mean time could eat a
    huge amount of space (4x to 25x) when writing out a file.

    There was also a 'zeros in file' bug reported for HAMMER which I
    believe these commits should get rid of.

					-Matt
					Matthew Dillon 
					<dillon <at> backplane.com>

Venkatesh Srinivas | 5 May 06:45 2012

virtio drivers!

Hi folks,

Tim Bisson ported FreeBSD's virtio drivers (from the bhyve project) to
DragonFly; I am currently working on reviewing and integrating them.
The virtio drivers should provide increased performance of DFly guests
in kvm, specifically block devices (virtio-blk) and network interfaces
(if_vtnet).

My work is available at git://github.com/vsrinivas/DragonFlyBSD.git,
in the 'virtual' branch, for anyone who wants to try it or contribute!
I am also tracking questions and thoughts in our bug tracker, bug 2360
(http://bugs.dragonflybsd.org/issues/2360). Any code review would be
helpful!

My 'virtual' tree also has some other related work, specifically
'vm_balloon'. Basically, vm-balloon is a VM level framework for
managing balloon devices; rather than having a balloon device respond
directly to host requests for pages without any consideration of guest
loads, we have talked about having the guest VM consider its own load
when releasing pages to a host. The vm-balloon code will encompass
this; it will also be able to drive the virtio-balloon device and a
newly created vkernel-balllon device, which will bring the same
functionality to vkernels. Any reviews or comments on vm_balloon.c or
the vkernel/virtio-balloon devices and in particular the interface
between vm_balloon and the devices would be great!

Take care,
-- vs
http://ops101.org/4k/

(Continue reading)

Venkatesh Srinivas | 5 May 07:15 2012

Re: Master branch and 3.0 release branch performance & disk use fixes

On Tue, May 1, 2012 at 10:21 PM, Matthew Dillon
<dillon <at> apollo.backplane.com> wrote:
>    The following fixes have gone into master and also been MFC'd to the
>    release branch (commitids for master):
>
>        b642a6c1f5bbb295e29522d99c65038f459288ac
>        kernel - Fix degenerate cluster_write() cases
>
>        66030e2b4635359f2d84f23298c9d8ce1e6af5da
>        HAMMER VFS - Only set B_CLUSTEROK on 64K buffers
>
>    This fixes a serious performance issue which we introduced when we
>    turned on the cluster buf code in HAMMER.  The code inadvertently
>    removed a file EOF check that caused fragmentory writes to the
>    end of a file to immediately flush to disk, greatly reducing filesystem
>    performance by causing excessive writes to disk as well as temporary
>    blocking conditions due to the I/O in progress.
>
>    Since HAMMER does allocate-on-flush this also caused bloated
>    filesystem space usage.  The space would be recovered automatically
>    by the nightly hammer cleanup code but in the mean time could eat a
>    huge amount of space (4x to 25x) when writing out a file.
>
>    There was also a 'zeros in file' bug reported for HAMMER which I
>    believe these commits should get rid of.

To make things clearer, this bug did not exist in the 3.0 release or
3.0.2. It was only present on the master branch and in the head of the
3.0 branch (DragonFly_RELEASE_3_0).

(Continue reading)

Justin Sherrill | 24 May 23:01 2012

Phoronix and Hammer - a followup

A while back, Phoronix published benchmarks for DragonFly 3.0.

http://www.phoronix.com/scan.php?page=news_item&px=MTA3MDA

They showed generally good numbers, but the disk test showed a
startling 250% DROP in disk speed with Hammer between 2.10 and 3.0.
The test is unpacking a certain version of the Linux kernel - pretty
simple test.  That kind of drop would be driving everyone crazy, but I
wanted to put some numbers behind saying "OMG that benchmark for my
favorite operating system is wrong and you are mean!"

There is bug 2288, which shows a small slowdown, but nothing of this magnitude:

http://bugs.dragonflybsd.org/issues/2288

So, I've been running the same test as the Phoronix one, at random
time on shiningsilence.com, before and after upgrading it from
DragonFly 2.10 to DragonFly 3.0.  Here's the command:

/bin/rm -rf linux-2.6.32; /usr/bin/time -o notes.txt -a tar xf
linux-2.6.32.tar.bz2

Here's the results, averaged:

39.791 - 2.10 real 	
28.509 - 2.10 user	
03.529 - 2.10 sys		

44.829 - 3.0 real	
26.137 - 3.0 user	
(Continue reading)

Loganaden Velvindron | 25 May 13:39 2012
Picon

Status of privsep in DragonflyBSD

I imported privsep.c & privsep_fdpass.c for syslogd. It compiles, but
there's much
more to do.

There are some historical divergences between syslogd in OpenBSD and
DragonflyBSD.

I hope to speed up a bit as soon as I'm done with a final exam on the 31st.

--

-- 
Brightest day,
Blackest night,
No bug shall escape my sight,
And those who worship evil's mind,
be wary of my powers,
puffy lantern's light !

Mihai Carabas | 26 May 14:59 2012
Picon

Re: GSoC: Add SMT/HT awareness to DragonFlyBSD scheduler

Hello,


The current status for week 1:
- I setup the working and testing environment (I needed a two-core HT aware CPU [1]).
- I setup a repo in github [2] and also I will make periodic updates to my repo on the leaf [3].
- I studied the sysctl interface through which I am going to modify the behaviour of the scheduler (it's more flexible this way then configs made at build time - thanks Alex for the tip).
- I read a paper on SMT aware schedulers [4]



On Wed, Apr 25, 2012 at 11:03 AM, Mihai Carabas <mihai.carabas <at> gmail.com> wrote:
Hello everyone!


My name is Mihai Carabas and this summer I will be working on the DragonFlyBSD scheduler. The goal of the project is to make the scheduler aware of the underlaying hyperthreading CPUs, in order to make better decisions.

First of all, I will have to build up the testing infrastructure (a physical machine with at least 2 physical cores, each of them having HT enable). At this step I will develop a test suite that stress up the scheduler and make different measurements. These measurements will be the reference for comparing the results obtained after modifying the scheduler.

In the next step, I have to develop a mechanism for grouping the CPUs in domains (scheduling domains or in other words: CPUs that are on the same physical core).
The actual implementation of the HT aware is described in a previous post [1] on the list and in my application [2]. To sum up, here I will treat different use cases (eg: "passive" scheduling - select a free physical CPU (if available) for the next thread to schedule; "active" scheduling" - if a physical CPU becomes idle and on another CPU are running two threads, migrate one of them on the idle physical cpu; etc.).



Vishesh Yadav | 26 May 15:37 2012
Picon

[GSoC] inotify and fs indexing service status

Hello everyone,

Current status for first week -

- Repository and wiki setup at Github[1]
- Made the basic skeleton for inotify interface - system calls, helper
functions, structures, headers and few basic stuff in there.
- Currently working on management of watches (will be using separate
file tables for watches). Once this is done, can write some working code.

In community bonding period, I setup my working environment, browse and
understood relevant kernel codebase and studied Linux early and recent
inotify implementation.

Now that exams are almost over, I hope I can catch up some pace now.

Regards,
Vishesh

[1] https://github.com/vishesh/DragonFlyBSD/

Ivan Sichmann Freitas | 27 May 04:37 2012
Picon

[GSoC] 32 bit api status

Hi everyone,

This is my first week status:

 - Environment setup at leaf (git repository, build and test
   environments).
 - Study of dragonfly's and freebsd's codebase (regarding syscalls and
   loading).

For the next week I will create a sketch of proposed changes and start
coding.

--

-- 
Ivan Sichmann Freitas
GNU/Linux user #509059

Gmane