GSOC 2012 Porting USB Stack

Hello everyone,
I am interested in porting USB stack of FreeBSD.
Do I have to do something other than writing a proposal?

--

-- 
Kind regards,
Olzhas Adiyatov
email: olzhas.adi <at> gmail.com

Re: GSOC 2012 Porting USB Stack

GSoC 2012: Port FreeBSD's USB stack to DragonFlyBSD

Re: [GSOC] Add SMT/HT awareness to DragonFlyBSD scheduler

VFS quota system now available in DragonFly 3.1

Hi,

DragonFly 3.1 now includes a filesystem-independent quota subsystem.

The VFS part in "VFS quota" means Virtual File System and is the cool
technology which allows modern Unix systems to use different sorts of file
systems in the same way.

Space limitations are mount-point specific and can be set per user, per group
or globally.

For example, the following command will stop /tmp from growing beyond ten
kilobytes:

    # vquota limit /tmp 10K

and this one will further limit the user johndoe to a maximum of 1800 bytes 
on the same filesystem:

    # vquota ulim /tmp johndoe 1800

VFS quotas can be enabled for most local filesystems. The complete list is 
constituted of ext2fs, hammer, hpfs, mfs, ntfs, nullfs, tmpfs and ufs.
PFSes are really nullfs under the hood and are supported.

VFS quotas are not enabled by default; a vfs.quota_enabled="1" line has to
be put into /boot/loader.conf and the system rebooted for them to become
operational.

This code is still a bit rough; both limits and disk usage counters are reset 
to zero on every reboot. Limits can only be set one user or group at a time,
etc...
I plan to improve all that in the coming months.

Should you want to give this a try, the vquota(8) manpage should contain all
the information you need to get started.
It is available online at this URL:
http://leaf.dragonflybsd.org/cgi/web-man?command=vquota&section=8

        Cheers,

--

-- 
Francois Tigeot

Re: VFS quota system now available in DragonFly 3.1

http://www.youtube.com/watch?v=LBduNcf1eQc

GSoC: Privilege separation in DragonflyBSD

Name: Loganaden Valaydon Velvindron

Email: loganaden <at> gmail.com

Physical address: 88, Avenue de Plevitz, Roches Brunes, Rose-Hill

Phone number (include country and area code):

(230) 9762817

Link to BSD work:

http://www.freshbsd.org/search?q=loganaden+velvindron

Links to prior code related to this area of work:

 ftp://ftp.irisa.fr/pub/OpenBSD/src/lib/libutil/imsg.c

ftp://ftp.irisa.fr/pub/OpenBSD/src/lib/libutil/imsg.h

http://www.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/syslogd/syslogd.c.diff?r1=1.64;r2=1.65;f=h

http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.sbin/syslogd/privsep_fdpass.c?rev=1.7;content-type=text%2Fplain

http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.sbin/tcpdump/privsep_pcap.c?rev=1.16;content-type=text%2Fplain

http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.sbin/tcpdump/privsep_fdpass.c?rev=1.7;content-type=text%2Fplain

http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.sbin/tcpdump/privsep.h?rev=1.7;content-type=text%2Fplain

http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.sbin/tcpdump/privsep.c?rev=1.30;content-type=text%2Fplain

Breakdown of the work you expect to do each week of the project:

The imsg API is a simple framework that was designed to make it easy
to write privileged-separated daemons. It is developed as part of the
OpenBSD project.

The advantages of the imsg framework is that it is minimalistic in
nature, and isactively being developed. Other privilege-separation
frameworks such as privman are no longer actively maintained and make
a number of assumptions that maynot apply to all daemons. imsg was
developed as a simple and reliable RPC mechanism, and it has been
found suitable for implementing privilege separation.

Another approach which is interesting to consider is postfix.
http://www.postfix.org/security.html Postfix uses separate processes
that communicate among themselves. However, this approach requires
significant re-write from scratch which may be difficult to complete
given our timeframe.

OpenBSD has also been using privilege separation since 2002, and their
code is considered very mature.

1st week, We need to import the imsg.{c,h} from OpenBSD into DragonflyBSD.

Then, we will need to sync with the latest changes in OpenBSD as some
daemons (dhclient. Dhcpcd) have moved to the imsg framework. Joerg
Sonnenberger & Hasso Tapper already started some of that work but it’s
a bit incomplete.

A cursory look at privsep.c in dhclient shows that it was last updated
in 2008. OpenBSD has been making fixes to privsep.c. Quote from a
recent commit message in 2011:

``Dump some useless calls to dhclient-script. i.e. MEDIUM, PREINIT,
ARPSEND, ARPCHECK. Drop support for 'media', 'medium' and 'alias'
specifications in dhclient.conf. Old leases still parse but these
options now have no effect.

Be more polite and decline all offers we don't accept. Fix a IMSG
length check. ''

If the import causes any conflict with the local patches, the latter
will need to

be adapted to fit with imsg framework. In case that these 2 conflict,
then the privilege separation may need to be loosen up a bit.

It's important to keep in sync with the latest privsep changes as
subtle bugs are

still being found.

As to how to quantify performance, some figures are already available:

http://www.citi.umich.edu/u/provos/papers/privsep.pdf

Chapter 6 shows that the performance penalty for the complex ssh
daemon was minimal as long as the data that needs to be moved from
master to slave process

through IPC is small. This will need to be kept in mind when applying privsep to

dntpd.

2nd-3rd week: Privilege separate syslogd by importing patches from openbsd.

syslogd could break when creating new files outside the chrooted
directory (/var/log/).

The only difference from existing behaviour is that if syslog.conf
changes and syslogd receives a HUP, it will re-exec itself and have
two new PIDs. A HUP with an unchanged config will make syslogd reopen
logfiles as before.

Additionally, it will involve splitting the code as dflybsd's syslogd
is monolithic compared to openbsd's syslogd. the tty handling code
will need to be moved, as

it requires special handling with the rpc.

 Once the code is properly split, we can start applying privilege separation.

4th-5th week: Privilege separate dntpd from scratch. A look at ntpd in
openbsd would be interesting as the latter was designed from scratch
and incorporates privilege separation.

logging & dns will break. The master will need to read the logs and
resolve dns for

the slave process. This will involve some strict checks as the slave
process is untrusted.

 Since the master calls the time system calls (adjtime, settime), a
delay is inevitable as those messages are sent through RPC. This has
been considered acceptable in openntpd.

6th-9th week: Privilege separate tcpdump. This will probably be the
most complicated among the demons due to the complexity inherent in
tcpdump. Patches from openbsd are available.

 dns & writing to a file using -W option can break since tcpdump is
chrooted to /var/log/

Master process needs to be able to do the following tasks and sent the results

through rpc to the slave process:

gethostbyaddr, ether_ntohost, getrprcbynumber, getservenetries, localtime.

From the commit message in openbsd:

Modified files:
      usr.sbin/tcpdump: Makefile addrtoname.c addrtoname.h gmt2local.c
                        interface.h print-atalk.c print-cnfp.c
                        print-sunrpc.c print-udp.c tcpdump.c util.c
Added files:
      usr.sbin/tcpdump: pf_print_state.c pfctl_osfp.c privsep.c
                        privsep.h privsep_fdpass.c privsep_pcap.c

A lot of rewrite of existing protocol handling is to be expected, and
these could break. Testing All those changes is going to be tricky as
protocols such as apple-talk are no longer popular.

Also, we need to check that tcpdump on pf isn't broken by this.

10th Last week: Commit to –current

Please list any needed equipment or accounts for your work:

I already have a machine running –current at home. I’ve already talked
to Angelos Oikonomopoulos who has shown some interest in reviewing the
patches.

A sane –current & users willing to test the code would certainly help!

--

-- 
Brightest day,
Blackest night,
No bug shall escape my sight,
And those who worship evil's mind,
be wary of my powers,
puffy lantern's light !

GSoC: Add SMT/HT awareness to DragonFlyBSD scheduler

[GSoC 2012] Implement inotify and filesystem indexing service

Hello everyone,

I'm Vishesh Yadav, a 3rd year Computer Science and Engineering student
from India and Google Summer of Code 2012 student. I'm interested in
Computer Architecture and Operating System development, which led me
applying for DFBSD.

The goal of my project is to implement Linux's inotify interface and to
write an indexing service for locate that will use inotify to keep
locate database more up-to-date. These new inotify system calls will be
exposed to Linux compatibility layer as well. I've appended my GSoC
proposal to this message.

This is the first time I'll be doing kernel programming, hence I would
really appreciate any advice that you may have. I will put in my best to
deliver what I promised for, and am looking forward enjoying hacking DFBSD.

[Proposal]
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/vishesh/20002

inotify System Calls and Indexing Service for Filesystem
========================================================

Name     : Vishesh Yadav
Email    : vishesh3y <at> gmail.com
Address  : A2/637, Himsagar Aptts
           Pocket P4
           Greater Noida (U.P)
           India - 201310

Abstract
--------

The goal of this project is to provide file system monitoring facilities
in DragonFly BSD. This project is divided in to two parts -

1. Implement inotify
2. Implement a Filesystem indexing service for 'locate' over inotify.

### inotify ###

DragonFly BSD provides kqueue/kevent interface to monitor file events
which works very well. However it comes with overhead of having unique
file descriptor for each watched file. Also, to watch changes in
directories each file inside that directory has to opened and watched
separately using kqueue/kevent. A system may reach the global/user file
descriptor limit when watching a large number of file.

To solve this problem, I propose to implement Linux's inotify interface.
 inotify interface can be used to monitor files and directories. Each
inotify instance use one file descriptor.

Implementing inotify will benefit various applications that use inotify,
eg.  Gamin, GIO, KIO etc... It will benefit developers who want to
implement file indexers, semantic desktop, malware scanners and
end-users who are looking for application compatibility with Linux.

### Filesystem indexing service ###

The second part of this project proposes to implement a Filesystem
indexing service. The service will prepare database that will be used by
locate utility.  Unlike the traditional updatedb program, this will
listen to filesystem changes and update the database instantly and
therefore the database would be more accurate and up to date.

If time allows (or after GSoC), I propose to extend the locate utility
to store additional information about files such as size, owner,
permissions etc.

Project Goals and Deliverables
------------------------------

During the GSoC period the goal is to -

### inotify ###
    * Implement inotify system calls.
    * Write manpages for inotify.
    * Write few tests.
    * Check well known softwares using inotify such as Gamin, GIO, KIO
    * Expose the new system calls through Linux ABI and test few native
Linux binaries against it.

### Filesystem Indexing Service ###
    * Write indexing service based on new inotify interface.
    * Extend locate to store additional information about the files.
    * Write tests for the new indexing service.
    * Update the man pages.

Implementation Details
----------------------

### inotify ###

There are essentially two ways by which we can have inotify on
DragonFly-BSD -

* Emulate over kqueue - This has been earlier tried in NetBSD (but not
in-kernel). However it still doesn't solve the problem of having an open
file descriptor for each monitored file (in kernel). Secondly inotify
provides few notifications that is not provided by kqueue. However this
approach will avoid much complexity and changing of VNODE structure and
hooking VNOPS. To avoid bloat of vnode structure and review from the
project mentor, this is the preferred method.

* Develop from scratch - In this approach we will have to add a couple
of members to VNODE structre, call inotify event functions in VNOPS.
VNOPS will forward the events to the inotify system. Each vnode will
keep a count and list of its watches. The approach is essentially same
as implemented in Linux where all file system events are sent to
fsnotify which is used by inotify/dnotify.

Overall -

* Three new system calls viz inotify_init, inotify_add_watch and
inotify_rm_watch will be implemented.
* Each open inotify instance is associated with an open inotify_device.
It keeps a list of watches and queued events.
* inotify_watch represents a watch request on specific node and is
associated with an inotify device and vnode.
* Lifetime of inotify_device and inotify_watch is managed by reference
count.
* inotify_kernel_event is an inotify event. A list of these is
associated with each inotify device.
* A basic pseudo filesystem called 'inotifyfs' will be created. It will
implement basic file operations associated such as read, ioctl, release.

Once inotify system is implemented, it will be made available at Linux
emulation layer.

### Filesystem Indexing Service ###

The filesystem indexing service will run as a daemon. We will maintain a
temporary database which will contain all changes since last update. For
every search this temporary database will be checked first for changes.
Whenever updatedb is run, this temporary database changes will be
committed to the main database. This will let us take benefit of already
mature updatedb, new inotify interface to make recent changes available
and avoid rewriting to huge database file everytime.

Optionally, if time allows (or after GSoC period) locate utility could
be further extended to store some additional information about files
such as owner, permissions, file type and other attributes. The user can
query and filter according to these attributes.

Milestones
----------

### Week 1 -
* Write headers and interface.
* Make dummy system calls (inotify_*)
* Implement inotify device and filesystem.
### Week 2-3 -
* Implement in kernel API to add/remove watches, queue events.
### Week 4-7 -
* Implement system calls inotfy_init, inotify_watch_add, inotify_watch_rm.
* Complete the implementation of inotify.
* Test the new interface. Write some tests.
### Week 8 -
* Write man pages.
* Document code.
* Test Gamin, GIO (and KIO if possible) against new interface.
* Prepare code for mid-term evaluation and review.
### Week 8 -
* Start working on Filesytem indexing service.
* Write basic interfaces and data types.
* Read config files and setup the service accordingly.
### Week 9-10 -
* Listen to filesystem changes.
* Write these changes to temporary database.
* Commit temporary database to master database whenever updatedb is run.
### Week 11-12 -
* Test the service.
* Write tests.
* Document code.
* Write/Update man pages.
* Prepare code for master.
* Get the code reviewed.
### Week 13-14
* Buffer period for any possible delays or weird bugs and more testing.

About Me
--------

I'm a 3rd year B.Tech Computer Science and Engineering student from
India. I have deep interest in Operating Systems, System Programming and
Open Source development. I'm well versed with C and UNIX C API and have
been working with it for quite few years. Apart from C, I also program
in C++, Python and Scheme.

Though I've never worked directly on kernel before, but I've good
understanding of Operating System internals. I've also taken Operating
System course from university. I have spent last few weeks studying the
architecture and implementation of DragonFly-BSD and Linux kernel
codebase as well as basic kernel development/debugging workflow.

I know String Matching algorithms and have rudimentary understanding of
ngrams which will be helpful while working with locate and indexing service.

My interest in Operating Systems and Open Source development made me
apply to DragonFly-BSD. I'm very excited and looking forward to be part
of team, contribute code and do my best.

I previously participated in GSoC 2011 for KDE and successfully
completed my project and am still maintaining it.

---------end proposal--------

Regards,
Vishesh Yadav

[GSoC 2012] Add 32 bit API for 64 bit kernels

Hello everyone,

I'm a computer engineering student, and my accepted project for GSoC
is about implementing a 32 bit API for 64 bit kernels (see [1] for the
complete application), thus enabling the latter to run 32 bit
applications.

The first step will be to develop a syscall translating layer, then
enable both loaders to be used simultaneously. Finally, the syscalls
themselves must be translated to its 64 bit counterparts.

[1] https://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/ivansichmann/8002

--

-- 
Ivan Sichmann Freitas
GNU/Linux user #509059