Maxime Villard | 20 Oct 15:38 2014
Picon

FFS: wrong superblock check ~> crash

Probably with the conviction I would find some bugs I opened ffs/ffs_vfsops.c
and something immediately stroke me:

918		error = bread(devvp, sblock_try[i] / DEV_BSIZE, SBLOCKSIZE, cred,
919			      0, &bp);

SBLOCKSIZE (=8192) bytes are read on the disk and put into bp->b_data
(allocated).

924		fs = (struct fs*)bp->b_data;
...
939			sbsize = fs->fs_sbsize;

'sbsize' is set to a value that was read on the disk.

976		/* Validate size of superblock */
977		if (sbsize > MAXBSIZE || sbsize < sizeof(struct fs))
978			continue;

Basic sanity checks. MAXBSIZE = 64 * 1024.

991	fs = kmem_alloc((u_long)sbsize, KM_SLEEP);
992	memcpy(fs, bp->b_data, sbsize);

And then comes this memcpy. The problem here is that the size of b_data is
8192, but the values of sbsize are located in [1376; 65536].

With a broken superblock the kernel will read far beyond the allocated
area, which mostly means it will crash.

(Continue reading)

Maxime Villard | 18 Oct 13:06 2014
Picon

Unreleased vnode in linux_sys_uselib()

Hi,
there seems to be a vnode issue in compat/linux/common/linux_uselib.c:

115	if (LINUX_N_MACHTYPE(&hdr) != LINUX_MID_MACHINE)
116		return ENOEXEC;

Here 'vp' is not released. Patch:

Index: linux_uselib.c
===================================================================
RCS file: /cvsroot/src/sys/compat/linux/common/linux_uselib.c,v
retrieving revision 1.30
diff -u -r1.30 linux_uselib.c
--- linux_uselib.c	28 Aug 2009 01:39:03 -0000	1.30
+++ linux_uselib.c	18 Oct 2014 10:48:53 -0000
 <at>  <at>  -103,17 +103,18  <at>  <at> 
 	if ((error = vn_rdwr(UIO_READ, vp, (void *) &hdr, LINUX_AOUT_HDR_SIZE,
 			     0, UIO_SYSSPACE, IO_NODELOCKED, l->l_cred,
 			     &rem, NULL))) {
-		vrele(vp);
-		return error;
+		goto out;
 	}

 	if (rem != 0) {
-		vrele(vp);
-		return ENOEXEC;
+		error = ENOEXEC;
+		goto out;
 	}
(Continue reading)

Emmanuel Dreyfus | 14 Oct 19:05 2014
X-Face
Picon

[PATCH] GOP_ALLOC and fallocate for puffs (round 2)

Hi

Here is the latest iteration of my patch to ensure we avoid sending data
to the page cache we cannot flush later.
http://ftp.espci.fr/shadow/manu/puffs-alloc.patch

- When writing, we make sure backend storage is allocated in two cases:
  1) the file is sparse, hence we never know if backend storage is here.
  2) the file grows: we know we need to allocated for the new area.
- sparse file detection is done at when updating metadata cache: if
  size is bigger than allocated blocks, it is sparse.
- I add fallocate FUSE operation (userland support is in another patch)
  for efficient backend storage allocation.
- If fallocate is unavailable, I fallback to explicit zero-filling. 
- puffs_vnop_write() is split into two functions, puffs_vnop_write_cache()
  that writes in cache, puffs_vnop_write_fs that writes to the FS

The change lets NetBSD pass GlusterFS quota test. Without it, process
writing over quota hang in DE state (exitting, disk I/O ongoing: it cannot
flush cached data overquota and therefore cannot exit).

Opinions?

One possible improvement: if PUFFS fallocate returne EOPNOTSUP / ENOSYS, 
we remove it from the operation array? This is dirty, but since we have 
a fallocate systemcalls that has no implementation for FFS, we are very
likely to encounter that case.

--

-- 
Emmanuel Dreyfus
(Continue reading)

Ilia Zykov | 13 Oct 12:57 2014
Picon

Ptyfs correction.

Hello!
This patch corrects some wrong comments and
little memory leaks inside "ptyfs" and "ptm" driver.
Can be applied on "netbsd-7" and "current".
Thank you.
Ilia.

 fs/ptyfs/ptyfs_subr.c        |    47
++++++++++++++++++++++++++++++++++++-----------
 fs/ptyfs/ptyfs_vfsops.c    |    1 +
 kern/tty_ptm.c                  |    18 ++----------------

 3 files changed, 39 insertions(+), 27 deletions(-)

Kamil Rytarowski | 11 Oct 22:52 2014
Picon

Tru64 AdvFS porting to NetBSD - 2. status 2014-10-11

Hello,

This is the second status [1] of porting AdvFS to NetBSD.

Thank you for your motivation support, including mails from outside the NetBSD world.

1. What is done
- Moved AdvFS files from src/sys/fs/msfs to src/sys/external/gpl2/msfs and updated the build machinery
- Picked missing dyn_hashtable functionality from the HP/UX port of AdvFS
- Designed new debugging & tracing system, with changeable levels (none, fatal asserts, debug asserts,
extensive checks) it's intended to replace the existing fine-grained debugging that is placed in the
original work almost everywhere and it's impractically difficult to port 1:1, as it's utilizing
Tru64-specific features -- the HP/UX port went with similar path; most debugging code (most notably
related to locking) will be gone
- Stopped using indent(1) as it introduces a lot of harm because of extensive usage of macros (missing
semicolons etc..)
- Overall: cleaned and squashed 377 proof-of-concept (aka throw-away) commits [2] into 115 cleaned
revisions (aka throw-away later) [3]
2. What is in progress
- Adapting locking code, with verification of the right path with the HP/UX port
- Adapting debugging for new design, removing unneeded code-complication and Tru64-specific debug solutions
- Converting macros, used as in-lined functions with side-effects, to functions
- Removing alternative compilation paths (exception for _KERNEL in general and MSFS_DEBUG in msfs/ms_assert.h)
- Other compatibility patches for a modern compiler and NetBSD
3. Issues
- Missing subsystems' details from Tru64, still no idea about definitions of functions from overlap.h,
missing quota's code (but not looked at it closely)
4. Next steps
- Virtual Memory porting
5. Pushed to NetBSD
(Continue reading)

Kamil Rytarowski | 9 Oct 00:38 2014
Picon

Re: Unification of common date/time macros

Hello,

For your interest there are already patches against current:
http://mail-index.netbsd.org/netbsd-bugs/2014/10/08/msg038523.html

Best regards,

Vinaykumar Bhat | 4 Oct 04:20 2014
Picon

Cross compiling NetBSD current

Hi All,

I am working on Compressed Cache project for NetBSD (https://wiki.netbsd.org/projects/project/compressed-cache/) . I am trying to set up the cross compilation environment for NetBSD current. Copying tech-kern in this email since my emails to current-users seems to be not getting delivered).

Host machine (where I am compiling): Ubuntu(64-bit) running on x86_64
Target architecture (where I intend to boot NetBSD): amd64

I am trying to compile tools using build.sh.
Command used: /build.sh -U -m amd64 tools

I see a bunch of compiler errors saying compilation terminated since a file was not found (#include <sys/disk.h> fails with file not found). Eventually the build fails with message "ERROR: Failed to make build_install in "tools"".

Attaching the last few lines of the build log. I was able to cross compile 6-1-4 release branch. hence I am guessing that I have all the necessary tools installed properly. Any help will be helpful and greatly appreciated.

Thanks,
Vinay


####################################################################################################################################
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f biosboot.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/biosboot.c &&  mv biosboot.d.tmp biosboot.d
/usr/NetBSD_Current/tools/gpt/../../sbin/gpt/biosboot.c:46:22: fatal error: sys/disk.h: No such file or directory
 #include <sys/disk.h>
                      ^
compilation terminated.
#    create  gpt/create.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f create.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/create.c &&  mv create.d.tmp create.d
#    create  gpt/dehumanize_number.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f dehumanize_number.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../lib/libc/gen/dehumanize_number.c &&  mv dehumanize_number.d.tmp dehumanize_number.d
#    create  gpt/destroy.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f destroy.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/destroy.c &&  mv destroy.d.tmp destroy.d
#    create  gpt/gpt.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f gpt.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/gpt.c &&  mv gpt.d.tmp gpt.d
/usr/NetBSD_Current/tools/gpt/../../sbin/gpt/gpt.c:43:22: fatal error: sys/disk.h: No such file or directory
 #include <sys/disk.h>
                      ^
compilation terminated.
#    create  gpt/gpt_uuid.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f gpt_uuid.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/gpt_uuid.c &&  mv gpt_uuid.d.tmp gpt_uuid.d
#    create  gpt/label.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f label.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/label.c &&  mv label.d.tmp label.d
#    create  gpt/map.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f map.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/map.c &&  mv map.d.tmp map.d
#    create  gpt/migrate.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f migrate.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/migrate.c &&  mv migrate.d.tmp migrate.d
#    create  gpt/opendisk.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f opendisk.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../lib/libutil/opendisk.c &&  mv opendisk.d.tmp opendisk.d
#    create  gpt/recover.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f recover.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/recover.c &&  mv recover.d.tmp recover.d
#    create  gpt/remove.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f remove.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/remove.c &&  mv remove.d.tmp remove.d
#    create  gpt/resize.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f resize.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/resize.c &&  mv resize.d.tmp resize.d
#    create  gpt/resizedisk.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f resizedisk.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/resizedisk.c &&  mv resizedisk.d.tmp resizedisk.d
#    create  gpt/set.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f set.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/set.c &&  mv set.d.tmp set.d
#    create  gpt/show.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f show.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/show.c &&  mv show.d.tmp show.d
#    create  gpt/type.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f type.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/type.c &&  mv type.d.tmp type.d
#    create  gpt/unset.d
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -f unset.d.tmp  --      -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64     /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/unset.c &&  mv unset.d.tmp unset.d
#    create  gpt/.depend
rm -f .depend
/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbhost-mkdep -s .lo\ .ln -d -f .depend add.d biosboot.d create.d dehumanize_number.d destroy.d gpt.d gpt_uuid.d label.d map.d migrate.d opendisk.d recover.d remove.d resize.d resizedisk.d set.d show.d type.d unset.d
#   compile  gpt/add.lo
cc -O -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -c -o add.lo.o    /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/add.c
mv add.lo.o add.lo
#   compile  gpt/biosboot.lo
cc -O -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -c -o biosboot.lo.o    /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/biosboot.c
/usr/NetBSD_Current/tools/gpt/../../sbin/gpt/biosboot.c:46:22: fatal error: sys/disk.h: No such file or directory
 #include <sys/disk.h>
                      ^
compilation terminated.

*** Failed target:  biosboot.lo
*** Failed command: cc -O -I/usr/NetBSD_Current/tools/gpt -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/nbinclude -I/usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/include/compat -I/usr/NetBSD_Current/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -c -o biosboot.lo.o /usr/NetBSD_Current/tools/gpt/../../sbin/gpt/biosboot.c
*** Error code 1

Stop.
nbmake[3]: stopped in /usr/NetBSD_Current/tools/gpt

*** Failed target:  dependall
*** Failed command: cd "/usr/NetBSD_Current/tools/gpt"; /usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbmake realall
*** Error code 1

Stop.
nbmake[2]: stopped in /usr/NetBSD_Current/tools/gpt

*** Failed target:  dependall-gpt
*** Failed command: _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*) this="${dir}/"; real="${dir}" ;; .) this=""; real="/usr/NetBSD_Current/tools" ;; *) this="${dir}/"; real="/usr/NetBSD_Current/tools/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+ (with: $ <at> )}"; cd "${real}" && /usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbmake _THISDIR_="${this}" "$ <at> " ${target}; }; _makedirtarget gpt dependall
*** Error code 1

Stop.
nbmake[1]: stopped in /usr/NetBSD_Current/tools

*** Failed target:  build_install
*** Failed command: _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*) this="${dir}/"; real="${dir}" ;; .) this=""; real="/usr/NetBSD_Current/tools" ;; *) this="${dir}/"; real="/usr/NetBSD_Current/tools/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+ (with: $ <at> )}"; cd "${real}" && /usr/NetBSD_Current/obj/tooldir.Linux-3.11.0-12-generic-x86_64/bin/nbmake _THISDIR_="${this}" "$ <at> " ${target}; }; _makedirtarget . dependall-disklabel dependall-gpt
*** Error code 1

Stop.
nbmake: stopped in /usr/NetBSD_Current/tools

ERROR: Failed to make build_install in "tools"
*** BUILD ABORTED ***

####################################################################################################################################


Justin Cormack | 3 Oct 14:01 2014

CTLTYPE_UINT?

Back in the sysctl discussion a while back, core group said:

http://mail-index.netbsd.org/tech-kern/2014/03/26/msg016779.html

a) What types are needed?  Currently, CTLTYPE_INT is a signed
   32-bit type, and CTLTYPE_QUAD is an unsigned 64-bit type.
   Perhaps all four possible combinations of signed/unsigned and
   32 bits/64 bits should be supported.

I noticed today that there are some cases where a CTLTYPE_INT is being
fed a uint32_t, for example in sys/netinet6/ip6_input.c this is the
case for ip6_temp_preferred_lifetime and similar values that are
uint32_t.

It seems to make sense to add a CTLTYPE_UINT for these types, although
there are other options. FreeBSD introduced them way back. We could
continue to ignore the signedness differences though...

Thoughts?

Justin

Emmanuel Dreyfus | 30 Sep 15:44 2014
X-Face
Picon

[PATCH] GOP_ALLOC and fallocate for PUFFS

Hello

When a PUFFS filesystem uses the page cache, data enters the
cache with no guarantee it will be flushed. If it cannot be flushed
(bcause PUFFS write requests get EDQUOT or ENOSPC), then the
kernel will loop forever trying to flush data from the cache,
and the filesystem cannot be unmounted without -f (and data loss).

In the attached patch, I add in PUFFS:
- support for the fallocate operation 
- a puffs_gop_alloe() function that use fallocate
- when writing through the page cache we call first GOP_ALLOC to make
  sure backend storage is allocated for the data we cache. debug printf
  show a sane behavior, GOP_ALLOC calling puffs_gop_alloc only when required.

If the filesystem does not implement fallocate, we keep the current 
behavior of filling the page cache with data we are not sure we can flush. 
Perhaps we can improve further: missing fallocate can be emulated by 
writing zeroed chuncks. I have implemented that in libperfuse, but 
we may want to have this in libpuffs, enabled by a mount option. Input
welcome.

-- 
Emmanuel Dreyfus
manu <at> netbsd.org
Index: lib/libpuffs/dispatcher.c
===================================================================
RCS file: /cvsroot/src/lib/libpuffs/dispatcher.c,v
retrieving revision 1.46.4.1
diff -U4 -r1.46.4.1 dispatcher.c
--- lib/libpuffs/dispatcher.c	24 Aug 2014 08:42:06 -0000	1.46.4.1
+++ lib/libpuffs/dispatcher.c	30 Sep 2014 09:40:28 -0000
 <at>  <at>  -1139,8 +1139,22  <at>  <at> 
 			    auxt->pvnr_attrname, pcr);
 			break;
 		}

+		case PUFFS_VN_FALLOCATE:
+		{
+			struct puffs_vnmsg_fallocate *auxt = auxbuf;
+
+			if (pops->puffs_node_fallocate == NULL) {
+				error = EOPNOTSUPP;
+				break;
+			}
+
+			error = pops->puffs_node_fallocate(pu,
+			    opcookie, auxt->pvnr_off, auxt->pvnr_len);
+			break;
+		}
+
 		default:
 			printf("inval op %d\n", preq->preq_optype);
 			error = EINVAL;
 			break;
Index: lib/libpuffs/opdump.c
===================================================================
RCS file: /cvsroot/src/lib/libpuffs/opdump.c,v
retrieving revision 1.36
diff -U4 -r1.36 opdump.c
--- lib/libpuffs/opdump.c	15 Mar 2012 02:02:21 -0000	1.36
+++ lib/libpuffs/opdump.c	30 Sep 2014 09:40:28 -0000
 <at>  <at>  -116,8 +116,9  <at>  <at> 
 	"PUFFS_VN_OPENEXTATTR",
 	"PUFFS_VN_DELETEEXTATTR",
 	"PUFFS_VN_SETEXTATTR",
 	"PUFFS_VN_CLOSEEXTATTR",
+	"PUFFS_VN_FALLOCATE",
 };
 size_t puffsdump_vnop_count = __arraycount(puffsdump_vnop_revmap);

 /* XXX! */
Index: lib/libpuffs/puffs.c
===================================================================
RCS file: /cvsroot/src/lib/libpuffs/puffs.c,v
retrieving revision 1.117
diff -U4 -r1.117 puffs.c
--- lib/libpuffs/puffs.c	14 Nov 2011 01:27:42 -0000	1.117
+++ lib/libpuffs/puffs.c	30 Sep 2014 09:40:28 -0000
 <at>  <at>  -105,8 +105,9  <at>  <at> 
 	FILLOP(getextattr,  GETEXTATTR);
 	FILLOP(setextattr,  SETEXTATTR);
 	FILLOP(listextattr, LISTEXTATTR);
 	FILLOP(deleteextattr, DELETEEXTATTR);
+	FILLOP(fallocate, FALLOCATE);
 }
 #undef FILLOP

 /*
Index: lib/libpuffs/puffs.h
===================================================================
RCS file: /cvsroot/src/lib/libpuffs/puffs.h,v
retrieving revision 1.124.10.1
diff -U4 -r1.124.10.1 puffs.h
--- lib/libpuffs/puffs.h	24 Aug 2014 08:42:06 -0000	1.124.10.1
+++ lib/libpuffs/puffs.h	30 Sep 2014 09:40:28 -0000
 <at>  <at>  -251,10 +251,12  <at>  <at> 
 	int (*puffs_node_reclaim2)(struct puffs_usermount *,
 	    puffs_cookie_t, int);
 	int (*puffs_node_open2)(struct puffs_usermount *,
 	    puffs_cookie_t, int, const struct puffs_cred *, int *);
+	int (*puffs_node_fallocate)(struct puffs_usermount *,
+	    puffs_cookie_t, off_t, off_t);

-	void *puffs_ops_spare[28];
+	void *puffs_ops_spare[27];
 };

 typedef	int (*pu_pathbuild_fn)(struct puffs_usermount *,
 			       const struct puffs_pathobj *,
 <at>  <at>  -413,9 +415,11  <at>  <at> 
 	    const struct puffs_cred *, int, int);			\
 	int fsname##_node_reclaim2(struct puffs_usermount *,		\
 	    puffs_cookie_t, int);					\
 	int fsname##_node_open2(struct puffs_usermount *,		\
-	    puffs_cookie_t, int, const struct puffs_cred *, int *);
+	    puffs_cookie_t, int, const struct puffs_cred *, int *);	\
+	int fsname##_node_fallocate(struct puffs_usermount *,		\
+	    puffs_cookie_t, int, off_t, off_t);

 
 #define PUFFSOP_INIT(ops)						\
     ops = malloc(sizeof(struct puffs_ops));				\
? sys/fs/puffs/diff
? sys/fs/puffs/ndiff
? sys/fs/puffs/out
Index: sys/fs/puffs/puffs_msgif.h
===================================================================
RCS file: /cvsroot/src/sys/fs/puffs/puffs_msgif.h,v
retrieving revision 1.80.14.1
diff -U 4 -r1.80.14.1 puffs_msgif.h
--- sys/fs/puffs/puffs_msgif.h	26 Aug 2014 23:15:12 -0000	1.80.14.1
+++ sys/fs/puffs/puffs_msgif.h	30 Sep 2014 13:36:52 -0000
 <at>  <at>  -85,13 +85,13  <at>  <at> 
 	PUFFS_VN_PRINT,		PUFFS_VN_ISLOCKED,	PUFFS_VN_PATHCONF,
 	PUFFS_VN_ADVLOCK,	PUFFS_VN_LEASE,		PUFFS_VN_WHITEOUT,
 	PUFFS_VN_GETPAGES,	PUFFS_VN_PUTPAGES,	PUFFS_VN_GETEXTATTR,
 	PUFFS_VN_LISTEXTATTR,	PUFFS_VN_OPENEXTATTR,	PUFFS_VN_DELETEEXTATTR,
-	PUFFS_VN_SETEXTATTR,	PUFFS_VN_CLOSEEXTATTR
+	PUFFS_VN_SETEXTATTR,	PUFFS_VN_CLOSEEXTATTR,	PUFFS_VN_FALLOCATE,
 	/* NOTE: If you add an op, decrement PUFFS_VN_SPARE accordingly */
 };
 #define PUFFS_VN_MAX PUFFS_VN_CLOSEEXTATTR
-#define PUFFS_VN_SPARE 32
+#define PUFFS_VN_SPARE 31

 /*
  * These signal invalid parameters the file system returned.
  */
 <at>  <at>  -665,8 +665,14  <at>  <at> 

 	struct puffs_kcred	pvnr_cred;			/* OUT	*/
 };

+struct puffs_vnmsg_fallocate {
+	struct puffs_req	pvn_pr;
+	off_t			pvnr_off;			/* OUT    */
+	off_t			pvnr_len;			/* OUT    */
+};
+
 /*
  * For cache reports.  Everything is always out-out-out, no replies
  */

Index: sys/fs/puffs/puffs_subr.c
===================================================================
RCS file: /cvsroot/src/sys/fs/puffs/puffs_subr.c,v
retrieving revision 1.66
diff -U 4 -r1.66 puffs_subr.c
--- sys/fs/puffs/puffs_subr.c	16 Nov 2008 19:34:30 -0000	1.66
+++ sys/fs/puffs/puffs_subr.c	30 Sep 2014 13:36:52 -0000
 <at>  <at>  -197,8 +197,15  <at>  <at> 

 	puffs_updatenode(VPTOPP(vp), uflags, 0);
 }

+int
+puffs_gop_alloc(struct vnode *vp, off_t off, off_t len,
+		int flags, kauth_cred_t cred)
+{
+	return _puffs_vnop_fallocate(vp, off, len);
+}
+
 void
 puffs_senderr(struct puffs_mount *pmp, int type, int error,
 	const char *str, puffs_cookie_t ck)
 {
Index: sys/fs/puffs/puffs_sys.h
===================================================================
RCS file: /cvsroot/src/sys/fs/puffs/puffs_sys.h,v
retrieving revision 1.84.4.2
diff -U 4 -r1.84.4.2 puffs_sys.h
--- sys/fs/puffs/puffs_sys.h	29 Aug 2014 11:55:34 -0000	1.84.4.2
+++ sys/fs/puffs/puffs_sys.h	30 Sep 2014 13:36:52 -0000
 <at>  <at>  -279,8 +279,10  <at>  <at> 
 void	puffs_mp_release(struct puffs_mount *);

 void	puffs_gop_size(struct vnode *, off_t, off_t *, int); 
 void	puffs_gop_markupdate(struct vnode *, int);
+int	puffs_gop_alloc(struct vnode *, off_t, off_t, int, kauth_cred_t);
+int	_puffs_vnop_fallocate(struct vnode *, off_t, off_t);

 void	puffs_senderr(struct puffs_mount *, int, int, const char *,
 		      puffs_cookie_t);

Index: sys/fs/puffs/puffs_vfsops.c
===================================================================
RCS file: /cvsroot/src/sys/fs/puffs/puffs_vfsops.c,v
retrieving revision 1.113.2.1
diff -U 4 -r1.113.2.1 puffs_vfsops.c
--- sys/fs/puffs/puffs_vfsops.c	29 Aug 2014 11:55:34 -0000	1.113.2.1
+++ sys/fs/puffs/puffs_vfsops.c	30 Sep 2014 13:36:52 -0000
 <at>  <at>  -73,11 +73,9  <at>  <at> 
 static const struct genfs_ops puffs_genfsops = {
         .gop_size = puffs_gop_size,
 	.gop_write = genfs_gop_write,
 	.gop_markupdate = puffs_gop_markupdate,
-#if 0
-	.gop_alloc, should ask userspace
-#endif
+	.gop_alloc = puffs_gop_alloc,
 };

 /*
  * Try to ensure data structures used by the puffs protocol
Index: sys/fs/puffs/puffs_vnops.c
===================================================================
RCS file: /cvsroot/src/sys/fs/puffs/puffs_vnops.c,v
retrieving revision 1.182.2.4
diff -U 4 -r1.182.2.4 puffs_vnops.c
--- sys/fs/puffs/puffs_vnops.c	11 Sep 2014 14:00:54 -0000	1.182.2.4
+++ sys/fs/puffs/puffs_vnops.c	30 Sep 2014 13:36:53 -0000
 <at>  <at>  -72,8 +72,9  <at>  <at> 
 int	puffs_vnop_symlink(void *);
 int	puffs_vnop_rename(void *);
 int	puffs_vnop_read(void *);
 int	puffs_vnop_write(void *);
+int	puffs_vnop_fallocate(void *);
 int	puffs_vnop_fcntl(void *);
 int	puffs_vnop_ioctl(void *);
 int	puffs_vnop_inactive(void *);
 int	puffs_vnop_print(void *);
 <at>  <at>  -112,9 +113,9  <at>  <at> 
         { &vop_getattr_desc, puffs_vnop_checkop },	/* getattr */
         { &vop_setattr_desc, puffs_vnop_checkop },	/* setattr */
         { &vop_read_desc, puffs_vnop_checkop },		/* read */
         { &vop_write_desc, puffs_vnop_checkop },	/* write */
-	{ &vop_fallocate_desc, genfs_eopnotsupp },	/* fallocate */
+	{ &vop_fallocate_desc, puffs_vnop_fallocate },	/* fallocate */
 	{ &vop_fdiscard_desc, genfs_eopnotsupp },	/* fdiscard */
         { &vop_fsync_desc, puffs_vnop_fsync },		/* REAL fsync */
         { &vop_seek_desc, puffs_vnop_checkop },		/* seek */
         { &vop_remove_desc, puffs_vnop_checkop },	/* remove */
 <at>  <at>  -2330,8 +2331,24  <at>  <at> 
 		if (ap->a_ioflag & IO_APPEND)
 			uio->uio_offset = vp->v_size;

 		origoff = uio->uio_offset;
+
+		/*
+		 * Attempt to allocate storage so that we do not
+		 * feed the page cache with data we cannot flush 
+		 * later. If we get EOPNOTSUPP it means FALLOCATE
+		 * is unimplemented in the filesystem: in that 
+		 * case we cary on without any guarantee that the
+		 * data we cache will be flushable
+		 */
+		error = GOP_ALLOC(vp, origoff, uio->uio_resid,
+				  0, curlwp->l_cred);
+		if (error == EOPNOTSUPP)
+			error = 0;
+		else
+			goto out;
+
 		while (uio->uio_resid > 0) {
 			oldoff = uio->uio_offset;
 			bytelen = uio->uio_resid;

 <at>  <at>  -2444,9 +2461,62  <at>  <at> 
 	uflags |= PUFFS_UPDATECTIME;
 	uflags |= PUFFS_UPDATEMTIME;
 	puffs_updatenode(VPTOPP(vp), uflags, vp->v_size);

+out:
+	mutex_exit(&pn->pn_sizemtx);
+	return error;
+}
+
+int
+_puffs_vnop_fallocate(struct vnode *vp, off_t pos, off_t len)
+{
+	PUFFS_MSG_VARS(vn, fallocate);
+	struct puffs_mount *pmp = MPTOPUFFSMP(vp->v_mount);
+	int error;
+
+	PUFFS_MSG_ALLOC(vn, fallocate);
+	fallocate_msg->pvnr_off = pos;
+	fallocate_msg->pvnr_len = len;
+	puffs_msg_setinfo(park_fallocate, PUFFSOP_VN,
+	    PUFFS_VN_FALLOCATE, VPTOPNC(vp));
+
+	PUFFS_MSG_ENQUEUEWAIT2(pmp, park_fallocate, vp->v_data,
+	    NULL, error);
+	error = checkerr(pmp, error, __func__);
+	PUFFS_MSG_RELEASE(fallocate);
+
+	return error;
+}
+
+int
+puffs_vnop_fallocate(void *v)
+{
+	struct vop_fallocate_args /* {
+		const struct vnodeop_desc *a_desc;
+		struct vnode *a_vp;
+		off_t a_pos;
+		off_t a_len;
+	} */ *ap = v;
+	struct vnode *vp = ap->a_vp;
+	struct puffs_node *pn = VPTOPP(vp);
+	int error;
+
+	mutex_enter(&pn->pn_sizemtx);
+	error = _puffs_vnop_fallocate(v, ap->a_pos, ap->a_len);
+	if (error) {
+		if (error == EAGAIN)
+			error = EIO;
+		goto out;
+	}
+
+	if (ap->a_pos + ap->a_len > vp->v_size) {
+		uvm_vnp_setsize(vp, ap->a_pos + ap->a_len);
+		puffs_updatenode(pn, PUFFS_UPDATESIZE, vp->v_size);
+	}
+out:
 	mutex_exit(&pn->pn_sizemtx);
+
 	return error;
 }

 int
Emmanuel Dreyfus | 25 Sep 10:43 2014
X-Face
Picon

syscall stub

Hi

I need to add a layer in libc for posix_allocate() because the
system call argument list does not match the prototype exposed
to userland. 

How can I do it? My problem is that the system call and the libc
function have the same name. Do I need to rename the system call,
to something like _posix_fallocate ? Or is there a trick to do
it otherwise.

--

-- 
Emmanuel Dreyfus
manu <at> netbsd.org

Emmanuel Dreyfus | 25 Sep 03:19 2014
Picon

[PATCH] fallocate() for FFS

Implementing fallocate for FFS seems quite obvious.  Is there anything I
missed in the patch below? Is it good enough to commit?

This lets me discover that posix_fallocate() libc stub is buggy. Using
it through #include <unistd.h> leads argument corruption. Defining it as
int      posix_fallocate(int, int, off_t, off_t); let it work (note the
second int pad, as in syscalls.master). I guess we need to add a libc
stub too.

Index: sys/ufs/ffs/ffs_vnops.c
===================================================================
RCS file: /cvsroot/src/sys/ufs/ffs/ffs_vnops.c,v
retrieving revision 1.125
diff -U 4 -r1.125 ffs_vnops.c
--- sys/ufs/ffs/ffs_vnops.c     25 Jul 2014 08:20:53 -0000      1.125
+++ sys/ufs/ffs/ffs_vnops.c     25 Sep 2014 01:09:50 -0000
 <at>  <at>  -114,9 +114,9  <at>  <at> 
        { &vop_getattr_desc, ufs_getattr },             /* getattr */
        { &vop_setattr_desc, ufs_setattr },             /* setattr */
        { &vop_read_desc, ffs_read },                   /* read */
        { &vop_write_desc, ffs_write },                 /* write */
-       { &vop_fallocate_desc, genfs_eopnotsupp },      /* fallocate */
+       { &vop_fallocate_desc, ffs_fallocate },         /* fallocate */
        { &vop_fdiscard_desc, genfs_eopnotsupp },       /* fdiscard */
        { &vop_ioctl_desc, ufs_ioctl },                 /* ioctl */
        { &vop_fcntl_desc, ufs_fcntl },                 /* fcntl */
        { &vop_poll_desc, ufs_poll },                   /* poll */
 <at>  <at>  -326,8 +326,41  <at>  <at> 
        return error;
 }

 int
+ffs_fallocate(void *v)
+{
+       struct vop_fallocate_args /* {
+               struct vnode *a_vp;
+               off_t a_pos;
+               off_t a_len;
+       } */ *ap = v;
+       struct vnode *vp;
+       struct mount *mp;
+       int error;
+
+
+       vp = ap->a_vp;
+       mp = vp->v_mount;
+
+       fstrans_start(mp, FSTRANS_LAZY);
+       error = UFS_WAPBL_BEGIN(vp->v_mount);
+       if (error)
+               goto out;
+
+       genfs_node_wrlock(vp);
+       error = GOP_ALLOC(vp, ap->a_pos, ap->a_len, 0, curlwp->l_cred);
+       genfs_node_unlock(vp);
+
+       UFS_WAPBL_UPDATE(vp, NULL, NULL, 0);
+       UFS_WAPBL_END(vp->v_mount);
+out:
+       fstrans_done(mp);
+
+       return error;
+}
+
+int
 ffs_fsync(void *v)
 {
        struct vop_fsync_args /* {
                struct vnode *a_vp;

--

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu <at> netbsd.org


Gmane