Massimiliano Ghilardi | 6 Jan 22:06 2014
Picon

bug in mmap() and mremap() return value - and a fix

Running 32-bit CCL on 64-bit Linux (but it can happen in any setup),
I noticed that (osicat-posix:mmap) does not correctly handle return
addresses larger than the maximum C "long" value, but instead throws
an exception.

In such cases, it means the underlying C mmap() function returned
successfully a pointer in the top half of the addressable virtual
memory, but the Lisp wrapper for mmap() and mremap() chokes on such 
pointers.

A patch that solves this problem is attached.

Feedback on the analysis and the solution is welcome :)

Best regards,

Massimiliano Ghilardi

------------------ longer explanation -----------------------------

While invoking

(osicat-posix:mmap (cffi-sys:null-pointer) 4096
             (logior osicat-posix:prot-read osicat-posix:prot-write)
             osicat-posix:map-shared
             *fd* 0)

with a file descriptor *fd* correctly open on a 4096-byte file,
I encountered the error:
-------------------------------------------------------------------
(Continue reading)

John Morrison | 31 Dec 02:11 2013

colons in pathnames - SBCL vs CCL

Ran into CCL (but not SBCL) choking on filenames that contain colons (it interprets them as logical hosts, regardless of the fact they're not recognized ones -- and in contravention of the CLHS?).  

Seems like the divergence in behavior between CCL and SBCL is in merge-pathnames - below the level of osicat.

My perhaps naive and overly simplistic expectation of behavior is that of SBCL (not choking, arguably closer adherence to CLHS).

[jm <at> jm7 ~]$ lx86cl64
Welcome to Clozure Common Lisp Version 1.9-r15870 (LinuxX8664)!
? (merge-pathnames "foo:bar" #P"" :newest)
> Error: "foo" is not a defined logical host
> While executing: CCL::PATHNAME-HOST-SSTR, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
1 > (apropos "host-translations")
CCL::%LOGICAL-HOST-TRANSLATIONS%, Value: (("ccl" (#P"ccl:l1;**;*.*" #P"ccl:level-1;**;*.*") (#P"ccl:l1f;**;*.*" #P"ccl:l1-fasls;**;*.*") (#P"ccl:ccl;*.*" #P"/opt/ccl/*.*") (#P"ccl:**;*.*" #P"/opt/ccl/**/*.*")) ("home" (#P"home:**;*.*" #P"/home/jm/**/*.*")))

Found this when rehabbing the ancient CLIM "indented-lists" tree widget, replacing the buggy (in CCL again) pathname hackery therein with osicat (which cleaned up the code).  Turns out dbus/dcop create these files/symlinks in one's home directory, so it's not like I went looking for corner cases...  If possible, I would of course prefer to be able to rely on osicat or the like to insulate me from system or implementation-dependent behavior.  Does anybody have any guidance?

Thanks in advance.

-jm

Max Rottenkolber | 11 May 21:15 2013

osicat:native-namestring undocumented.

Hi,

I was about to write a portable NATIVE-NAMESTRING function when I noticed
the package conflict with OSICAT. Since I don't have texinfo-docstrings I
can only guess what's missing so here is a diff:

diff --git a/doc/osicat.texinfo b/doc/osicat.texinfo
index 53df0bf..05965de 100644
--- a/doc/osicat.texinfo
+++ b/doc/osicat.texinfo
 <at>  <at>  -99,6 +99,7  <at>  <at> 
  <at> include include/macro-osicat-with-temporary-file.texinfo
  <at> include include/fun-osicat-make-link.texinfo
  <at> include include/fun-osicat-read-link.texinfo
+ <at> include include/fun-osicat-native-namestring.texinfo

  <at> node Users
  <at> section Users

Cheers,
max

Luís Oliveira | 15 Jul 03:57 2012
Picon

Re: [osicat-devel] bug in UNMERGE-PATHNAMES when called with #P

On Mon, Jul 2, 2012 at 5:09 PM,  <max <at> mr.gy> wrote:
> I understand now. You are right it's not a bug. Instead of having arbitrary no-op behaviour, signalling
errors forces the programmer to
> be explicit.

Yeah, when in doubt I tend to go that way. I thought about this a
little bit more; the argument for the no-op behaviour is that it would
make unmerge-pathnames work more like the inverse of merge-pathnames,
which has no trouble mixing relative and absolute pathnames. But that
just makes it clear that unmerge-pathnames is a misnomer, since it
only deals with directories and ignores other components. A better
name for this function would perhaps reflect its similarities with
enough-namestring.

Another problem with unmerge-pathnames only taking the directory
component into account is that it fails to do the right thing on
Windows, where the device component is particularly relevant for this
operation.

Do you feel like taking a stab at these issues?

Cheers,

--

-- 
Luís Oliveira
http://r42.eu/~luis/

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

max | 22 Jun 03:01 2012

[osicat-devel] bug in UNMERGE-PATHNAMES when called with #P"" for second argument

Hi,

I believe UNMERGE-PATHNAMES should not signal an error if called with #P"" as its
second
argument. Instead it should return its first argument.

I attached a possible bugfix.

Cheers,
max 

Attachment (unmerge-pathnames-fixed.lisp): application/octet-stream, 713 bytes
_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs
Vladimir Sedach | 16 Apr 01:29 2012
Picon

[osicat-devel] Unicode error for Clozure when calling osicat-posix:mkdir on existing directory

I get a strange error when calling osicat-posix:mkdir on an already
existing directory in Clozure.

(osicat-posix:mkdir "/this/directory/exists/" #o777)

An error is thrown:

--------------------------------------------

#<error printing EEXIST #x1985838E>
   [Condition of type OSICAT-POSIX:EEXIST]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [*ABORT] Return to SLIME's top level.
 2: [ABORT-BREAK] Reset this thread
 3: [ABORT] Kill this thread

Backtrace:
  0: (OSICAT-POSIX:POSIX-ERROR 17 NIL OSICAT-POSIX:MKDIR)
  1: (CCL::CALL-CHECK-REGS OSICAT-POSIX:MKDIR #P"/home/viper/tmp/bar/" 511)
  2: (CCL::CHEAP-EVAL (OSICAT-POSIX:MKDIR (MERGE-PATHNAMES "bar/"
"/home/viper/tmp/") 511))
  3: (SWANK::EVAL-REGION "(osicat-posix:mkdir (merge-pathnames
\"bar/\" \"/home/viper/tmp/\") #o777)\n")

--------------------------------------------

Select restart 1 (abort), another error comes up:

--------------------------------------------

Illegal :UTF-8 character starting at position 0.
   [Condition of type BABEL-ENCODINGS:INVALID-UTF8-STARTER-BYTE]

Restarts:
 0: [*ABORT] Return to SLIME's top level.
 1: [ABORT-BREAK] Reset this thread
 2: [ABORT] Kill this thread

Backtrace:
  0: (BABEL-ENCODINGS::UTF-8-DECODER #<A Foreign Pointer
[stack-allocated] #x7F60D08> 0 3 "^ <at> ^ <at> ^ <at> "  0)
  1: (CFFI:FOREIGN-STRING-TO-LISP #<A Foreign Pointer
[stack-allocated] #x7F60D08> :OFFSET 0 :COUNT NIL :MAX-CHARS 16777215
:ENCODING :UTF-8)
  2: (OSICAT-POSIX:STRERROR 17)
  3: ((:INTERNAL (PRINT-OBJECT (OSICAT-POSIX:POSIX-ERROR T))))
  4: (CCL::%PRINT-UNREADABLE-OBJECT #<EEXIST #<error printing
OSICAT-POSIX:EEXIST #x19797F2E> #<STRING-OUTPUT-STREAM  #x1979BB66> T
NIL #<COMPILED-LEXICAL-CLOSURE (:INTERNAL ..))
  5: (#<STANDARD-METHOD PRINT-OBJECT (OSICAT-POSIX:POSIX-ERROR T)>
#<EEXIST #<error printing OSICAT-POSIX:EEXIST #x19797F2E>
#<STRING-OUTPUT-STREAM  #x1979BB66>)
  6: (CCL::%CALL-NEXT-METHOD (NIL #<STANDARD-METHOD PRINT-OBJECT
(OSICAT-POSIX:POSIX-ERROR T)> . 4670269))
  7: (CCL::%%STANDARD-COMBINED-METHOD-DCODE
(#<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT :AROUND (T T)>
#<STANDARD-METHOD PRINT-OBJECT (OSICAT-POSIX:POSIX-ERROR T)>) 4670269)
  8: (NIL #<Unknown Arguments>)
  9: (CCL::WRITE-A-FROB #<EEXIST #<error printing OSICAT-POSIX:EEXIST
#x19797F2E> #<STRING-OUTPUT-STREAM  #x1979BB66> 536870911 NIL)
 10: (CCL::WRITE-INTERNAL #<STRING-OUTPUT-STREAM  #x1979BB66> #<EEXIST
#<error printing OSICAT-POSIX:EEXIST #x19797F2E> 536870911 NIL)
 11: (CCL::WRITE-1 #<EEXIST #<error printing OSICAT-POSIX:EEXIST
#x19797F2E> #<STRING-OUTPUT-STREAM  #x1979BB66> 536870911)
 12: (PRIN1 #<EEXIST #<error printing OSICAT-POSIX:EEXIST #x19797F2E>
#<STRING-OUTPUT-STREAM  #x1979BB66>)
 13: (PRIN1-TO-STRING #<EEXIST #<error printing OSICAT-POSIX:EEXIST #x19797F2E>)
 14: (SWANK:EVAL-FOR-EMACS (SWANK:LISTENER-EVAL "(osicat-posix:mkdir
(merge-pathnames \"bar/\" \"/home/viper/tmp/\") #o777)\n")
"COMMON-LISP-USER" 63)

--------------------------------------------

This doesn't come up in SBCL or CLISP - they both print the EEXIST
error normally.

The environment is Clozure Version 1.9-dev-r15251M  (LinuxX8632) on
SLIME on Ubuntu, both locale and ccl:*default-file-character-encoding*
are utf-8.

Vladimir

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Nicolas Martyanoff | 22 Feb 08:29 2012
Picon

[osicat-devel] prctl() support and test-related patches


Hi,

I added support for the prctl() Linux syscall to Osicat, and made two
test-related fixes.

The patches are available on my Github Osicat fork [1], more
precisely:

https://github.com/galdor/osicat/commit/736bb5c10c9c55ed76def0d58da31c1ad1eeff05
https://github.com/galdor/osicat/commit/bede645f2a3d15d825160c7518c295c95bed4271
https://github.com/galdor/osicat/commit/288dc2ee6a6861ea272ea75bb3e346cda6ca85a7

You may be interested in picking them in the official Osicat repository
:)

[1] https://github.com/galdor/osicat

Regards,

--

-- 
Nicolas Martyanoff
http://wandrian.net
khaelin <at> gmail.com

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Achilles Xu | 6 Feb 10:22 2012
Picon

[osicat-devel] walk-directory does not support unicode filename and diretoryname?

It seems walk-directory will skip files and directories whose name is in Unicode, such as Chinese.


Is this a bug?

Test platform: Mac OS 10.7 Lion, Clozure Common Lisp 1.8

Test Code: 

(require "asdf")
(require "osicat")

(defpackage :com.losttemple.zip-db
  (:use :common-lisp :osicat))

(in-package :com.losttemple.zip-db)

(walk-directory
 (current-directory)
 #'(lambda (x) (format t "~a~%" (absolute-pathname x)))
 :test #'(lambda (x) (format t "---~a~%" (absolute-pathname x)) t)
 :directories :depth-first)

(in-package :common-lisp-user)
(quit)

--
---------------------------
Achilles Xu

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs
Timo Myyrä | 12 Jan 12:20 2012
Picon

[osicat-devel] Fix out-getuid for Clisp

Clisp doesn't have (posix:getuid) function, instead it uses (posix:uid).
Here's small patch to fix it in osicat tests.

Timo

diff --git a/tests/tests.lisp b/tests/tests.lisp
index 2710a8f..7d0c399 100644
--- a/tests/tests.lisp
+++ b/tests/tests.lisp
 <at>  <at>  -59,6 +59,6  <at>  <at> 
 (defun our-getuid ()
   #+sbcl (sb-unix:unix-getuid)
   #+cmu (unix:unix-getuid)
-  #+clisp (posix:getuid)
+  #+clisp (posix:uid)
   #+allegro (excl.osi:getuid)
   #-(or sbcl cmu clisp allegro) 0) ; A sane enough default for testing?

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Timo Myyrä | 8 Jan 10:55 2012
Picon

[osicat-devel] Patch to fix osicat on OpenBSD

Hi,

Here's diff to make osicat work on OpenBSD.

OpenBSD doesn't define blksize_t and blkcnt_t so define them as
'long'. Not sure if this is correct but seems to work.
Neither does OpenBSD have the timer functions so omit them on OpenBSD.
The tests need to be tweaked for OpenBSD too, its similar to darwin in
this. Testing with other BSD's would be welcome to see if it its
needed there as well.

The funcall-getpw function seems to handle return values incorrectly.
As result it would cause exception on OpenBSD with incorrect entries
and not nil value as expected. The fix below works on OpenBSD but
could use some testing on other platforms as well.

Hopefully gmail won't break the diff.

Timo

diff --git a/posix/basic-unixint.lisp b/posix/basic-unixint.lisp
index 3ee3710..88371ae 100644
--- a/posix/basic-unixint.lisp
+++ b/posix/basic-unixint.lisp
 <at>  <at>  -286,12 +286,18  <at>  <at> 
 (ctype dev "dev_t")
 (ctype ino "ino_t")

-#-windows
+#-(or windows openbsd)
 (progn
   (ctype nlink "nlink_t")
   (ctype blksize "blksize_t")
   (ctype blkcnt "blkcnt_t"))

+#+openbsd
+(progn
+  (ctype nlink "nlink_t")
+  (ctype blksize "long")
+  (ctype blkcnt "long"))
+
 (cstruct stat "struct stat"
   (dev     "st_dev"     :type #-mips dev #+mips :unsigned-long)
   (ino     "st_ino"     :type ino)
diff --git a/posix/unix.lisp b/posix/unix.lisp
index 702d32d..f65ff55 100644
--- a/posix/unix.lisp
+++ b/posix/unix.lisp
 <at>  <at>  -394,8 +394,10  <at>  <at> 
     (with-foreign-object (ts 'timespec)
       (with-foreign-slots ((sec nsec) ts timespec)
         (%clock-settime clock-id ts)
-        (values sec nsec))))
+        (values sec nsec)))))

+#-(or darwin openbsd)
+(progn
   (defsyscall ("timer_create" %timer-create) :int
     (clockid clockid)
     (sigevent :pointer)
 <at>  <at>  -646,21 +648,22  <at>  <at>  than C's printf) with format string FORMAT and
arguments ARGS."
   (result  :pointer))

 (defun funcall-getpw (fn arg)
+  ;; http://pubs.opengroup.org/onlinepubs/009695399/functions/getpwnam.html
+  ;; "Applications wishing to check for error situations should set
+  ;; errno to 0 before calling getpwnam(). If getpwnam() returns null
+  ;; pointer and errno is non-zero, an error occured.
+  (set-errno 0)
   (with-foreign-objects ((pw 'passwd) (pwp :pointer))
     (with-foreign-pointer (buf 4096 bufsize)
       (with-foreign-slots ((name passwd uid gid gecos dir shell) pw passwd)
         (let ((ret (funcall fn arg pw buf bufsize pwp)))
-          ;; Darwin's getpwnam_r() is broken a returns -1 when the
-          ;; user is not found.  Not sure if it returns the error
-          ;; number as specified by posix.
-          #+darwin
-          (when (= ret -1)
-            (return-from funcall-getpw nil))
-          (if (zerop ret)
-              (if (null-pointer-p (mem-ref pwp :pointer))
-                  nil
-                  (values name passwd uid gid gecos dir shell))
-              (posix-error ret)))))))
+          (cond ((and (null-pointer-p (mem-ref pwp :pointer))
+                      (not (zerop (get-errno))))
+                 (posix-error ret))
+                ((and (null-pointer-p (mem-ref pwp :pointer))
+                      (zerop (get-errno)))
+                 nil)
+                (t (values name passwd uid gid gecos dir shell))))))))

 (defun getpwuid (uid)
   "Gets the password-entry of a user, by user id."
diff --git a/posix/unixint.lisp b/posix/unixint.lisp
index 43273c9..cee4337 100644
--- a/posix/unixint.lisp
+++ b/posix/unixint.lisp
 <at>  <at>  -105,6 +105,7  <at>  <at> 
   (int "sival_int" :type :int)
   (ptr "sival_ptr" :type :pointer))

+#-openbsd
 (cstruct sigevent "struct sigevent"
   (notify            "sigev_notify"            :type :int)
   (signo             "sigev_signo"             :type :int)
diff --git a/tests/posix.lisp b/tests/posix.lisp
index 32a4eef..de4f35d 100644
--- a/tests/posix.lisp
+++ b/tests/posix.lisp
 <at>  <at>  -139,9 +139,9  <at>  <at> 
 (define-posix-test mkdir.error.2
     (handler-case
         (nix:mkdir "/" 0)
-      (#+darwin nix:eisdir
+      (#+(or darwin openbsd) nix:eisdir
        #+windows nix:eacces
-       #-(or darwin windows) nix:eexist () 'failed))
+       #-(or darwin windows openbsd) nix:eexist () 'failed))
   failed)

 (define-eacces-test mkdir.error.3
 <at>  <at>  -189,9 +189,9  <at>  <at> 
 (define-posix-test rmdir.error.3
     (handler-case
         (nix:rmdir "/")
-      (#+darwin nix:eisdir
+      (#+(or darwin openbsd) nix:eisdir
        #+windows nix:eacces
-       #-(or darwin windows) nix:ebusy () 'failed))
+       #-(or darwin windows openbsd) nix:ebusy () 'failed))
   failed)

 (define-posix-test rmdir.error.4
 <at>  <at>  -206,7 +206,7  <at>  <at> 
           (nix:rmdir dir)
           ;; documented by POSIX
           (not (null (member (system-error-identifier c)
-                             '(:eexist :enotempty #+darwin :enonet
+                             '(:eexist :enotempty #+(or darwin openbsd) :enonet
                                #+windows :enosr)))))))
   t)

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Timo Myyrä | 2 Jan 13:43 2012
Picon

[osicat-devel] blksize_t and blkcnt_t undefined on OpenBSD

Hi,

Osicat fails to compile on OpenBSD because of missing defines for
blksize_t and blkcnt_t.

>>> Actual error output <<<

External process exited with code 1.
Command was: "cc" "-m64"
"-I/home/zmyrgel/quicklisp/dists/quicklisp/software..." "-fPIC" "-o"
"/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openb..."
"/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openb..."
Output was:
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:
In function 'main':
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2041:
error: 'blksize_t' undeclared (first use in this function)
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2041:
error: (Each undeclared identifier is reported only once
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2041:
error: for each function it appears in.)
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2053:
error: 'blkcnt_t' undeclared (first use in this function)

Timo

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs


Gmane