Achilles Xu | 6 Feb 10:22
Picon
Gravatar

[osicat-devel] walk-directory does not support unicode filename and diretoryname?

It seems walk-directory will skip files and directories whose name is in Unicode, such as Chinese.


Is this a bug?

Test platform: Mac OS 10.7 Lion, Clozure Common Lisp 1.8

Test Code: 

(require "asdf")
(require "osicat")

(defpackage :com.losttemple.zip-db
  (:use :common-lisp :osicat))

(in-package :com.losttemple.zip-db)

(walk-directory
 (current-directory)
 #'(lambda (x) (format t "~a~%" (absolute-pathname x)))
 :test #'(lambda (x) (format t "---~a~%" (absolute-pathname x)) t)
 :directories :depth-first)

(in-package :common-lisp-user)
(quit)

--
---------------------------
Achilles Xu

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs
Timo Myyrä | 12 Jan 12:20
Picon
Gravatar

[osicat-devel] Fix out-getuid for Clisp

Clisp doesn't have (posix:getuid) function, instead it uses (posix:uid).
Here's small patch to fix it in osicat tests.

Timo

diff --git a/tests/tests.lisp b/tests/tests.lisp
index 2710a8f..7d0c399 100644
--- a/tests/tests.lisp
+++ b/tests/tests.lisp
@@ -59,6 +59,6 @@
 (defun our-getuid ()
   #+sbcl (sb-unix:unix-getuid)
   #+cmu (unix:unix-getuid)
-  #+clisp (posix:getuid)
+  #+clisp (posix:uid)
   #+allegro (excl.osi:getuid)
   #-(or sbcl cmu clisp allegro) 0) ; A sane enough default for testing?

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Timo Myyrä | 8 Jan 10:55
Picon
Gravatar

[osicat-devel] Patch to fix osicat on OpenBSD

Hi,

Here's diff to make osicat work on OpenBSD.

OpenBSD doesn't define blksize_t and blkcnt_t so define them as
'long'. Not sure if this is correct but seems to work.
Neither does OpenBSD have the timer functions so omit them on OpenBSD.
The tests need to be tweaked for OpenBSD too, its similar to darwin in
this. Testing with other BSD's would be welcome to see if it its
needed there as well.

The funcall-getpw function seems to handle return values incorrectly.
As result it would cause exception on OpenBSD with incorrect entries
and not nil value as expected. The fix below works on OpenBSD but
could use some testing on other platforms as well.

Hopefully gmail won't break the diff.

Timo

diff --git a/posix/basic-unixint.lisp b/posix/basic-unixint.lisp
index 3ee3710..88371ae 100644
--- a/posix/basic-unixint.lisp
+++ b/posix/basic-unixint.lisp
@@ -286,12 +286,18 @@
 (ctype dev "dev_t")
 (ctype ino "ino_t")

-#-windows
+#-(or windows openbsd)
 (progn
   (ctype nlink "nlink_t")
   (ctype blksize "blksize_t")
   (ctype blkcnt "blkcnt_t"))

+#+openbsd
+(progn
+  (ctype nlink "nlink_t")
+  (ctype blksize "long")
+  (ctype blkcnt "long"))
+
 (cstruct stat "struct stat"
   (dev     "st_dev"     :type #-mips dev #+mips :unsigned-long)
   (ino     "st_ino"     :type ino)
diff --git a/posix/unix.lisp b/posix/unix.lisp
index 702d32d..f65ff55 100644
--- a/posix/unix.lisp
+++ b/posix/unix.lisp
@@ -394,8 +394,10 @@
     (with-foreign-object (ts 'timespec)
       (with-foreign-slots ((sec nsec) ts timespec)
         (%clock-settime clock-id ts)
-        (values sec nsec))))
+        (values sec nsec)))))

+#-(or darwin openbsd)
+(progn
   (defsyscall ("timer_create" %timer-create) :int
     (clockid clockid)
     (sigevent :pointer)
@@ -646,21 +648,22 @@ than C's printf) with format string FORMAT and
arguments ARGS."
   (result  :pointer))

 (defun funcall-getpw (fn arg)
+  ;; http://pubs.opengroup.org/onlinepubs/009695399/functions/getpwnam.html
+  ;; "Applications wishing to check for error situations should set
+  ;; errno to 0 before calling getpwnam(). If getpwnam() returns null
+  ;; pointer and errno is non-zero, an error occured.
+  (set-errno 0)
   (with-foreign-objects ((pw 'passwd) (pwp :pointer))
     (with-foreign-pointer (buf 4096 bufsize)
       (with-foreign-slots ((name passwd uid gid gecos dir shell) pw passwd)
         (let ((ret (funcall fn arg pw buf bufsize pwp)))
-          ;; Darwin's getpwnam_r() is broken a returns -1 when the
-          ;; user is not found.  Not sure if it returns the error
-          ;; number as specified by posix.
-          #+darwin
-          (when (= ret -1)
-            (return-from funcall-getpw nil))
-          (if (zerop ret)
-              (if (null-pointer-p (mem-ref pwp :pointer))
-                  nil
-                  (values name passwd uid gid gecos dir shell))
-              (posix-error ret)))))))
+          (cond ((and (null-pointer-p (mem-ref pwp :pointer))
+                      (not (zerop (get-errno))))
+                 (posix-error ret))
+                ((and (null-pointer-p (mem-ref pwp :pointer))
+                      (zerop (get-errno)))
+                 nil)
+                (t (values name passwd uid gid gecos dir shell))))))))

 (defun getpwuid (uid)
   "Gets the password-entry of a user, by user id."
diff --git a/posix/unixint.lisp b/posix/unixint.lisp
index 43273c9..cee4337 100644
--- a/posix/unixint.lisp
+++ b/posix/unixint.lisp
@@ -105,6 +105,7 @@
   (int "sival_int" :type :int)
   (ptr "sival_ptr" :type :pointer))

+#-openbsd
 (cstruct sigevent "struct sigevent"
   (notify            "sigev_notify"            :type :int)
   (signo             "sigev_signo"             :type :int)
diff --git a/tests/posix.lisp b/tests/posix.lisp
index 32a4eef..de4f35d 100644
--- a/tests/posix.lisp
+++ b/tests/posix.lisp
@@ -139,9 +139,9 @@
 (define-posix-test mkdir.error.2
     (handler-case
         (nix:mkdir "/" 0)
-      (#+darwin nix:eisdir
+      (#+(or darwin openbsd) nix:eisdir
        #+windows nix:eacces
-       #-(or darwin windows) nix:eexist () 'failed))
+       #-(or darwin windows openbsd) nix:eexist () 'failed))
   failed)

 (define-eacces-test mkdir.error.3
@@ -189,9 +189,9 @@
 (define-posix-test rmdir.error.3
     (handler-case
         (nix:rmdir "/")
-      (#+darwin nix:eisdir
+      (#+(or darwin openbsd) nix:eisdir
        #+windows nix:eacces
-       #-(or darwin windows) nix:ebusy () 'failed))
+       #-(or darwin windows openbsd) nix:ebusy () 'failed))
   failed)

 (define-posix-test rmdir.error.4
@@ -206,7 +206,7 @@
           (nix:rmdir dir)
           ;; documented by POSIX
           (not (null (member (system-error-identifier c)
-                             '(:eexist :enotempty #+darwin :enonet
+                             '(:eexist :enotempty #+(or darwin openbsd) :enonet
                                #+windows :enosr)))))))
   t)

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Timo Myyrä | 2 Jan 13:43
Picon
Gravatar

[osicat-devel] blksize_t and blkcnt_t undefined on OpenBSD

Hi,

Osicat fails to compile on OpenBSD because of missing defines for
blksize_t and blkcnt_t.

>>> Actual error output <<<

External process exited with code 1.
Command was: "cc" "-m64"
"-I/home/zmyrgel/quicklisp/dists/quicklisp/software..." "-fPIC" "-o"
"/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openb..."
"/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openb..."
Output was:
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:
In function 'main':
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2041:
error: 'blksize_t' undeclared (first use in this function)
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2041:
error: (Each undeclared identifier is reported only once
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2041:
error: for each function it appears in.)
/home/zmyrgel/.cache/common-lisp/sbcl-1.0.54.openbsd-bsd-x64/home/zmyrgel/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c:2053:
error: 'blkcnt_t' undeclared (first use in this function)

Timo

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Peter Denno | 27 Nov 01:00
Picon

never mind... Problem compiling osicat-posix with quicklisp on x86_64 with 32-bit lisp....

Hi Osicat developers,

Disregard my previous note about problems compiling osicat-posix on 64-bit for 32-bit lispworks. Thing work just fine if you install the 32-bit gcc, and just let it happen!

Best regards,
   Peter

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs
Peter Denno | 26 Nov 23:49
Picon

Problem compiling osicat-posix with quicklisp on x86_64 with 32-bit lispworks

Hi,

I ran into a problem compiling osicat-posix under the environment described in the subject line. What I get:

[package osicat-posix]; /usr/bin/cc -m32 -I/home/pdenno/quicklisp/dists/quicklisp/software/cffi_0.10.6/ -fPIC -o /home/pdenno/.cache/common-lisp/lw-6.0.1-linux-x86/home/pdenno/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint /home/pdenno/.cache/common-lisp/lw-6.0.1-linux-x86/home/pdenno/quicklisp/dists/quicklisp/software/osicat-20110619-git/posix/basic-unixint.c

External process exited with code 1.
Command was: "/usr/bin/cc" "-m32" "-I/home/pdenno/quicklisp/dists/quicklisp/software/..." "-fPIC" "-o" "/home/pdenno/.cache/common-lisp/lw-6.0.1-linux-x86..." "/home/pdenno/.cache/common-lisp/lw-6.0.1-linux-x86..."
Output was:
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /usr/lib64/gcc/x86_64-suse-linux/4.3/libgcc.a when searching for -lgcc
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: cannot find -lgcc
collect2: ld returned 1 exit status

I thought the -m32 a bit suspicious. Perhaps this is a cffi problem? I get
CL-USER>  cffi-grovel::*cpu-word-size-flags*
("-m32")

... which seems odd.

Osicat, and everything else compiles fine on sbcl.

Best regards,
  Peter


_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs
Elias Pipping | 19 Sep 18:40

open-temporary-file

Hello,

the function `open-temporary-file` currently returns a stream and
nothing else. Consequently, `with-temporary-file` only exposes a
stream to its body. From my understanding it is (intentionally,
because streams are an abstraction) not possible to tell what the name
of the corresponding temporary file is just by that.

Could open-temporary-file be made to return the name of that file as
well?

I'd like to be able to write code like the following

(osicat:with-temporary-file (stream filename)
  (format stream "some content")
  (= 0 (sb-ext:process-exit-code (sb-ext:run-program "./test.sh" (list filename)))))

In other words, I'd like to pass strings in the form of (temporary)
files to a script/program that reads from the file whose name it is
passed and use `with-temporary-file` for that. Is that reasonable?

Best regards,

Elias Pipping

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

MON KEY | 17 Aug 20:00

Re: osicat dotted-namestrings and absolute-pathname interaction around directory iterators

Very sorry for needing to do this again.
Where previous mail said:

,----
| Again, I would prefer that (osicat:list-directory ".") not return a
| value that is CL:EQUAL (osicat:absolute-pathname "./").
`----

It should have said:

Again, I would prefer that (osicat:list-directory ".") not return a
value that is CL:EQUAL (osicat:list-directory "./").

--
/s_P\

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

MON KEY | 12 Aug 21:36

osicat dotted-namestrings and absolute-pathname interaction around directory iterators

I initially neglected to send this register an account w/ osicat-devel
and mailed the following exchange to Nikodemus directly
with the subject line: "osicat:list-directory vs. cl-fad:list-directory"

No doubt Nikodemus has plenty on his plate right now as it is and
needn't be solely responsible for fielding this verobisty :)

I should have taken the time to register an osicat-devel account.
Having done so I am now forwarding a transcript of my previous exchange.

--------------
On 12 August 2011 19:46, MON KEY wrote:
I noticed today that on SBCL these return differently:

 (cl-fad:list-directory ".")  (osicat:list-directory ".")
 (cl-fad:list-directory "..") (osicat:list-directory "..")

Assuming this is a bug, then i'm pretty sure its origin is around the
reliance on osicat:with-directory-iterator.

osicat:call-with-directory-iterator binds *d-p-d* to the value of the
dir var before setting cwd in the protected form of its
unwind-protect.

Further up in osicat:call-with-directory-iterator there is a merge
with *d-p-d* via:

 (osicat:absolute-pathname <PATHSPEC>)

When the PATHSPEC arg to osicat:list-directory is a dotted
namestring e.g. "."  or ".."  the merge with *d-p-d* in
osicat:call-with-directory-iterator will yield the equivalent of:

 (merge-pathnames "." *default-pathname-defaults*)

such that for the duration of each #'one-iter *d-p-d* gets dynamically
funky bound.

--------------

On Fri, Aug 12, 2011 at 1:07 PM, Nikodemus Siivola wrote:
> The docstrings for many of the osicat functions mirror cl-fad's, so
> I am ssuming this is a bug, then i'm pretty sure its origin is around

Well, Osicat predates CL-FAD by a year or so, so not really.

A quick look at results from Osicat looks like just what I would
expect -- what exactly is the difference you think is a bug?

Possibly you're looking for :bare-pathnames t?

---------------
> On 12 August 2011 19:46, MON KEY  wrote:
>> The docstrings for many of the osicat functions mirror cl-fad's, so
>> I am assuming this is a bug, then i'm pretty sure its origin is around
>
> Well, Osicat predates CL-FAD by a year or so, so not really.

Would you not agree that they are curiosly similar?

> Possibly you're looking for :bare-pathnames t?

No, I didn't find :bare-pathnames t necessarily applicable w/r/t
dotted-namestrings, and neither of the respcective below pairs do what
I would expect:

 (osicat:list-directory ".")
 (osicat:list-directory "."  :bare-pathnames t)

 (osicat:list-directory "..")
 (osicat:list-directory ".." :bare-pathnames t)

Note however, that these forms do retrun what i would expect:

 (osicat:list-directory "./")
 (osicat:list-directory "." :bare-pathnames t)

 (osicat:list-directory "../")
 (osicat:list-directory "../" :bare-pathnames t)

When a namestring is given as a dotted-namestring and a trailing #\/
(solidus) is _not_ present, I would expect Osicat to resolve
dotted-namestrings their cl:truename before executing the inspection
PATHSPEC in the iterations.

When a namestring is given as a dotted-namestring and a trailing #\/
(solidus) _is_ present I would expect Osicat to _not_ resolve the
dotted-namestrings.

>
> A quick look at results from Osicat looks like just what I would
> expect -- what exactly is the difference you think is a bug?
>

Following evaulated from SBCL "1.0.47.1":

*default-pathname-defaults*
;=> #P"/home/me/long-path/here/subdir/sub-subdir/"

(cl:pathname-type #P".")
;=> NIL

(cl:pathname-name #P".")
;=> "."

(cl:pathname-name "..")
;=> "."

(cl:pathname-name "..")
;=> "."

(cl:pathname-name "../")
=> NIL

(cl:pathname-name "./")
=> NIL

(cl:pathname-type ".")
;=> NIL

(cl:pathname-type "..")
;=> ""

(cl:pathname-type "./")
;=> NIL

(cl:pathname-type "../")
;=> NIL

(cl:pathname-directory ".")
;=> NIL

(cl:pathname-directory "..")
;=> NIL

(pathname-directory "./")
;=> (:RELATIVE ".")

(pathname-directory "../")
;=> (:RELATIVE :UP)

(cl:truename ".")
;=> #P"/home/me/long-path/here/subdir/sub-subdir/"

(cl:truename "..")
;=> #p"/home/sp/hg-repos/cl-repo-hg/cl-mon-code/"

(cl:truename "./")
;=> #P"/home/me/long-path/here/subdir/sub-subdir/"

(cl:truename "../")
;=> #P"/home/me/long-path/here/subdir/"

(cl:probe-file ".")
;=> #P"/home/me/long-path/here/subdir/sub-subdir/"

(cl:probe-file "..")
;=> #P"/home/me/long-path/here/subdir/"

(directory ".")
;=> (#P"/home/me/long-path/here/subdir/sub-subdir/")

(directory "..")
;=> (#P"/home/me/long-path/here/subdir/")

(osicat:absolute-pathname ".")
;=> #P"/home/me/long-path/here/subdir/sub-subdir/."

(osicat:absolute-pathname "..")
;=> #P"/home/me/long-path/here/subdir/sub-subdir/.."

(osicat:file-kind ".")
;=> :DIRECTORY

(osicat:file-kind "..")
;=> :DIRECTORY

(osicat:file-kind (osicat:absolute-pathname ".."))
;=> :DIRECTORY

(osicat:file-kind (osicat:absolute-pathname "."))
;=> :DIRECTORY

(osicat:directory-pathname-p ".")
;=> NIL

(osicat:directory-pathname-p "..")
;=> NIL

(osicat:pathname-as-directory ".")
;=> #P"./"

(osicat:pathname-as-directory "..")
;=> #P"../"

(osicat:pathname-directory-pathname ".")
;=> #P""

(osicat:pathname-directory-pathname "..")
;=> #P""

Disregarding the following assertion from the README of Quicklisp's
dist of Osicat as to the extent of its Posixness:

,----
|
| Osicat is a lightweight operating system interface for Common Lisp
| on Unix-platforms. It is not a POSIX-style API, but rather a simple
| lispy accompaniment to the standard ANSI facilities.
|
`---- :SOURE osicat-20110619-git/README

It may be worth considering osicats behaviour w/r/t following from
POSIX.1-2008 apropos dotted namestrings:

,----
|
| 4.12 Pathname Resolution
|  {...}
|
| The special filename dot shall refer to the directory specified by
| its predecessor. The special filename dot-dot shall refer to the
| parent directory of its predecessor directory. As a special case, in
| the root directory, dot-dot may refer to the root directory itself.
|
| The Open Group Base Specifications Issue 7 IEEE Std 1003.1-2008
|
`---- :SOURCE (URL `http://pubs.opengroup.org/onlinepubs/9699919799/')

Following may also be applicable:
 (URL `http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#dot')

As mentioned on #lisp, I'm not interested in advocating that Osicat
be changed if it isn't broken.

The reliable portablility offered by Osicat underscores my interest in
it and I am currently in the process of converting the CL-FAD and SBCL
specific routines use in my code to use Osicat instead :) As such, my
concern/curiousity is to learn the rationale for its implementation so
as to avoid unnecessary future surprises.

AFAICT Osicat absolute-pathname treats the dotted namestring as
pathname-name whereas these:
 (osicat:file-kind "..")
 (osicat:file-kind ".")
consider them to be pathname-directory.

So long as Osicat allows pathspec args to take occur as a namestring
(as opposed to a #P"." or #P"..") then there remains a certain
ambiguity as to what type of pathname component the user intended the
dotted-namestring to resolve to.

To the extent with which the following behave similiarly:
 (osicat:list-directory ".")  shell> ls .
 (osicat:list-directory ".")  shell> ls ..

Osicats dotted-namestring seems sane.

However, barring `scandir` `readdir` where there isn't really a POSIX
equivalent to osicat:mapdir, osicat:walk-directory,
osicat:with-directory-iterator I would advocate that Osicat behave as
SBLC does and resolve the dotted-namestring to its cl:truename.

--
/s_P\

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs

Alexander Kahl | 6 Jul 10:42
Picon
Gravatar

Patch for getgrnam / getgrgid members

I've created a patch for the missing gr_mem field of cstruct group that
fetches the group members properly as a list of strings. Who can I send
the patch or where can I get git push rights, alternatively?

Alex

--

-- 
Alexander Kahl    █▉     www: http://fsfe.org
FSFE Fellow     █▉█▉█▉   e-mail: e-user <at> fsfe.org
Lambda Hacker     ▉▉     xmpp: e-user <at> jabber.fsfe.org

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs
Luís Oliveira | 28 May 22:03
Picon
Gravatar

Using github

Hello,

I'm moving a bunch of repositories to github, to make projects more
visible, keep track of forks, etc. Unless there are any objections, I
can put osicat there as well as an organization and add current
committers as owners.

Cheers,

--

-- 
Luís Oliveira
http://r42.eu/~luis/

_______________________________________________
pg-cvs site list
pg-cvs <at> common-lisp.net
http://common-lisp.net/mailman/listinfo/pg-cvs


Gmane