Mackram Raydan | 12 Apr 12:51 2014
Picon

Patch for the handling of symbols in clsql

Hey everyone,

Based on a previous thread, I have submitted a pull request to handle 
the output of symbols when saved with clsql. In line with the discussion 
with Russ, the functionality I submitted continues to use intern but 
makes checks on whether there are packages or not in the saved symbol 
and acts accordingly.

One thing I have missed is writing test cases because I was not sure how 
best to proceed (although I have done tests on my end to verify 
everything is in order). The current tests call directly the high level 
functions such as select or update and not the lower functions like 
read-sql-value. I was not sure if I should change some of the 
def-view-classes to add a symbol column, or to add a new def-view-class 
for this specific test or to basically put the tests that I did directly 
on read-sql-value. Any suggestions on what you would like me to do?

Finally I did the pull request to the git repository found at github.com 
even though the change is small (I am just used to pull requests), if 
someone prefers another method I have no problem in resubmitting in a 
different way.

Thank you

Mackram Raydan
Jussi Lahdenniemi | 8 Apr 16:16 2014
Picon
Picon

Problem with the new db-mysql makefile

Hello,

I just upgraded to the latest clsql with quicklisp, after which the 
db-mysql module failed to compile. Digging into the problem, the culprit 
was the new Makefile with its logic for finding out the ld flags:

LDFLAGS:=$(LDFLAGS) $(shell dpkg-buildflags --get LDFLAGS | sed 
"/-Wl,\(.*\)/ s//\1/g")

I was building clsql in the AWS EC2 environment; there dpkg-buildflags 
--get LDFLAGS results in the following:

-Wl,-Bsymbolic-functions -Wl,-z,relro

The regex above removes the first -Wl, but not the second one, which 
makes the linker unhappy. I fixed this by changing the regex to 
"/-Wl,\([^ ]*\)/ s//\1/g", you might want to incorporate something 
similar into the official version as well.

--

-- 
Jussi Lahdenniemi
Mackram Raydan | 8 Apr 15:08 2014
Picon

Possible new issue introduced with 20140316 release

Hey Everyone,

I would like to point out that during the release of 20140316 which was pushed to quicklisp as the most recent clsql implementation, a new behavior is taking place for the treatment of symbols . This could introduce some bugs in the running code of people if not pointed out, let me illustrate.

In the new version file oodml.lisp line number 624 will read a string from the db and translate into a symbol using the following function:
(intern (symbol-name-default-case val))

Previous version of clsql used to translate a symbol using the following function. Please note this is taken from oodml.lisp line number 690 version 20131211
(read-from-string val)

These two behaviours are quite different from each other specifically consider a function hello-world defined in package alpha if I were to save the symbol 'alpha::hello-world during the previous versions then I would be able to call that function as the returned symbol would be alpha::hello-world however in the new version it would be the symbol common-lisp-user:|alpha::hello-world| which is not callable as a function since it is not defined.

Can someone please help by shedding some light on why the change was done?

Thank you.

_______________________________________________
CLSQL mailing list
CLSQL@...
http://lists.b9.com/cgi-bin/mailman/listinfo/clsql
Nicolas Neuss | 20 Feb 17:14 2014
Picon

Bug in sql/oodml.lisp

Hello,

I got bitten by the following problem: When I have
*print-length* set to some number for preventing output of undesired
length elsewhere, lists longer than that number are not written
correctly to the database when using UPDATE-RECORDS-FROM-INSTANCE.

The correct remedy for this special problem is probably to change the
PROGV special variable list in certain method definitions of
DATABASE-OUTPUT-SQL-AS-TYPE in the file "sql/oodml.lisp" to include also
setting *PRINT-LENGTH* to NIL, i.e.

(defmethod database-output-sql-as-type ((type (eql 'list)) val database db-type)
  (declare (ignore database db-type))
  (progv '(*print-circle* *print-array*) '(t t)
  ...))

becomes

(defmethod database-output-sql-as-type ((type (eql 'list)) val database db-type)
  (declare (ignore database db-type))
  (progv '(*print-circle* *print-array* *print-length*) '(t t nil)
  ...))

Note that the same change has to be performed for the specializations of
this generic function to vectors and arrays later in that file:

  (defmethod database-output-sql-as-type ((type (eql 'vector)) val ...) ...)
  (defmethod database-output-sql-as-type ((type (eql 'array)) val ...) ...)

Yours,

Nicolas

P.S.: What about *PRINT-BASE*?
Aaron Burrow | 7 Jan 05:49 2014
Picon

unsigned 64 bit integers patch

I do not believe unsigned 64 bit integers behave properly.

CL-USER> (setf d (clsql:connect '("127.0.0.1" "dt0" "root" "letmein" 3306)      
                                :database-type :mysql                           
                                :if-exists :new                                 
                                :make-default nil))                             
 <snip>                                                                         
#<CLSQL-MYSQL:MYSQL-DATABASE 127.0.0.1:3306/dt0/root OPEN {10068A59C3}>         
CL-USER> (clsql:query "create table tt (x bigint unsigned)" :database d)        
NIL                                                                             
NIL                                                                             
CL-USER> (clsql:query "select * from tt" :database d)                           
NIL                                                                             
("x")                                                                           
CL-USER> (clsql:query (format nil "insert into tt values (~A), (~A), (~A)"      
                             (expt 2 64) (1- (expt 2 64)) (expt 2 63))          
                      :database d)                                              
NIL                                                                             
NIL                                                                             
CL-USER> (clsql:query "select * from tt" :database d)                           
((-1) (-1) (-9223372036854775808))                                              
("x")      

The issue seemed to stem from indiscriminately using make-64-bit-integer(...) which was treating everything like it was signed.  My patch uses strtoull and strtoll in an attempt to match the style seen nearby.  Why do you not use parse-integer(...) ?

patch:

diff --git a/uffi/clsql-uffi.lisp b/uffi/clsql-uffi.lisp                        
index 79d423f..942c69d 100644                                                   
--- a/uffi/clsql-uffi.lisp                                                      
+++ b/uffi/clsql-uffi.lisp                                                      
<at> <at> -71,6 +71,18 <at> <at>                                                              
      (radix :int))                                                             
   :returning :unsigned-long)                                                   
                                                                                
+(uffi:def-function ("strtoull" c-strtoull)                                     
+    ((str (* :unsigned-char))                                                  
+     (endptr (* :unsigned-char))                                               
+     (radix :int))                                                             
+  :returning :unsigned-long-long)                                              
+                                                                               
+(uffi:def-function ("strtoll" c-strtoll)                                       
+    ((str (* :unsigned-char))                                                  
+     (endptr (* :unsigned-char))                                               
+     (radix :int))                                                             
+  :returning :unsigned-long-long)                                              
+                                                                               
 (uffi:def-function "atol"                                                      
     ((str (* :unsigned-char)))                                                 
   :returning :long)                                                            
<at> <at> -108,6 +120,16 <at> <at>                                                            
            (type char-ptr-def char-ptr))                                       
   (c-strtoul char-ptr uffi:+null-cstring-pointer+ 10))                         
                                                                                
+(defun strtoull (char-ptr)                                                     
+  (declare (optimize (speed 3) (safety 0) (space 0))                           
+           (type char-ptr-def char-ptr))                                       
+  (c-strtoull char-ptr uffi:+null-cstring-pointer+ 10))                        
+                                                                               
+(defun strtoll (char-ptr)                                                      
+  (declare (optimize (speed 3) (safety 0) (space 0))                           
+           (type char-ptr-def char-ptr))                                       
+  (c-strtoll char-ptr uffi:+null-cstring-pointer+ 10))                         
+                                                                               
 (defun convert-raw-field (char-ptr type &key length encoding)                  
  (declare (optimize (speed 3) (safety 0) (space 0))                            
           (type char-ptr-def char-ptr))                                        
<at> <at> -122,17 +144,14 <at> <at>                                                           
        (atol char-ptr))                                                        
       (:int32                                                                  
        (atoi char-ptr))                                                        
+      (:int64                                                                  
+       (strtoll char-ptr))                                                     
       (:uint32                                                                 
        (strtoul char-ptr))                                                     
       (:uint                                                                   
        (strtoul char-ptr))                                                     
-      ((:int64 :uint64)                                                        
-       (uffi:with-foreign-object (high32-ptr :unsigned-int)                    
-         (let ((low32 (atol64 char-ptr high32-ptr))                            
-               (high32 (uffi:deref-pointer high32-ptr :unsigned-int)))         
-           (if (zerop high32)                                                  
-               low32                                                           
-               (make-64-bit-integer high32 low32)))))                          
+      (:uint64                                                                 
+       (strtoull char-ptr))                                                    
       (:blob                                                                   
        (if length                                                              
            (uffi:convert-from-foreign-usb8 char-ptr length)       
_______________________________________________
CLSQL mailing list
CLSQL@...
http://lists.b9.com/cgi-bin/mailman/listinfo/clsql
Kiss Kálmán | 20 Nov 09:18 2013
Picon

Re: ccl getenv fix


Good remark, modify the "#+mcl..." line then, like:
#+(or mcl ccl) (ccl::getenv var)

Take care
Kami

-------- Eredeti üzenet --------
Tárgy: Re: [CLSQL] ccl getenv fix
Feladó: Ralf Mattes <rm <at> seid-online.de>
Címzett: Kiss Kálmán <kami <at> zalaszam.hu>
CC: clsql <at> b9.com
Dátum: 2013.11.14. 17:12

> On Thu, Nov 14, 2013 at 03:03:08PM +0100, Kiss Kálmán wrote:
>>
>> At least the Oracle driver uses the implementation specific getenv.
>>
>> --- sql/utils.lisp    2013-11-14 14:42:54.000000000 +0100
>> +++ sql/utils.lisp.new    2013-11-14 14:28:46.000000000 +0100
>>  <at>  <at>  -376,7 +376,8  <at>  <at> 
>>                  :key #'string))
>>      #+lispworks (lw:environment-variable (string var))
>>      #+mcl (ccl::getenv var)
>> -  #+sbcl (sb-ext:posix-getenv var))
>> +  #+sbcl (sb-ext:posix-getenv var)
>> +  #+ccl (getenv var))
>
> Please make this:
>
>   +  #+ccl (ccl:getenv var))
>
> An unqualified symbol outside cl might by bound to surprising functions ...
>
>   Cheers, Ralf Mattes
>

_______________________________________________
CLSQL mailing list
CLSQL <at> b9.com
http://lists.b9.com/cgi-bin/mailman/listinfo/clsql
Kiss Kálmán | 14 Nov 15:03 2013
Picon

ccl getenv fix


At least the Oracle driver uses the implementation specific getenv.

--- sql/utils.lisp    2013-11-14 14:42:54.000000000 +0100
+++ sql/utils.lisp.new    2013-11-14 14:28:46.000000000 +0100
 <at>  <at>  -376,7 +376,8  <at>  <at> 
                :key #'string))
    #+lispworks (lw:environment-variable (string var))
    #+mcl (ccl::getenv var)
-  #+sbcl (sb-ext:posix-getenv var))
+  #+sbcl (sb-ext:posix-getenv var)
+  #+ccl (getenv var))

  (eval-when (:compile-toplevel :load-toplevel :execute)
    (when (char= #\a (schar (symbol-name '#:a) 0))
Victor | 20 Oct 20:38 2013

Re: Migrating from UFFI to CFFI

On Sun, 20 Oct 2013 05:39:35 +0300, Kevin Rosenberg <kevin@...> wrote:

> On Oct 19, 2013, at 2:33 PM, Victor <bobbie@...> wrote:
>> I think that it is possible to get rid of this library: the only
>> useful function that it provides is "atol64" that converts a string to
>> a 64 bit integer.
>
> Yes, that is its only function.
>
>> "atol64" is used in the "CONVERT-RAW-FIELD" function. As far as I
>> understand it could be replaced by calling PARSE-INTEGER on the string
>> in Lisp. This will make the native library not really needed.
>
> The decision to undergo to effort to write and test the native C
> function was not taken lightly. If one wants to take time to try a version
> of CLSQL using an all Lisp implementation and benchmark it against the
> current version on a number of platforms and see that it performs well,
> I'd love to drop the C interface. For my application, this is a function
> is called hundreds of millions of times and optimization is important.
>
>> Another option is to use a standard C/C++ function to do this task as
>> described in this StackOverflow thread:
>>
>> http://stackoverflow.com/questions/7502926/what-is-atoi-equivalent-for-64bit-integeruint64-t-in-c-that-works-on-both-unix/7503328#7503328
>
> The problem is supporting both 32-bit and 64-bit lisp implementations.
> At the time I wrote this, not all of the implementations had a way to handle
> 64-bit results on a 32-bit implementation.
>
>>  What do you think? Are there better ways out?
>
> We can discuss a lot of pros and cons of various approachs, but I
> don't think any of them seem will be as time efficient as you
> getting the current C library compiled on your platform.
> Over the years, I compiled this on very many platforms without much effort.
> I think the uffi/Makefile.msvc would be a useful starting point.
> Perhaps a search of the mail list archive would identify other
> instructions, but I am not certain that this compilation
> step has been discussed. Also, the last I heard was Microsoft
> had a free "express" version of their C compiler available.
>

Hi Kevin,

I tried to work without the clsql_uffi library on Linux (Ubuntu) and
will try these modifications on the "problematic" Windows 7 system
tomorrow. Diff for the changes is here:

https://github.com/vityok/clsql/commit/eabede9da3e61ef5191b601dcf9b30f30f613f3e

Actually, I would like to avoid dependencies as much as possible.

I would like to ask participants of this mailing list who have
interest in this issue to review the modifications. Your comments are
very welcome!

Thanks,
Victor
Victor | 19 Oct 22:33 2013

Re: Migrating from UFFI to CFFI


On Sat, 19 Oct 2013 05:53:10 +0300, Kevin Rosenberg <kevin@...> wrote:

> On Oct 18, 2013, at 4:22 AM, Victor <bobbie@...> wrote:
>> I've got yet new issues that I did not anticipate: on Windows 7 with
>> Clozure CL 1.9 loading clsql-sqlite3 fails due to the missing .dll:
>>
>> | The program can't start because MSVCR71.dll is missing from your
>> | computer. Try reinstalling the program to fix this problem.
>>
>> If this error is ignored, the next one pops up:
>>
>> | clsql_uffi.lib is either not designed to run on Windows or it contains
>> | an error.
>>
>> As far as I understand the vcr71 library is a deliverable component
>> of the Visual Studio that the clsql_uffi.dll relies on.
>
> That is a microsoft visual C runtime library, though I don't know if the
> version being asked for relates to the binary of clsql_uffi.dll or some
> other component of your windows installation.  I don't develop for windows,
> but a simple character match in the binary doesn't find MSVCR71.
>
> I see the file data of clsql_uffi.dll is over 5 years ago. You may do
> well to compile clsql_uffi.c with a more current C compiler -- perhaps
> using the same compiler that you are using for SQLite.
>
> clsql_uffi.c is a simple file with only a single short function that
> should be easy to compatible with most all windows C compilers.
>

Hi Kevin,

Unfortunately compiling the C sources is not really an option: I did
not do this on Windows long enough and I think that it is not really
needed.

I think that it is possible to get rid of this library: the only
useful function that it provides is "atol64" that converts a string to
a 64 bit integer.

"atol64" is used in the "CONVERT-RAW-FIELD" function. As far as I
understand it could be replaced by calling PARSE-INTEGER on the string
in Lisp. This will make the native library not really needed.

Another option is to use a standard C/C++ function to do this task as
described in this StackOverflow thread:

http://stackoverflow.com/questions/7502926/what-is-atoi-equivalent-for-64bit-integeruint64-t-in-c-that-works-on-both-unix/7503328#7503328

What do you think? Are there better ways out?

Thanks,
Victor
Munawar Cheema | 10 Oct 16:20 2013
Picon

Is CCL a supported implementation CLSQL?

I wanted to try out clsql but didn't see any mention of Clozure implementation.  Is it supported? I am new to lisp so maybe asking something obvious.
MC

--
Kind Regards
Munawar

_______________________________________________
CLSQL mailing list
CLSQL@...
http://lists.b9.com/cgi-bin/mailman/listinfo/clsql
Nicolas Neuss | 15 Oct 15:27 2013
Picon

SQL numeric -> CL rational?

Hello,

I just observe that numbers like 1.5 (of type 'numeric' in my Postgresql
database) appear as rationals 3/2 in SELECT queries or VIEW-CLASS
objects.

Was this changed recently?  If yes, is there a way to revert to the old
behavior?  I have an old application which I am reactivating and which
breaks at this point, and I am just wondering how I should fix it in the
most reasonable way.

Thank you,

Nicolas

Gmane