Faré | 21 Sep 01:41 2013
Picon

patch from qres

I have a patch for babel as used by qres.

It was originally written by Scott McKay and Dan Weinreb (RIP), to
1- add support for CESU-8 (as infamously used by Oracle).
2- speed up babel (and measure the speed)
Plus by me to
3- support XCVB.

I've removed some extraneous type declarations from SWM&DLW,
where some other macro already declared a type, and also removed
some handling for surrogate codes that was already added upstream since.

The patch is attached (against HEAD on github).

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Invent a clever saying, and your name shall live forever.
        — Anonymous
Attachment (x.diff): application/octet-stream, 64 KiB
Frank | 19 Aug 10:17 2013
Picon

Introspection

Hello,

I would like to determine the (max) number of bytes a character/units in
a certain encoding consumes.  Unfortunately there does not seem to be an
interface exposed for it (or I just don't see it).

I'm doing it this way:

(defun max-code-units (encoding)
  (babel::enc-max-units-per-char (babel::get-character-encoding
encoding)))

(defun code-unit-size (encoding)
  (ash (babel::enc-code-unit-size (babel::get-character-encoding
encoding)) -3))

(defun max-code-size (encoding)
  (let ((e (babel::get-character-encoding encoding)))
    (* (babel::enc-max-units-per-char e)
       (ash (babel::enc-code-unit-size e) -3))))

Is this a good idea to to that way?  Please comment.

Thanks,
Frank

Michel Deloncle | 11 Apr 23:04 2013

Votre Convention collective

Si vous avez des difficultés de lire ce message, visualisez-le grâce à ce chemin.




Votre convention collective

La Convention Collective nationale dont vous dépendez est disponible.
Elle vous est livrée dans son classeur.

  • Sommaire thématique
    (classification, durée de travail, formation, salaire...)
La mise à jour de votre Convention sera assurée sans frais
pendant 1 an.*

Cet e-mail commercial est conforme à la législation en vigueur et aux délibérations de la CNIL des 22 et 30 mars deux milles cinq sur la prospection par courrier électronique dans le cadre professionnel. Conformément à l’article 34 de la loi 78-17 du 6 janvier mille neuf cent soixante dix huit relative à l’informatique, aux fichiers et aux libertés,vous disposez d’un droit d’accès, de rectification de données nominative vous concernant.
se désabonner
Stelian Ionescu | 11 Apr 14:57 2013

Test

Test

--

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib

Erik Huelsmann | 21 Oct 19:05 2012
Picon

BABEL-TESTS on ABCL

Hi Luis,


Today we've been working on ABCL being able to compile and load BABEL; actually, it could do that already, but didn't work with BABEL-TESTS. We've committed some fixes which handle reader issues uncovered by the library, but we're now stuck: there's a bit of code commented out on CCL. It seems we're running into the same problems as CCL, due to strictness of interpretation of code points.

Could you mark the #-ccl section in tests/tests.lisp as #-(or ccl abcl)? That solves our problem with the #\udcf0 code point.

Thanks in advance!

Bye,


Erik.
_______________________________________________
armedbear-devel mailing list
armedbear-devel@...
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
harven | 23 Mar 16:35 2012
Picon

error downloading babel

Hello,

I encounter a problem trying to download babel using quicklisp. 

I am on debian squeeze and I use the last version of sbcl 1.0.55 that I
downloaded 5mns ago from the website, together with a fresh install of
quicklisp. I can reproduce the error using the version of sbcl 1.0.40
in the debian repositories. 

The relevant part of the log which is attached to this message seems to be
* (ql:quickload "babel")
To load "babel":
  Load 1 ASDF system:
    babel
; Loading "babel"

; file: /home/harven/.quicklisp/dists/quicklisp/software/babel-20120208-git/src/enc-unicode.lisp
; in: DEFINE-UTF-16 :UTF-16
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

A similar error has been reported on github 
https://github.com/cl-babel/babel/issues/9
where it was suggested to run the command
(alexandria:format-symbol t '#:~a-code-point-counter "UTF16") 
Here is what I get.

* (ql:quickload "alexandria")
To load "alexandria":
  Load 1 ASDF system:
    alexandria
; Loading "alexandria"

("alexandria")
* (alexandria:format-symbol t '#:~a-code-point-counter "UTF16") 

debugger invoked on a SB-KERNEL:CASE-FAILURE in thread
#<THREAD "initial thread" RUNNING {AB2C889}>:
  #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
  Wanted one of (STRING SIMPLE-STRING).

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(SB-KERNEL:CASE-FAILURE
 ETYPECASE
 #:~A-CODE-POINT-COUNTER
 (STRING SIMPLE-STRING))
0] 0

Any help ?

* (ql:quickload "babel")
To load "babel":
  Load 1 ASDF system:
    babel
; Loading "babel"

; file: /home/harven/.quicklisp/dists/quicklisp/software/babel-20120208-git/src/enc-unicode.lisp
; in: DEFINE-UTF-16 :UTF-16
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UTF-16 :UTF-16LE
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16LE :LE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16LE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UTF-16 :UTF-16BE
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16BE :BE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16BE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UTF-32
;     (BABEL-ENCODINGS::DEFINE-UCS :UTF-32 4)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UTF-32 ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UTF-32LE
;     (BABEL-ENCODINGS::DEFINE-UCS :UTF-32LE 4 :LE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UTF-32LE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UTF-32BE
;     (BABEL-ENCODINGS::DEFINE-UCS :UTF-32BE 4 :BE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UTF-32BE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UCS-2
;     (BABEL-ENCODINGS::DEFINE-UCS :UCS-2 2 NIL 65536)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UCS-2 ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UCS-2LE
;     (BABEL-ENCODINGS::DEFINE-UCS :UCS-2LE 2 :LE 65536)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UCS-2LE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UCS-2BE
;     (BABEL-ENCODINGS::DEFINE-UCS :UCS-2BE 2 :BE 65536)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UCS-2BE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

debugger invoked on a ASDF:COMPILE-ERROR in thread
#<THREAD "initial thread" RUNNING {AB2C889}>:
  Error while invoking #<COMPILE-OP (:VERBOSE NIL) {B72ECF9}> on
  #<CL-SOURCE-FILE "babel" "src" "enc-unicode">

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY ] Retry compiling #<CL-SOURCE-FILE "babel" "src" "enc-unicode">.
  1: [ACCEPT] Continue, treating
              compiling #<CL-SOURCE-FILE "babel" "src" "enc-unicode"> as having
              been successful.
  2: [ABORT ] Give up on "babel"
  3:          Exit debugger, returning to top level.

((SB-PCL::FAST-METHOD ASDF:PERFORM (ASDF:COMPILE-OP ASDF:CL-SOURCE-FILE))
 #<unavailable argument>
 #<unavailable argument>
 #<ASDF:COMPILE-OP (:VERBOSE NIL) {B72ECF9}>
 #<ASDF:CL-SOURCE-FILE "babel" "src" "enc-unicode">)
0] 

Sincerely,
--

-- 
Harven
_______________________________________________
babel-devel mailing list
babel-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel
Luís Oliveira | 2 Jan 23:05 2012
Picon

Problem with gbk-map.lisp

Hello Wenpeng,

I missed an obvious problem with your patch, it depends on Lisps
loading gbk-map.lisp using the UTF-8 encoding. AFAIK, there's no good
way to portably enforce that via ASDF, so I've converted the file to
an ASCII representation. It's not as pretty as the previous version,
but it seems to work.

Cheers,

--

-- 
Luís Oliveira
http://r42.eu/~luis/
Anton Vodonosov | 28 Dec 22:21 2011
Picon

Unit tests failures on different lisps

Hello.

I am running tests of some most often Quicklisp-downloaded libraries, including babel.

Babel tests have different number of failures/errors on different Lisps (about 8, 9 or 5).

You may find the results here: http://common-lisp.net/project/cl-test-grid/pivot_ql-lib_lisp.html

Clicking the ok/fail status refer to the library test logs where you may find what failures
occurred.

Best regards,
- Anton
Xiaofeng Yang | 22 Dec 15:55 2011
Picon

Re: Submit a GBK patch



I suggest this for both GB2312 and GBK encoding (this 2 encodings are very important for Chinese users):
http://trac.clozure.com/ccl/changeset/14911
I've tested this for CCL using the encoding tables and tests from GNU's iconv.

Most babel code came from CCL. I think to integrated this for babel is easily and more suitable though it just only for CCL now.

     Best regards,
Xiaofeng Yang


2011/12/21 levin <levin108 <at> gmail.com>
HI,all

Days ago, I found this project babel, and tried to use it in my project, as I'm Chinese,
I found it doesn't support GBK encoding in babel, so I wrote a patch for babel to make
it support GBK.

I just used common lisp for not a long time, so if there was something not
good enough, please let me know, I also hope you can accept this patch or help me
modify it to make babel support GBK, so we can use it freely to process Chinese text.

Thanks.

--
levin

_______________________________________________
babel-devel mailing list
babel-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel



_______________________________________________
babel-devel mailing list
babel-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel
levin | 21 Dec 12:21 2011
Picon

Submit a GBK patch

HI,all

Days ago, I found this project babel, and tried to use it in my project, as I'm Chinese,
I found it doesn't support GBK encoding in babel, so I wrote a patch for babel to make
it support GBK.

I just used common lisp for not a long time, so if there was something not
good enough, please let me know, I also hope you can accept this patch or help me
modify it to make babel support GBK, so we can use it freely to process Chinese text.

Thanks.

--
levin


Attachment (0002-New-encoding-GBK.patch): application/octet-stream, 101 KiB
_______________________________________________
babel-devel mailing list
babel-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel
Dmitry Ignatiev | 27 Oct 15:46 2011
Picon

Few fixes for ucs-2 and utf-32

Hi there.


There were some BOM-related bugs in unicode decoders.

patch attached
Attachment (enc-unicode.diff): application/octet-stream, 1402 bytes
_______________________________________________
babel-devel mailing list
babel-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

Gmane