Karl Hegbloom | 25 May 05:26 2016

"<varspace>" in red?

In my zotero-texmacs-integration, when I set the CSL style to ISO-690, it outputs "ISBN" followed by a two-byte character, followed by the actual ISBN number. I'm taking the text it sends me, which is output via an output format specification that wraps the text as LaTeX "bbl", and transforming it into a TeXmacs tree with: (latex->texmacs (parse-latex str_text)).

In the result, I see, in red, <varspace>. When I catch the offending character in an Emacs terminal buffer, and run it through Guile's string->list, I get:  (#\302 #\240); and (map char->integer (string->list " ")) yields (194 160). As hexadecimal, those are 0xC2 0xA0. That is UTF-8 encoded NO-BREAK SPACE.

What this tells me is that TeXmacs is converting the UTF-8 Unicode character into one of it's internal things, but instead of it being rendered as an actual non-breakable space, I see the string <varspace> in red letters. I see the same thing when I paste that character into a document.

What's the matter? How can I fix this? I think that it's not seeing the encoding as being UTF-8 when it parses the LaTeX because the heuristics fail in this case, since it's not a whole document, but only a snippet of LaTeX. I want to control the encoding or run an explicit UTF-8 to Cork first.
Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 19 May 03:42 2016

Segv in smart_font_rep:

I don't know what triggers this. svn trunk as of today.

Thread 1 "texmacs.bin" received signal SIGSEGV, Segmentation fault.
0x0000000000d6fd01 in unicode_font_rep::get_xpositions (this=0x50b26b0, s=..., xpos=0x7fffffffb0d4, ligf=true)
    at ./Plugins/Freetype/unicode_font.cpp:309
309        for (int j= start; j<i; j++) xpos[j]= x;
(gdb) where full
#0  0x0000000000d6fd01 in unicode_font_rep::get_xpositions(string, int*, bool) (this=0x50b26b0, s=..., xpos=0x7fffffffb0d4, ligf=true) at ./Plugins/Freetype/unicode_font.cpp:309
        j = 4043
        start = 0
        pc = 4294967295
        next = 0x39ca5f0
        i = 21442
        n = 20
        x = 0
        uc = 60
#1  0x0000000000d6fe03 in unicode_font_rep::get_xpositions(string, int*) (this=0x50b26b0, s=..., xpos=0x7fffffffb0d4)
    at ./Plugins/Freetype/unicode_font.cpp:319
#2  0x0000000000c54a36 in smart_font_rep::get_xpositions(string, int*) (this=0x4964718, s=..., xpos=0x7fffffffb0d0)
    at ./Graphics/Fonts/smart_font.cpp:1168
        nr = 4
        r = {rep = 0x4f03af8}
        start = 1
        x = 11197
        i = 21
        n = 21
#3  0x0000000000ca0284 in text_box_rep::find_box_path(int, int, int, bool, bool&) (this=
    0x0, x=0, y=0, delta=0, force=false, found= <at> 0x0: <error reading variable>) at ./Typeset/Boxes/Basic/text_boxes.cpp:194
Python Exception <class 'gdb.error'> value requires 85772 bytes, which is more than max-value-size:
        xpos = #4  0x0000000000000000 in  ()

Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 9 May 02:33 2016

[patch] configure.in and configure Qt plugins check syntax fix

When only one slash is in the expression, it can only handle a list of 2 plugin names. Use two slashes to make it remove all of the commas so it can test each item of the entire list.
Attachment (configure-syntax-fix.diff): text/x-patch, 1033 bytes
Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 8 May 00:54 2016

Development help request: Zotero interface, citations that make footnotes, other questions...

I'm working on a connector for TeXmacs with Juris-M / Zotero that will utilize the pre-existing TCP protocol that's there for the OpenOffice integration. The wire protocol is documented here:


The project is on Github. I've ported the guile json library to guile-1.8 so it can be used from TeXmacs, and I made a start on an rtf to texmacs converter, though I think it will be better to have Zotero send output as LaTeX (bbl), or even as TeXmacs scheme. It's easy enough to create a new output format. I have forked and patched versions of zotero and better-bibtex-for-zotero in my GitHub as well. Some patches to better-bibtex-for-zotero have been accepted; the one to Juris-M / Zotero is not ready until it's tested and working. I told them about it, and created a pull request, but said I don't expect them to apply it yet, just look at it and advise me. Also in my GitHub repositories is a "git svn" mirror of TeXmacs itself.

I'm not asking for anyone else to write the program for me... But if somebody makes a fork and writes something significant that this thing needs, I won't complain, no worries... mainly I need help understanding how to do some things that I'm having difficulty figuring out for myself through code reading and experimentation.

There's a scheme-session.tm file in my project directory that shows that I can send and receive commands with Zotero. I'm having trouble figuring out how to do the protocol loop. When I initiate the sequence by sending, e.g., an "addCitation" integration command, Zotero responds with a word processor command, to which the integration code must respond after processing it. It must stay in an IO loop with Zotero until it is issued a Document_complete and sends the JSON null response to that. At that point it does not need to listen for connections from Zotero until another integration command is sent by TeXmacs to Zotero. While it is in the word processor command mode, TeXmacs is not expected to accept input from the user until Zotero is done with the sequence of operations.

Q: How do I devise the loop that runs to handle the incoming word processor commands? Do I need to use 'delayed'? Please advise.

By 'delayed' here I mean the Scheme "delayed" macro I see all over the GUI widget code, and in the tm-server and tm-client codes. How does this interact with "select" to wait for information to arrive on a socket port? I need help designing this IO loop so that it won't cause TeXmacs to hang, and so it will be robust in case of things like Firefox crashing or being closed by the user, etc.

The other problem that I need to solve is that I'll have to save data in the document. I think I can mostly figure that out by looking over the database code, and the bibliography handling, saving it in the aux data or with a set-binding... I think that set-binding is for setting the strings that are displayed by an in-text citation... but for some CSL citation styles, the citation belongs in a footnote. Switching styles from a "note" style to an "in-text" style should make the citations flip between being in footnotes (or end-notes; see below) or being displayed in-text at the point where the cite tag appears.

My initial experiments with using set-binding from inside TeXmacs, right there in the buffer, show that I can set a binding to a footnote, and then later recall it with get-binding. But when I define one, then move up above it and define another one, then call on get-binding in a different order, the footnote numbers are not updated to reflect the new document order... So I think what I need is delayed evaluation... as in quote-arg with later eval...  Or what?

Should I use the already available 'label' -> 'reference' mechanism to display the citation text? Or is there a lower-level thing that it uses which I should use more directly?  When you look at the "scheme-session.tm" there, you'll see what it sends and what it expects to be kept, in a hidden part of the citation field, for later sending back to Zotero to re-initialize the state. I think that if I just use 'label' and 'reference' that it's not good since sometimes a citation needs to make it's own footnote, and really expanding 'reference' ought to only give back some text to display in-line, not a tree that makes a footnote. A tree with formatting, maybe...

Ideas please! I need help understanding this.

Thank you.
    Karl M. Hegbloom

Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 12 Apr 16:58 2016

Fwd: Pdf-writer, tagged pdf?

[Gal, Can you join the texmacs-dev list to discuss it?]
[texmacs-dev, Please Cc Gal until and unless he joins the list.]

When I produce a PDF with TeXmacs (latest from svn) and use the "read aloud" feature of pdf-max pro on Android, it doesn't "see" the typographic ligatures, and pronounces words wrong. For example, "offense" becomes "o ense", dropping the "ff" ligature. I wrote to the author of the pdf-writer library that TeXmacs uses after reading a little about the problem using a Google search, and learning that there exists something called "tagged pdf" which I don't know much about yet; I'm too busy to read about it as much as I'd like to.

I hereby "put the phones together" on this.

---------- Forwarded message ---------
From: Gal Kahana <gal.kahana <at> hotmail.com>
Date: Tue, Apr 12, 2016 at 1:31 AM
Subject: RE: Pdf-writer, tagged pdf?
To: Karl Hegbloom <karl.hegbloom <at> gmail.com>

PDF-Writer can allow adding tagged pdf content. yes.
However, for the problem at hand i think that the toUnicode map of a font should suffice, and in fact the library creates it when generating font representations and using the regular text characters.

Its either that what you are seeing is a bug in creating a unicode map in case of typographic ligatures usage, or that for some reason it is not created at all (maybe through and alternative text setup).

We need someone from TeXmacs to help us pinpoint the issue. Do you know anyone there and can communicate this to them?


Date: Sun, 10 Apr 2016 15:54:52 -0600
Subject: Pdf-writer, tagged pdf?
From: karl.hegbloom <at> gmail.com
To: gal.kahana <at> hotmail.com

I wonder if it supports creation of "tagged pdf"? My interest is due to your library being used by the TeXmacs editor, and when I export to pdf and have the pdf reader app perform text to speech (read aloud) it skips typographic ligatures and thus mispronounces words that contain them. I think the solution involves emitting tagged pdf that has typographic ligatures transliterated to their multi character unicode equvalent for the voice-reader and highlight + clipboard, while still using the ligature for display and printing.

Karl.Hegbloom <at> gmail.com

Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 21 Mar 18:56 2016

Paragraphsign in PDF outlines

I've got a document style sheet that puts the paragraphsign  ¶  in front of the paragraph number. It shows up fine in the document, both at the start of the paragraph and in the table of contents, but in the PDF outlines, it shows up as  ű.

If I then run the command:

pdftk inputfile.pdf dump_data_utf8  | sed -r -e 's,ű,¶,g'  | pdftk inputfile.pdf update_info_utf8 - output outputfile.pdf

... the resulting PDF has the paragraphsign in the outlines the way it's supposed to. I guess if I actually used that thing that looks like a u with a double quote above it in the text of the document in any section or paragraph headings, there would be paragraph signs instead of that letter there, which would not be correct.

Because PDF accepts that character fine when I run that sed filter, I suspect that the problem is somewhere in TeXmacs, in how it translates the text from it's internal coding to the one in the PDF.

It seems to me that it used to work fine. It might be something that broke with one of the recent changes to the PDF writer, but I'm not certain. I've not used that paragraphsign prefixed paragraph numbering much yet.

Any idea how to fix it?
Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 14 Mar 20:59 2016

Example "secure" not functioning in TeXmacs 1.99.4 svn 9936

When I compile the "secure" plugin example, the \inv macro works fine, but the \latexer one causes texmacs.bin to hit 100% CPU and do nothing else. I do not know how to fix it or what is the matter and I need some help.

When it does this, secure.bin has been launched as a subprocess below texmacs.bin, but it is not using any CPU.

Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Karl Hegbloom | 11 Mar 01:25 2016

Cacheing of http concretised documents

... really ought to watch the http headers to know when to refresh. I am using tree-import to call the BetterBibTeX for Zotero "cayw" (cite as you write) GET interface,  which operates only to localhost, and causes Zotero to open a dialog box for searching to find references, which are then returned. So the URL doesn't change from call to call,  but the data returned does. The workaround was to add an extra GET argument to the URL with a timestamp. Luckily the service it's calling doesn't care about extra args.

I realize that when TeXmacs has guile 2, it's web modules can replace the use of wget.

Q: is TeXmacs threadsafe? Can multiple guile threads be utilized in the future? 
Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Stephen Crowley | 7 Mar 20:35 2016

QT gui tool bar positions not persisting

Just latest svn revision 9974 from svn://svn.savannah.gnu.org/texmacs 
and compiled against qt4. WHen I move the toolbar positions, so all 3 
are stacked horizontally on one line, then exit texmacs and start it 
again, the toolbars go back to being distributed 3 stacked vertically on 
the left side...

How do I make the changes persistent? If it can be done with the current 
system, where would I look for the code that handles that?


p.s. I apologize in advice for this draconian nonsense in the footer 
attached to this message

This message is intended only for the use of the individual or entity to which it is addressed and may contain
information which is privileged, confidential or subject to copyright. Internet communications
cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost,
arrive late or contain viruses. The information contained in this e-mail is drawn from sources believed
to be reliable, but the accuracy and completeness of the information is not guaranteed, nor in providing
it does Canaccord Genuity Corp. or its subsidiaries, or affiliated companies, "The Firm" assume any

Canaccord Genuity Corp., its subsidiaries or affiliated companies, disclaims all responsibility and
accepts no liability (including negligence) for the consequences for any person acting, or refraining
from acting, on such information. Unless otherwise stated, this transmission is neither an offer nor the
solicitation of an offer to sell or purchase any investment. As a solicitation, this e-mail was intended
for distribution in those jurisdictions where The Firm is registered as advisors or dealers in
securities. Any unauthorized use, disclosure, distribution or copying of this communication by anyone
other than the intended recipient is strictly prohibited. 

When addressed to our clients, any opinions or advice contained in this email are subject to the terms and
conditions expressed in the governing Canaccord Genuity Corp., its subsidiaries or affiliated
companies' client agreements.  If you have received this message in error, please notify us immediately
by return email and destroy and delete the message from your computer.  (Disclaimer)
Karl Hegbloom | 1 Mar 19:22 2016

Guile 2.2, "lazy catch"

I spent some hobby-time yesterday looking into what it will take to port TeXmacs to Guile 2.2, since it has a fast virtual machine and I think it will make TeXmacs much more useable. One of the first differences that I find is that the "lazy catch" thing is no longer present.

I realize that the only real way to do this is going to involve spending more hobby-time on learning the internals of guile-1.8 vs guile-2.2 wrt libguile/throw.[hc] and then the internals of TeXmacs itself, to see what's the right thing to do. But if anyone has a quick answer, or any insight that might help me with that study, please, post it here.
Texmacs-dev mailing list
Texmacs-dev <at> gnu.org
Marduk Bolaños | 17 Feb 00:10 2016

Style files and cache

Dear all,

I just learned that changes made to a style file with a text
editor will not be recognized by TeXmacs. It is necessary to
rebuild the cache to make TeXmacs aware of the changes.

When editing the file with TeXmacs, it takes care of updating
the cache.

It would be of great help to include this information in the

Best regards,