headers
Erik Osheim | 22 Apr 2008 17:08
Favicon

utf8 and curses/urwid

Hey Ian,

So, I have been developing an emacs-like, curses-based editor in python
(Pmacs[1]) for a couple of years now (which I do all my work in at this
point). After much reflection, I realized that I should actually try to
support Unicode so that non-English speakers would have the option to
use it. This may have been a mistake.

I've seen some emails you sent around 2006 detailing problems you found
with Python's curses bindings. I'm running into some of those same
problems now. However, based on the version of Urwid I have installed
(0.9.7.1) it seems like you have it figured out... my test[2] shows
both the "print" and the "urwid" case working, whereas the "curses"
case falls down.

I read Urwid's curses_display module, but I didn't see anything
substantially different in your use of curses from mine. Do you have
any advice (other than giving up on curses and using Uriwd[3])? I
noticed you still have disclaimers about UTF-8 on your main page--what
is the state of the curses module in your opinion?

Thanks,

-- Erik

[1] http://www.bearhome.net/pmacs

[2] Attached script, run under python 2.4.4 and 2.6a2 on xterm (with
UTF8 enabled). curses module linked to ncursesw (for what it's worth).
LANG=en_US.utf8, TERM=xterm.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Erik Osheim | 22 Apr 2008 22:04
Favicon

Re: utf8 and curses/urwid

This email was originally to Ian, but he requested I resend it to the
list. I'm just replying because when I bounced the email via mutt the
"To" and "Reply-To" fields were still to Ian.

-- Erik

On Tue, Apr 22, 2008 at 11:08:58AM -0400, Erik Osheim wrote:
> Hey Ian,
> 
> So, I have been developing an emacs-like, curses-based editor in python
> (Pmacs[1]) for a couple of years now (which I do all my work in at this
> point). After much reflection, I realized that I should actually try to
> support Unicode so that non-English speakers would have the option to
> use it. This may have been a mistake.
> 
> I've seen some emails you sent around 2006 detailing problems you found
> with Python's curses bindings. I'm running into some of those same
> problems now. However, based on the version of Urwid I have installed
> (0.9.7.1) it seems like you have it figured out... my test[2] shows
> both the "print" and the "urwid" case working, whereas the "curses"
> case falls down.
> 
> I read Urwid's curses_display module, but I didn't see anything
> substantially different in your use of curses from mine. Do you have
> any advice (other than giving up on curses and using Uriwd[3])? I
> noticed you still have disclaimers about UTF-8 on your main page--what
> is the state of the curses module in your opinion?
> 
> Thanks,
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Ian Ward | 23 Apr 2008 02:49
Favicon
Gravatar

Re: utf8 and curses/urwid

On Tue, 2008-04-22 at 16:04 -0400, Erik Osheim wrote:
> > I've seen some emails you sent around 2006 detailing problems you found
> > with Python's curses bindings. I'm running into some of those same
> > problems now. However, based on the version of Urwid I have installed
> > (0.9.7.1) it seems like you have it figured out... my test[2] shows
> > both the "print" and the "urwid" case working, whereas the "curses"
> > case falls down.
> > 
> > I read Urwid's curses_display module, but I didn't see anything
> > substantially different in your use of curses from mine. Do you have
> > any advice (other than giving up on curses and using Uriwd[3])? I
> > noticed you still have disclaimers about UTF-8 on your main page--what
> > is the state of the curses module in your opinion?

Hi Erik,

I don't see any difference between your use of curses and Urwid's
either.  You might try commenting out parts of what Urwid does when
initializing curses until it stops working too :-)

The problem I've run into is that you can't rely on your users' version
of Python having been linked against ncursesw (the one that works with
UTF8 and other interesting encodings.)  This was one of the reasons I
started work on the raw_display module.

raw_display just assumes a basic subset of terminal escape sequences and
tosses them at stdout.  It's hackish, but it avoids problems like the
one you're running into as well as TERM misconfiguration or missing
termcap entries.
(Continue reading)

Permalink | Reply | 0 comments |
headers
George Goh | 23 Apr 2008 03:58

Re: utf8 and curses/urwid

Hi Erik,

I'm interested in pmacs, but got a 404 on [1].

George

On Tue, 2008-04-22 at 11:08 -0400, Erik Osheim wrote:
> Hey Ian,
> 
> So, I have been developing an emacs-like, curses-based editor in python
> (Pmacs[1]) for a couple of years now (which I do all my work in at this
> point). After much reflection, I realized that I should actually try to
> support Unicode so that non-English speakers would have the option to
> use it. This may have been a mistake.
> 
> I've seen some emails you sent around 2006 detailing problems you found
> with Python's curses bindings. I'm running into some of those same
> problems now. However, based on the version of Urwid I have installed
> (0.9.7.1) it seems like you have it figured out... my test[2] shows
> both the "print" and the "urwid" case working, whereas the "curses"
> case falls down.
> 
> I read Urwid's curses_display module, but I didn't see anything
> substantially different in your use of curses from mine. Do you have
> any advice (other than giving up on curses and using Uriwd[3])? I
> noticed you still have disclaimers about UTF-8 on your main page--what
> is the state of the curses module in your opinion?
> 
> Thanks,
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Karsten Schulz | 23 Apr 2008 06:43
Picon
Picon

Re: utf8 and curses/urwid

Hi George,

> I'm interested in pmacs, but got a 404 on [1].

try <http://www.bearhome.net/erik/pmacs/>

Karsten
Permalink | Reply | 0 comments |
headers
Max E. Kuznecov | 23 Apr 2008 22:37
Picon
Gravatar

Escape sequences

Hi,
I'm using urwid in my project where I want to create a flexible way of
binding key-sequences (like Ctrll-X, Meta-G, F1 etc) to some actions.
On my FreeBSD 7 box in xterm
test program with get_input(True) pressing F1 returns (['f1'], [27,
79, 80]), i. e. correctly determines F1 pressed.
Things getting a bit worse when I run the same test program in pure
console, where terminal type is cons25r, in this case I get (['meta
[', 'M'], [27, 91, 77]).
xterm termcap entry says:
:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:\ ...
which apparently matches escape.py's:
('OP','f1'),('OQ','f2'),('OR','f3'),('OS','f4'),

But sure there is no matching entry for cons25r.
So the question is: does urwid recognize only those hardcoded in
escape.py  sequences?
And what is the best way to handle another terminal types?
Thanks.

--

-- 
~syhpoon
Permalink | Reply | 8 comments |
headers
Ian Ward | 23 Apr 2008 22:57
Favicon
Gravatar

Re: Escape sequences

Max E. Kuznecov wrote:
> xterm termcap entry says:
> :k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:\ ...
> which apparently matches escape.py's:
> ('OP','f1'),('OQ','f2'),('OR','f3'),('OS','f4'),
> 
> But sure there is no matching entry for cons25r.
> So the question is: does urwid recognize only those hardcoded in
> escape.py  sequences?

That is correct.

> And what is the best way to handle another terminal types?

I'm happy to accept a patch that adds the FreeBSD console escape 
sequences to escape.py.  A better solution would be to query the termcap 
info and build the sequences from that.

I don't use the curses library escape sequence detection because it 
misses many keys (likely just incomplete termcap entries) and it pauses 
for 1 second if the user just presses ESC.

Ian
Permalink | Reply | 5 comments |
headers
Max E. Kuznecov | 23 Apr 2008 23:28
Picon
Gravatar

Re: Escape sequences

2008/4/23, Ian Ward <ian <at> excess.org>:
>  > And what is the best way to handle another terminal types?
>
>
> I'm happy to accept a patch that adds the FreeBSD console escape
>  sequences to escape.py.  A better solution would be to query the termcap
>  info and build the sequences from that.

Ok, I'll prepare a patch as for now. Maybe later I will come up with
something more flexible.

--

-- 
~syhpoon
Permalink | Reply | 4 comments |
headers
Max E. Kuznecov | 24 Apr 2008 14:04
Picon
Gravatar

Re: Escape sequences

2008/4/24, Max E. Kuznecov <mek <at> mek.uz.ua>:
>
>
> Ok, I'll prepare a patch as for now. Maybe later I will come up with
>  something more flexible.
>

It seems some of the cons25 terminal escape codes conflicting with
already defined in escape.py:
:k1=\E[M for F1 and this already defined as ('[M', 'mouse')
as well as kN=\E[G for page_down already exists in escape.py ('[G','5')
And a a result urwid raises:
AssertionError: trie conflict detected

Is there some way to solve this issue?

--

-- 
~syhpoon
Permalink | Reply | 3 comments |
headers
Ian Ward | 24 Apr 2008 18:05
Favicon
Gravatar

Re: Escape sequences

Max E. Kuznecov wrote:
> 2008/4/24, Max E. Kuznecov <mek <at> mek.uz.ua>:
>>
>> Ok, I'll prepare a patch as for now. Maybe later I will come up with
>>  something more flexible.
>>
> 
> It seems some of the cons25 terminal escape codes conflicting with
> already defined in escape.py:
> :k1=\E[M for F1 and this already defined as ('[M', 'mouse')
> as well as kN=\E[G for page_down already exists in escape.py ('[G','5')
> And a a result urwid raises:
> AssertionError: trie conflict detected
> 
> Is there some way to solve this issue?

I guess we have to do it properly then -- try the values from termcap 
first, then fall back to the default interpretation.

Ian
Permalink | Reply | 2 comments |

Gmane