UTF8 and connection encoding

Hi All,

if someone changes connection setting in main.php from UTF-8 to 
something else, all content from DB will be delivered with this encoding 
to users Browser, but we say the document is UTF-8 and strings coming 
from us (language files) are in UTF-8

so we end up with a mixed encoding in the HTML output

there are two options:

- setting PHP output encoding

http://php.net/manual/en/mbstring.http.php

do not know if this takes care of already mixed encodings

- do not allow switching connection charset

why is this required anyway?

i would vote for dropping, as is see no requirement for this, any modern 
browser should be UTF-8 capable

any comments suggestions?

--

-- 
Sebastian Mendel

------------------------------------------------------------------------------

UTF8 and connection encoding

Hi All,

if someone changes connection setting in main.php from UTF-8 to 
something else, all content from DB will be delivered with this encoding 
to users Browser, but we say the document is UTF-8 and strings coming 
from us (language files) are in UTF-8

so we end up with a mixed encoding in the HTML output

there are two options:

- setting PHP output encoding

http://php.net/manual/en/mbstring.http.php

do not know if this takes care of already mixed encodings

- do not allow switching connection charset

why is this required anyway?

i would vote for dropping, as is see no requirement for this, any modern 
browser should be UTF-8 capable

any comments suggestions?

--

-- 
Sebastian Mendel

------------------------------------------------------------------------------

Re: UTF8 and connection encoding

On 08.02.2009 16:02, Sebastian Mendel wrote:
> Hi All,
>
> if someone changes connection setting in main.php from UTF-8 to
> something else, all content from DB will be delivered with this encoding
> to users Browser, but we say the document is UTF-8 and strings coming
> from us (language files) are in UTF-8
>
> so we end up with a mixed encoding in the HTML output
>
>
> there are two options:
>
> - setting PHP output encoding
>
> http://php.net/manual/en/mbstring.http.php
>
> do not know if this takes care of already mixed encodings
>
>
> - do not allow switching connection charset
>
> why is this required anyway?
>
>
> i would vote for dropping, as is see no requirement for this, any modern
> browser should be UTF-8 capable
>
>
> any comments suggestions?

Re: UTF8 and connection encoding

Hi Sebastian and list,

Sebastian Mendel schrieb:
> 
> if someone changes connection setting in main.php from UTF-8 to 
> something else, all content from DB will be delivered with this encoding 
> to users Browser, but we say the document is UTF-8 and strings coming 
> from us (language files) are in UTF-8

Who changed this anyway? In my original MySQL 4.1 compatibility code,
the option to change the connection charset was intentionally not given
to the user - only for dump files, afaik.

> so we end up with a mixed encoding in the HTML output

Right, we would need to either recode the language files or on the fly
any string that comes from MySQL. But I guess, we would destroy the
"intention" of that feature - see my explanation below.

> there are two options:
> 
> - setting PHP output encoding
> 
> http://php.net/manual/en/mbstring.http.php
> 
> do not know if this takes care of already mixed encodings

It doesn't, afaik.

> - do not allow switching connection charset

Re: UTF8 and connection encoding

On 09.02.2009 10:47, cand. inf. Alexander M. Turek wrote:
> Hi Sebastian and list,
>
> Sebastian Mendel schrieb:
>> if someone changes connection setting in main.php from UTF-8 to
>> something else, all content from DB will be delivered with this encoding
>> to users Browser, but we say the document is UTF-8 and strings coming
>> from us (language files) are in UTF-8
>
> Who changed this anyway? In my original MySQL 4.1 compatibility code,
> the option to change the connection charset was intentionally not given
> to the user - only for dump files, afaik.
>
>> so we end up with a mixed encoding in the HTML output
>
> Right, we would need to either recode the language files or on the fly
> any string that comes from MySQL. But I guess, we would destroy the
> "intention" of that feature - see my explanation below.
>
>> there are two options:
>>
>> - setting PHP output encoding
>>
>> http://php.net/manual/en/mbstring.http.php
>>
>> do not know if this takes care of already mixed encodings
>
> It doesn't, afaik.
>
>> - do not allow switching connection charset

Re: UTF8 and connection encoding

cand. inf. Alexander M. Turek a écrit :
> Hi Sebastian and list,
> 
> Sebastian Mendel schrieb:
>> if someone changes connection setting in main.php from UTF-8 to 
>> something else, all content from DB will be delivered with this encoding 
>> to users Browser, but we say the document is UTF-8 and strings coming 
>> from us (language files) are in UTF-8
> 
> Who changed this anyway? In my original MySQL 4.1 compatibility code,
> the option to change the connection charset was intentionally not given
> to the user - only for dump files, afaik.

Alexander,
you did this:
http://phpmyadmin.svn.sourceforge.net/viewvc/phpmyadmin/tags/RELEASE_2_6_0PL3/phpMyAdmin/main.php?r1=6062&r2=6068

for this bug:
https://sourceforge.net/tracker2/index.php?func=detail&aid=984912&group_id=23067&atid=377408

where you said:

"My suggestion is to let the user choose his connection
character set on main.php. In addition, we should imho force
the users to convert their pma DBs to UTF-8. As these tables
contain metadata and should be able to store strings in any
language, this would be the best solution anyway."

So it was useful back then, but nowadays we could remove this feature or 
at least make it configurable and not display it by default.

Re: UTF8 and connection encoding

Sebastian Mendel a écrit :
> On 09.02.2009 10:47, cand. inf. Alexander M. Turek wrote:
>> Hi Sebastian and list,
>>
>> If you decide to keep that "feature", it is imho to prominent where it
>> is right now and you should add warnings with tons of exclamation marks
>> against using it.
> 
> in short: drop this in phpMyAdmin 3?
> 
Let's target 3.2 and either:

- add a configuration parameter that controls this feature and disables 
it by default, or

- add a message on the main page when the connection collation has been 
changed, to explain the possible consequences

Marc

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: UTF8 and connection encoding

Hi Marc & list,

Marc Delisle schrieb:
> cand. inf. Alexander M. Turek a écrit :
>>
>> Who changed this anyway? In my original MySQL 4.1 compatibility code,
>> the option to change the connection charset was intentionally not given
>> to the user - only for dump files, afaik.
> 
> Alexander,
> you did this:

Just read the docs to refresh my memory. Changing the connection charset
is fine. It is used for comparisions and should not affect the results.
In the current PMA, the client charset is changed instead (and this is
the evil thing I was talking about). I'm having a look at this, hang on.

Regards,

Alexander

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: UTF8 and connection encoding

Hi again,

cand. inf. Alexander M. Turek schrieb:
> 
> Just read the docs to refresh my memory. Changing the connection charset
> is fine. It is used for comparisions and should not affect the results.
> In the current PMA, the client charset is changed instead (and this is
> the evil thing I was talking about). I'm having a look at this, hang on.

The evil change has happened here:

http://phpmyadmin.svn.sourceforge.net/viewvc/phpmyadmin/trunk/phpMyAdmin/libraries/database_interface.lib.php?r1=10984&r2=10990

Shame on you, cybot_tm! 

The bug should be fixed in SVN, please confirm.

Regards,

Alexander

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: UTF8 and connection encoding

On 09.02.2009 16:27, cand. inf. Alexander M. Turek wrote:
> Hi again,
>
> cand. inf. Alexander M. Turek schrieb:
>> Just read the docs to refresh my memory. Changing the connection charset
>> is fine. It is used for comparisions and should not affect the results.
>> In the current PMA, the client charset is changed instead (and this is
>> the evil thing I was talking about). I'm having a look at this, hang on.
>
> The evil change has happened here:
>
> http://phpmyadmin.svn.sourceforge.net/viewvc/phpmyadmin/trunk/phpMyAdmin/libraries/database_interface.lib.php?r1=10984&r2=10990
>
> Shame on you, cybot_tm! 

uh, someone must have hacked into my account ...

--

-- 
Sebastian Mendel

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com