headers
Niklas Lampén | 2 Jun 2010 10:15
Picon

Cannot send UTF-8 chars to MSSQL 2008

Hi,

I'm trying to send a string "Найти" (which is Russian, I don't know what it means) from PHP to MSSQL
server using FreeTDS, but I end up with empty column in MSSQL. Iäm using a stored procedure called
Mod_Text_Put_Content with three parameters  <at> Content (containing the string),  <at> ContentId and  <at> SiteId.

The string is originally in UTF-8. ISO-8859-1 compatible strings like "abc" are stored to database as expected.

I'm trying this with freetds-0.83.dev.20100522. Freetds-0.82 gives an error and does not send empty
string to database.

I tested that my iconv works as expected by creating a file with string "Найти" in it and did:

# iconv -futf-8 -tucs2 --verbose -oucs2.txt utf8.txt
# iconv -fucs2 -tutf8 --verbose -outf8.txt_2 ucs2.txt
# hexdump utf8.txt
0000000 9dd0 b0d0 b9d0 82d1 b8d0 000a
000000b
# hexdump utf8.txt_2
0000000 9dd0 b0d0 b9d0 82d1 b8d0 000a
000000b
# hexdump ucs2.txt
0000000 041d 0430 0439 0442 0438 000a
000000c

My freetds.conf looks like this:
---
[global]
        text size = 64512
        port = 1433
(Continue reading)

Permalink | Reply | 13 comments |
headers
sekoene | 2 Jun 2010 10:40
Picon

Wrong result character conversion with sqsh

Hi,

I have problems reading output from queries with sqsh.

I already posted the problem on souceforge but it looks like a like a 
library problem ?

In short:

I use sqsh for our sybase ASE severs. From now on we also use MS SQL 
server 2005. So I build freetds and it works on both MS SQL Server 2005 
and Sybase ASE.

It should be possible to build sqsh with freetds libraries and use it 
for both servers. And it does ! and I am able to logon on both servers 
:)  but ... I get unreadable output :(

http://sourceforge.net/projects/sqsh/forums/forum/365923/topic/3688396

Maybe you can help ?

Thank you in advance, sek44
Permalink | Reply | 0 comments |
headers
Daniel Fazekas | 2 Jun 2010 10:47
Picon
Favicon

Re: Cannot send UTF-8 chars to MSSQL 2008

On Jun 2, 2010, at 10:15, Niklas Lampén wrote:

> I'm trying to send a string "Найти" (which is Russian, I don't know what it means)

Google translate says it means Search. :)

> The string is originally in UTF-8.

Then leave it that way and tell FreeTDS that you use UTF-8:

> [AavaDb2]
>        host = *****
>        client charset = UCS2

Simply change the client charset line to read UTF-8.

Wide-char encodings like UCS2/UTF-16/UTF-32 are not supported for the client side.
_______________________________________________
FreeTDS mailing list
FreeTDS <at> lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Permalink | Reply | 12 comments |
headers
Niklas Lampén | 2 Jun 2010 11:29
Picon

Re: Cannot send UTF-8 chars to MSSQL 2008

> -----Original Message-----
> From: freetds-bounces <at> lists.ibiblio.org [mailto:freetds-
> bounces <at> lists.ibiblio.org] On Behalf Of Daniel Fazekas
> Sent: 2. kesäkuuta 2010 11:47
>
> > [AavaDb2]
> >        host = *****
> >        client charset = UCS2
> 
> Simply change the client charset line to read UTF-8.
>

Thank you for your reply.

Unfortunately this did not solve my problem, thought it made a small change:

rpc.c:171:dbrpcparam(0x8626028,  <at> Content, 0x0, 35, -1, 11, 0x8b45790)

changed to 

rpc.c:171:dbrpcparam(0x845ab40,  <at> Content, 0x0, 35, -1, 10, 0x8acda44)

but the error is still the same.

I've tried to change settings in php.ini (mssql.charset), too, but that doesn't seem to have any effect
unless charset is set to "UCS2", which makes queries to fail. I guess that is something to be expected.


Niklas
_______________________________________________
(Continue reading)

Permalink | Reply | 11 comments |
headers
Sylvain Viart - Gmail | 2 Jun 2010 12:06
Picon

Re: mssql_execute(): stored procedure execution failed

Hi Robert,

On 28/05/2010 17:44, Robert Gonzalez wrote:
> The fix is to free your result resources before sending another init request
> to the server. I think you are running into something that has been causing
> issues for a while, most recently in PDO as an unbuffered query error. What
> is happening is the server is sending back a result but the response is
> being held open on it until the next direct query is sent. But since you are
> not sending a query and rather sending an init call the server is choking on
> the request.
>
> After every one of your queries and subsequent result reads, try adding in a
> call to mssql_free_result($yourResultVar). That should make your problem go
> away. At least it did for me in my testing just now
Yes, may be.
I'm gonna try that.

One try we made, which totaly crash the lib (may be somewhere in apache 
+ persistent connection  + freetds)

Was to replace the most failing mssql_init() call with a function wrapper :

here's the wrapper code

//  $rt = call_proc("linsyn_mail_exclusion_get",
//      " <at> id", $in,
//      " <at> type", $tag_mail
//  );
function call_proc($procname)
{
(Continue reading)

Permalink | Reply | 2 comments |
headers
Craig A. Berry | 2 Jun 2010 14:33
Picon

Re: Cannot send UTF-8 chars to MSSQL 2008


On Jun 2, 2010, at 4:29 AM, Niklas Lampén wrote:

>> -----Original Message-----
>> From: freetds-bounces <at> lists.ibiblio.org [mailto:freetds-
>> bounces <at> lists.ibiblio.org] On Behalf Of Daniel Fazekas
>> Sent: 2. kesäkuuta 2010 11:47
>>
>>> [AavaDb2]
>>>       host = *****
>>>       client charset = UCS2
>>
>> Simply change the client charset line to read UTF-8.
>>
>
> Thank you for your reply.
>
> Unfortunately this did not solve my problem

You might want to set TDSDUMPCONFIG and TDSDUMP in your environment  
and review the resulting logs to see what client character set you're  
actually getting.  You might also try applying the patch from:

<http://lists.ibiblio.org/pipermail/freetds/2010q2/026008.html>

which I think should do a better job than the existing code at  
allowing freetds.conf to override the character set, though no  
guarantees it will work in your situation (however, it would be nice  
to have the confirmation if it does).
(Continue reading)

Permalink | Reply | 10 comments |
headers
Niklas Lampén | 2 Jun 2010 14:38
Picon

Re: Cannot send UTF-8 chars to MSSQL 2008

> You might want to set TDSDUMPCONFIG and TDSDUMP in your environment
> and review the resulting logs to see what client character set you're
> actually getting.  You might also try applying the patch from:
> 
> <http://lists.ibiblio.org/pipermail/freetds/2010q2/026008.html>
> 
> which I think should do a better job than the existing code at
> allowing freetds.conf to override the character set, though no
> guarantees it will work in your situation (however, it would be nice
> to have the confirmation if it does).

I'll do that and post you the results. Here's some more info about this issue without the patch. I tried to
insert UTF-8 chars from shell using tsql, but it fails too. I do not get any errors, but result is not expected:

# tsql -S AavaDb2 -U ***** -P ***** -D *****
locale is "en_US.UTF-8"
locale charset is "UTF-8"
Default database being set to *****
1> UPDATE Mod_Texts SET ContentData='Найти' WHERE ContentId=6824
2> go
1> SELECT ContentData FROM Mod_Texts WHERE ContentId=6824
2> go
ContentData
?????
(1 row affected)

The string is inserted as question marks to database. If I set text like 'foo Найти', it will be 'foo ?????'.

By analyzing the log it seems that FreeTDS thinks it should send data as CP1252. Or is it just for single byte
strings and UCS-2 is used otherwise?
(Continue reading)

Permalink | Reply | 0 comments |
headers
Niklas Lampén | 2 Jun 2010 15:16
Picon

Re: Cannot send UTF-8 chars to MSSQL 2008

> You might want to set TDSDUMPCONFIG and TDSDUMP in your environment
> and review the resulting logs to see what client character set you're
> actually getting.

I applied the patch, build freetds again and tried and...not luck.

Commands run:

# tsql -S AavaDb2 -U ***** -P ***** -D *****
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
Default database being set to *****
1> UPDATE Mod_Texts SET ContentData='foo Найти' WHERE ContentId=6824;
2> SELECT ContentData FROM Mod_Texts WHERE ContentId=6824;
3> GO
ContentData
foo ?????
(1 row affected)
1> exit

Here's what TDSDUMPCONFIG says:
-----
log.c:196:Starting log file for FreeTDS 0.83.dev.20100522
        on 2010-06-02 16:06:59 with debug flags 0x4fff.
config.c:185:Getting connection information for [AavaDb2].
config.c:189:Attempting to read conf files.
config.c:346:... $FREETDSCONF not set.  Trying $FREETDS/etc.
config.c:359:... $FREETDS not set.  Trying $HOME.
config.c:285:Could not open '/root/.freetds.conf' ((.freetds.conf)).
(Continue reading)

Permalink | Reply | 8 comments |
headers
Daniel Fazekas | 2 Jun 2010 16:32
Picon
Favicon

Re: Cannot send UTF-8 chars to MSSQL 2008

On Jun 2, 2010, at 15:16, Niklas Lampén wrote:

> 1> UPDATE Mod_Texts SET ContentData='foo Найти' WHERE ContentId=6824;
> 2> SELECT ContentData FROM Mod_Texts WHERE ContentId=6824;
> 3> GO
> ContentData
> foo ?????

Is that field even capable of storing Cyrillic letters?
It's possible it's not an NTEXT field and the server isn't set to use a Cyrillic single-byte character set.

_______________________________________________
FreeTDS mailing list
FreeTDS <at> lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
Permalink | Reply | 7 comments |
headers
Robert Gonzalez | 2 Jun 2010 19:12
Favicon

Re: Cannot send UTF-8 chars to MSSQL 2008

Are you sure PHP is handling the charset correctly? Just a thought, if PHP
is doing something to it before it ever gets handed off to the database then
it would make sense that the DB is not getting it properly.

On Wed, Jun 2, 2010 at 7:32 AM, Daniel Fazekas <fdsubs <at> t-online.hu> wrote:

> On Jun 2, 2010, at 15:16, Niklas Lampén wrote:
>
> > 1> UPDATE Mod_Texts SET ContentData='foo Найти' WHERE ContentId=6824;
> > 2> SELECT ContentData FROM Mod_Texts WHERE ContentId=6824;
> > 3> GO
> > ContentData
> > foo ?????
>
> Is that field even capable of storing Cyrillic letters?
> It's possible it's not an NTEXT field and the server isn't set to use a
> Cyrillic single-byte character set.
>
> _______________________________________________
> FreeTDS mailing list
> FreeTDS <at> lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds
>

--

-- 
Robert Gonzalez
http://www.robert-gonzalez.com
http://twitter.com/RobertGonzalez
http://www.linkedin.com/in/robertanthonygonzalez
_______________________________________________
(Continue reading)

Permalink | Reply | 1 comment |

Gmane