Encoding in ruby/tk internals

[Nikolay Ponomarenko]

Hello ruby-talk peoples,

It seems that something is wrong in ruby/tk libraries:

I'm trying to work with cp1251 encoding. Everything works fine, until
i try to use "percent substitutions" - params values are passed in
unicode, ignoring Tk.encoding value.

In pure Tcl all works normal -

encoding system "cp1251"
-command[list tblCmd $table(array) %i %C %s]

%s value is passing to tblCmd in cp1251
but when i try to do something like that in ruby

Tk.encoding = 'CP1251'
@entry = TkEntry.new
@entry.configure('validatecommand',[proc {|s| v(s)},'%s'])

%s is passing to v in utf-8 :(

Such behavior is bug, feature or i'm doing something wrong?
It there any way to retrive "percent substitutions" values in correct
encoding?

--
Best regards,
Nikolay

[Hidetoshi NAGAI]

Hi,

From: Nikolay Ponomarenko <ts@pstu.edu>
Subject: Encoding in ruby/tk internals
Date: Wed, 19 Nov 2003 17:39:02 +0900
Message-ID: <807410826.20031119104245@pstu.edu>

> In pure Tcl all works normal -
>
> encoding system "cp1251"
> -command[list tblCmd $table(array) %i %C %s]
>
> %s value is passing to tblCmd in cp1251
> but when i try to do something like that in ruby
>
> Tk.encoding = 'CP1251'
> @entry = TkEntry.new
> @entry.configure('validatecommand',[proc {|s| v(s)},'%s'])
>
> %s is passing to v in utf-8 :(

After calling Tk.encoding_system = 'cp1251',
do you get the same result?
--
Hidetoshi NAGAI (nagai@ai.kyutech.ac.jp)

[Nikolay Ponomarenko]

Hello Hidetoshi,

Wednesday, November 19, 2003, 11:23:19 AM, you wrote:

>> In pure Tcl all works normal -
>> encoding system "cp1251"
>> -command[list tblCmd $table(array) %i %C %s]
>>
>> %s value is passing to tblCmd in cp1251
>> but when i try to do something like that in ruby
>>
>> Tk.encoding = 'CP1251'
>> @entry = TkEntry.new
>> @entry.configure('validatecommand',[proc {|s| v(s)},'%s'])
>>
>> %s is passing to v in utf-8 :(

HN> After calling Tk.encoding_system = 'cp1251',
HN> do you get the same result?

Yes.
something like

Tk.encoding_system = 'cp1251' # or Tk.encoding = 'cp1251'
$btn = TkButton.new($root) {
text "and here text in cp1251"
}
works right

Problem is only with "%" values :(




--
Best regards,
Nikolay

[Hidetoshi NAGAI]

Hi,

From: Nikolay Ponomarenko <ts@pstu.edu>
Subject: Re: Encoding in ruby/tk internals
Date: Thu, 20 Nov 2003 15:58:37 +0900
Message-ID: <80918300.20031120090222@pstu.edu>

> HN> After calling Tk.encoding_system = 'cp1251',
> HN> do you get the same result?
>
> Yes.
> something like
>
> Tk.encoding_system = 'cp1251' # or Tk.encoding = 'cp1251'
> $btn = TkButton.new($root) {
> text "and here text in cp1251"
> }
> works right
>
> Problem is only with "%" values :(

Well, could you try the following patch?

Index: tkentry.rb
================================================== =================
RCS file: /src/ruby/ext/tk/lib/tkentry.rb,v
retrieving revision 1.19
diff -u -r1.19 tkentry.rb
--- tkentry.rb 3 Aug 2003 22:07:46 -0000 1.19
+++ tkentry.rb 20 Nov 2003 08:41:54 -0000
@@ -27,9 +27,10 @@

class ValidateArgs
VARG_KEY = 'disvPSVW'
- VARG_TYPE = 'nxsssssw'
+ VARG_TYPE = 'nxeseesw'

def self.scan_args(arg_str, arg_val)
+ enc = Tk.encoding
arg_cnv = []
arg_str.strip.split(/\s+/).each_with_index{|kwd,idx|
if kwd =~ /^%(.)$/
@@ -39,6 +40,12 @@
arg_cnv << TkComm::number(arg_val[idx])
when ?s
arg_cnv << TkComm::string(arg_val[idx])
+ when ?e
+ if enc
+ arg_cnv << Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
+ else
+ arg_cnv << TkComm::string(arg_val[idx])
+ end
when ?w
arg_cnv << TkComm::window(arg_val[idx])
when ?x

--
Hidetoshi NAGAI (nagai@ai.kyutech.ac.jp)

[Nikolay Ponomarenko]

Hello Hidetoshi,

Thursday, November 20, 2003, 10:44:10 AM, you wrote:

HN> Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)

Seems that something wrong here.

I try to replace it with this for testing and debug purpose

puts Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
puts Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
arg_cnv << Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))

works right

--
Best regards,
Nikolay

[Hidetoshi NAGAI]

Hi,

From: Nikolay Ponomarenko <ts@pstu.edu>
Subject: Re: Encoding in ruby/tk internals
Date: Thu, 20 Nov 2003 18:25:58 +0900
Message-ID: <1649750029.20031120112934@pstu.edu>

> HN> Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
> Seems that something wrong here.
>
> I try to replace it with this for testing and debug purpose
>
> puts Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
> puts Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
> arg_cnv << Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
>
> works right

Hmmm...
Could you try to replace
'Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)'
to 'Tk.fromUTF8(TkComm::string(arg_val[idx]), enc)'?

# Is the value of enc is 'cp1251'?
--
Hidetoshi NAGAI (nagai@ai.kyutech.ac.jp)

[Nikolay Ponomarenko]

Hello, Hidetoshi!
You wrote on Fri, 21 Nov 2003 02:04:10 +0900:

??>> I try to replace it with this for testing and debug purpose
??>>
??>> puts Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
??>> puts Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
??>> arg_cnv << Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
??>>
??>> works right

HN> Hmmm...
HN> Could you try to replace
HN> 'Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)'
HN> to 'Tk.fromUTF8(TkComm::string(arg_val[idx]), enc)'?

Works right. Thanks a lot. Will this fix became official?

HN> # Is the value of enc is 'cp1251'?

Yes, of course.

By the way, why such functions as
Tk.encoding_convertto
Tk.fromUTF8

are not described anywhere?

With best regards,
Nikolay Ponomarenko.

[Hidetoshi NAGAI]

Hi,

From: "Nikolay Ponomarenko" <pnv82@pisem.net>
Subject: Re: Encoding in ruby/tk internals
Date: Fri, 21 Nov 2003 16:17:14 +0900
Message-ID: <3fbdbbf9$1@news.itcom.net.ua>

> HN> Hmmm...
> HN> Could you try to replace
> HN> 'Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)'
> HN> to 'Tk.fromUTF8(TkComm::string(arg_val[idx]), enc)'?
> Works right. Thanks a lot. Will this fix became official?

Thank you for your help to debug the problem.
I'll commit the patch.
It will be included in 1.8.1-preview3.
--
Hidetoshi NAGAI (nagai@ai.kyutech.ac.jp)