FreeTDS Development Group

[Frediano Ziglio <freddyz77 AT tin.it>]

Il dom, 2003-03-30 alle 21:06, James K. Lowden ha scritto:
> Now that we're going to use iconv for more than ascii-ucs2 conversion, we
> need better names and more flexible functions.  tdsiconvinfo can be
> simplified as follows:
> 
> typedef struct tdsiconvinfo
> {
>       int use_iconv;
> #if HAVE_ICONV
>       int bytes_per_char;
>       char client_charset[64];
>       iconv_t cdto_ucs2;   /* conversion from client charset to UCS2LE
> MSSQLServer */
>       iconv_t cdfrom_ucs2; /* conversion from UCS2LE MSSQLServer to client
> charset */
>       iconv_t cdto_srv;    /* conversion from client charset to SQL Server
> ASCII charset */
>       iconv_t cdfrom_srv;  /* conversion from SQL Server ASCII charset  to
> client charset */
> #endif
> } TDSICONVINFO;
> 
> becomes:
> 
> typedef struct tdsiconvinfo
> {
>       char client_charset[64];
>       char server_charset[64];
>       iconv_t to_wire;   /* conversion from client charset to server's 
> format
> */
>       iconv_t from_wire; /* conversion from server's format to client 
> charset
> */
> } TDSICONVINFO;
> 

Yes, but wire can be unicode and single-byte, this is the reason for 4
iconv_t. Also bytes_per_char is not still used but required for
allocating row...

> If either iconv_t is NULL, that's the equivalent of use_iconv == 0.  
> 

iconv_t is not a pointer, use (iconv_t)-1 (like file handle...)

> Then, instead of 
> 
> char* tds7_ascii2unicode(TDSSOCKET * tds, const char *in_string, 
>                        char *out_string, int maxlen);
> 
> and its complement, I propose:
> 
> size_t tds_iconv (iconv_t cd, const char *input, size_t * input_size,
>                        char *out_string, size_t maxlen);
> 

Yes and not. To skip a character (required to handle inconvertible
characters without truncating) you should know how to skip it. This is
not simple (this at ucs2 and singlebyte first), also some conversion
need a state (same sequence of bytes can be interpreted differently...
like utf8 and big5)
iconv do not help very much doing this (there aren't simple methods to
copy state, check if charset use a fixed bytes and how many or skip
characters..)

> [I don't understand why iconv uses char* instead of unsigned char*.]
> 

I think cause it convert characters...

> If "cd" is NULL, fallback to #if !HAVE_ICONV behavior.  Whether you're
> converting from or to wire format is controlled by passing the to_wire or
> from_wire descriptor.  
> 
> I've eliminated bytes_per_char, too, since utf-8 has no fixed value for
> it.  If we need to estimate how big a buffer to use in conversion, 4
> bytes/char is always safe for the foreseeable future.  
> 

Yes, but it's a waste of space. Every column in row occupy 4 times. For
store a char(10) we should need 40 bytes (compare it with current
10...).
Another solution is use always dynamic allocated buffers for all
characters. This keep code to read text similar to code for read
varchar.

> These changes will remove assumptions about the server's format and TDS
> version (allowing migration to UTF-8 as a Sybase server character set). 
> Please tell me if I've overlooked something, or if you have a better idea,
> or if you think I've lost my mind.  
> 
> Regards, 
> 
> --jkl

bye
  freddy77