Skip to Content.
Sympa Menu

freetds - RE: [freetds] Problem with data type conversions and FreeTDS cont inued...

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Thompson, Bill D (London)" <ThompBil AT exchange.uk.ml.com>
  • To: "'FreeTDS Development Group'" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] Problem with data type conversions and FreeTDS cont inued...
  • Date: Fri, 1 Aug 2003 11:14:30 +0100

Hi James,

I haven't got much time to look at this in any detail today, sorry!

My investigations so far show that we are not processing the data row
properly.

The unknown marker problem is caused by the fact that we THINK we have fully
processed the data in the row, and are moving on to what we believe to be
the next row, but it isn't.
The data 0x10 0xfb is still part of the current row.

Our understanding of the results meta data is correct, witness all the log
entries like this one...

2003-08-01 09:43:57.896845 tds7_get_data_info:1083:
type = 108 (numeric)
column_varint_size = 1
colname = _numeric
colnamelen = 8

This code seems to be interperting all the meta data correctly.

Something is obviously throwing us when we get to the data row, however.

We hit the 0x10 0xfb thinking we have fully processed the row. however this
is (I think) 47 bytes before the end of the row.

Therefore we are somehow omitting to process 47 bytes of data somewhere down
the line.

We *shouldn't* have any problems in this area, unless there's a data type
that we haven't fully understood to date.

The only one of the data types that I'm not too sure of is the variant type,
so I'd be suspicious of that.

Could you try the test without the variant type ?

Bill

> -----Original Message-----
> From: James Vanns [SMTP:jimv AT canterbury.ac.uk]
> Sent: 01 August 2003 09:50
> To: FreeTDS Development Group
> Subject: Re: [freetds] Problem with data type conversions and FreeTDS
> continued...
>
> OK, here goes. Warning - quite a nasty lot of data. I can't see anything
> obvious as to why the conversion isn't happening...
>
> Received packet @ 2003-08-01 09:43:57.896220
> 0000 81 17 00 00 00 08 00 38 04 50 00 4b 00 45 00 59 |.......8
> .P.K.E.Y|
> 0010 00 00 00 08 00 7f 07 5f 00 62 00 69 00 67 00 69 |......._
> .b.i.g.i|
> 0020 00 6e 00 74 00 00 00 09 00 ad 32 00 07 5f 00 62 |.n.t....
> ..2.._.b|
> 0030 00 69 00 6e 00 61 00 72 00 79 00 00 00 08 00 32 |.i.n.a.r
> .y.....2|
> 0040 04 5f 00 62 00 69 00 74 00 00 00 08 00 af 0a 00 |._.b.i.t
> ........|
> 0050 09 04 d0 00 34 05 5f 00 63 00 68 00 61 00 72 00 |....4._.
> c.h.a.r.|
> 0060 00 00 08 00 3d 09 5f 00 64 00 61 00 74 00 65 00 |....=._.
> d.a.t.e.|
> 0070 74 00 69 00 6d 00 65 00 00 00 08 00 6a 11 12 00 |t.i.m.e.
> ....j...|
> 0080 08 5f 00 64 00 65 00 63 00 69 00 6d 00 61 00 6c |._.d.e.c
> .i.m.a.l|
> 0090 00 00 00 08 00 3e 06 5f 00 66 00 6c 00 6f 00 61 |.....>._
> .f.l.o.a|
> 00a0 00 74 00 00 00 09 00 22 ff ff ff 7f 12 00 61 00 |.t....."
> ......a.|
> 00b0 6e 00 6f 00 74 00 68 00 65 00 72 00 5f 00 74 00 |n.o.t.h.
> e.r._.t.|
> 00c0 65 00 73 00 74 00 5f 00 74 00 61 00 62 00 6c 00 |e.s.t._.
> t.a.b.l.|
> 00d0 65 00 06 5f 00 69 00 6d 00 61 00 67 00 65 00 00 |e.._.i.m
> .a.g.e..|
> 00e0 00 08 00 38 04 5f 00 69 00 6e 00 74 00 00 00 08 |...8._.i
> .n.t....|
> 00f0 00 3c 06 5f 00 6d 00 6f 00 6e 00 65 00 79 00 00 |.<._.m.o
> .n.e.y..|
> 0100 00 08 00 ef 14 00 09 04 d0 00 34 06 5f 00 6e 00 |........
> ..4._.n.|
> 0110 63 00 68 00 61 00 72 00 00 00 09 00 63 fe ff ff |c.h.a.r.
> ....c...|
> 0120 7f 09 04 d0 00 34 12 00 61 00 6e 00 6f 00 74 00 |.....4..
> a.n.o.t.|
> 0130 68 00 65 00 72 00 5f 00 74 00 65 00 73 00 74 00 |h.e.r._.
> t.e.s.t.|
> 0140 5f 00 74 00 61 00 62 00 6c 00 65 00 06 5f 00 6e |_.t.a.b.
> l.e.._.n|
> 0150 00 74 00 65 00 78 00 74 00 00 00 08 00 6c 11 12 |.t.e.x.t
> .....l..|
> 0160 00 08 5f 00 6e 00 75 00 6d 00 65 00 72 00 69 00 |.._.n.u.
> m.e.r.i.|
> 0170 63 00 00 00 08 00 e7 64 00 09 04 d0 00 34 09 5f |c......d
> .....4._|
> 0180 00 6e 00 76 00 61 00 72 00 63 00 68 00 61 00 72 |.n.v.a.r
> .c.h.a.r|
> 0190 00 00 00 08 00 3b 05 5f 00 72 00 65 00 61 00 6c |.....;._
> .r.e.a.l|
> 01a0 00 00 00 08 00 3a 0f 5f 00 73 00 6d 00 61 00 6c |.....:._
> .s.m.a.l|
> 01b0 00 6c 00 6c 00 64 00 61 00 74 00 65 00 74 00 69 |.l.l.d.a
> .t.e.t.i|
> 01c0 00 6d 00 65 00 00 00 08 00 34 09 5f 00 73 00 6d |.m.e....
> .4._.s.m|
> 01d0 00 61 00 6c 00 6c 00 69 00 6e 00 74 00 00 00 08 |.a.l.l.i
> .n.t....|
> 01e0 00 7a 0b 5f 00 73 00 6d 00 61 00 6c 00 6c 00 6d |.z._.s.m
> .a.l.l.m|
> 01f0 00 6f 00 6e 00 65 00 79 00 00 00 09 00 62 49 1f |.o.n.e.y
> .....bI.|
> 0200 00 00 0c 5f 00 73 00 71 00 6c 00 5f 00 76 00 61 |..._.s.q
> .l._.v.a|
> 0210 00 72 00 69 00 61 00 6e 00 74 00 00 00 09 00 23 |.r.i.a.n
> .t.....#|
> 0220 ff ff ff 7f 09 04 d0 00 34 12 00 61 00 6e 00 6f |........
> 4..a.n.o|
> 0230 00 74 00 68 00 65 00 72 00 5f 00 74 00 65 00 73 |.t.h.e.r
> ._.t.e.s|
> 0240 00 74 00 5f 00 74 00 61 00 62 00 6c 00 65 00 05 |.t._.t.a
> .b.l.e..|
> 0250 5f 00 74 00 65 00 78 00 74 00 00 00 08 00 30 08 |_.t.e.x.
> t.....0.|
> 0260 5f 00 74 00 69 00 6e 00 79 00 69 00 6e 00 74 00 |_.t.i.n.
> y.i.n.t.|
> 0270 00 00 08 00 24 10 11 5f 00 75 00 6e 00 69 00 71 |....$.._
> .u.n.i.q|
> 0280 00 75 00 65 00 69 00 64 00 65 00 6e 00 74 00 69 |.u.e.i.d
> .e.n.t.i|
> 0290 00 66 00 69 00 65 00 72 00 d1 01 00 00 00 cf 19 |.f.i.e.r
> ........|
> 02a0 46 96 02 00 00 00 ff ff 01 0a 00 74 65 73 74 20 |F.......
> ...test |
> 02b0 20 20 20 20 20 6b 93 00 00 04 52 d6 00 05 01 17 | k..
> ..R.....|
> 02c0 00 00 00 a4 70 3d 0a d7 40 93 40 00 d2 1e 00 00 |....p=..
> @.@.....|
> 02d0 00 00 00 00 44 d6 12 00 14 00 28 00 6e 00 63 00 |....D...
> ..(.n.c.|
> 02e0 68 00 61 00 72 00 29 00 20 00 20 00 20 00 10 fe |h.a.r.). . .
> ...|
> 02f0 ff 9e 2e 00 00 00 00 9c 00 00 00 01 00 00 00 64 |........
> .......d|
> 0300 75 6d 6d 79 54 53 00 00 00 00 00 05 01 11 09 00 |ummyTS..
> ........|
> 0310 00 14 00 28 00 6e 00 76 00 61 00 72 00 63 00 68 |...(.n.v
> .a.r.c.h|
> 0320 00 61 00 72 00 29 00 5a 72 f6 42 6b 93 00 00 22 |.a.r.).Z
> r.Bk..."|
> 0330 00 78 0e 56 00 00 00 00 00 10 fb ff 9f 2e 00 00 |.x.V....
> ........|
> 0340 00 00 9c 00 00 00 01 00 01 00 64 75 6d 6d 79 54 |........
> ..dummyT|
> 0350 53 00 00 00 00 00 02 10 58 78 4a f4 e3 8b 7b 42 |S.......
> XxJ...{B|
> 0360 99 29 87 19 4b 66 65 ec fd 10 00 c1 00 01 00 00 |.)..Kfe.
> ........|
> 0370 00 |.|
>
>
> 2003-08-01 09:43:57.896586 processing result tokens. marker is
> 81(TDS7_RESULT)
> 2003-08-01 09:43:57.896623 tds7_get_data_info:1083:
> type = 56 (int)
> column_varint_size = 0
> colname = PKEY
> colnamelen = 4
> 2003-08-01 09:43:57.896641 tds7_get_data_info:1083:
> type = 127 (long long)
> column_varint_size = 0
> colname = _bigint
> colnamelen = 7
> 2003-08-01 09:43:57.896658 tds7_get_data_info:1083:
> type = 45 (binary)
> column_varint_size = 2
> colname = _binary
> colnamelen = 7
> 2003-08-01 09:43:57.896674 tds7_get_data_info:1083:
> type = 50 (bit)
> column_varint_size = 0
> colname = _bit
> colnamelen = 4
> 2003-08-01 09:43:57.896690 tds7_get_data_info:1083:
> type = 47 (char)
> column_varint_size = 2
> colname = _char
> colnamelen = 5
> 2003-08-01 09:43:57.896706 tds7_get_data_info:1083:
> type = 61 (datetime)
> column_varint_size = 0
> colname = _datetime
> colnamelen = 9
> 2003-08-01 09:43:57.896731 tds7_get_data_info:1083:
> type = 106 (decimal)
> column_varint_size = 1
> colname = _decimal
> colnamelen = 8
> 2003-08-01 09:43:57.896748 tds7_get_data_info:1083:
> type = 62 (float)
> column_varint_size = 0
> colname = _float
> colnamelen = 6
> 2003-08-01 09:43:57.896765 tds7_get_data_info:1083:
> type = 34 (image)
> column_varint_size = 4
> colname = _image
> colnamelen = 6
> 2003-08-01 09:43:57.896781 tds7_get_data_info:1083:
> type = 56 (int)
> column_varint_size = 0
> colname = _int
> colnamelen = 4
> 2003-08-01 09:43:57.896797 tds7_get_data_info:1083:
> type = 60 (money)
> column_varint_size = 0
> colname = _money
> colnamelen = 6
> 2003-08-01 09:43:57.896813 tds7_get_data_info:1083:
> type = 47 (char)
> column_varint_size = 2
> colname = _nchar
> colnamelen = 6
> 2003-08-01 09:43:57.896829 tds7_get_data_info:1083:
> type = 35 (text)
> column_varint_size = 4
> colname = _ntext
> colnamelen = 6
> 2003-08-01 09:43:57.896845 tds7_get_data_info:1083:
> type = 108 (numeric)
> column_varint_size = 1
> colname = _numeric
> colnamelen = 8
> 2003-08-01 09:43:57.896861 tds7_get_data_info:1083:
> type = 39 (varchar)
> column_varint_size = 2
> colname = _nvarchar
> colnamelen = 9
> 2003-08-01 09:43:57.896877 tds7_get_data_info:1083:
> type = 59 (real)
> column_varint_size = 0
> colname = _real
> colnamelen = 5
> 2003-08-01 09:43:57.896892 tds7_get_data_info:1083:
> type = 58 (smalldatetime)
> column_varint_size = 0
> colname = _smallldatetime
> colnamelen = 15
> 2003-08-01 09:43:57.896908 tds7_get_data_info:1083:
> type = 52 (smallint)
> column_varint_size = 0
> colname = _smallint
> colnamelen = 9
> 2003-08-01 09:43:57.896924 tds7_get_data_info:1083:
> type = 122 (smallmoney)
> column_varint_size = 0
> colname = _smallmoney
> colnamelen = 11
> 2003-08-01 09:43:57.896956 tds7_get_data_info:1083:
> type = 98 (variant)
> column_varint_size = 4
> colname = _sql_variant
> colnamelen = 12
> 2003-08-01 09:43:57.896974 tds7_get_data_info:1083:
> type = 35 (text)
> column_varint_size = 4
> colname = _text
> colnamelen = 5
> 2003-08-01 09:43:57.896990 tds7_get_data_info:1083:
> type = 48 (tinyint)
> column_varint_size = 0
> colname = _tinyint
> colnamelen = 8
> 2003-08-01 09:43:57.897007 tds7_get_data_info:1083:
> type = 36 (uniqueidentifier)
> column_varint_size = 1
> colname = _uniqueidentifier
> colnamelen = 17
> 2003-08-01 09:43:57.897078 processing result tokens. marker is d1(ROW)
> 2003-08-01 09:43:57.897095 processing row tokens. marker is d1(ROW)
> 2003-08-01 09:43:57.897108 processing row. column is 0 varint size = 0
> 2003-08-01 09:43:57.897121 processing row. column size is 4
> 2003-08-01 09:43:57.897133 clearing column 0 NULL bit
> 2003-08-01 09:43:57.897145 processing row. column is 1 varint size = 0
> 2003-08-01 09:43:57.897158 processing row. column size is 8
> 2003-08-01 09:43:57.897170 clearing column 1 NULL bit
> 2003-08-01 09:43:57.897182 processing row. column is 2 varint size = 2
> 2003-08-01 09:43:57.897195 processing row. column size is 0
> 2003-08-01 09:43:57.897208 setting column 2 NULL bit
> 2003-08-01 09:43:57.897219 processing row. column is 3 varint size = 0
> 2003-08-01 09:43:57.897232 processing row. column size is 1
> 2003-08-01 09:43:57.897244 clearing column 3 NULL bit
> 2003-08-01 09:43:57.897256 processing row. column is 4 varint size = 2
> 2003-08-01 09:43:57.897269 processing row. column size is 10
> 2003-08-01 09:43:57.897281 clearing column 4 NULL bit
> 2003-08-01 09:43:57.897293 processing row. column is 5 varint size = 0
> 2003-08-01 09:43:57.897305 processing row. column size is 8
> 2003-08-01 09:43:57.897317 clearing column 5 NULL bit
> 2003-08-01 09:43:57.897329 processing row. column is 6 varint size = 1
> 2003-08-01 09:43:57.897342 processing row. column size is 5
> 2003-08-01 09:43:57.897354 clearing column 6 NULL bit
> 2003-08-01 09:43:57.897366 swapping numeric data...
> 2003-08-01 09:43:57.897380 processing row. column is 7 varint size = 0
> 2003-08-01 09:43:57.897392 processing row. column size is 8
> 2003-08-01 09:43:57.897405 clearing column 7 NULL bit
> 2003-08-01 09:43:57.897416 processing row. column is 8 varint size = 4
> 2003-08-01 09:43:57.897429 processing row. column size is 0
> 2003-08-01 09:43:57.897451 setting column 8 NULL bit
> 2003-08-01 09:43:57.897463 processing row. column is 9 varint size = 0
> 2003-08-01 09:43:57.897475 processing row. column size is 4
> 2003-08-01 09:43:57.897487 clearing column 9 NULL bit
> 2003-08-01 09:43:57.897499 processing row. column is 10 varint size = 0
> 2003-08-01 09:43:57.897512 processing row. column size is 8
> 2003-08-01 09:43:57.897524 clearing column 10 NULL bit
> 2003-08-01 09:43:57.897536 processing row. column is 11 varint size = 2
> 2003-08-01 09:43:57.897548 processing row. column size is 20
> 2003-08-01 09:43:57.897560 clearing column 11 NULL bit
> 2003-08-01 09:43:57.897573 processing row. column is 12 varint size = 4
> 2003-08-01 09:43:57.897586 processing row. column size is 0
> 2003-08-01 09:43:57.897598 setting column 12 NULL bit
> 2003-08-01 09:43:57.897610 processing row. column is 13 varint size = 1
> 2003-08-01 09:43:57.897622 processing row. column size is 5
> 2003-08-01 09:43:57.897634 clearing column 13 NULL bit
> 2003-08-01 09:43:57.897646 swapping numeric data...
> 2003-08-01 09:43:57.897657 processing row. column is 14 varint size = 2
> 2003-08-01 09:43:57.897670 processing row. column size is 20
> 2003-08-01 09:43:57.897682 clearing column 14 NULL bit
> 2003-08-01 09:43:57.897694 processing row. column is 15 varint size = 0
> 2003-08-01 09:43:57.897707 processing row. column size is 4
> 2003-08-01 09:43:57.897719 clearing column 15 NULL bit
> 2003-08-01 09:43:57.897731 processing row. column is 16 varint size = 0
> 2003-08-01 09:43:57.897744 processing row. column size is 4
> 2003-08-01 09:43:57.897756 clearing column 16 NULL bit
> 2003-08-01 09:43:57.897768 datetime4 107 147 0 0
> 2003-08-01 09:43:57.897779 processing row. column is 17 varint size = 0
> 2003-08-01 09:43:57.897790 processing row. column size is 2
> 2003-08-01 09:43:57.897801 clearing column 17 NULL bit
> 2003-08-01 09:43:57.897811 processing row. column is 18 varint size = 0
> 2003-08-01 09:43:57.897823 processing row. column size is 4
> 2003-08-01 09:43:57.897834 clearing column 18 NULL bit
> 2003-08-01 09:43:57.897844 processing row. column is 19 varint size = 4
> 2003-08-01 09:43:57.897855 processing row. column size is 0
> 2003-08-01 09:43:57.897866 setting column 19 NULL bit
> 2003-08-01 09:43:57.897876 processing row. column is 20 varint size = 4
> 2003-08-01 09:43:57.897887 processing row. column size is 0
> 2003-08-01 09:43:57.897898 setting column 20 NULL bit
> 2003-08-01 09:43:57.897908 processing row. column is 21 varint size = 0
> 2003-08-01 09:43:57.897936 processing row. column size is 1
> 2003-08-01 09:43:57.897948 clearing column 21 NULL bit
> 2003-08-01 09:43:57.897959 processing row. column is 22 varint size = 1
> 2003-08-01 09:43:57.897970 processing row. column size is 0
> 2003-08-01 09:43:57.897981 setting column 22 NULL bit
> 2003-08-01 09:43:57.898114 inside tds_convert_money()
> 2003-08-01 09:43:57.898125 mymoney = ld
> 2003-08-01 09:43:57.898223 processing row tokens. marker is 10()
> 2003-08-01 09:43:57.898235 inside tds_process_default_tokens() marker is
> 10()
> Unknown marker: 16(10)!!
> 2003-08-01 09:43:57.898253 processing result tokens. marker is fb()
> 2003-08-01 09:43:57.898265 inside tds_process_default_tokens() marker is
> fb()
> Unknown marker: 251(fb)!!
>
> Many thanks! Maybe someone can see something I can't!
>
> Regards
>
> Jim
>
>
> On Thu, 2003-07-31 at 21:53, Frediano Ziglio wrote:
> > Il gio, 2003-07-31 alle 15:19, James Vanns ha scritto:
> > > Background:
> > >
> > > Linux to M$ SQL Server 2000 (using FreeTDS 0.61 and TDS==8.0)
> > > Problem printing images/binaries and Unique Identifiers.
> > > Code below.
> > >
> > > OK, I have discovered more. The function tds_get_null() seems to think
> > > that the field containing the UID is null - when it isn't. SO in fact
> > > the tds_convert function is never called on that value.
> > >
> > > Why does tds_get_null think that the current row,column field is NULL
> > > when it contains that {45535-HHKSJOD-3456} thing!?
> > >
> > > Here's the code again:
> > > const string& tds::connection::print_field_as_string (const int
> &column) const {
> > > static string str;
> > > unsigned char *row = results->current_row;
> > > const int type = results->columns[column]->column_type;
> > > const int offset = results->columns[column]->column_offset;
> > > const int size = results->columns[column]->column_cur_size;
> > > const int conversion_type = tds_get_conversion_type (type, size);
> > >
> > > str.clear ();
> > >
> > > if (tds_get_null (row, column))
> > > str = "NULL";
> > > else {
> > > CONV_RESULT convert;
> > > unsigned char *value = row + offset;
> > >
> > > if (is_blob_type (type))
> > > value = (unsigned char*) ((TDSBLOBINFO *)
> value)->textvalue;
> > >
> > > if (tds_convert (context, conversion_type, (TDS_CHAR *)
> value, size, SYBVARCHAR, &convert) < 0)
> > > throw tds::error ("Failed to convert row type at
> column %d.", column + 1);
> > >
> > > str = convert.c;
> > > free (convert.c);
> > > }
> > >
> > > return str;
> > > }
> > >
> > >
> > > Any ideas? Thanks!
> > >
> > > Jim
> >
> > Hi!
> > No, no idea. Try to set TDSDUMP and post results (you can remove all
> > lines before your query for security reasons).
> >
> > freddy77
> >
> >
> > _______________________________________________
> > FreeTDS mailing list
> > FreeTDS AT lists.ibiblio.org
> > http://lists.ibiblio.org/mailman/listinfo/freetds
> --
> James Vanns BSc (Hons) MCP
> Linux Systems Administrator
> Senior Software Engineer (Linux / C & C++)
> Canterbury Christ Church University College
> Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x24045370
>
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds





Archive powered by MHonArc 2.6.24.

Top of Page