[freetds] Truncated data for real data type in Sybase.
manoranjan_sahu at agilent.com
manoranjan_sahu at agilent.com
Tue Nov 22 19:53:23 EST 2005
I appreciate your response.
But here is the interesting thing.
Actual data
insert into t1 values(1, 123456789.123456789)
insert into t1 values(2, 12345678.123456789)
insert into t1 values(3, 1234567.123456789)
insert into t1 values(4, 123456.123456789)
insert into t1 values(5, 12345.123456789)
>From SYBASE isql
========================
1> select c1, c2 from t1 order by 1
2> go
c1 c2
----------- --------------------
1 123456792.000000
2 12345678.000000
3 1234567.125000
4 123456.125000
5 12345.123047
>From Freetds tsql
============================
1> select c1, c2 from t1 order by 1
2> go
c1 c2
1 1.234568e+08
2 1.234568e+07
3 1234567
4 123456.1
5 12345.12
Please see the difference in data in Sybase isql and freetds tsql.
So my guess is freetds is doing something different from sybase isql.
Any comments?
Thanks,
Manoranjan
-----Original Message-----
From: freetds-bounces at lists.ibiblio.org [mailto:freetds-bounces at lists.ibiblio.org] On Behalf Of Lowden, James K
Sent: Tuesday, November 22, 2005 2:33 PM
To: FreeTDS Development Group
Subject: Re: [freetds] Truncated data for real data type in Sybase.
> From: manoranjan_sahu at agilent.com
> Sent: Tuesday, November 22, 2005 2:14 PM
>
> I am using unixODBC with freetds v0.63RC10.
>
> The issue I have is that my application is truncating the real
> number data.
>
> I was able to see this behavior through tsql as well. Here is the
> test case.
>
[modified to use temporary table]
create table #t1 (c1 integer, c2 real)
go
delete from #t1
go
insert into #t1 values(1, 123456789.123456789)
insert into #t1 values(2, 12345678.123456789)
insert into #t1 values(3, 1234567.123456789)
insert into #t1 values(4, 123456.123456789)
insert into #t1 values(5, 12345.123456789)
go
select c1, c2 from #t1 order by c1
go
Here are the results in sqsh:
[30] mpquant.testdb.1> select c1, c2 from #t1 order by c1
[30] mpquant.testdb.2> select cast(123456789.123456789 as real)
[30] mpquant.testdb.3> go
c1 c2
----------- --------------------
1 123456792.000000
2 12345678.000000
3 1234567.125000
4 123456.125000
5 12345.123047
(5 rows affected)
--------------------
123456792.000000
In Microsoft's isql:
2> select c1, c2 from #t1 order by c1
3> go
c1 c2
----------- --------------------
1 1.234567920000e+008
2 1.234567800000e+007
3 1.234567125000e+006
4 123456.125000
5 12345.123047
> Please note that the actual data is truncated.
Right. The real datatype holds 7 digits of precision, irrespective of
where the decimal is. That's why "select cast(123456789.123456789 as
real)" produces a rounded result; it also explains all of your results.
cf. http://msdn.microsoft.com/library/en-us/tsqlref/ts_fa-fz_6r3g.asp
As you can see, this isn't a library issue. It's a simple question of
how many bits you're using to represent your number, and there's no
configuration option to stuff more than one bit into a bit. ;-)
If you need more precision, I recommend float instead.
--jkl
-----------------------------------------
The information contained in this transmission may be privileged and
confidential and is intended only for the use of the person(s) named
above. If you are not the intended recipient, or an employee or agent responsible
for delivering this message to the intended recipient, any review, dissemination,
distribution or duplication of this communication is strictly prohibited. If you are
not the intended recipient, please contact the sender immediately by reply e-mail
and destroy all copies of the original message. Please note that we do not accept
account orders and/or instructions by e-mail, and therefore will not be responsible
for carrying out such orders and/or instructions. If you, as the intended recipient
of this message, the purpose of which is to inform and update our clients, prospects
and consultants of developments relating to our services and products, would not
like to receive further e-mail correspondence from the sender, please "reply" to the
sender indicating your wishes. In the U.S.: 1345 Avenue of the Americas, New York,
NY 10105.
_______________________________________________
FreeTDS mailing list
FreeTDS at lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
More information about the FreeTDS
mailing list