Skip to Content.
Sympa Menu

freetds - Re: [freetds] UTF-8 issue

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Peter Deacon <peterd AT iea-software.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] UTF-8 issue
  • Date: Tue, 8 Jan 2019 09:27:53 -0800 (Pacific Standard Time)

On Tue, 8 Jan 2019, Deepa Umesh wrote:

Hi Peter,
I set ClientCharset=CP1252 in odbc.ini (unixODBC)
But the result is as previous :

[FreeTDS][SQL Server]Some character(s) could not be converted into
client's character set. Unconverted bytes were changed to question
marks ('?')

Hi Deepa,

Normally this encoding causes everything to be mapped across so above would not occur. Recommend checking FreeTDS logs to confirm this is actually encoding used by FreeTDS.

Also make sure FreeTDS uses libiconv when compiling.

regards,
Peter

So do I need to reinstall freetds with this configuration in freetds.conf ?


Request you to provide additional info on installation stuff.
Also is there any config flags to be set?

Thanks,
Deepa

-----Original Message-----
From: FreeTDS <freetds-bounces AT lists.ibiblio.org> On Behalf Of Peter Deacon
Sent: Tuesday, January 8, 2019 1:50 PM
To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
Subject: Re: [freetds] UTF-8 issue

On Tue, 8 Jan 2019, Deepa Umesh wrote:

Hi ,
I am trying to rephrase my question with more clarity,

We are using UTF-8 as encoding in our product, which uses single byte
of character data, But when we attempt to use non US ASCII UTF-8
charecters , we are having encoding problems similar to the following,

Insert query : insert into altitem values ('あなたの犬を散歩したいと思います');

[FreeTDS][SQL Server]Some character(s) could not be converted into
client's character set. Unconverted bytes were changed to question
marks ('?')

[sqlprog1]
Driver=/usr/local/freetds-1.00.97/lib/libtdsodbc.so
Server=172.16.127.134
Database=sqlprog1_catdev19
Port=1433
TDS_Version=7.3
Client Charset = UTF-8.

Hi Deepa,

Set ClientCharset=CP1252

regards,
Peter
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
https://lists.ibiblio.org/mailman/listinfo/freetds
This message contains information that is confidential and/or may be
privileged. If you are not the intended recipient, you are hereby notified
that any use, dissemination, distribution or copy of this message and/or its
attachments is strictly prohibited. If you have received this message in
error, please advise the sender immediately by replying to this email and
delete this message. We are committed to your privacy. For more information,
please see our Privacy Policy at http://www.aptean.com/privacy-statement
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
https://lists.ibiblio.org/mailman/listinfo/freetds

From Deepa.Umesh AT aptean.com Wed Jan 9 01:52:08 2019
Return-Path: <Deepa.Umesh AT aptean.com>
X-Original-To: freetds AT lists.ibiblio.org
Delivered-To: freetds AT lists.ibiblio.org
Received: from mx0b-0014cc01.pphosted.com (mx0a-0014cc01.pphosted.com
[208.84.65.236])
by lists.ibiblio.org (Postfix) with ESMTP id 9749620141EE
for <freetds AT lists.ibiblio.org>; Wed, 9 Jan 2019 01:52:08 -0500 (EST)
Received: from pps.filterd (m0041843.ppops.net [127.0.0.1])
by mx0a-0014cc01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id
x096nl5E026442
for <freetds AT lists.ibiblio.org>; Wed, 9 Jan 2019 01:52:07 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aptean.com;
h=from : to : subject :
date : message-id : references : in-reply-to : content-type :
content-transfer-encoding : mime-version; s=own;
bh=3X5oPfbAIc2iUR9VqoichAbKTPb9zpPDCyFzTTijKhE=;
b=JcIPlBSIInzR74jbQlznfGAKn0HBF1DvqTHXyE4DldwfH4PsoduhdF/uFq8NrppVJ9/4
X1wZ0y26HEvvJirHlRrIxvwow51F/cIhFqgv+YwP8x9/ysK+JfH7jJd5yrUOfnnilyDd
cjhVxlVR5s9CzsbJTfZv38TUsZeDJ96+CMC9FtzJUTIwWv0y4eCdTbIUeKokE+tTIc0h
x+tFhbnAyDsApxzJGyhazxHCWmj6Vn/65FwOLrlJ3hAa5c9UxfGkUS219LeZGd+oVgSP
YWoUyB0DVqTy+YQW9dtrOoFZHUpBtGKC6vzzwroU6kRooBA/lQ4kHPyqEha+wyy8VjlC 7w== Received: from nam04-co1-obe.outbound.protection.outlook.com
(mail-co1nam04lp2054.outbound.protection.outlook.com [104.47.45.54])
by mx0a-0014cc01.pphosted.com with ESMTP id 2pu41yupfe-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)
for <freetds AT lists.ibiblio.org>; Wed, 09 Jan 2019 01:52:07 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=ApteanOnline.onmicrosoft.com; s=selector1-aptean-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=3X5oPfbAIc2iUR9VqoichAbKTPb9zpPDCyFzTTijKhE=;
b=Klz91rLo8+74HwqAmd9qRP7UIT2nqczCuuPW/rB0xFlXB6g5m6J9w2FTCi5fPSULO3IZbn5IJYkKald/5JQbUfiwao5zMFScqzqs/ReyVYB/+hFwbYMaPmALVTXXGW9lS75Oji0vQXbYugqnXoN4qnM9hr/qky/4eCsbSqmj91Y=
Received: from SN1PR18MB2272.namprd18.prod.outlook.com (52.132.199.146) by
SN1PR18MB2125.namprd18.prod.outlook.com (52.132.196.33) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.1495.6; Wed, 9 Jan 2019 06:52:05 +0000
Received: from SN1PR18MB2272.namprd18.prod.outlook.com
([fe80::19f6:93d5:72ab:2016]) by SN1PR18MB2272.namprd18.prod.outlook.com
([fe80::19f6:93d5:72ab:2016%2]) with mapi id 15.20.1495.011; Wed, 9 Jan 2019
06:52:05 +0000
From: Deepa Umesh <Deepa.Umesh AT aptean.com>
To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
Thread-Topic: [freetds] UTF-8 issue
Thread-Index:
AdSiYVwEraYquvosRHi6rL0N1eOqrgBrMAuAAMLsXxAABEe5AAAEybXQAA5aPoAAGWr8AA==
Date: Wed, 9 Jan 2019 06:52:05 +0000
Message-ID:
<SN1PR18MB22725F961072602990782C1C878B0 AT SN1PR18MB2272.namprd18.prod.outlook.com>
References:
<SN1PR18MB227294BE18628A9B0A8AC372878C0 AT SN1PR18MB2272.namprd18.prod.outlook.com>
<CAHt6W4eenkvZhuxHbb=z0nF8u507ZLa+Jz8_ODPQwhn1jtgT3g AT mail.gmail.com>
<BL0PR18MB2259BB651E5A3E778A42AE05878A0 AT BL0PR18MB2259.namprd18.prod.outlook.com>
<alpine.WNT.2.21.1.1901080014290.2516@smurf>
<SN1PR18MB22722370CC38E6F669BD77C4878A0 AT SN1PR18MB2272.namprd18.prod.outlook.com>
<alpine.WNT.2.21.1.1901080918570.2516@smurf>
In-Reply-To: <alpine.WNT.2.21.1.1901080918570.2516@smurf>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [115.110.248.130]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; SN1PR18MB2125;
20:v1DkaCdGSnRUWoXx4f9jVZeBdiVCoeW2RKlmKVSyXDxFnrLTYuyuTncHaXZFKgw7OXSBNRI18CvavhO38AkQsSe018NbMWZe7MNEydB2S3GSBYPILdsWrw17ePphEm+LFAfSSsGeVmGvx6WpuCjTBKKCV82eTIHH7ke55/rlYqg=
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: c4fa5104-27a0-4bad-fa54-08d675fef771
x-microsoft-antispam: BCL:0; PCL:0;
RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(2017052603328)(7153060)(7193020);
SRVR:SN1PR18MB2125; x-ms-traffictypediagnostic: SN1PR18MB2125:
x-microsoft-antispam-prvs:
<SN1PR18MB2125EA23144A2CD3C596A766878B0 AT SN1PR18MB2125.namprd18.prod.outlook.com>
x-forefront-prvs: 0912297777
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(346002)(376002)(366004)(39860400002)(136003)(396003)(55514002)(13464003)(189003)(199004)(476003)(229853002)(53936002)(7696005)(76176011)(26005)(86362001)(106356001)(93886005)(8936002)(25786009)(102836004)(55016002)(6306002)(9686003)(186003)(305945005)(68736007)(7736002)(316002)(3846002)(19627235002)(6116002)(66066001)(2906002)(486006)(256004)(14444005)(71200400001)(71190400001)(14454004)(5024004)(74316002)(478600001)(99286004)(6916009)(5660300001)(11346002)(446003)(6436002)(6506007)(81166006)(81156014)(6246003)(53546011)(8676002)(105586002)(72206003)(966005)(33656002)(97736004);
DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR18MB2125;
H:SN1PR18MB2272.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en;
PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: aptean.com does not designate
permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info:
W6V/i1/mbx8pC1H1AiJ7Nv5ASv0mhx4rDuZnD5MFN3XgnqUNA34XGwn6rTmHEt4KEN15Wbx6XM7RcBP3WP+89WnRj9ETHvk00mmLmZ1LngryEWM2nyak7elTnbDg3osfay6pZR0e+bBKwLhDySLs1dOelLJt5G+yT5L9fw8McKGm3dZ2LyisJYV7YXq/Q1XF5EDhLtmwJKOOQmHYm5iyOYPNHXyEYEDttfNyEoxfbwqLUhtPSfAp3q6DTz3IBcKMWKbd3dJUj7GcUCbbV+fiKMoXNyt1sFOyuNCFTnnrkRkXZezYSQfEvMxfvYr9Ngjk
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-2022-jp"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: aptean.com
X-MS-Exchange-CrossTenant-Network-Message-Id:
c4fa5104-27a0-4bad-fa54-08d675fef771
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jan 2019 06:52:05.1995 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 560ec2b0-df0c-4e8c-9848-a15718863bb6
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR18MB2125
X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9130
signatures=668680
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0
priorityscore=1501 malwarescore=0
suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015
lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000
definitions=main-1901090058
Subject: Re: [freetds] UTF-8 issue
X-BeenThere: freetds AT lists.ibiblio.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FreeTDS Development Group <freetds.lists.ibiblio.org>
List-Unsubscribe: <https://lists.ibiblio.org/mailman/options/freetds>,
<mailto:freetds-request AT lists.ibiblio.org?subject=unsubscribe>
List-Archive: <https://lists.ibiblio.org/sympa/arc/freetds/>
List-Post: <mailto:freetds AT lists.ibiblio.org>
List-Help: <mailto:sympa AT lists.ibiblio.org?subject=HELP>
List-Subscribe: <https://lists.ibiblio.org/mailman/listinfo/freetds>,
<mailto:freetds-request AT lists.ibiblio.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Jan 2019 06:52:08 -0000

Hi Peter,

Let me summarize my installation and configuration settings :

We are using FreeTDS(freetds-1.00.99) with UnixODBC(unixODBC-2.2.13) in Lin=
ux Platform and provided the following configuration in
Odbc.ini in UnicODBC is as below:

[sqlprog1]
Driver=3D/usr/local/freetds-1.00.97/lib/libtdsodbc.so
Server=3D172.16.127.134
Database=3Dsqlprog1_catdev19
Port=3D1433
TDS_Version=3D7.3
Client Charset =3D CP1252

CP1252 is used in Windows Platform , is there anything similar to CP1252 in=
Linux?

Thanks,
Deepa

-----Original Message-----
From: FreeTDS <freetds-bounces AT lists.ibiblio.org> On Behalf Of Peter Deacon
Sent: Tuesday, January 8, 2019 10:58 PM
To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
Subject: Re: [freetds] UTF-8 issue

On Tue, 8 Jan 2019, Deepa Umesh wrote:

Hi Peter,
I set ClientCharset=3DCP1252 in odbc.ini (unixODBC) But the result is as
previous :

[FreeTDS][SQL Server]Some character(s) could not be converted into
client's character set. Unconverted bytes were changed to question
marks ('?')

Hi Deepa,

Normally this encoding causes everything to be mapped across so above would=
not occur. Recommend checking FreeTDS logs to confirm this is actually en=
coding used by FreeTDS.

Also make sure FreeTDS uses libiconv when compiling.

regards,
Peter

So do I need to reinstall freetds with this configuration in freetds.conf=
?


Request you to provide additional info on installation stuff.
Also is there any config flags to be set?

Thanks,
Deepa

-----Original Message-----
From: FreeTDS <freetds-bounces AT lists.ibiblio.org> On Behalf Of Peter
Deacon
Sent: Tuesday, January 8, 2019 1:50 PM
To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
Subject: Re: [freetds] UTF-8 issue

On Tue, 8 Jan 2019, Deepa Umesh wrote:

Hi ,
I am trying to rephrase my question with more clarity,

We are using UTF-8 as encoding in our product, which uses single byte
of character data, But when we attempt to use non US ASCII UTF-8
charecters , we are having encoding problems similar to the
following,

Insert query : insert into altitem values ('=1B$B$"$J$?$N8$$r;6Jb$7$?$$$=
H;W$$$^$9=1B(B');

[FreeTDS][SQL Server]Some character(s) could not be converted into
client's character set. Unconverted bytes were changed to question
marks ('?')

[sqlprog1]
Driver=3D/usr/local/freetds-1.00.97/lib/libtdsodbc.so
Server=3D172.16.127.134
Database=3Dsqlprog1_catdev19
Port=3D1433
TDS_Version=3D7.3
Client Charset =3D UTF-8.

Hi Deepa,

Set ClientCharset=3DCP1252

regards,
Peter
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
https://lists.ibiblio.org/mailman/listinfo/freetds
This message contains information that is confidential and/or may be
privileged. If you are not the intended recipient, you are hereby
notified that any use, dissemination, distribution or copy of this
message and/or its attachments is strictly prohibited. If you have
received this message in error, please advise the sender immediately
by replying to this email and delete this message. We are committed to
your privacy. For more information, please see our Privacy Policy at
http://www.aptean.com/privacy-statement
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
https://lists.ibiblio.org/mailman/listinfo/freetds

_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
https://lists.ibiblio.org/mailman/listinfo/freetds
This message contains information that is confidential and/or may be privil=
eged. If you are not the intended recipient, you are hereby notified that a=
ny use, dissemination, distribution or copy of this message and/or its atta=
chments is strictly prohibited. If you have received this message in error,=
please advise the sender immediately by replying to this email and delete =
this message. We are committed to your privacy. For more information, pleas=
e see our Privacy Policy at http://www.aptean.com/privacy-statement




Archive powered by MHonArc 2.6.24.

Top of Page