Skip to Content.
Sympa Menu

freetds - [freetds] A new issue with bcp/freebcp

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Thompson, William" <bill.d.thompson AT baml.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: [freetds] A new issue with bcp/freebcp
  • Date: Tue, 31 May 2011 21:22:16 +0100

Hi All,

I've had a new issue reported to me on freebcp.
This has come from someone migrating from Sybase to SQL Server, using freetds
as their "enabler" - an increasing trend in the use of freetds, I'm told.
So we have some solid evidence - "This works using Sybase bcp, but it doesn't
work with freebcp" - and although it's an edge case, I think it should be
fixed.

So the problem occurs with freebcp using a format file. It evidences itself
as a dbconvert() overflow error...

The problematic column is CHAR(6) on the database
The line from the format file for the column looks like this:

22 SYBCHAR 0 6 "~" 22 cost_center

and a snippet from the problematic line from the data file, showing the
problematic host file data looks like this:

...~123456 ~...

So in the format file, we've given a host file data length as 6 (the fourth
token in the format file), but we've also said the data is terminated (by a
"~")
However, there's a couple of extra spaces on the end of that data in the data
file, giving it a total length of 8 characters.

Well I took a look at this, and the Sybase manual pages for format files
tells me the following about the host file data length:


"Host file data length refers to the maximum number of bytes to copy for the
field. To decide how much data to copy in or out, bcp uses one of:

- The maximum field length
- The prefix length, if any
- The field terminator string, if any

If more than one method of field length specification is given, bcp chooses
the one that copies the least amount of data."
This caused me to delve into _bcp_read_hostfile() (in src/dblib/bcp.c). To
be honest, it shows the signs of having "evolved" to deal with the myriad of
possibilities available to the innocent or pragmatic user. I'm probably to
blame for much of that ad-hoc evolution, and here's another mutation to add
to the mix. It works, and doesn't make things significantly worse than they
they currently are:
Add the following lines to the function after approximately line 1421 of
bcp.c (some context given):

/*
* FIXME bcpcol->bcp_column_data->data && bcpcol->column_size
??
* It seems a buffer overflow waiting...
*/

+ /* If column length specified (in a format file), then it
*might* */
+ /* be less than the data read from file (if the column was
also */
+ /* described as a terminated field...) in this case we need to
only */
+ /* take the number of characters specified in the column
length */
+
+ if (hostcol->column_len > 0) {
+ collen = (hostcol->column_len < collen) ?
hostcol->column_len : collen;
+ }

bcpcol->bcp_column_data->datalen =
dbconvert(dbproc, hostcol->datatype, (const BYTE *)
coldata, collen, desttype,
bcpcol->bcp_column_data->data,
bcpcol->column_size);

if (bcpcol->bcp_column_data->datalen == -1) {


In penance, I may re-work the function at some point soon. The code is going
to keep me awake at nights otherwise...

Bill



.


Bill Thompson

GMO Technology
Bank of America Merrill Lynch
5 Canada Square
London E14 5AQ
Phone: +44 (0)207 995 8307
Email: bill.d.thompson AT baml.com


----------------------------------------------------------------------
This message w/attachments (message) is intended solely for the use of the
intended recipient(s) and may contain information that is privileged,
confidential or proprietary. If you are not an intended recipient, please
notify the sender, and then please delete and destroy all copies and
attachments, and be advised that any review or dissemination of, or the
taking of any action in reliance on, the information contained in or attached
to this message is prohibited.
Unless specifically indicated, this message is not an offer to sell or a
solicitation of any investment products or other financial product or
service, an official confirmation of any transaction, or an official
statement of Sender. Subject to applicable law, Sender may intercept,
monitor, review and retain e-communications (EC) traveling through its
networks/systems and may produce any such EC to regulators, law enforcement,
in litigation and as required by law.
The laws of the country of each sender/recipient may impact the handling of
EC, and EC may be archived, supervised and produced in countries other than
the country in which you are located. This message cannot be guaranteed to be
secure or free of errors or viruses.

References to "Sender" are references to any subsidiary of Bank of America
Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are
Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a
Condition to Any Banking Service or Activity * Are Not Insured by Any Federal
Government Agency. Attachments that are part of this EC may have additional
important disclosures and disclaimers, which you should read. This message is
subject to terms available at the following link:
http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you
consent to the foregoing.



  • [freetds] A new issue with bcp/freebcp, Thompson, William, 05/31/2011

Archive powered by MHonArc 2.6.24.

Top of Page