Skip to Content.
Sympa Menu

freetds - Re: [freetds] results processing in freetds

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "James K. Lowden" <jklowden AT schemamania.org>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] results processing in freetds
  • Date: Thu, 16 Oct 2003 20:49:46 -0400

On Wed, 8 Oct 2003, "Thompson, Bill D (London)" <bill_d_thompson AT ml.com>
wrote:
>
> 1. submit sql batch (which might contain multiple statements)
> 2. While no more results
> 2.1 receive and store results metadata
> 2.2 receive and process data relating to the result set
> 2.3 receive and store "end" data
>
> We knew pretty much that a given connection would only have
> to handle one
> set of results metadata at a time.
> Each time we received a new set of results metdata, we could
> safely discard the last set and start afresh.

Maybe we can simplify some of this, or educate me why not?

I think we discard the metadata too late. Seems to me we should discard
the
metadata as soon as we know we're not going to get any more rows
(NO_MORE_ROWS, TDS_BUFSTAT_ATTNACK).

> As a result, we now have the following in the tds_socket structure :
>
> TDSRESULTINFO *curr_resinfo; /* pointer to one of the below... */
> TDSRESULTINFO *res_info; /* "main" results metadata */
> TDS_INT num_comp_info; /* number of compute results */
> TDSCOMPUTEINFO **comp_info; /* array of compute results metadata */
> TDSPARAMINFO *param_info; /* parameter results metadata */

To illustrate, here's a boiled-down log. The query:

select indid from sysindexes
where indid in (0,1)
order by indid
compute sum(indid), count(indid) by indid

$ grep 'processing r.* tokens' dump \
|perl -ne'if (/marker is d1\(ROW\)/) \
{ $d1++; } else {$d1=0;} \
print if $d1 <4;'
10:41:51.131139 processing result tokens. marker is 81(TDS7_RESULT)
10:41:51.131925 processing result tokens. marker is
88(TDS7_COMPUTE_RESULT)
10:41:51.134062 processing result tokens. marker is a9(ORDERBY)
[metadata ^^^]
10:41:51.136047 processing row tokens. marker is d1(ROW)
10:41:51.152915 processing row tokens. marker is d1(ROW)
10:41:51.220191 processing row tokens. marker is d3(CMP_ROW)
10:41:51.228653 processing row tokens. marker is d1(ROW)
10:41:51.230592 processing row tokens. marker is d1(ROW)
10:41:51.265537 processing row tokens. marker is d3(CMP_ROW)
[interleaved regular and compute rows ^^^]
10:41:51.270848 processing result tokens. marker is fd(DONE)

> I can envisage a new model where :
>
> tds_process_result_tokens, when it reads a set of results metadata,
> allocates the storage for that data, but returns the storage
> to the calling function for management.
> If an API function, say ct_results or dbresults is calling
> tds_process_result_tokens, it could then save the allocated results
> structure to it's own control structure - the DBPROCESS structure for
> dblibrary or the CS_COMMAND structure for ctlibrary
> If an internal function such as tds_set_spid or
> tds_process_simple_query is
> calling tds_process_result_tokens, it could hold the results data
> temporarily.
>
> tds_process_row_tokens, when called to retrieve a data row,
> could be passed
> the appropriate results metadata structure to allow it to
> interpret the "row" data correctly.

I think we should avoid pushing this up to the client layers, Bill. I
don't
like the idea of libtds allocating memory and db-lib managing it, and I
think there's more overlap than not among the client libraries.

I should mention, though, that you're doing the impossible, which is
interesting to say the least. I don't know how you managed to coerce the
server into holding two result sets for a connection....

Here's an alternative to your idea. It's similar to how I handled
parmeters
and charsets.

* Let tds_socket::res_info become an array of TDSRESULTINFO structures.
* Remove tds_socket::comp_info et. al. They should be in TDSRESULTINFO.
* On opening a cursor, add an element to tds_socket::res_info. Return
its
index to the caller, for future reference. I presume ct-lib would stash
the
index in the CS_COMMAND structure.
* Fetches through cursors require the caller to pass back the
TDSRESULTINFO
index.
* Closing a cursor invalidates its tds_socket::res_info element.

To avoid reallocations, don't free elements. Mark it as closed, and
re-use
it on the next cursor open. That way, if cursor #2 is closed, the index
to
#3 stays valid.

Note that tds_socket::res_info remains a TDSRESULTINFO*, not a
TDSRESULTINFO**. Non-cursor work can remain as is, because
tds->res_info[0]
== *tds->res_info.

The client has very little to manage this way, and the existing client
infrastructure is unaffected. For example, dbnextrow() can still call
tds_process_row_tokens(), but tds_process_row_tokens() becomes a wrapper
for
[notionally] tds_process_row_tokens2(). The latter takes one more
parameter: the res_info index. tds_process_row_tokens() obviously simply
passes it a 0.

How does that sound?

--jkl




Archive powered by MHonArc 2.6.24.

Top of Page