Skip to Content.
Sympa Menu

sm-sorcery-bugs - [SM-Sorcery-Bugs] [Bug 8209] improvements to the downloading and url handling code

sm-sorcery-bugs AT lists.ibiblio.org

Subject: Bugs for Sorcery are reported here

List archive

Chronological Thread  
  • From: bugzilla-daemon AT bugs.sourcemage.org
  • To: sm-sorcery-bugs AT lists.ibiblio.org
  • Subject: [SM-Sorcery-Bugs] [Bug 8209] improvements to the downloading and url handling code
  • Date: Mon, 11 Apr 2005 19:35:32 -0700 (PDT)

http://bugs.sourcemage.org/show_bug.cgi?id=8209





------- Additional Comments From acedit AT armory.com 2005-04-11 19:35 -------
so, the things i want to change are mostly centered around moving code around
because I think certain things are in the wrong place. Also I want to make the
code less ignorant of the differences between downloading a file vs
downloading
a tree or files.

url handlers do too much work, they currently handle the following things
parse urls
actually download the urls
run netselect
package cache trees

Theres also code above the url level that expands urls to the various sorcery
mirrors.

Im going to split url_handlers into two seperate blobs of code, url_handlers
and
dl_handlers. The url handlers will handle higher level tasks: parsing urls,
netselect, url expanding/translating, dl_handlers are focused around actual
downloading programs (wget, cvs, svn, rsync, etc). Url handlers will pick the
dl_handler for a given url, so there could be a dl_handler for several
protocols
, or there could be one of several dl_handlers for a protocol (wget, curl,
proz)
. I think this is a good distinction to make. Url handlers will then be given
the responsibility of choosing a dl_handler for a given url and the next layer
up will use each dl_handler until one succeeds, the common case will be just
one, but we shouldnt be limited to that.

Im going to push the code for repackaging tree checkouts up out of the
url_handler/liburl code since its mostly redundant for different url_handlers
and they're really not in a position to necessarily detect the callers intent.
It feels like a layering violation to me.

Im giving up on the one true function to rule them all mentality that is
download_files(), or at least, in its current form I am. There are three
points
in sorcery where we do downloading and they all have different requirements
(sorcery update, scribe add/update, summon). Download_files was mostly summon
centric, so im going to push the tree download/tarball repackaging there,
neither sorcery or scribe care about that in particular so they dont need it.

In terms of passing information up from the url handlers, the return data
will be:
success/failure, location of downloaded file or tree, type of thing downloaded
(file or tree (we may not know until we try)).

There might also be an additional SOURCE_URL_HINTS parameter in DETAILS to get
extra information to the url/dl handler about stuff it would otherwise be
annoying to stuff into the url.

Finally, there will be a DOWNLOAD file that will be run instead of the
standard
operation of downloading urls.

And this concludes my braindump. Mostly a lot of shuffling code around, but
this
will let us solve some other bugs more easily (8052 3265 1777)

--
Configure bugmail: http://bugs.sourcemage.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.




Archive powered by MHonArc 2.6.24.

Top of Page