Public SourceMage Discussion List

[Remko van der Vossen <wich AT stack.nl>]

On Mon, Jun 02, 2008 at 03:52:34PM +0200, "Andraž 'ruskie' Levstik" wrote:
> On 15:47:22 2008-06-02 Thomas Orgis <thomas-forum AT orgis.org> wrote:
> > Am Mon, 02 Jun 2008 15:39:12 +0200
> > schrieb "Andraž 'ruskie' Levstik" <ruskie AT codemages.net>:  
> > 
> > > He should have called verify_source (same params as unpack_source)
> > 
> > Oh, great, so that's the RTFM I missed (I added default_pre_build,
> > wich fortunately (?) does _not_ break when it cannot unpack).
> > 
> 
> Ups it's verify_file... as is unpack_file...
> 
> default_pre_build is:
> mk_source_dir $SOURCE_DIRECTORY  &&
> unpack_file ''
> 
> Anything more advanced of that requires you to handle it on your own...

Don't mind me too much, just musing on various possibilities...

I recently encountered the whole source verification thing in PRE_BUILD
with the netpbm spell. New releases of netpbm are not distributed as
tarballs anymore, but only as a source tree (in this case from an
subversion repository). The release is supposed to be stable, so source
verification is possible, but not as a simple hash on a single file.
Instead source verification has to be done on the extracted source tree.

In this case it is simply impossible to do verification before
unpacking. So while I think it might be worthwile to separate
verification, unpacking and stuff like patching, it is not really
possible to do this in a predefined order. So either it should be left
as-is, i.e. a monolithic verification, unpacking, patching step, or make
two verification steps on either side of the unpacking.

But verification also needs to be overridable from the spell for the
cases where sorcery can't handle the source verification, like in the
netpbm case. A possible way to do this would be to have ARCHIVE_VERIFY,
UNPACK, TREE_VERIFY and PATCH (or PRE_BUILD) scripts executed in that
order, then default ARCHIVE_VERIFY is a tarball hash check and default
TREE_VERIFY would be [true]. In the case of needed tree verification,
the spell would have an ARCHIVE_VERIFY which simple returns true and a
TREE_VERIFY which would do the hash check on the actual source tree.

The danger with this however is that one might accidentally leave an
ARCHIVE_VERIFY containing simply true after removing a post verify,
leaving the spell without source checking, like in the initial problem.

I think the best fix for this would be a sorcery call verify_success
which would indicate to sorcery that verification has been successfully
done, if this call has not been made (either by default sorcery code or
spell scripts) by the time BUILD is started, then the spell fails. This
will make sure that either sorcery takes care of the verification, or if
the mage disables that, be it purposely or accidentally, the spell
scripts are forced to take care of verification. In this case it would
also not matter whether we maintain a monolithic PRE_BUILD or end up
splitting this into a ARCHIVE_VERIFY, UNPACK, TREE_VERIFY and PATCH (or
PRE_BUILD).

Of course another option would be to include tree hash checking in
sorcery and only provide tarball hash and tree hash verification, which
would be selected in DETAILS by a variable. Then sorcery could make sure
the verification happens in the correct way. But of course unpacking and
patching would then have to be separated, otherwise sorcery can't do the
tree check in between.

Well, enough babble from me, choices are up to the sorcery devs.

Remko.

Attachment: smime.p7s
Description: S/MIME cryptographic signature