Public SourceMage Discussion List

[Andrew <afrayedknot AT thefrayedknot.armory.com>]

> 
> We already have prometheuse spell that can automatically check spells. 
> It's broken right now, but its author says it could be fixed.
> 
> Furthermore, as I build the ISO, I was trying to write a tool that could 
> check that what's used by the ISO (ie a running sourcemage system) was 
> working properly. I used several tests:
> 
> 
> - running cast on each and every spell. This is what prometheuse does 
> too. First, it would be cool if cast returns the status of the 
> installation. It returns 0 even when cast failed.

You are looking in the wrong place for the status of a spell, this
problem was solved in prometheus by looking in the activity log, besides
if you cast several things due to dependencies its easy to tease out
the details from the activity log rather than relying on a relatively
obscure return code, this was already hashed out the summer before last
when we started on prometheus. However in either case, cast should have
a non-zero return status if a spell fails.

Nonetheless, I think it would be better to utilize as much of prometheus
as possible for this. It was written in a very modular way so that
individual pieces could be re-written or adjusted without too much
turbulence to the other components. Its all very well documented in the
source tarball. The only part thats broken is the pm-autocast component
which automates answering casts questions and ensuring that cast isnt
going to do something bad, such as casting a spell we know is broken and
failing, people may remember a number of duplicate bugs were filed
because a spell a number of other spells depended on was broken). In
order to fix pm-autocast someone just needs to step up to the plate and
work on it, it just needs to know about the new messages from sorcery,
its in perforce, I will be happy to explain what to do in more detail,
how to do it, and to review the code later, i just do NOT have time to
do it myself, sorcery has a much higher priority for me right now.

> Moreover, a lot of 
> spells are not correctly using BUILD_API=2 or BUILD_API=1 and blocks 
> forever. Some other spells blocks forever for no reason (it might be 
> because the cast scripts is copied to /tmp/sorcery and later removed 
> before the script end)

What spells might these be and why havent you filed bugs? This sounds
like very odd behavior.

> 
> Anyway, I think we should have the following tool:
> 
> - check-spell-syntax : a script that check the syntax of spell's file. 
> It could check that BUILD_API=1 spells are calling "prepare_install" and 
> that BUILD_API=2 spells are not calling it. It could check the syntax of 
> the DETAILS file.

Theres probably some bash linting tools floating around that could find
silly syntax errors, ive found a few spells with some minor syntax errors
a while ago that i need to get around to fixing...

> 
> - check-spell-compile: a script that check that a spell is compiling 
> with all the combination of dependency and options. It could also check 
> for aliens.
this is exactly what prometheus was designed to do, please dont waste
my time by re-implementing the wheel.

> 
> - check-grimoire : a script that check all spells in a grimoire and that 
> might do a "sorcery rebuild" test.
isnt this prometheus again? if you read the documentation it has options
to check all spells in a grimoire, check all spells in a section,
it can notice when spells are updated and re-add those to the queue of
spells to test. Really, the only thing it doesnt do is try all the
possible options for a spell, instead it can pick random answers which
proved to be good enough when run a couple of times.

> 
> Having such tools (which need to be defined more accurately) would help 
> in having a working system and would help in finding bugs as soon as 
> possible. Right now, people are complaining about bugs in the ISO, but 
> the ISO is just a snapshot of the stable grimoire, which comes from test 
> and from devel...

I agree, but chatting about this is really pretty useless unless someone
steps up to the plate and actually leads the effort. Like i said before
I can help give help/advice (my day job is in qa fwiw), I can also review
programs for their correctness, but im spread too thin to actually lead
the effort right now.

Also, let us not forget that theres a number of bugs filed against the
installer that I think need to be addressed. If users cant install and
at least get online after the install, then all this qa work on installed
systems is rather pointless IMO.

-Andrew