Public SourceMage Discussion List

[Remko van der Vossen <wich AT yuugen.jp>]

On Sat, Sep 06, 2014 at 12:12:30PM -0400, Sukneet Basuta wrote:
> On Wed, Sep 3, 2014 at 6:41 AM, Remko van der Vossen <wich AT yuugen.jp> wrote:
> 
>     As far as I know there is no (existing) way to force a certain casting
>     order in sorcery, the best we can do is improve our DEPENDS files so
>     that we get a more complete dependency tree. That still leaves problems
>     with cyclic dependencies though, which we need to break as much as
>     possible. Automatic cycle breaking in sorcery by first installing
>     without optional dependencies and later with would be nice, but I don't
>     see it happening anytime soon. (Also you'd have to take into account sub
>     dependencies, so you might even get more than two rounds for some
>     spells.)
> 
> I tried to fix things in devel-xorg by fixing the DEPENDS files first, but 
> it
> resulted in way too many cyclical dependencies. i.e. Spell A depends on 
> Spell
> B, Spell B depends on Spell C, but Spell C depends on Spell A. That's 
> probably
> a bad example, but I kept running into stuff like that.

Yes, there are still problems with cyclic dependencies, though this is
probably exacerbated by the extraneous link dependencies caused by
libtool and possibly incorrectly placed dependencies.

Still, we'll have cyclic dependencies left which we need to resolve in
some way, either automatically, which might be difficult, or with a
reasonable writeup people can follow to work out the cyclic
dependencies.

Ultimately I'd like to see automatic dependency cycle breaking in
sorcery, but this is a major development, which is, as far as I see it,
not going to happen unless we first get momentum going again within the
Source Mage project.

> This is already happening in the libpthread-stubs UP_TRIGGERS but it doesn't
> get all the spells that need to be recast. I was only adding spells to
> UP_TRIGGERS that weren't being found. (I believe there was a reason for 
> this,
> but I can't remember right now. I'll try to look into it again.) Perhaps
> linking with "-as-needed" will fix this problem.

Hmmn, that is extremely curious, the only spells that truly break
because of an ABI incompatible change in spell foo are those spells that
have a link dependency with spell foo. Granted, without as-needed this
set can be huge, but with as-needed it should be manegable. Of course
another spell may not work because one of its dependencies does have a
link dependency on foo, but that is not resolved by recasting that
spell, in fact that recast will fail. Instead the spell that has a link
dependency on foo needs to be recast and that should also solve the
problem in the spells higher up the dependency tree.

Are you sure that by "this is already happening" you mean the readelf
based link dependency finging as in devel-libpng's libpng/UP_TRIGGERS
and not something else? That should definitely catch all link dependers.
I'd be very interested to know of any other failure mode that we have to
address.

>     I would much rather we invest time in making sure that we recast the
>     right stuff upon ABI incompatible changes in the right way, which means;
>     1) actively rolling out as-needed
>     2) starting work on getting better quality DEPENDS files
>     3) creating some mechanism in sorcery which allows us to recast
>        dependers upon ABI incompatible changes
>     4) creating a mechanism in sorcery to automatically break dependency
>        cycles
> 
>     We can start working on 1) right now, 2) and 3) can be partially worked
>     around using smart UP_TRIGGERS scripts. We'll have work in the long run
>     for 2), 3) and 4). Note that in this case it also becomes easy when to
>     remove any special UP_TRIGGERS scripts; when 2) and 3) are done.
> 
> 1) Is there any problems with doing this? Why isn't done by default?
>     Looks like Gentoo has been doing this as the default for a while. They 
> just
> had to disable as-needed for a some ebuilds.

As far as I can see there are no major problems. As I've said in some
other mail, personally I didn't want to go the as-needed route several
years ago as it forces a decision on the user which is something we
don't like to do and technically it is possible to deal with the problem
without as-needed. Over the past years however I've come to the
conclusion that not forcing as-needed is too much for the Source Mage
project to take on. Therefore I switched to as-needed on all my boxes
about a year ago, and as it turns out so did Treeve at some point in the
past. I'm not sure about Treeve, but I can say that I did not run into
too many problems, there were a couple of spells that failed to build
but all were easily resolved, mostly by just updating the spell to a
more recent version. So in my opinion; no there are no problems in going
for as-needed by default and it is in fact what I am suggesting to do
and which is why I made the sorcery patch to have the as-needed option
in the sorcery optimization options which got into sorcery devel several
months ago.

Regards, Remko