Source Mage code commit list

[Remko van der Vossen <wich AT yuugen.jp>]

On Wed, Mar 26, 2014 at 05:48:05PM +0100, Florian Franzmann wrote:
> On Mi 2014-03-26 17:15:08 +0100, Treeve Jelbert wrote:
> > I would like to propose that devel-gcc-4.8, devel-icu and 
> > devel-libpng-1.6 be 
> > merged immediately into devel-xorg-modular, which can stabliise for a 
> > short 
> > period and then merge  devel-xorg-modular to master.
> I'm all in favour of that.

+1 the only chance of us getting through this mess is just to bite the
bullet. Better alternatives are possible but would require large efforts
and/or extensive changes to sorcery both of which aren't going to happen
anytime soon.

> The problem with these branches is not that
> there are any spells that don't build with the development branches.
> The problem is that we don't have a migration path from our current
> master to the development branches. For example, migrating to the new
> version of icu will break lots and lots of installed spells since,
> unlike gentoo, we do not preserve old libraries until they aren't
> needed anymore. In principle, cleanse --fix should be able to take
> care of the situation, however, it fails to do so.

I suspect the problem is, just as it is for libpng, as it was for libxcb
some years back, (and as it may again be now,) that there is deep
DT_NEEDED linking because of the use of libtool and its braindamaged way
of working.

The issue is that all spells that directly or indirectly depend (well
actually, that should be "directly or indirectly link with", so there
will be automagic dependencies there as well) on a library with a soname
change need to be recast in the exact order of the tree defined by the
DT_NEEDED entries in the ELF objects in the dependent spells. This of
course doesn't happen because; firstly, not all dependencies are
properly recorded in the spells and secondly, sorcery doesn't
necessarily need to enforce the dependency order as the dependencies are
already installed. Let alone the problems of circular dependencies.

linker flag --as-needed will help here because it will prevent indirect
dependents from including DT_NEEDED entries even though libtool may
issue indirect dependencies during the linking phase of those indirect
dependents. --as-needed only has effect though, when all direct and
indirect dependents have already been compiled with --as-needed prior to
upgrading a library with a soname change. Also note that --as-needed
will not solve everything, there may still be circular depdendencies
related to the direct dependents or missing dependency data amongst
direct dependents, but the problem will be massively reduced and cleanse
--fix will have a much better chance of actually fixing any remaining
issues.

> Honestly, I don't think that we will come up with a working migration
> path. We've had this situation for over a year now and nobody has
> has been able to solve it.

yes, a true solution just isn't going to happen anytime soon. I do
expect though that --as-needed will help immensely, so maybe we should
first do a bit more testing with --as-needed amongst ourselves as
--as-needed may have its own issues with certain spells. And then a
month or two down the line when hopefully many people have tested
--as-needed on many spells and thus also have most spells built with
--as-needed merge all these devel branches into test.

When we do take the plunge, whether now or later, I would suggest
advising to use sorcery devel and enable --as-needed in the optimization
menu, if not for the current upgrades, then at least for the next ones.
If you want the least amount of trouble with the soname changes I would
suggest, if you haven't already done so, to first do a full rebuild with
--as-needed enabled, or at least a rebuild of all direct and indirect
dependents of the soname changing libraries, before upgrading the
libraries themselves. Though of course the latter choice may miss some
spells that do have DT_NEEDED entries on the soname changing libraries
because of missing dpendency records and automagic optional dependency
linking.

> I haven't tried the libpng branch so far but I think it has similar
> problems as the icu branch.

Yes, the soname change is the problem for libpng. I suspect that
devel-xorg may have similar problems with libxcb, libx11 or others.

Regards, Remko