Public SourceMage Discussion List

[Remko van der Vossen <wich AT yuugen.jp>]

We have a devel-libpng that is collecting mothballs, I've been wanting to 
invest time since forever, but whenever I want to do so I run into so many 
other problems that I need to resolve first that in the end I don't have any 
time anymore for libpng.

The big problem is massive amounts of undocumented linking dependencies, 
mostly caused by the infernal libtool. Many libs and programs that indirectly 
depend on libpng but don't have any direct code dependencies link to libpng 
via DT_NEEDED anyway. When libpng gets updated all these spells need to be 
recast, which is why UP_TRIGGERS in the devel-libpng branch version of libpng 
tries to find all those spells that need to be updated. However, since many 
other link dependencies are not properly present in the grimoire the cast 
order is not good enough and a hell of a lot of casts fail leading to many 
cycles of cleanse --fix to try and get stuff working. And then we haven't 
even tackled any circular link dependencies.

The best fix would be libtool being smart enough that on a modern linux 
platform deep linking is unnecessary and hence would't do so, but that hasn't 
been happening for a long time now and I don't see it happening.

The next best fix would be updating sorcery to keep track of link 
dependencies of spells in tablet and having any ABI incompatible upgrade 
trigger recasts of any link dependent spells in the correct casting order. 
Then we also need to have a workable cyclic dependency resolver however 
otherwise we would still be in trouble. There is however an underlying 
problem of determining the link dependencies. For ELF shared objects we at 
least have the DT_NEEDED entries, but those aren't salvation either, as 1) 
sorcery would need to have knowledge of shared object resolving by the 
dynamic linker and its cache, which isn't really very standardized and 2) 
sorcery can't know about wrapper scripts that set stuff like LD_LIBRARY_PATH 
or LD_PRELOAD that mess stuff up. And that doesn't even consider non-ELF 
binary objects, let alone stuff like scripting languages, but at least most 
of those won't fail during cast when their dependencies are unable to 
run/link.

So, that leaves us with --as-needed, which causes the linker to omit any 
DT_NEEDED entries for shared objects which are not directly used by the 
dynamic object. This basically solves the problem that libtool causes and 
makes sure that we only need to recast libpng's direct dependers. Though it 
still leaves possible cyclic dependency resolving issues. --as-needed also 
has two other problems though, 1) we are forcing a choice on the user which 
is not something we tend to do, and 2) it breaks some spells. Some spells are 
fixable, but some may not be. The most common problem is a spell directly 
using a dependency, but not linking to it directly, which can in certain 
cases lead to a problem.

In any case, I think our best course of action for us now is to go the 
--as-needed route, at least in the short term. I've already been running with 
--as-needed in local sorcery LD_FLAGS on a couple systems. I've fixed a few 
spells, but mostly stuff just works. Though I have to say that I generally 
run pretty lean systems, so I would like to see more testing. I'd like to 
have sorcery have an as-needed option in the sorcery option/feature menu, 
which will default to off. Then if some of us will test it we can move the 
default to enabled and send out the word.

When any serious sorcery development is going to happen we could also pursue 
the second option, which I think would still be a good thing to do.

Regards, Remko

> On 2013/12/02, at 8:46, Treeve Jelbert <treeve AT scarlet.be> wrote:
> 
>> On Monday 02 December 2013 00:49:36 Javier Vasquez wrote:
>> Hi all,
>> 
>> I just realized that the test version of libpng is pretty old
>> (1.2.50), when one might use 1.6.7 now a days, :-)
>> 
>> I tried updating, and the 1.6.7 spell compiles and install with no 
>> problems.
>> 
>> However when recompiling just the spells that directly depend (1st
>> dependency level) on libpng, several fail.
>> 
>> I'm wondering if there's a recipe one can follow to update libpng,
>> like a series of planar casts (no matter if some have to be repeated),
>> till everything stabilize.
>> 
>> Or perhaps just soft linking the 1.6 libraries into 1.2 ones does the 
>> trick?
>> 
>> Thanks,
> everything on my system that  needs libpng builds for me
> 
> $ gzd libpng
> cairo:on:optional
> calligra:on:required
> cups-filters:on:optional
> freeglut:on:optional
> gdk-pixbuf2:on:required
> ghostscript:on:required
> kdelibs:on:required
> khtml:on:required
> kwin5:on:required
> leptonica:on:optional
> librsvg2:on:required
> opencv:on:optional
> openjpeg:on:required
> pango:on:required
> poppler:on:optional
> pstoedit:on:optional
> qt4:on:optional
> qtbase:on:optional
> slang:on:optional
> vlc:on:optional
> weston:on:required
> 
> There may be some old  spells which fail, but I have not tried them
> _______________________________________________
> SM-Discuss mailing list
> SM-Discuss AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/sm-discuss