Public SourceMage Discussion List

[Remko van der Vossen <wich AT yuugen.jp>]

On Fri, Sep 05, 2014 at 10:32:47PM +0200, Thomas Orgis wrote:
> Am Thu, 4 Sep 2014 21:30:37 +0200
> schrieb Remko van der Vossen <wich AT yuugen.jp>: 
> > - Devel branch owners should create a good UP_TRIGGERS for spells with
> >   ABI incompatible changes compared to test and prepare for pushing to
> >   test after stable-rc has been prepared
> 
> We once had a testing approach that started with basessytem to ensure
> one doesn't get irrevocably fritzed, then we wanted to expand this ...
> what is the base install we start with for testing the upgrade from
> current test to the devel stuff? Shall we include Xorg and some
> graphics libs? Excluding those would nicely get around the
> pthread-stubs issue, but I suppose you're not aiming for that easy way
> out.

yeah, unfortunately that process died a slow death. If me or anyone else
gets some time to make some baseline tarballs for stable and  that'd be
great. But it's six or even eight tarballs to maintain which I am not
sure we can really keep up, even two or four might be a bridge too far
at the moment. This time I guess we should just rely on mages to test
their work properly.  systems for I guess basesystem, X11+WM, and maybe
a DE or two.

> > Why are our dependency trees so inaccurate?  1) we choose to omit
> > dependencies that are covered by basesystem 2) we don't check our
> > DEPENDS files for correctness, as long as it compiles we assume it's
> > correct
> 
> Didn't we have some scripting already to find missing depends? I
> remember some thoughts from lynx back when attending SMEG (anyone
> else?).

I had been working on some scripting several years back to find missing
dependencies based on ELF link dependencies, but it was tough going
because of all the libfool madness, difficulty with wrapper scripts and
the general mess in determining which shared object is actually linked
against.

I do plan to continue work on this to try and improve our DEPENDS files.
If we use as-needed as default things will become a lot more manageable
and I've had some ideas to improve shared object resolution which I'd
like to work out.

> > Switching to as-needed already takes care of a very, very large part
> > of the dependency tree problem.
> 
> What is your view on the number of spells that have issues with
> as-needed?

I switched to as-needed by adding it to custom LDFLAGS about a year ago.
I encountered a few spells that failed casting due to it, but they were
mostly solved simply by updating the spell to a newer version. We're
hugely helped by other distros having switched to as-needed long before
us. On the whole there don't seem to be too many problems running with
as-needed. Though it has to be said that I don't run much more than
X11, a minimal WM, a graphical browser and various console apps.

> > So, what do we do about this problem? Again, we cheat! We find all
> > dependers, not by our dependency tree, but by checking all installed
> > spells for dynamic ELF objects 
> 
> I sense that this should be either done once for the whole system
> (using cleanse) and cached, further automatically on each cast after
> INSTALL, right? That way, using `gaze from` style look-up, we can even
> auto-suggest additions to DEPENDS. Actually ... how does that relate
> to `cleanse --delint`?

No, this would mean big changes to sorcery which need to be developed
and thoroughly tested. This isn't easy to do right, different spells may
install different SOs with the same soname. The cached data needs to be
kept up to date with spell casts and dispels. To be honest, this was
actually the idea I had several years back, I wanted sorcery to maintain
link dependency information in tablet which is essentially this. I just
don't see this happening anytime soon. This is why I suggested we cheat,
we do it on demand in UP_TRIGGERS. It won't be terribly fast, but if we
do it right it only needs to happen upon actual ABI incompatible
upgrades.

> > 1) We have to get as-needed everywhere. Without as-needed we have no
> > hope in hell of getting any reasonable casting orders and thus
> > always end up in upgrade hell. So, to everyone I say, get latest
> > devel sorcery on your box
> 
> So, before upgrading to new test grimoire, users should rebuild with
> as-needed or face consequences. How do we communicate that (assuming
> there are users that are not reading this list, one can dream)? Or,
> well, I guess it makes sense to do the rebuild with the fresh stable
> (aka test now) for a slightly higher chance to find source tarballs
> (I'm trying to fill up my mirror and promise not to delete things
> before it runs out of space).

Yes, good question, I had been thinking about that myself. I guess we
should at least send an announcement mail on sm-announce, but I feel we
have to have some kind of message inline to the update process. If worst
comes to worst I guess we could put a message in Changelog itself so
that it would appear at scribe update. Perhaps we can do better in some
other way though.

> >, (maybe we should make a new stable sorcery,)
> 
> +1

lynx, can we count on a bit of help from you? I guess devel has stewed
long enough to make it up to stable.

> > I'll take care of devel-libpng, who will take ownership of the other
> > two?
> 
> Ah, back we are: I wondered who these "owners" are. I do have some
> interest in the xorg branch, as it's the only thing that works for me.
> I have a hard time with the decisions there, though. Is mesalib-1x
> supposed to be renamed to mesalib again? The old spell (and thus, the
> MESALIB provider) is useless now. Also, we should settle treatment of
> llvm upgrades (see sm-commit), as it's a sensitive requirement for
> current mesalib.

I think both devel-xorg-modular and devel-icu were created by Treeve,
but perhaps it's a bit too much to lay everything on him, though of
course if he wants to I'm not going to stand in his place. The
devel-xorg-modular branch is quite a bit larger than the devel-libpng
and devel-icu branches so it'd be good if that one has a few more
testers.

llvm (as well as newer versions of gcc) are a diffrent matter, we can't
solve this with as-needed, dependency trees, or UP_TRIGGERS. I'm afraid
those will simply require grunt work. As for mesalib and stuff, I don't
have any particular opinions on that at the moment, that's a separate
problem to solve.

> Am I right in assuming that the "only" troubles are libpthread-stubs
> and the newer drivers? I am in the "lucky" position to use the three
> main video drivers (intel, radeon, nouveau). I'm not sure about
> libxcb, I didn't seem to have trouble with it.

I'm not sure about the current state of xorg, I'm still using X11 from
test, which works well enough for my purposes at the moment. I mentioned
libxcb because we had major headaches with it in the past, it could be
that it's fine right now though, I haven't looked at it.

> I hope to get away without linux-pam, as I loathe it's foolishly easy
> ways of seriously configuring it badly.

I've been running all my systems without linux-pam for the past couple
of years because it broke my system on every upgrade. Hopefully this can
be resolved with a good UP_TRIGGERS that (successfully) recasts all ELF
link dependers. I still won't be using as I simply don't need it, but
it's something that a lot of people use so it should be working without
too much hassle I feel.

Regards, Remko