Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] [RFC] Compressed filetype detection (Bug #16011)

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Vlad Glagolev <stealth AT tiberian.ru>
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: Re: [SM-Discuss] [RFC] Compressed filetype detection (Bug #16011)
  • Date: Wed, 18 May 2016 15:48:10 -0400

+1 from GL

On Wed, 18 May 2016 21:41:25 +0200
Jaka Kranjc <smgl AT lynxlynx.info> wrote:

> On Sunday 15 of May 2016 19:31:55 Ismael Luceno wrote:
> > On 15/May/2016 23:37, Jaka Kranjc wrote:
> > > On Sunday 15 of May 2016 17:37:25 Ismael Luceno wrote:
> > > > On 15/May/2016 21:31, Jaka Kranjc wrote:
> > > > > On Sunday 15 of May 2016 16:04:39 Ismael Luceno wrote:
> > > > > > I am thinking about having the uncompression, checksumming,
> > > > > > signature
> > > > > > verification, etc. all reworked and split into a separate unit, so
> > > > > > that we can make it more flexible and streamline all tasks related
> > > > > > to the inspection, packing and unpacking of spell sources, which I
> > > > > > hope makes it easier to build better tools, specially to automate
> > > > > > spell creation and inspection, and which could remove the need to
> > > > > > specify information like BUILD_DIRECTORY into the spells.
> > > > >
> > > > > They're already in separate "units" — libraries. How could they be
> > > > > any
> > > > > easier to reuse? If you're missing some more high-level interfaces,
> > > > > that
> > > > > is trivial to add.
> > > >
> > > > It is only simpler to reuse within sorcery. Doing complex changes,
> > > > experiments, with sorcery is nowadays not simple.
> > >
> > > Eh? Nowadays? What did you try to do and what stopped you? Maybe as a
> > > counter- example, when I wrote the update mode for quill, sorcery hardly
> > > changed to make that possible.
> > >
> > > If you for some (odd) reason don't want to load all sorcery, you can
> > > still
> > > only load what you need. Quill does that for faster startup and then
> > > loads
> > > the rest.
> >
> > For example: I've been interested in adding a new configuration
> > mechanism for spells, and that does require far more changes than it
> > should; switching back and forth between both isn't an option, but it
> > could be if sorcery were designed in a more loosely-coupled fashion,
> > if we could simply make configuration an add-on.
> It's a pretty integral part of what sorcery does and the design decision to
> go
> for interactive mode by default was probably made right at the start. There
> is
> such a thing as too many levels of indirection.
>
> If that's what you mean by configuration mechanism?
>
> > > > The problem I see with sorcery is that it is too tightly coupled,
> > > > it's exactly the opposite of what unix is meant to be, and for no
> > > > good reason.
> > >
> > > Examples, proof? I'm sure some functions could be split further and
> > > libmisc
> > > cleaned up, but that's bordering on nitpicking. Code is reused when a
> > > need
> > > is shown. Most of it is in thematic library functions, there's plenty of
> > > modularity and layering, so it's pretty bizzare that you accuse it of
> > > tight coupling. Especially since it is written in bash where it is
> > > trivial to redefine functions at runtime.
> >
> > Implementing new functionality is a pain: proper binary support,
> What do you mean by proper?
>
> > reproducible builds,
> Is this really a sorcery problem or a limit of the toolchains? (I
> understood
> it as binary equal successive builds)
>
> > changing the flow of the build mechanism,
> This is simple, just create build_api3 if the individual overrides are not
> enough.
>
> > any behaviour and about any change big or small requires to modify sorcery
> > itself in incompatible ways,
> [citation needed]
>
> > and thus forces us to add more and more
> > code paths and compatibility workarounds that would be unnecessary if
> > we could just replace components at run-time.
> But you *can* replace a whole lot at run time. Obviating the default build
> api
> or individual spell files out of it, compressors, downloaders, castfs and
> trivial things like the pager and editor come to mind.
>
> As far as workarounds go, most are there due to the support for multiple
> choices and we can't really move that upstream, so even more replaceability
> would probably make it worse, not improve the situation.
>
> > An example of this is
> > the new switch for comparing spell versions, if sorcery were designed
> > differently, it would have required no changes, just a new utility,
> > completely independent from the rest of sorcery.
> This example is not good for your case. The switch is only there since you
> deliberately changed the default behaviour. External version comparison
> utilities are already on all the systems, namely sort -V, so there wouldn't
> be
> much to gain (for reuse) with externalising the simple awk script.
>
> The user would still have to configure sorcery somewhere to tell it to use
> that
> particular comparison utility, so there'd be no real user experience gain
> compared to lets say one option to always ignore downgrades and disregard
> errors due to odd version schemes (obviating the need to pass the flag).
>
> > Having a more unixy implementation, we could simply redefine parts
> > of sorcery, switch back and forth, and compare results effortlessly
> > in the more intrusive cases, while new functionality that is not
> > vital could be provided by separate packages that could be provided
> > by spells, and none of this would alter the core installation, nor
> > it's functioning, everything becomes explicit.
> We should be reducing complexity, not increasing it. Too many
> interchangeable
> parts and (te)stability flies out the window.
>
> If you're a developer, switching parts around is already simple, just
> another
> wrapper away and you can redefine as many functions as you want, then
> switch by
> arbitrary means between them.
>
> > Competing features and implementations of functionalities could be
> > installed and work side-by-side unhindered.
> >
> > We can work towards that.
> But why? Why not settle on the best thing like it was done all the years
> before? Things were externalised when needed, not just for the sake of it.
> Besides the version comparator, you haven't given an example for extraction
> yet.
>
> > > > > The only reason I can think of for a separate BUILD_DIRECTORY var is
> > > > > that
> > > > > some archives don't extract to expected locations. In case they
> > > > > create
> > > > > more than one top-level directory, the correct one can't be
> > > > > deterministically guessed, so I doubt the var can go away. I guess
> > > > > it's
> > > > > also partly a verification mechanism.
> > > >
> > > > For the few spells that are not tarbombs or similar, we can add an
> > > > extra line in BUILD, the rest can be automatically fixed.
> > > >
> > > > This could also be a step forward to allow easier incompatible
> > > > parallel
> > > > builds, sharing the rest of the system, which could in turn be unified
> > > > with the cross-building support for a more uniform approach.
> > >
> > > Parallel builds of the same spell sounds like a bad idea. Perhaps ok for
> > > the individual level, but imagine it being part of a queue. And an
> > > automatically different build dir wouldn't solve much, since you still
> > > need a different install and potentially track root. There's no way to
> > > guess that, so what's an extra var to specify when invoking cast?
> >
> > It is just one of the prerequisites to implement support for binary
> > spells, along with isolation, repeatable builds, and some verification
> > infrastructure among other things.
> I doubt it's a prerequisite, but that's a debate for a different thread.
>
> > > > Also, testing of different steps of the build, with changes in the
> > > > middle, could be easier.
> > >
> > > What do you miss in delve? AFAIK it was created for exactly this reason.
> >
> > I meant changes in sorcery, not in the spell. Those changes could be
> > made external to sorcery if it weren't a single entity.
> I'm not sure what you mean. It sounds like you want to replace the build
> api
> by developing it somewhere sorcery won't find it. And expecting it to.
>
> And it's not a single entity. ><
>
> Btw, we even provide pre/post hooks for all the steps if you don't want to
> look at logs or use a debugger. These can easily be provided by spells if
> one
> so desires.
>
> LP
> --
> To err is humour
> www.gemrb.org
> _______________________________________________
> SM-Discuss mailing list
> SM-Discuss AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/sm-discuss


--
Vlad Glagolev <stealth AT tiberian.ru>

Attachment: pgp_wbzLqgg1Q.pgp
Description: PGP signature




Archive powered by MHonArc 2.6.24.

Top of Page