Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] [RFC] Compressed filetype detection (Bug #16011)

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Jaka Kranjc <smgl AT lynxlynx.info>
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: Re: [SM-Discuss] [RFC] Compressed filetype detection (Bug #16011)
  • Date: Wed, 18 May 2016 21:41:25 +0200

On Sunday 15 of May 2016 19:31:55 Ismael Luceno wrote:
> On 15/May/2016 23:37, Jaka Kranjc wrote:
> > On Sunday 15 of May 2016 17:37:25 Ismael Luceno wrote:
> > > On 15/May/2016 21:31, Jaka Kranjc wrote:
> > > > On Sunday 15 of May 2016 16:04:39 Ismael Luceno wrote:
> > > > > I am thinking about having the uncompression, checksumming,
> > > > > signature
> > > > > verification, etc. all reworked and split into a separate unit, so
> > > > > that we can make it more flexible and streamline all tasks related
> > > > > to the inspection, packing and unpacking of spell sources, which I
> > > > > hope makes it easier to build better tools, specially to automate
> > > > > spell creation and inspection, and which could remove the need to
> > > > > specify information like BUILD_DIRECTORY into the spells.
> > > >
> > > > They're already in separate "units" — libraries. How could they be any
> > > > easier to reuse? If you're missing some more high-level interfaces,
> > > > that
> > > > is trivial to add.
> > >
> > > It is only simpler to reuse within sorcery. Doing complex changes,
> > > experiments, with sorcery is nowadays not simple.
> >
> > Eh? Nowadays? What did you try to do and what stopped you? Maybe as a
> > counter- example, when I wrote the update mode for quill, sorcery hardly
> > changed to make that possible.
> >
> > If you for some (odd) reason don't want to load all sorcery, you can still
> > only load what you need. Quill does that for faster startup and then loads
> > the rest.
>
> For example: I've been interested in adding a new configuration
> mechanism for spells, and that does require far more changes than it
> should; switching back and forth between both isn't an option, but it
> could be if sorcery were designed in a more loosely-coupled fashion,
> if we could simply make configuration an add-on.
It's a pretty integral part of what sorcery does and the design decision to
go
for interactive mode by default was probably made right at the start. There
is
such a thing as too many levels of indirection.

If that's what you mean by configuration mechanism?

> > > The problem I see with sorcery is that it is too tightly coupled,
> > > it's exactly the opposite of what unix is meant to be, and for no
> > > good reason.
> >
> > Examples, proof? I'm sure some functions could be split further and
> > libmisc
> > cleaned up, but that's bordering on nitpicking. Code is reused when a need
> > is shown. Most of it is in thematic library functions, there's plenty of
> > modularity and layering, so it's pretty bizzare that you accuse it of
> > tight coupling. Especially since it is written in bash where it is
> > trivial to redefine functions at runtime.
>
> Implementing new functionality is a pain: proper binary support,
What do you mean by proper?

> reproducible builds,
Is this really a sorcery problem or a limit of the toolchains? (I understood
it as binary equal successive builds)

> changing the flow of the build mechanism,
This is simple, just create build_api3 if the individual overrides are not
enough.

> any behaviour and about any change big or small requires to modify sorcery
> itself in incompatible ways,
[citation needed]

> and thus forces us to add more and more
> code paths and compatibility workarounds that would be unnecessary if
> we could just replace components at run-time.
But you *can* replace a whole lot at run time. Obviating the default build
api
or individual spell files out of it, compressors, downloaders, castfs and
trivial things like the pager and editor come to mind.

As far as workarounds go, most are there due to the support for multiple
choices and we can't really move that upstream, so even more replaceability
would probably make it worse, not improve the situation.

> An example of this is
> the new switch for comparing spell versions, if sorcery were designed
> differently, it would have required no changes, just a new utility,
> completely independent from the rest of sorcery.
This example is not good for your case. The switch is only there since you
deliberately changed the default behaviour. External version comparison
utilities are already on all the systems, namely sort -V, so there wouldn't
be
much to gain (for reuse) with externalising the simple awk script.

The user would still have to configure sorcery somewhere to tell it to use
that
particular comparison utility, so there'd be no real user experience gain
compared to lets say one option to always ignore downgrades and disregard
errors due to odd version schemes (obviating the need to pass the flag).

> Having a more unixy implementation, we could simply redefine parts
> of sorcery, switch back and forth, and compare results effortlessly
> in the more intrusive cases, while new functionality that is not
> vital could be provided by separate packages that could be provided
> by spells, and none of this would alter the core installation, nor
> it's functioning, everything becomes explicit.
We should be reducing complexity, not increasing it. Too many interchangeable
parts and (te)stability flies out the window.

If you're a developer, switching parts around is already simple, just another
wrapper away and you can redefine as many functions as you want, then switch
by
arbitrary means between them.

> Competing features and implementations of functionalities could be
> installed and work side-by-side unhindered.
>
> We can work towards that.
But why? Why not settle on the best thing like it was done all the years
before? Things were externalised when needed, not just for the sake of it.
Besides the version comparator, you haven't given an example for extraction
yet.

> > > > The only reason I can think of for a separate BUILD_DIRECTORY var is
> > > > that
> > > > some archives don't extract to expected locations. In case they create
> > > > more than one top-level directory, the correct one can't be
> > > > deterministically guessed, so I doubt the var can go away. I guess
> > > > it's
> > > > also partly a verification mechanism.
> > >
> > > For the few spells that are not tarbombs or similar, we can add an
> > > extra line in BUILD, the rest can be automatically fixed.
> > >
> > > This could also be a step forward to allow easier incompatible parallel
> > > builds, sharing the rest of the system, which could in turn be unified
> > > with the cross-building support for a more uniform approach.
> >
> > Parallel builds of the same spell sounds like a bad idea. Perhaps ok for
> > the individual level, but imagine it being part of a queue. And an
> > automatically different build dir wouldn't solve much, since you still
> > need a different install and potentially track root. There's no way to
> > guess that, so what's an extra var to specify when invoking cast?
>
> It is just one of the prerequisites to implement support for binary
> spells, along with isolation, repeatable builds, and some verification
> infrastructure among other things.
I doubt it's a prerequisite, but that's a debate for a different thread.

> > > Also, testing of different steps of the build, with changes in the
> > > middle, could be easier.
> >
> > What do you miss in delve? AFAIK it was created for exactly this reason.
>
> I meant changes in sorcery, not in the spell. Those changes could be
> made external to sorcery if it weren't a single entity.
I'm not sure what you mean. It sounds like you want to replace the build api
by developing it somewhere sorcery won't find it. And expecting it to.

And it's not a single entity. ><

Btw, we even provide pre/post hooks for all the steps if you don't want to
look at logs or use a debugger. These can easily be provided by spells if one
so desires.

LP
--
To err is humour
www.gemrb.org




Archive powered by MHonArc 2.6.24.

Top of Page