Public SourceMage Discussion List

[Ismael Luceno <ismael AT iodev.co.uk>]

On 01/Oct/2020 17:44, Thomas Orgis wrote:
> Am Thu, 1 Oct 2020 02:23:16 +0200
> schrieb Ismael Luceno <ismael AT iodev.co.uk>: 
> 
> > Neither distributing binaries nor having these tools compile stuff out
> > of our control is acceptable.
> > 
> > Cargo nor "final binaries" as they call them, can freely decide
> > which library versions to use.
> 
> To be clear on the terms: As I see it, the package defines its
> dependencies, preferrably with version numbers (in Cargo.toml or such),
> cargo fetch downloads them into $CARGO_HOME. I package that, which is
> all sources.
> 
> The build then builds all the deps and then the final program, which
> gets all deps linked statically.

Is it only the sources though?

If you have A and B depending on C, C only gets built once...

> 
> > Also it's really unsettling the results from cdynlib and rlib in the
> > system, which to me seem to have very little reason to exist.
> 
> Can you elaborate?

rlibs are static and cdynlibs are dynamic libs with static dependencies
built-in.

And the problem is we need to tell which versions are used to apply
policies.

If you start pinning commits from git repos then it's chaos.

> > There's probably a way to provide a package for each individual
> > dependency installed in a global location... of course we have to
> > duplicate all the effort unless we manage to automate it all.
> 
> The question is if this is practicable with the differing versions
> involved. Do we dare to force certain grimoire versions of dependencies
> despite the package's preference? I don't see us being able to handle
> the fallout from that. Who keeps stable API if users can pick and
> choose which exact version (git snapshot) they use?

We need to force versions for sure (replace/edit .lock files), because
upstream often can't be trusted to make the right choices, but we need to
be able to install multiple versions too.

Not a problem if we let cargo manage it under a special target
directory, I guess, one per rust version.

> Are some packages stable enough that we can provide a (or just a few)
> binary version of them already installed in the system, speeding up
> builds, too?

I would say no, even the lang ABI isn't stable.

> 
> > Do we need to keep only the CARGO_TARGET_DIR or is it necessary to keep
> > CARGO_HOME too?
> 
> I don't know enough about that. I guess keeping only the target is
> messing with cargo internals … it's used to manage it all, after all.

Well, it isn't like we have an option, we need to understand it at least
superficially to make it cooperate.

The way I see it, we need one prefix per rust release, so:

rust-{ver}-{pkg}-{ver} spells

And we copy them all at each upgrade; the rust spell should also be
versioned.

Looking more in depth it looks like we need to provide a synthetic
CARGO_HOME.

It seems it would be enough to fill:
- registry/index/
- registry/cache/
- git/db/

This will be produced at each PRE_BUILD from the downloaded sources.

Then I guess the spells only need to keep build artifacts around (that's
CARGO_TARGET_DIR), and somehow workaround cargo's dependency management,
to avoid unintended side-effects (i.e. it must only build the current
package and if a dep is missing fail).