sm-discuss AT lists.ibiblio.org
Subject: Public SourceMage Discussion List
List archive
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds
- From: Ismael Luceno <ismael AT iodev.co.uk>
- To: Thomas Orgis <thomas-forum AT orgis.org>
- Cc: sm-discuss AT lists.ibiblio.org
- Subject: Re: [SM-Discuss] How to handle the rust ecosystem with offline builds
- Date: Thu, 1 Oct 2020 18:46:31 +0200
On 01/Oct/2020 17:44, Thomas Orgis wrote:
> Am Thu, 1 Oct 2020 02:23:16 +0200
> schrieb Ismael Luceno <ismael AT iodev.co.uk>:
>
> > Neither distributing binaries nor having these tools compile stuff out
> > of our control is acceptable.
> >
> > Cargo nor "final binaries" as they call them, can freely decide
> > which library versions to use.
>
> To be clear on the terms: As I see it, the package defines its
> dependencies, preferrably with version numbers (in Cargo.toml or such),
> cargo fetch downloads them into $CARGO_HOME. I package that, which is
> all sources.
>
> The build then builds all the deps and then the final program, which
> gets all deps linked statically.
Is it only the sources though?
If you have A and B depending on C, C only gets built once...
>
> > Also it's really unsettling the results from cdynlib and rlib in the
> > system, which to me seem to have very little reason to exist.
>
> Can you elaborate?
rlibs are static and cdynlibs are dynamic libs with static dependencies
built-in.
And the problem is we need to tell which versions are used to apply
policies.
If you start pinning commits from git repos then it's chaos.
> > There's probably a way to provide a package for each individual
> > dependency installed in a global location... of course we have to
> > duplicate all the effort unless we manage to automate it all.
>
> The question is if this is practicable with the differing versions
> involved. Do we dare to force certain grimoire versions of dependencies
> despite the package's preference? I don't see us being able to handle
> the fallout from that. Who keeps stable API if users can pick and
> choose which exact version (git snapshot) they use?
We need to force versions for sure (replace/edit .lock files), because
upstream often can't be trusted to make the right choices, but we need to
be able to install multiple versions too.
Not a problem if we let cargo manage it under a special target
directory, I guess, one per rust version.
> Are some packages stable enough that we can provide a (or just a few)
> binary version of them already installed in the system, speeding up
> builds, too?
I would say no, even the lang ABI isn't stable.
>
> > Do we need to keep only the CARGO_TARGET_DIR or is it necessary to keep
> > CARGO_HOME too?
>
> I don't know enough about that. I guess keeping only the target is
> messing with cargo internals … it's used to manage it all, after all.
Well, it isn't like we have an option, we need to understand it at least
superficially to make it cooperate.
The way I see it, we need one prefix per rust release, so:
rust-{ver}-{pkg}-{ver} spells
And we copy them all at each upgrade; the rust spell should also be
versioned.
Looking more in depth it looks like we need to provide a synthetic
CARGO_HOME.
It seems it would be enough to fill:
- registry/index/
- registry/cache/
- git/db/
This will be produced at each PRE_BUILD from the downloaded sources.
Then I guess the spells only need to keep build artifacts around (that's
CARGO_TARGET_DIR), and somehow workaround cargo's dependency management,
to avoid unintended side-effects (i.e. it must only build the current
package and if a dep is missing fail).
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Thomas Orgis, 10/01/2020
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Ismael Luceno, 10/01/2020
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Thomas Orgis, 10/01/2020
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Ismael Luceno, 10/01/2020
- Re: [SM-Discuss] How to handle the rust ecosystem with offline builds, Thomas Orgis, 10/02/2020
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Ismael Luceno, 10/01/2020
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Thomas Orgis, 10/01/2020
-
Re: [SM-Discuss] How to handle the rust ecosystem with offline builds,
Ismael Luceno, 10/01/2020
Archive powered by MHonArc 2.6.24.