Add BuildTargets.md

[lukewagner]

This PR adds BuildTargets.md to define this new concept of "build targets" as presented in both CG-06 and WASI-06-12, which itself was a revision of the earlier "wasit2" idea that came up in WASI/#595.

Currently, BuildTargets.md only defines one build target, wasm32, but with the intention of adding more in the future (e.g., wasm64 and wasmgc). See BuildTargets.md in this PR for more details.

The goal, mentioned at the end of BuildTargets.md, is that wasi-sdk and various other currently-component-producing toolchains would be able to emit core modules that matched wasm32 as defined here (with "componentification" being merely a final linker option of whether to run wasm-ld or wasm-component-ld). I plan to leave this open until we can get some implementation feedback to test this out.

[elewis787]

Would love to see this added and I am happy to test this. Let me know how/if I can support.

[lukewagner]

@elewis787 Great to hear! Probably the next thing we need for this PR to merge is a concrete implementation just to validate what's here makes sense. In particular, one great proof-of-concept would be extending the wit-component crate to accept any core module targeting wasm32 as defined here and emit a wasip2 component that runs on unmodified wasmtime or jco.

[elewis787]

@lukewagner sounds great.

A very basic example showing wasip1 and wasip2 and using WIT to generate bindings can be found here https://github.com/elewis787/wasip1-wasip2.

The goal is to show that Tinygo ( dev branch supporting wasip1 and wasip2 ) can be used to target a component or a module. Once these are compiled, this example shows that the wasmtime-go bindings ( currently wasip1 support only ) can be used for core modules, and wasmtime-cli ( or rust wasmtime, or jco ) can be used for components.

Let me know if this is on the right track for what you would like to see. It may be a slightly different use case. This focuses more on proving that a wit component ( with limitations ) can be a core module per wasip1.

Regardless, I will start digging into the wit-component crate.

[jeff1010322]

I am looking into this as well. Does this mean there would be build target support for generating modules (wasip1 only) versus generating components (wasip2) from compilers?

I have been testing out WebAssembly modules with JCO and through their transpile command I can convert the component into a core module that can be run in runtimes that don't support components like wasmtime-go. I just have to make sure the runtime links any imports called by the core module. In JCO you can manually disable the WASI P2 features with a few flags.

But is there a plan to make core modules without WASI P2 imports a build target for things like this?

[alexcrichton]

From a technical perspective almost all components today start as single core modules. This is the case for nearly all compiles for TinyGo, Rust, and C (e.g. LLVM-based toolchains). In all of these cases LLVM emits a single core wasm module which is then fed into wit-component which generates a component. So in that sense the answer to:

But is there a plan to make core modules without WASI P2 imports a build target for things like this?

is "yes that's sort of already supported today". I say "sort of" in that the way this typically works is that the original binary is using a mixture of wasi_snapshot_preview1 APIs plus WIT-bound APIs. That means that it's sort of a hybrid p1/p2 module which doesn't have necessarily clear semantics. The componentization process papers over all of this to produce a single component out the other end.

The goal of this proposal is to formalize the core-module-to-component step of tooling. It makes the core module artifact (which today becomes a component) a first-class artifact. It additionally defines what it means to import/export various names from a core module and this proposal means that most core modules emitted by compilers today are not valid to become a component because they import `wasi_snapshot_preview1.

This means that one big chunk in implementing this proposal is going to be updating language standard libraries to exclusively use WASIp2 APIs instead of a mixture of WASIp1 and WASIp2. This would likely start with wasi-libc and start bubbling up from there for example.

[elewis787]

Thanks @alexcrichton! This is helpful and clear.

as a naive example, does this demonstrate what you are discussing?

import "wasi_snapshot_preview1" "fd_write" with wasip1
vs
import "wasi:filesystem/[email protected]" "[method]descriptor.write" with wasip2

If so, is a byproduct of this a core module that is produced but using wasip2 imports before componentization? This means that in the example I provided the obvious difference is that the module produced does not use wasip2 syntax.

Are there any plans for taking a component to a module to be backward compatible with wasip1? I know wasip1 does not include some of the wasip2 system interfaces and many have changed but I have been able to side step some of the tooling support by implementing what is needed by the core module. This is still useful.

Thanks for walking through regardless!

[alexcrichton]

Indeed yeah the differences will come down to imports. The first one you listed is WASIp1, and the second one is WASIp2-as-defined-by-wit-component-today which is not a standard and does not match what this document is proposing. This document would use a WASIp2 target that looks like:

(import "cm32p2|ns:wasi:filesystem/[email protected]" "[method]descriptor.write" (func (param ...)))

Notably the prefix cm32p2|ns: on the import moudle plus the lack of trailing .0 in the import module as well.

If so, is a byproduct of this a core module that is produced but using wasip2 imports before componentization?

Yes, and this document is describing the exact name of component imports/exports to be "componentized", optionally, later on. This document is a formalization of the names used today which is intended to become standard (as opposed to whatever was implemented to begin with)

Are there any plans for taking a component to a module to be backward compatible with wasip1?

Not currently, but that's also sort of out of the scope of this PR

[elewis787]

@alexcrichton, thanks for explaining this.

@jeff1010322 and I are both in the jco exploration phase. As mentioned, we are playing around with the transpile flags to produce core modules. As a next step, do you feel it is helpful to look into the generation of the core module and take a stab at outlining the behavior listed here?

As far as backward compatibility, do you know of a place to discuss/contribute to this?

After using WIT and a few various tools, it seems like there may be some options/benefits in the bindings generated. Candidly, I am still getting up to speed with WASIp2 and may be incorrect.

[alexcrichton]

Oh no worries! Always great to have more folks help and happy to help out where I can!

I think the first step is what you're already doing here which is to make sure you feel comfortable with the current process of how a component is created. The high-level overview of that is that source code is written with the help of wit-bindgen frequently, compiled with a language's standard library, and assembled with LLVM's wasm-ld to produce a core wasm module. This module is fed through wit-component to produce a component.

The next step is going to be working backwards in this pipeline from the end back to the front. For example if you were to change the core wasm module today nothing would be able to turn it into a component. In that sense you'll want to do as @lukewagner suggested which is to start with wit-component, the final layer of this stack. That lives in the wasm-tools repository and you'll basically be updating various locations of where a core wasm "thing" is matched up to a component model "thing" to match the conventions outlined in this document. That repository has a whole bunch of handwritten tests in *.wat syntax which you can add to for this.

Once that's all implemented the next step is to probably work on wit-bindgen. That'll be updating all of its generated code to generate references to new names for imports/exports. That'll probably feel more natural once wit-component implemented.

From there the next hard part will be to flesh out language standard libraries, but that's probably best tackled once we're closer to that.

[elewis787]

Great overview. Thanks again!

I'll start looking into this. I am still getting up to date on the various tools but I believe I understand what's required now.