feat: WASI target support for func (build + deploy)#3560
feat: WASI target support for func (build + deploy)#3560cardil wants to merge 25 commits intoknative:mainfrom
Conversation
Initial design proposal for WASI/WebAssembly support in func CLI. Covers build pipeline, deploy pipeline, WasmModule CRD integration, and func.yaml schema changes. Assisted-by: 🤖 Claude Opus/Sonnet 4.5
- Remove explicit --builder/--deployer flags from diagram (auto-inferred) - Replace 'Runtime Detection' with 'Runtime Selection' section - Clarify runtime is chosen at func create time, not auto-detected Assisted-by: 🤖 Claude Sonnet 4.5
Implements foundational WASM support infrastructure: - Add github.com/cardil/knative-serving-wasm dependency - Create pkg/wasm/ package with: - Constants for builder, deployer, and OCI media type - WASI runtime identifiers (rust-wasi, go-wasi, etc.) - IsWasiRuntime() helper function - Type aliases for WasmModule CRD network configuration - Add Wasm constant to pkg/builders/builders.go - Include unit tests with 83.3% coverage - Fix trailing whitespace in design document All tests pass (make test) and linting passes (make check). Phase 1 focuses on core types and constants. Builder and Deployer implementations will follow in subsequent phases. Assisted-by: 🤖 Claude Opus/Sonnet 4.5
… checks Self-registering builders and deployers via explicit Registry struct; WASI runtimes auto-infer wasm builder/deployer; incompatible combinations rejected at build/deploy time with wrapped ErrIncompatibility. Assisted-by: 🤖 Claude Sonnet 4.5
- Add templates/rust-wasi/http/ with Cargo.toml, src/lib.rs, README.md - Add templates/go-wasi/http/ with go.mod, function.go, function_test.go, README.md - Fix yaml.Decoder EOF bug in repository.go: comment-only manifest.yaml is valid - Update test expected lists to include go-wasi and rust-wasi runtimes - Regenerate embedded FS Assisted-by: 🤖 Claude Sonnet 4.6
Add build and deploy support for go-wasi and rust-wasi runtimes. Builder dispatch (pkg/wasm/builder.go) routes to TinyGo (go-wasi) or cargo (rust-wasi). TinyGo uses -no-debug to strip debug info (1.4MB→438KB). OCI push packs .wasm as a wasm/wasi layer (pkg/wasm/oci.go). Self-registration hooks builder and deployer into the registry. Fix inference never running: builder/deployer were always non-empty due to viper static defaults, making InferBuilder/InferDeployer dead code. Track explicit origin (flag or func.yaml) via BuilderExplicit/DeployerExplicit before Configure() overwrites with the static default. Fix func.yaml pollution: inferred values (e.g. 'wasm') are not written back to func.yaml. Restore the pre-Configure value before f.Write() when inference was used. Traditional runtimes still persist 'pack'/'knative'. e2e tests cover rust-wasi and go-wasi builds (renamed from e2e_wasm_test.go to avoid the GOARCH=wasm Go build constraint on _wasm_ filenames). First build uses --verbose; second build verifies func.yaml is not polluted. Assisted-by: 🤖 Claude Sonnet 4.6
…enerator gitignore - cmd/client.go: wire wasm remover, describer, lister in NewClient() - pkg/wasm: rename API fields TcpSpec→TCPSpec, UdpSpec→UDPSpec, AllowIpNameLookup→AllowIPNameLookup (.Tcp→.TCP, .Udp→.UDP) - bump github.com/cardil/knative-serving-wasm to latest (env var fix) - generate/templates/main.go: respect .gitignore in embedded FS generator (pass trailing slash for dirs so 'target/' pattern matches correctly) - pkg/filesystem/filesystem_test.go: apply same gitignore filtering to loadLocalFiles, initOSFS, initGitFS so they match the embedded FS - regenerate zz_filesystem_generated.go (drops rust-wasi/http/target/) - e2e: deployer tests, cluster.sh improvements - docs: regenerated reference pages
- Remove types.go (unused type aliases) - Rename types_test.go -> wasm_test.go (stale name) - Convert deployer_test.go to package wasm_test (black-box) - Export BuildNetworkSpec for external test access - Add t.Parallel() to compatibility_test.go tests - serving_wasm() uses stable kubectl apply from release manifest - Add KnativeServingWasm to component-versions
- Deployer readiness polling mirrors Knative deployer pattern - Pod watcher goroutine detects CrashLoopBackOff/ImagePullBackOff fast - ServiceUnavailable treated as transient (not terminal) - In-cluster registry (Deployment+NodePort) replaces external container - Insecure registry ConfigMap for WASM runner pods - Podman PID limit raised to 8192 (Knative+wasmtime needs >2048) - E2e liveness checks with withContentMatch for WASI templates - Update serving-wasm to v0.2.0 (insecure registries + terminal failures) Assisted-by: 🤖 Claude Opus/Sonnet 4.6
- Add pkg/wasm/wit.go: ProvisionWIT downloads WIT deps from OCI into wit/<key>/ using wasm-tools, compares wit/.versions to skip unchanged entries, writes per-subdir .gitignore to prevent accidental commits - Add pkg/wasm/wit_test.go: unit tests for diffVersions, load/save versions, writeGitignore, and ProvisionWIT no-op cases - Update pkg/wasm/builder.go: call ProvisionWIT before compiler when build.builderImages is non-empty - Update pkg/wasm/builder_go.go: recursive+parallel hasGoGenerateDirective, run go generate when //go:generate found; add -wit-package/-wit-world boson tinygo flags when wit/ dir present - Add templates/go-wasi/http/wit/world.wit: static source WIT world (package boson:function; world boson includes wasi:http/proxy@0.2.3) - Rewrite templates/go-wasi/http/function.go: WIT bindings handler via wit-bindgen-go generated bindings; pure greet() helper for testability - Rewrite templates/go-wasi/http/function_test.go: test greet() directly - Update templates/go-wasi/http/go.mod: go 1.24, tool directive for go.bytecodealliance.org/cmd/wit-bindgen-go - Add templates/go-wasi/http/.gitignore: ignore gen/ and module.wasm - Update e2e/e2e_wasi_test.go: GoDeploy now expects success with liveness - Update docs/design/wasi-integration.md: document WIT build pipeline Assisted-by: 🤖 Claude Sonnet 4.6
Replace atomic.Int32 + manual done channel with sync.WaitGroup to prevent double-close panic when goroutines finish between WalkDir callbacks. Move semaphore release inside the acquired path to prevent deadlock on context cancellation. Add builder_go_test.go with unit tests for hasGoGenerateDirective and fileHasGoGenerate, including stress tests exercising the concurrent fan-out with the race detector. Assisted-by: 🤖 Claude Opus/Sonnet 4.6
….2.3 wit.go — rewrite WIT provisioning pipeline: - Download OCI artifacts to temp dir, restructure into flat wit/deps/<pkg>/ - restructureWITDeps: main packages always overwrite, transitive deps skip existing dirs (prevents HTTP's stub cli.wit from clobbering full CLI package) - Add copyFile helper; pre-allocate provisioned slice builder_go.go — fix build pipeline: - Add runGoModTidy before go generate (template ships without go.sum) - Fix tinygo -o flag: use relative 'module.wasm' not absolute wasmPath (cmd.Dir=root makes -o relative to root, avoiding double-nesting) manifest.yaml — add builderImages for WASI deps: - cli: ghcr.io/webassembly/wasi/cli:0.2.3 - http: ghcr.io/webassembly/wasi/http:0.2.3 world.wit — define composite world for TinyGo: - include wasi:http/proxy@0.2.3 (HTTP handler export) - include wasi:cli/imports@0.2.3 (TinyGo runtime needs environment, filesystem, sockets beyond what proxy provides) function.go — fix wit-bindgen-go code generation: - go:generate flags: --world boson --out gen --package-root function/gen - Fix cm API: Value() not Unwrap(), cm.None/cm.OK not types.None/Ok, intermediate vars for pointer receivers, cm.ToList for byte slices, ResourceDrop() not Drop() builder_go_test.go — fix lint: check error returns in test helpers Regenerated embedded FS and docs. Assisted-by: 🤖 Claude Opus/Sonnet 4.6
The versionsScriptTemplate was missing knative_serving_wasm_version. The shell script had stale v0.1.0 while component-versions.json said v0.2.0, causing the cluster to deploy a controller without config-runner insecure registries support. Assisted-by: 🤖 Claude Opus/Sonnet 4.6
- Sort builderImages keys for deterministic iteration order - Track direct deps to prevent transitive overrides - Add __WASI_WIT_PROVIDER_MODE config in builderImages - strict (default): byte-diff is fatal error - forgiving: same-size skip, larger wins (with warnings) - Enable forgiving mode for go-wasi template - Detect auto-fetched vs user-vendored dep dirs Assisted-by: 🤖 Claude Opus/Sonnet 4.6
On systems with podman-docker installed, /usr/bin/docker is a shell wrapper that exec's podman. The previous default (CONTAINER_ENGINE=docker) caused the PID limit fix in cluster.sh to never fire, leading to PID exhaustion and containerd crashes under WASM workloads. Now common.sh detects the actual runtime via 'docker --version' output and falls back to podman when docker is absent. Assisted-by: 🤖 Claude Opus/Sonnet 4.6
Assisted-by: 🤖 Claude Opus/Sonnet 4.6
- Add Args []string to RunSpec in pkg/functions/function.go - Map f.Run.Args to spec.Args in pkg/wasm/deployer.go buildWasmModuleSpec - Add TestDeploy_Args and TestDeploy_Args_Empty to deployer_test.go - Regenerate schema/func_yaml-schema.json (adds args, NetworkSpec, TcpNetworkSpec, UdpNetworkSpec) - Document run.args in docs/reference/func_yaml.md Assisted-by: 🤖 Claude Sonnet 4.6
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: cardil The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/hold This should all be already working, but I need to run a review of all the changes. |
| github.com/alecthomas/jsonschema v0.0.0-20220216202328-9eeeec9d044b | ||
| github.com/blang/semver/v4 v4.0.0 | ||
| github.com/buildpacks/pack v0.38.2 | ||
| github.com/cardil/knative-serving-wasm v0.2.0 |
There was a problem hiding this comment.
Rely on my personal WASI runner.
TBD what to do with it...
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3560 +/- ##
==========================================
- Coverage 56.13% 54.05% -2.09%
==========================================
Files 180 202 +22
Lines 20465 22220 +1755
==========================================
+ Hits 11489 12010 +521
- Misses 7781 9033 +1252
+ Partials 1195 1177 -18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…ation # Conflicts: # go.mod # go.sum
Use json.Decoder token parsing to verify key ordering from raw JSON bytes instead of iterating a map (non-deterministic in Go). Assisted-by: 🤖 Claude Opus/Sonnet 4.6
The Knative Configuration may not have created its first Revision yet during cold-start, reporting Ready=False with Reason=RevisionMissing. This is a transient condition (like ServiceUnavailable) that resolves once the backing ksvc boots. Treating it as terminal causes false deploy failures in CI. Assisted-by: 🤖 Claude Opus/Sonnet 4.6
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Adds an alternative build/deployment mode for func that uses WASI modules instead of container images.
What it does
func create --runtime rust-wasiorfunc create --runtime go-wasicreates WASI function projectsfunc buildcompiles to wasm32-wasip2 and pushes OCI WASM artifacts to registryfunc deploycreates WasmModule CRs (from knative-serving-wasm) with readiness pollingfunc list,func describe,func deletework with WasmModule resources*-wasiruntime suffixKey components
pkg/wasm/— Builder, deployer, lister, remover, describer, OCI artifact pushertemplates/rust-wasi/http/— Rust WASI HTTP function template (wasi:http/proxy)templates/go-wasi/http/— Go WASI HTTP function template (TinyGo + WIT bindings)pkg/functions/registry.go— Self-registering builder/deployer system with constraintspkg/functions/compatibility.go— Runtime/builder/deployer compatibility validationhack/cluster.sh— E2e cluster setup with knative-serving-wasm controllerSupported runtimes
rust-wasi, go-wasi (others return "not yet implemented")
Dependencies
github.com/cardil/knative-serving-wasm v0.2.0(WasmModule CRD types)cargo+wasm32-wasip2target for Rust,tinygofor GoJira: SRVOCF-750
Assisted-by: 🤖 Claude Opus/Sonnet 4.6