feat: add builtin runtime support#4430
Conversation
abdulrahman11a
requested review from
efiacor,
kispaljr,
liamfallon and
mozesl-nokia
as code owners
✅ Deploy Preview for kptdocs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
dosubot
bot
added
size:XL
There was a problem hiding this comment.
Pull request overview
This PR implements a built-in runtime for curated KRM functions (apply-replacements and starlark) inside kpt itself, allowing them to run without Docker by registering them in a thread-safe in-process registry. When a function image matches a registered builtin (by normalized name, ignoring tags/digests), the builtin is used; otherwise the existing Docker/WASM fallback is preserved.
Changes:
- New self-registration registry (
internal/builtins/registry/) with thread-safeRegister/Lookupusing image name normalization - Built-in implementations for
apply-replacementsandstarlarkfunctions, wired viainit()blank imports - Integration into
fnruntime/runner.goto check the builtin registry before falling back to Docker/WASM
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
internal/builtins/registry/registry.go |
Thread-safe builtin function registry with image name normalization |
internal/builtins/applyreplacements/apply_replacements.go |
Built-in apply-replacements KRM function implementation |
internal/builtins/starlark/starlark.go |
Built-in starlark KRM function entry point |
internal/builtins/starlark/config.go |
Starlark config parser supporting StarlarkRun, Run, and ConfigMap kinds |
internal/builtins/starlark/processor.go |
Starlark processing logic |
internal/builtins/starlark/config_test.go |
Tests for starlark config parsing |
internal/builtins/BuiltinRuntime.go |
Blank-imports all builtins to trigger init() registration |
internal/fnruntime/runner.go |
Wires builtin registry lookup between pkg-context and Docker/WASM |
go.mod / go.sum |
Adds starlark catalog dependency and upgrades fn SDK to v1.0.2 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| func Process(resourceList *fn.ResourceList) (bool, error) { | ||
| err := func() error { | ||
| sr := &Run{} | ||
| if err := sr.Config(resourceList.FunctionConfig); err != nil { | ||
| return err | ||
| } | ||
| return sr.Transform(resourceList) | ||
| }() | ||
|
|
||
| if err != nil { | ||
| resourceList.Results = []*fn.Result{ | ||
| { | ||
| Message: err.Error(), | ||
| Severity: fn.Error, | ||
| }, | ||
| } | ||
| return false, nil | ||
| } | ||
|
|
||
| return true, nil | ||
| } |
There was a problem hiding this comment.
The config_test.go only tests config parsing but there are no tests for Process or Transform in the starlark package. Given that the existing pkg_context_test.go tests the full Run pipeline (input → Run → output), please add end-to-end tests for the starlark Process/Transform flow with a simple starlark script to verify actual resource transformation works correctly.
| // Copyright 2026 The kpt Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| // Package builtins registers all built-in KRM functions into the builtin registry. | ||
| package builtins | ||
|
|
||
| import ( | ||
| // Register built-in functions via init() | ||
| _ "github.com/kptdev/kpt/internal/builtins/applyreplacements" | ||
| _ "github.com/kptdev/kpt/internal/builtins/starlark" | ||
| ) |
There was a problem hiding this comment.
The filename BuiltinRuntime.go uses PascalCase, which deviates from the Go convention and the naming pattern used throughout this codebase (e.g., pkg_context.go, apply_replacements.go, runner.go). Consider renaming to builtin_runtime.go for consistency.
| func normalizeImage(image string) string { | ||
| if idx := strings.Index(image, "@"); idx != -1 { | ||
| image = image[:idx] | ||
| } | ||
| parts := strings.Split(image, "/") | ||
| if len(parts) > 0 { | ||
| last := parts[len(parts)-1] | ||
| if idx := strings.Index(last, ":"); idx != -1 { | ||
| parts[len(parts)-1] = last[:idx] | ||
| } | ||
| } | ||
| return strings.Join(parts, "/") |
There was a problem hiding this comment.
The normalizeImage function strips tags and digests, which means any version of a registered image (e.g., starlark:v0.1.0, starlark:v99.0.0) will match the single built-in implementation. If a user pins an older version expecting specific behavior from that version, they'll silently get the built-in instead. Consider at minimum logging a warning when the requested tag differs from a known/supported version, so users aren't surprised by version mismatches.
| func Register(fn BuiltinFunction) { | ||
| mu.Lock() | ||
| defer mu.Unlock() | ||
| registry[normalizeImage(fn.ImageName())] = fn | ||
| } | ||
|
|
||
| func Lookup(imageName string) BuiltinFunction { | ||
| mu.RLock() | ||
| defer mu.RUnlock() | ||
| return registry[normalizeImage(imageName)] | ||
| } | ||
|
|
||
| func List() []string { | ||
| mu.RLock() | ||
| defer mu.RUnlock() | ||
| names := make([]string, 0, len(registry)) | ||
| for name := range registry { | ||
| names = append(names, name) | ||
| } | ||
| return names | ||
| } | ||
|
|
||
| func normalizeImage(image string) string { | ||
| if idx := strings.Index(image, "@"); idx != -1 { | ||
| image = image[:idx] | ||
| } | ||
| parts := strings.Split(image, "/") | ||
| if len(parts) > 0 { | ||
| last := parts[len(parts)-1] | ||
| if idx := strings.Index(last, ":"); idx != -1 { | ||
| parts[len(parts)-1] = last[:idx] | ||
| } | ||
| } | ||
| return strings.Join(parts, "/") | ||
| } |
There was a problem hiding this comment.
There are no unit tests for the registry package (registry.go). Given that the existing pkg_context_test.go has tests for the other builtin, and the registry is a critical component (incorrect normalization or lookup would silently break function dispatch), please add tests for Register, Lookup, List, and especially normalizeImage covering edge cases like images with digests (@sha256:...), tags (:v1.0), both, and no tag/digest.
| func applyReplacements(rl *fn.ResourceList) (bool, error) { | ||
| r := &Replacements{} | ||
| return r.Process(rl) | ||
| } | ||
|
|
||
| type Replacements struct { | ||
| Replacements []types.Replacement `json:"replacements,omitempty" yaml:"replacements,omitempty"` | ||
| } | ||
|
|
||
| func (r *Replacements) Config(functionConfig *fn.KubeObject) error { | ||
| if functionConfig.IsEmpty() { | ||
| return fmt.Errorf("FunctionConfig is missing. Expect `ApplyReplacements`") | ||
| } | ||
| if functionConfig.GetKind() != fnConfigKind || functionConfig.GetAPIVersion() != fnConfigAPIVersion { | ||
| return fmt.Errorf("received functionConfig of kind %s and apiVersion %s, only functionConfig of kind %s and apiVersion %s is supported", | ||
| functionConfig.GetKind(), functionConfig.GetAPIVersion(), fnConfigKind, fnConfigAPIVersion) | ||
| } | ||
| r.Replacements = []types.Replacement{} | ||
| if err := functionConfig.As(r); err != nil { | ||
| return fmt.Errorf("unable to convert functionConfig to replacements:\n%w", err) | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| func (r *Replacements) Process(rl *fn.ResourceList) (bool, error) { | ||
| if err := r.Config(rl.FunctionConfig); err != nil { | ||
| rl.LogResult(err) | ||
| return false, nil | ||
| } | ||
| transformedItems, err := r.Transform(rl.Items) | ||
| if err != nil { | ||
| rl.LogResult(err) | ||
| return false, nil | ||
| } | ||
| rl.Items = transformedItems | ||
| return true, nil | ||
| } | ||
|
|
||
| func (r *Replacements) Transform(items []*fn.KubeObject) ([]*fn.KubeObject, error) { | ||
| var transformedItems []*fn.KubeObject | ||
| var nodes []*yaml.RNode | ||
| for _, obj := range items { | ||
| objRN, err := yaml.Parse(obj.String()) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| nodes = append(nodes, objRN) | ||
| } | ||
| transformedNodes, err := replacement.Filter{ | ||
| Replacements: r.Replacements, | ||
| }.Filter(nodes) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| for _, n := range transformedNodes { | ||
| obj, err := fn.ParseKubeObject([]byte(n.MustString())) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| transformedItems = append(transformedItems, obj) | ||
| } | ||
| return transformedItems, nil | ||
| } |
There was a problem hiding this comment.
There are no unit tests for the apply-replacements builtin. The starlark builtin has config_test.go, and the existing pkg_context builtin has pkg_context_test.go. Please add tests for the ApplyReplacementsRunner covering at least the Config, Transform, and Run methods to maintain consistency with the test coverage of other builtins in this package.
Signed-off-by: Abdulrahman Fikry <abdulrahmanfikry1@gmail.com>
abdulrahman11a
force-pushed
the
feature/builtin-runtime
branch
from
446b055 to
4bdb97c
Compare
Summary
Closes #4307
This PR implements a built-in runtime for curated KRM functions inside kpt itself,
allowing
apply-replacementsandstarlarkto run without pulling images from Docker Hub.Approach
As discussed in the issue thread, this implementation avoids reintroducing a circular
dependency between Porch and kpt by moving the built-in runtime concept into kpt directly.
The architecture is intentionally simple and extensible — new functions can be added
incrementally by registering them in the builtin registry.
How it works
A thread-safe self-registration registry (
internal/builtins/registry) allows KRM functionimplementations to register themselves via
init(). Thefnruntimerunner checks thisregistry before falling back to Docker or WASM, so existing behavior is fully preserved
for unregistered functions.
Priority order in
fnruntime/runner.go:pkg-contextbuiltin (existing)Functions included
ghcr.io/kptdev/krm-functions-catalog/apply-replacementsghcr.io/kptdev/krm-functions-catalog/starlarkFiles changed
internal/builtins/registry/registry.gointernal/builtins/applyreplacements/apply_replacements.gointernal/builtins/starlark/starlark.gointernal/builtins/starlark/config.goStarlarkRunandRunkinds)internal/builtins/starlark/processor.gointernal/builtins/BuiltinRuntime.goinit()internal/fnruntime/runner.goVerified locally
Both functions execute without Docker running.
Next steps