diff --git a/README.md b/README.md index 131195b..a3b2606 100644 --- a/README.md +++ b/README.md @@ -184,10 +184,16 @@ You can also connect using your IDE's remote debugging features (VS Code, GoLand # Unit tests make test -# E2E tests +# E2E tests (requires Kind cluster with Gitea) +make create-kind-cluster # Sets up cluster with Gitea make test-e2e + +# Bundle tests +make test-e2e-bundle ``` +E2E tests use an in-cluster Gitea instance instead of GitHub, providing complete test isolation. See [Gitea Integration](docs/development/gitea-integration.md) for details on the test infrastructure. + ### Linting ```bash diff --git a/docs/development/gitea-integration.md b/docs/development/gitea-integration.md new file mode 100644 index 0000000..034e782 --- /dev/null +++ b/docs/development/gitea-integration.md @@ -0,0 +1,200 @@ +# Gitea Integration for E2E Tests + +## Overview + +The e2e test suite uses an in-cluster Gitea instance to provide git repository functionality. This eliminates external dependencies on GitHub and provides complete isolation for testing. + +## Architecture + +- **Gitea Installation**: Deployed via Helm during cluster setup +- **Network Access**: NodePort on port 30000 (HTTP) and 30022 (SSH) +- **Service Discovery**: ConfigMap in kube-public namespace contains endpoint +- **Authentication**: Admin user (giteaadmin/giteapass) for test operations +- **Provider Abstraction**: Tests use `RepositoryProvider` interface, allowing easy switching between Gitea/GitHub/GitLab + +## Using Repository Provider in Tests + +### Basic Pattern + +```go +var ( + repoURL string + repoDir string +) + +BeforeEach(func() { + var err error + + // Create repository provider resources with automatic cleanup + username, password, _, cleanup, err := repoProvider.CreateRandomUser() + Expect(err).NotTo(HaveOccurred()) + DeferCleanup(cleanup) + + _, repoURL, cleanup, err = repoProvider.CreateRandomRepo(username, false) + Expect(err).NotTo(HaveOccurred()) + DeferCleanup(cleanup) + + // Initialize with function code + repoDir, err = InitializeRepoWithFunction(repoURL, username, password, "go") + Expect(err).NotTo(HaveOccurred()) + DeferCleanup(os.RemoveAll, repoDir) +}) +``` + +### Available Helper Methods + +**RepositoryProvider Interface:** +- `CreateUser(username, password, email string) (cleanup func(), err error)` - Create user with cleanup function +- `CreateRandomUser() (username, password, email string, cleanup func(), err error)` - Create user with random credentials +- `CreateRepo(owner, name string, private bool) (url string, cleanup func(), err error)` - Create repository +- `CreateRandomRepo(owner string, private bool) (name, url string, cleanup func(), err error)` - Create repo with random name +- `CreateAccessToken(username, password, tokenName string) (string, error)` - Generate access token + +**E2E Helper Functions:** +- `InitializeRepoWithFunction(url, user, pass, lang)` - Clone, init function, push +- `CommitAndPush(repoDir, msg, file, ...otherFiles)` - Commit and push files + +### DeferCleanup Pattern + +All Create methods return cleanup functions that can be used with Ginkgo's `DeferCleanup`: + +```go +username, password, _, cleanup, err := repoProvider.CreateRandomUser() +Expect(err).NotTo(HaveOccurred()) +DeferCleanup(cleanup) // Automatically deletes user after test +``` + +This ensures resources are cleaned up even if tests fail. + +## Accessing Gitea UI + +During development, you can access the Gitea web UI: + +1. Get the Gitea endpoint: + ```bash + kubectl get configmap gitea-endpoint -n kube-public -o jsonpath='{.data.http}' + ``` + +2. Open in browser and login: + - Username: `giteaadmin` + - Password: `giteapass` + +## Testing Private Repositories + +```go +// Create private repo +_, repoURL, cleanup, err := repoProvider.CreateRandomRepo(username, true) +Expect(err).NotTo(HaveOccurred()) +DeferCleanup(cleanup) + +// Create access token +token, err := repoProvider.CreateAccessToken(username, password, "test-token") + +// Use token in Function spec +Spec: functionsdevv1alpha1.FunctionSpec{ + Source: functionsdevv1alpha1.FunctionSpecSource{ + RepositoryURL: repoURL, + Credentials: &functionsdevv1alpha1.Credentials{ + Token: token, + }, + }, +} +``` + +## Bundle Test Pattern + +Bundle tests use the `TestNamespace` struct for better organization: + +```go +type TestNamespace struct { + Name string + RepoURL string +} + +func createNamespaceAndDeployFunction() TestNamespace { + // Creates namespace and deploys function + // Returns TestNamespace with both namespace and repo URL +} + +// Usage +testNs := createNamespaceAndDeployFunction() +CreateFunctionAndWaitForReady(testNs) +``` + +## Troubleshooting + +### Gitea not accessible + +```bash +kubectl get pods -n gitea +kubectl logs -n gitea deployment/gitea +``` + +### ConfigMap missing + +```bash +kubectl get configmap -n kube-public +./hack/create-kind-cluster.sh # Recreate cluster +``` + +### Test failures with git operations + +- Check Gitea pod is running +- Verify network connectivity to NodePort: + ```bash + GITEA_IP=$(kubectl get configmap gitea-endpoint -n kube-public -o jsonpath='{.data.http}') + curl $GITEA_IP + ``` +- Check git credentials in test + +### Repository cleanup issues + +If tests leave behind repositories, you can clean them manually: + +```bash +# List all users +curl -u giteaadmin:giteapass http:///api/v1/admin/users + +# Delete user (includes their repos) +curl -X DELETE -u giteaadmin:giteapass http:///api/v1/admin/users/ +``` + +## Implementation Details + +### Cluster Setup + +Gitea is installed during cluster creation in `hack/create-kind-cluster.sh`: + +```bash +helm install gitea gitea-charts/gitea --namespace gitea --create-namespace \ + --set service.http.type=NodePort \ + --set service.http.nodePort=30000 \ + --set gitea.admin.username=giteaadmin \ + --set gitea.admin.password=giteapass \ + --set persistence.enabled=false +``` + +### Service Discovery + +The cluster setup creates a ConfigMap with the Gitea endpoint: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: gitea-endpoint + namespace: kube-public +data: + http: "http://:30000" + ssh: ":30022" +``` + +Tests read this ConfigMap to discover where Gitea is running. + +### Networking + +- **Kind Node IP**: Retrieved via Docker inspect of control-plane node +- **NodePort**: Fixed ports 30000 (HTTP) and 30022 (SSH) +- **Accessibility**: Node IP is reachable from both host machine and cluster pods + +This allows the same repository URL to work in both test code (running on host) and Function resources (running in cluster). \ No newline at end of file diff --git a/go.mod b/go.mod index 30769a0..82757c0 100644 --- a/go.mod +++ b/go.mod @@ -3,6 +3,7 @@ module github.com/functions-dev/func-operator go 1.25.0 require ( + code.gitea.io/sdk/gitea v0.23.2 github.com/go-git/go-git/v6 v6.0.0-20260114124804-a8db3a6585a6 github.com/go-logr/logr v1.4.3 github.com/onsi/ginkgo/v2 v2.27.5 @@ -21,6 +22,7 @@ require ( require ( cel.dev/expr v0.24.0 // indirect dario.cat/mergo v1.0.2 // indirect + github.com/42wim/httpsig v1.2.3 // indirect github.com/BurntSushi/toml v1.5.0 // indirect github.com/Masterminds/semver/v3 v3.4.0 // indirect github.com/Microsoft/go-winio v0.6.2 // indirect @@ -37,6 +39,7 @@ require ( github.com/coreos/go-semver v0.3.1 // indirect github.com/cyphar/filepath-securejoin v0.6.1 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect + github.com/davidmz/go-pageant v1.0.2 // indirect github.com/docker/cli v28.3.0+incompatible // indirect github.com/docker/distribution v2.8.3+incompatible // indirect github.com/docker/docker-credential-helpers v0.9.3 // indirect @@ -46,6 +49,7 @@ require ( github.com/felixge/httpsnoop v1.0.4 // indirect github.com/fsnotify/fsnotify v1.9.0 // indirect github.com/fxamacker/cbor/v2 v2.9.0 // indirect + github.com/go-fed/httpsig v1.1.0 // indirect github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect github.com/go-git/gcfg/v2 v2.0.2 // indirect github.com/go-git/go-billy/v5 v5.6.2 // indirect @@ -77,6 +81,7 @@ require ( github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 // indirect github.com/google/uuid v1.6.0 // indirect github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 // indirect + github.com/hashicorp/go-version v1.7.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect github.com/json-iterator/go v1.1.12 // indirect diff --git a/go.sum b/go.sum index 6148abb..9146fd2 100644 --- a/go.sum +++ b/go.sum @@ -2,8 +2,12 @@ cel.dev/expr v0.24.0 h1:56OvJKSH3hDGL0ml5uSxZmz3/3Pq4tJ+fb1unVLAFcY= cel.dev/expr v0.24.0/go.mod h1:hLPLo1W4QUmuYdA72RBX06QTs6MXw941piREPl3Yfiw= cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs= cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= +code.gitea.io/sdk/gitea v0.23.2 h1:iJB1FDmLegwfwjX8gotBDHdPSbk/ZR8V9VmEJaVsJYg= +code.gitea.io/sdk/gitea v0.23.2/go.mod h1:yyF5+GhljqvA30sRDreoyHILruNiy4ASufugzYg0VHM= dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8= dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA= +github.com/42wim/httpsig v1.2.3 h1:xb0YyWhkYj57SPtfSttIobJUPJZB9as1nsfo7KWVcEs= +github.com/42wim/httpsig v1.2.3/go.mod h1:nZq9OlYKDrUBhptd77IHx4/sZZD+IxTBADvAPI9G/EM= github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg= github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg= @@ -72,6 +76,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davidmz/go-pageant v1.0.2 h1:bPblRCh5jGU+Uptpz6LgMZGD5hJoOt7otgT454WvHn0= +github.com/davidmz/go-pageant v1.0.2/go.mod h1:P2EDDnMqIwG5Rrp05dTRITj9z2zpGcD9efWSkTNKLIE= github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= github.com/docker/cli v28.3.0+incompatible h1:s+ttruVLhB5ayeuf2BciwDVxYdKi+RoUlxmwNHV3Vfo= @@ -112,6 +118,8 @@ github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c= github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU= github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= +github.com/go-fed/httpsig v1.1.0 h1:9M+hb0jkEICD8/cAiNqEB66R87tTINszBRTjwjQzWcI= +github.com/go-fed/httpsig v1.1.0/go.mod h1:RCMrTZvN1bJYtofsG4rd5NaO5obxQ5xBkdiS7xsT7bM= github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI= github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic= github.com/go-git/gcfg/v2 v2.0.2 h1:MY5SIIfTGGEMhdA7d7JePuVVxtKL7Hp+ApGDJAJ7dpo= @@ -204,6 +212,8 @@ github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 h1:+ngKgrYPPJr github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 h1:NmZ1PKzSTQbuGHw9DGPFomqkkLWMC+vZCkfs+FHv1Vg= github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3/go.mod h1:zQrxl1YP88HQlA6i9c63DSVPFklWpGX4OWAc9bFuaH4= +github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY= +github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c= github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= @@ -402,6 +412,9 @@ go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8= golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.47.0 h1:V6e3FRj+n4dbpw86FJ8Fv7XVOql7TEwpHapKoMJ/GO8= golang.org/x/crypto v0.47.0/go.mod h1:ff3Y9VzzKbwSSEzWqJsJVBnWmRwRSHt/6Op5n9bQc4A= @@ -409,6 +422,8 @@ golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 h1:R84qjqJb5nVJMxqWYb3np9L5Z golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0/go.mod h1:S9Xr4PYopiDyqSyp5NjCrhFrqg6A5zA2E/iPHPhqnS8= golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c= golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o= golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8= @@ -416,6 +431,8 @@ golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw= golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -427,6 +444,8 @@ golang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.39.0 h1:RclSuaJf32jOqZz74CkPA9qFuVTX7vhLlpfj/IGWlqY= golang.org/x/term v0.39.0/go.mod h1:yxzUCTP/U+FzoxfdKmLaA0RV1WgE0VY7hXBwKtY/4ww= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.33.0 h1:B3njUFyqtHDUI5jMn1YIr5B0IE2U0qck04r6d4KPAxE= golang.org/x/text v0.33.0/go.mod h1:LuMebE6+rBincTi9+xWTY8TztLzKHc/9C1uBCG27+q8= diff --git a/hack/create-kind-cluster.sh b/hack/create-kind-cluster.sh index a0331f0..9920c2e 100755 --- a/hack/create-kind-cluster.sh +++ b/hack/create-kind-cluster.sh @@ -164,6 +164,40 @@ function install_keda() { kubectl wait deployment --all --timeout=-1s --for=condition=Available --namespace keda } +function install_gitea() { + header_text "Installing Gitea" + + helm repo add gitea-charts https://dl.gitea.com/charts/ + helm repo update + helm install gitea gitea-charts/gitea --namespace gitea --create-namespace \ + --set service.http.type=NodePort \ + --set service.http.nodePort=30000 \ + --set service.ssh.type=NodePort \ + --set service.ssh.nodePort=30022 \ + --set gitea.admin.username=giteaadmin \ + --set gitea.admin.password=giteapass \ + --set gitea.admin.email=admin@gitea.local \ + --set persistence.enabled=false + + header_text "Waiting for Gitea to become ready" + kubectl wait deployment --all --timeout=-1s --for=condition=Available --namespace gitea + + # Get Gitea endpoint for tests + GITEA_NODE_IP=$(docker inspect kind-control-plane --format '{{.NetworkSettings.Networks.kind.IPAddress}}') + + # Create ConfigMap with Gitea endpoint info + kubectl apply -f - <