Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions apps/docs/content/guides/backup.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: "Backup on Zerops"
description: "Zerops auto-backs up databases and storage daily (00:00-01:00 UTC) with X25519 encryption; backups are retained for 7 days minimum after service/project deletion."
---


## Keywords
backup, restore, snapshot, daily backup, cron backup, encryption, retention, backup schedule

## TL;DR
Zerops auto-backs up databases and storage daily (00:00-01:00 UTC) with X25519 encryption; backups are retained for 7 days minimum after service/project deletion.

## Supported Services
MariaDB, PostgreSQL, Qdrant, Elasticsearch, NATS, Meilisearch, Shared Storage.

**Not supported**: Runtimes, Object Storage (use S3 lifecycle policies), Valkey/KeyDB (in-memory).

## Schedule Options
- No backups
- Once a day (default: 00:00-01:00 UTC)
- Once a week
- Once a month
- Custom CRON: `minute hour day month weekday`

## Tagging
- Auto tags: `daily` (every backup), `weekly` (first Monday UTC), `monthly` (1st of month UTC)
- User tags: Up to 24 chars (letters, numbers, `:-_`)
- **Protected tags**: Exempt from automatic deletion — use for critical snapshots

## Storage Limits
| Plan | Backup Storage | Egress |
|------|---------------|--------|
| Lightweight | 5 GB | 100 GB |
| Serious | 25 GB | 3 TB |
| Technical max | 1 TiB per project (shared across all services) |

## Retention Defaults
- Minimum kept: 7 daily + 4 weekly + 3 monthly
- Maximum per service: 50 backups

## Encryption
End-to-end with X25519 per-project keys. Decrypted only on download.

## Grace Period
7 days after service or project deletion before backups are permanently removed.

## Backup Formats by Service
| Service | Format |
|---------|--------|
| PostgreSQL | pg_dump |
| MariaDB | mysqldump |
| Elasticsearch | elasticdump (.gz) |
| Meilisearch | .dump |
| Qdrant | .snapshot |
| NATS | .tar.gz |
| Shared Storage | filesystem archive |

## Gotchas
1. **Object Storage has no Zerops backup**: Use S3 lifecycle policies or external backup
2. **Valkey/KeyDB not backed up**: In-memory data — use persistence or application-level backup
3. **Backup storage is shared**: All services in a project share the backup quota

## See Also
- zerops://themes/core — platform infrastructure
- zerops://themes/services — database service cards
- zerops://guides/scaling
103 changes: 103 additions & 0 deletions apps/docs/content/guides/build-cache.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: "Build Cache"
description: "Zerops uses a two-layer build cache: base layer (OS + prepareCommands) and build layer (buildCommands output). The `cache:` attribute in zerops.yml controls which files persist between builds. Changing `build.os`, `build.base`, `build.prepareCommands`, or `build.cache` invalidates both layers (cascade)."
---


## Keywords
build cache, cache, invalidation, prepareCommands, buildCommands, base layer, build layer, node_modules, vendor, cache paths, cache true, cache false, build speed, build optimization, two-layer cache, cascade invalidation

## TL;DR
Zerops uses a two-layer build cache: base layer (OS + prepareCommands) and build layer (buildCommands output). The `cache:` attribute in zerops.yml controls which files persist between builds. Changing `build.os`, `build.base`, `build.prepareCommands`, or `build.cache` invalidates both layers (cascade).

---

## Two-Layer Architecture

| Layer | Contains | Cached when |
|-------|----------|-------------|
| **Base layer** | OS, installed packages, prepareCommands output | prepareCommands unchanged |
| **Build layer** | Files from `cache:` attribute after buildCommands | cache config unchanged |

Both layers are currently **coupled** -- invalidating the base layer also invalidates the build layer (cascade invalidation).

## Cache Lifecycle

1. **Restoration**: cached files moved from `/build/cache` to `/build/source` (no-clobber -- source files win)
2. **Build execution**: buildCommands run with cached + source files
3. **Preservation**: specified cache files moved from `/build/source` to `/build/cache`

No compression or network transfer -- fast directory rename operations within the container.

---

## Configuration

### Path-Specific Caching (Recommended)

```yaml
build:
cache: node_modules # single path
cache: [node_modules, .next] # multiple paths
```

All paths resolve relative to `/build/source`. Supports Go `filepath.Match` patterns (e.g., `"subdir/*.txt"`, `"package*"`). Forms `./node_modules`, `node_modules`, `node_modules/` are equivalent.

### System-Wide Caching

- **`cache: true`** -- preserves entire build container state. Best for global package managers (Go modules, pip)
- **`cache: false`** -- only prevents caching within `/build/source`. Files outside (e.g., `$GOPATH`) **remain cached**

---

## Cache Invalidation

### Automatic Triggers

Any change to these zerops.yml fields invalidates **both layers**:

- `build.os`
- `build.base`
- `build.prepareCommands`
- `build.cache`

**DO NOT** add trivial changes to `prepareCommands` (e.g., adding `vim`) without understanding this will also invalidate cached `node_modules`, `vendor/`, etc.

### Manual Triggers

- **GUI**: Service detail -> Pipelines & CI/CD Settings -> Invalidate build cache
- **API**: `DELETE /service-stack/{id}/build-cache`
- **Version restore**: Activating a backup app version also invalidates cache

---

## Per-Runtime Cache Recommendations

| Runtime | Recommended `cache:` paths |
|---------|---------------------------|
| Node.js / Bun | `node_modules`, `.next`, `.turbo`, `package-lock.json` |
| Go | `cache: true` (modules live outside /build/source) |
| PHP | `vendor`, `composer.lock` |
| Python | `cache: true` (pip installs globally) or `.venv` |
| Rust | `target` |
| Java | `cache: true` (.m2 lives outside /build/source) |
| .NET | `cache: true` (NuGet outside /build/source) |

---

## Build Container Specs

CPU 1-5 cores, RAM 8 GB fixed, Disk 1-100 GB, Timeout 60 min. User `zerops` with **sudo**. Default OS: **Alpine** (use `apt-get` with `os: ubuntu`).

---

## Common Pitfalls

1. **Cascade invalidation**: Changing `prepareCommands` wipes build-layer cache too (e.g., adding `sqlite` to prepare also clears cached `node_modules`)
2. **`cache: false` is misleading**: Only clears `/build/source` cache. Globally installed packages (Go modules, pip packages) persist in the base layer
3. **No-clobber restore**: If source repo contains a file also in cache, **source wins** -- the cached version is silently skipped (logged but does not fail)
4. **Lock file caching**: Cache lock files (`package-lock.json`, `composer.lock`) alongside dependency directories for consistent installs

## See Also
- zerops://themes/core -- zerops.yml schema and cache attribute syntax
- zerops://guides/deployment-lifecycle -- full build and deploy pipeline sequence
69 changes: 69 additions & 0 deletions apps/docs/content/guides/cdn.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: "CDN on Zerops"
description: "Zerops CDN has 6 global regions with a **fixed 30-day cache TTL** (HTTP Cache-Control headers are ignored by CDN but still affect browsers). Built on Nginx + Cloudflare geo-steering."
---


## Keywords
cdn, cache, edge, content delivery, static assets, object storage cdn, geo-steering, purge, cache invalidation

## TL;DR
Zerops CDN has 6 global regions with a **fixed 30-day cache TTL** (HTTP Cache-Control headers are ignored by CDN but still affect browsers). Built on Nginx + Cloudflare geo-steering.

## Regions
1. **EU (Prague, CZ)** — Primary + all-region failover
2. **EU (Falkenstein, DE)** — Secondary European
3. **UK (London)**
4. **AU (Sydney)**
5. **SG (Singapore)**
6. **CA (Beauharnois, Canada)**

DNS TTL: 30 seconds. Geo-steering routes to nearest node. EU Prague is fallback if all others down.

## CDN Modes

### Object Storage CDN
- URL: `https://storage.cdn.zerops.app/bucket/path`
- Env var: `${storageCdnUrl}`
- Direct from Object Storage through CDN

### Static CDN
- URL: `https://static.cdn.zerops.app/domain.com/path`
- Env var: `${staticCdnUrl}`
- For custom domains on static/nginx services
- **Wildcard domains NOT supported**

### API CDN
- Coming soon
- Env var: `${apiCdnUrl}`

## Cache Behavior
- TTL: **Fixed 30 days** (not configurable)
- HTTP `Cache-Control` headers: Affect browser caching, **NOT CDN caching**
- Eviction: LRU when storage capacity reached
- First request: Fetched from origin and cached

## Purge Patterns
```
/* # All content
/dir/* # Directory contents
/file$ # Specific file (exact match)
/prefix* # Pattern prefix match
```
Wildcard must be at end. Use `$` suffix for exact file match.

### Purge via zsc
```bash
zsc cdn purge /* # Purge all cached content
zsc cdn purge /images/* # Purge directory
zsc cdn purge /style.css$ # Purge exact file
```

## Gotchas
1. **30-day fixed TTL**: Cannot be changed — `Cache-Control: max-age=3600` has no effect on CDN
2. **No wildcard domains on static CDN**: `*.domain.com` is not supported
3. **Purge wildcards at end only**: `/images/*.jpg` is invalid — use `/images/*`

## See Also
- zerops://themes/services — Object Storage service card
- zerops://guides/public-access
41 changes: 41 additions & 0 deletions apps/docs/content/guides/choose-cache.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: "Choosing a Cache on Zerops"
description: "**Use Valkey.** KeyDB development has stalled and is effectively deprecated on Zerops."
---


## Keywords
cache, redis, valkey, keydb, in-memory, session, key-value, choose cache, which cache

## TL;DR
**Use Valkey.** KeyDB development has stalled and is effectively deprecated on Zerops.

## Decision Matrix

| Need | Choice | Why |
|------|--------|-----|
| **Any caching need** | **Valkey** (default) | Active development, full HA, Redis-compatible |
| Legacy KeyDB apps | KeyDB | Only if migrating existing KeyDB deployment |

## Valkey (Default Choice)

- Redis-compatible drop-in replacement
- HA: 3 nodes (1 master + 2 replicas) with automatic failover
- Ports: 6379 (non-TLS), 6380 (TLS), 7000 (read replica non-TLS), 7001 (read replica TLS)
- Connection: `redis://${user}:${password}@${hostname}:6379`
- HA detail: Ports 6379/6380 on replicas forward traffic to current master (Zerops-specific, not native Valkey)

## KeyDB (Deprecated)

- Development activity has slowed significantly
- Port: 6379
- **Do not use for new projects**

## Gotchas
1. **HA replication is async**: Brief data loss possible during master failover
2. **Port forwarding is Zerops-specific**: Replicas forward 6379/6380 to master — this is not standard Redis/Valkey behavior
3. **Read replicas use different ports**: 7000/7001 for direct replica reads

## See Also
- zerops://themes/services — Valkey, KeyDB service cards and wiring
- zerops://decisions/choose-database
49 changes: 49 additions & 0 deletions apps/docs/content/guides/choose-database.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: "Choosing a Database on Zerops"
description: "**Use PostgreSQL** for everything unless you have a specific reason not to. It's the best-supported database on Zerops with full HA, read replicas, and pgBouncer."
---


## Keywords
database, postgresql, mariadb, clickhouse, sql, relational, columnar, analytics, postgres, mysql, choose database, which database

## TL;DR
**Use PostgreSQL** for everything unless you have a specific reason not to. It's the best-supported database on Zerops with full HA, read replicas, and pgBouncer.

## Decision Matrix

| Need | Choice | Why |
|------|--------|-----|
| **General-purpose** | **PostgreSQL** (default) | Full HA, read replicas, pgBouncer, best Zerops support |
| MySQL compatibility | MariaDB | MaxScale routing, async replication |
| Analytics / OLAP | ClickHouse | Columnar storage, ReplicatedMergeTree, 4 protocol ports |

## PostgreSQL (Default Choice)

- HA: 3 nodes (1 primary + 2 replicas)
- Ports: 5432 (primary), 5433 (read replicas), 6432 (external TLS via pgBouncer)
- Connection: `postgresql://${user}:${password}@${hostname}:5432/${db}`
- Read scaling: Use port 5433 for read-heavy workloads

## MariaDB

- HA: MaxScale routing with async replication
- Port: 3306
- Connection: `mysql://${user}:${password}@${hostname}:3306/${db}`
- Use when: Application requires MySQL wire protocol

## ClickHouse

- HA: 3 data nodes, replication factor 3
- Ports: 9000 (native), 8123 (HTTP), 9004 (MySQL), 9005 (PostgreSQL)
- Requires `ReplicatedMergeTree` engine in HA mode
- Use when: Analytics, time-series, OLAP workloads

## Gotchas
1. **HA mode is immutable**: Cannot switch HA/NON_HA after creation — delete and recreate
2. **No internal TLS**: Use `http://hostname:port` internally — VPN provides encryption
3. **PostgreSQL URI scheme**: Some libraries need `postgres://` not `postgresql://` — create a custom env var

## See Also
- zerops://themes/services — PostgreSQL, MariaDB, ClickHouse service cards and wiring
- zerops://decisions/choose-cache
51 changes: 51 additions & 0 deletions apps/docs/content/guides/choose-queue.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: "Choosing a Message Queue on Zerops"
description: "**Use NATS** for most cases (simple, fast, JetStream persistence). Use **Kafka** only for enterprise event streaming with guaranteed ordering and unlimited retention."
---


## Keywords
queue, message queue, kafka, nats, event, stream, pub-sub, broker, choose queue, which queue, messaging

## TL;DR
**Use NATS** for most cases (simple, fast, JetStream persistence). Use **Kafka** only for enterprise event streaming with guaranteed ordering and unlimited retention.

## Decision Matrix

| Need | Choice | Why |
|------|--------|-----|
| **General messaging** | **NATS** (default) | Simple auth, JetStream built-in, fast |
| Enterprise event streaming | Kafka | SASL auth, 3-broker HA, unlimited retention |
| Lightweight pub/sub | NATS | Low overhead, 8MB default messages |
| Event sourcing / audit logs | Kafka | Indefinite topic retention, strong ordering |

## NATS (Default Choice)

- Ports: 4222 (client), 8222 (HTTP monitoring)
- Auth: user `zerops` + auto-generated password
- Connection: `nats://${user}:${password}@${hostname}:4222`
- JetStream: Enabled by default (`JET_STREAM_ENABLED=1`)
- Storage: Up to 40GB memory + 250GB file store
- Max message: 8MB default, 64MB max (`MAX_PAYLOAD`)
- Health check: `GET /healthz` on port 8222
- **Config changes require restart** (no hot-reload)

## Kafka

- Port: 9092 (SASL PLAIN auth)
- Auth: `user` + `password` env vars (auto-generated)
- Bootstrap: `${hostname}:9092`
- HA: 3 brokers, 6 partitions, replication factor 3
- Storage: Up to 40GB RAM + 250GB persistent
- Topic retention: **Indefinite** (no time or size limits)
- Schema Registry: Port 8081 (if enabled)

## Gotchas
1. **NATS config changes need restart**: No hot-reload — changing env vars requires service restart
2. **Kafka single-node has no replication**: 1 broker = 3 partitions but zero redundancy
3. **NATS JetStream HA sync interval**: 1-minute sync across nodes — brief data lag possible
4. **Kafka SASL only**: No anonymous connections — always use the generated credentials

## See Also
- zerops://themes/services — NATS, Kafka service cards and wiring
- zerops://decisions/choose-database
Loading
Loading