skills/deploy-server/SKILL.md ``` name: deploy-server description: Use this skill to provision or update a Welley server on a supported cloud (currently DigitalOcean) when the user asks to deploy a new instance or run a server update. Covers single-instance deployments and SSH-based updates, including interactive collection of missing credentials, SSH keys, domain, region, and sizing. ``` # Deploy Server ## Overview This skill provisions a single-instance Welley stack on DigitalOcean and can run updates over SSH. It guides the agent to gather missing inputs, fetch available instance specs from a defined source, create the VM, configure the host, deploy with `wlycli deployment create`, and validate. Command names must match the CLI: - `wlycli deployment create` - `wlycli deployment update` - `wlycli deployment delete` ## Defaults (DigitalOcean) - Region: `nyc3` - Image: `latest LTS available via API` (resolve via `images?type=distribution`) - Droplet name: randomly generated as `wlsly--` (e.g., `wlsly-bright-hopper`). Use the name generator snippet from `references/digitalocean.md`. Use the same name for the managed DB if created. - Plan target: Basic / Regular (Shared CPU) - Size target: 2 vCPU / 4 GB RAM / 80 GB SSD / 4 TB transfer (~$24/mo) ## Default Override Rule If the user specifies any of the defaults (region, image, droplet name, project, size), the agent must: 1. Echo the override back verbatim. 2. Ask for confirmation before proceeding. ## Interactive Input Template Ask only what’s missing. Use these exact prompts to reduce ambiguity. Ask in iterative rounds. Round 1: max 2 questions per turn. Round 2 may present a bundled “accept all defaults” option. If the runtime supports choices, present 2–3 options per question (A/B/C). Avoid free-form unless necessary. Round 1 questions (identity + access): 1. “Do you have a DigitalOcean API token to use, or should I wait for you to provide one?” 2. “Which domain should this server use (e.g., `super-puper.example.com`)? Do you control its DNS?” 3. “Do you already have an SSH public key to use, or should I create and upload one?” 4. “Do you want to assign this droplet to an existing DigitalOcean project? If yes, provide the project name or ID.” 5. “DNS requirement: you must create A/AAAA records for `` and `storage.` (if no CDN). Can you do that before certbot runs?” 6. “Email DNS requirement: the `--smtp-from` domain needs SPF, DMARC and DKIM records, or invitation and confirmation emails to new recipients may be silently dropped. Are they in place, or should I list the records your SMTP provider needs?” 7. If the user asks where to get a DO token, provide this link: ``` https://cloud.digitalocean.com/account/api/tokens ``` Round 2 questions (sizing + location + deploy options): 1. “Do you want the default region `nyc3`, or a different region?” 2. “Do you want the default size (2 vCPU / 2 GB), or a different CPU/RAM target?” 3. “Should I use the latest Ubuntu LTS available in the API, or a specific Ubuntu version?” 4. “Which SSH user should we use after bootstrap (default: `wlsly`)? You can keep the default.” 5. “Do you want a managed Postgres DB, or use the bundled Postgres container?” 6. If managed Postgres is chosen: “Which DB region and size should I use? I can default to `nyc3` and 1 vCPU / 2 GB.” 7. If managed Postgres is chosen: “Do you want me to proceed with creation at the provider’s listed monthly price?” 8. “Please provide required SMTP settings (host, user, password, from). OTEL is optional — provide it only if needed.” 9. “Admin email is required for create. Provide it now (flag: `--email` / `-e`). If not provided, I will prompt twice to confirm it.” Offer an “accept all defaults” option in Round 2 to speed up input collection. If any answer is missing, ask a focused follow-up (one or two questions). ## Workflow (DigitalOcean, single instance) 1. **Collect required inputs (ask if missing).** - DigitalOcean API token - Project name or ID (optional; if missing or not found, use default project) - Region - Instance size target (e.g., 4 vCPU / 16 GB RAM) - Domain name - SSH public key (or confirmation to create and register one) - Admin email for first login and password recovery (required for create; `--email` / `-e`). If missing, prompt twice and confirm match. - Optional: custom TLS certs vs certbot - DNS access confirmation (must be able to create A/AAAA records for `` and `storage.` if used) 1. **Fetch instance types and pick a size.** - Use the sizes snippet from `references/digitalocean.md` to list and filter by CPU/RAM. - Confirm with the user if multiple matches. 2b. **Resolve Ubuntu image slug.** - Call `GET /v2/images?type=distribution&per_page=200` and filter for Ubuntu. - If the user asked for a specific version, verify its slug exists and confirm. - Otherwise pick the latest LTS slug from the list and confirm with the user before creation. 1. **Create the droplet.** - Use the DigitalOcean API directly (doctl is optional). - Image: prefer user-specified; otherwise pick the latest LTS from the API list. - Attach the SSH key fingerprint or create and upload a key. - Enable monitoring. - If project name/ID was provided but not found, create the droplet without a project (default project). - Note: Managed DB creation can run in parallel with droplet creation. ## Parallelization Guidance The agent should overlap independent tasks to reduce wall‑clock time: - While gathering user inputs (Round 2), start droplet creation if token, region, size, image, and SSH key are already confirmed. - While droplet is provisioning, start managed Postgres creation (if requested). - While droplet is starting, continue collecting remaining inputs (SMTP, OTEL, admin email). - If any critical input is missing for a step, pause that step and continue with other parallelizable tasks. - If DNS is managed in DigitalOcean, create/update A/AAAA records as soon as the droplet IP is available (do not wait for the full deployment step). 1. **DNS / domain.** - Ask whether to create or update DNS in DigitalOcean or leave external DNS. - If using DigitalOcean DNS, create A/AAAA records to the droplet IP. 1. **Provision the host via SSH.** - Create a non-root user (e.g., `wlsly`), grant sudo, and copy SSH authorized keys. - **Exit root and re-login as the non-root user** before continuing. - Install Docker + docker compose. - Fetch the Welley CLI binary from AppStore and make it executable: - Base URL: `https://appstore.allroads.social/v1/social.wellesley.wlycli/latest?develop=stable&platform=/` - Choose the correct `platform` value by OS/arch: - Linux x86\_64: `linux/amd64` - Linux arm64: `linux/arm64` - macOS x86\_64: `darwin/amd64` - macOS arm64: `darwin/arm64` - Windows x86\_64: `windows/amd64` - Example (Linux x86\_64): ``` curl -fL 'https://appstore.allroads.social/v1/social.wellesley.wlycli/latest?develop=stable&platform=linux/amd64' -o wlycli chmod +x wlycli ``` - Expected response headers include: - `x-artifact-display-name: Welley CLI` - `x-artifact-full-name: social.wellesley.wlycli` - `x-artifact-kind: cli` - `x-artifact-platform: /` such as `linux/amd64` - `x-artifact-sha256: ` - `x-artifact-version: ` - `x-channel: develop` - Before installing, prefer checking headers with a HEAD request and verify that artifact name, kind, platform, and channel match the requested binary. - Verify CLI binary: run `wlycli -v` and `wlycli deployment --help` to confirm expected build info. 1. **Deploy Welley.** - Run `wlycli deployment create` with the required options. - Use single-instance defaults (no external Postgres unless user asks). - If managed Postgres is requested, create it first (DigitalOcean DB) and then use the returned host/port/db/user/password in `--postgres-*` flags. - If the DB user password is not returned by the API/UI, reset the DB user auth to obtain fresh credentials before running `wlycli`. - Non-interactive runs must pass `--email` / `-e` or the command will fail. - `--base-dir` points to the parent directory; the CLI creates the project subdirectory automatically. - The CLI refuses to run if the project directory or containers already exist; cleanup is required before retrying. 1. **Certificates (mandatory).** - Default: use Certbot standalone (non-interactive). - **User must point DNS A/AAAA records for the domain (and `storage.` if used) to the droplet IP before certbot runs.** - CLI supports `--public-ip` (override detected IP) and `--skip-dns-preflight` if you must bypass the check. - **Never assume DNS is configured. Ask and wait for explicit confirmation.** - **Never use `--skip-dns-preflight` unless the user explicitly requests it.** - If CDN is not configured, certbot must include `storage.` in the certificate request. - Use self-signed certificates only if the user explicitly asks and pass `--self-signed-certs` (generates self-signed certs in Go). - Provide the user with DNS setup instructions and wait for confirmation: - “Create A/AAAA records so `` resolves to the droplet IP.” - “If `storage.` is used, create an A/AAAA record for it too.” - “Once DNS propagates, tell me and I’ll continue. I cannot observe propagation unless you ask me to verify resolution.” - The agent may verify DNS propagation itself using `ping -c 1 ` and compare the IP to the droplet. If it matches, proceed. 7b. **Email DNS (SPF / DMARC / DKIM).** - Mail from a domain without SPF and DMARC is often dropped by Gmail and Outlook for first-time recipients, even though the SMTP provider reports it as delivered. - `deployment create` prints a `Warning:` when the `--smtp-from` domain has no SPF record, or no DMARC record at `_dmarc.` or a parent domain. It does not block the deploy. Never ignore the warning: show it to the user and give them the records to add: - `` TXT `v=spf1 include: ~all` (the exact include comes from the SMTP provider; merge into an existing SPF record, never add a second one) - `_dmarc.` TXT `v=DMARC1; p=none; rua=mailto:dmarc@` - DKIM: the `._domainkey.` records the SMTP provider generates when the domain is verified - Check with `dig +short TXT ` and `dig +short TXT _dmarc.`. - The CLI cannot check DKIM (the selector is provider-specific). Always ask the user to confirm DKIM is verified at the SMTP provider, even when `create` printed no warning. With Amazon SES and similar providers, SPF is checked for the provider's own envelope domain, so only DKIM can make DMARC pass. In SES: _Verified identities → `` → DKIM_ must read _Successful_. 1. **Validate and report.** - Check `docker ps` and service health endpoints. - Provide final connection details and next steps. ## SSH Update Flow When user asks for “update server”: 1. Ask for target host/IP and SSH user/key. 2. Run `wlycli deployment update --domain [--force-restart]`. 3. Verify containers and Nginx status. ## Resources - DigitalOcean API snippets and examples: `references/digitalocean.md`. ## Secrets to Flags Use these flag names for secrets: - SMTP host: `--smtp-host` - SMTP user: `--smtp-user` - SMTP password: `--smtp-password` - SMTP from: `--smtp-from` - OTEL endpoint: `--otel-endpoint` - OTEL headers: `--otel-headers` ## Minimal Create Command Required flags: - `--domain` - `--email` - `--smtp-host` - `--smtp-user` - `--smtp-password` - `--smtp-from` Minimal example: ``` wlycli deployment create \ --domain example.com \ --email admin@example.com \ --smtp-host smtp.example.com \ --smtp-user user \ --smtp-password pass \ --smtp-from noreply@example.com ``` ## Managed Postgres Defaults (DigitalOcean vs CLI) When using DigitalOcean managed Postgres, do not rely on CLI defaults. Set all `--postgres-*` flags explicitly: - DO typical defaults: - Port: `25060` - Database: `defaultdb` - User: `doadmin` - CLI defaults (if you omit flags): - Port: `5432` - Database: `wlsly` - User: `wlsly` ## CLI Usage Notes - Version check: `wlycli -v` or `wlycli --version` (flag). There is no `version` subcommand. - Use `--base-dir` only if you want a non-default location (default is `~/.welley`). The CLI will create the directory if it has permission. - Non-interactive SSH runs should include `--email` and any required flags to avoid prompts. - If the user provides a CLI binary on the server (e.g. `~/wcli`), use it directly. Do not relocate it unless asked. - **Copy snippets verbatim** from `references/digitalocean.md` for all DO API calls. Do not rewrite, adapt, or improvise the Python parsing — even small changes (e.g. escaping, quoting) can break across shells. - **SSH remote scripts:** Always use `ssh host 'bash -s' << 'EOF'` (quoted heredoc). Never use `ssh host "..."` with double quotes — zsh expands `$(...)` and `[...]` locally before sending to the remote host. ## Failure Recovery If `deployment create` fails and you need to retry: - Use `wlycli deployment delete --domain --yes-please-delete-all-the-data` to clean up. - Delete will attempt container-based cleanup of root-owned files. If it still fails, remove the project directory under `--base-dir` with elevated permissions and re-run. ## Preflight Port-80 Test (Optional) If certbot fails, test port 80 reachability before re-running: ``` docker run --rm -d -p 80:80 --name test-http nginx:alpine curl -s http://:80 docker rm -f test-http ``` ## Other documents - [**Welley CLI** docs/index.md](https://welley.com/administrators/docs/cli/) - [**Quick Start: Minimal Welley Deployment** docs/quick-start.md](https://welley.com/administrators/docs/quick-start/) - [**openai.yaml** skills/deploy-server/agents/openai.yaml](https://welley.com/administrators/docs/deploy-server-openai-yaml/) - [**DigitalOcean Reference** skills/deploy-server/references/digitalocean.md](https://welley.com/administrators/docs/deploy-server-digitalocean/)