feat: add reader vote widget and vote-service

The skills-review desk is static, so "which draft would you ship?" needs a
stateful counterpart. vote-service is a small Go API on its own pod backed by
a JSON file on a ReadWriteOnce PVC, with one active vote per skill per source
IP as the anti-abuse rule and CORS (ALLOWED_ORIGIN) as the caller boundary.

Deployment notes that differ from the obvious path, all confirmed against the
live cluster: the image is side-loaded with `ctr image import` plus
`imagePullPolicy: Never` because kubelet has no credentials for the Nexus ref;
the pod is pinned to `kubernets` because the hostpath PV takes a nodeAffinity
for whichever node first binds it; and public exposure is Caddy on the VPS,
not the cloudflared tunnel.

The ingress controller runs with `use-forwarded-headers` off, so nginx
overwrites X-Forwarded-For with its own peer — every visitor would collapse
into one voter and each skill would cap at one vote overall. Caddy stamps the
true remote address into X-Client-IP, which nginx forwards untouched, and
clientIP() reads that first. Scoped to this app rather than flipping the
global flag, which would change client-IP handling for every other ingress.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Marcos Paulo
2026-09-04 23:49:55 +00:00
parent d12d301a1a
commit e2bcfff5ab
14 changed files with 714 additions and 8 deletions
+16
View File
@@ -0,0 +1,16 @@
FROM golang:1.22-alpine AS build
WORKDIR /src
COPY go.mod ./
COPY main.go ./
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/vote-service . \
&& mkdir -p /out/data
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /out/vote-service /vote-service
# distroless has no shell/chown; carry a pre-owned dir from the build stage
# so the nonroot user (65532) can write votes.json even without a mounted
# PVC (e.g. local `docker run` smoke tests).
COPY --from=build --chown=nonroot:nonroot /out/data /data
VOLUME ["/data"]
EXPOSE 8080
ENTRYPOINT ["/vote-service"]
+152
View File
@@ -0,0 +1,152 @@
# vote-service
Tiny Go HTTP API backing the "prefer original / prefer improved" vote widget
on `skills-review/`. One binary, no external dependencies, one JSON file on
disk as the store — proportionate to workshop-scale traffic, not a general
voting platform.
## Why a separate service
`netcracker.pages.marcospaulo.dev.br` is a static Pages Server: it serves
files, it cannot run server code or remember state. Any real vote count needs
a small stateful service reachable from the visitor's browser, so this lives
outside the static repo and runs as its own pod.
## Anti-abuse: IP, not MAC
A MAC address is a link-layer detail; it never reaches a server across the
internet, so it cannot be used here. "Same source" is approximated by client
IP (`X-Forwarded-For` / `X-Real-IP` behind the ingress, else the raw remote
address). One IP holds at most one active vote per skill — casting again
updates that vote instead of stacking a second one. This is imperfect (NAT,
VPNs, shared networks collapse to one vote; IP changes let someone vote
again) but matches the ask and needs no cookies, accounts, or client secrets.
A `X-Voter-Id` header (a random id the frontend keeps in `localStorage`) is
layered on only so a browser can display "you already voted X" — it is never
trusted as the sole anti-abuse signal, since `localStorage` is trivially
resettable.
## API
| Method | Path | Body | Response |
| :--- | :--- | :--- | :--- |
| `GET` | `/api/votes` | — | `{ "tallies": { "<skillId>": { "original": n, "improved": n } } }` |
| `GET` | `/api/votes?skillId=X` | — | adds `"you": "original"\|"improved"` when the caller's IP already voted on `X` |
| `POST` | `/api/votes` | `{"skillId":"X","choice":"original"\|"improved"}` | `{"skillId","original","improved","you"}` |
| `GET` | `/healthz` | — | `200` |
## Run locally
```bash
go run . # PORT=8080 VOTE_DB_PATH=/tmp/votes.json ALLOWED_ORIGIN=http://localhost:4173
```
## Build and publish the image
Pushed to this cluster's Nexus registry (docker-hosted repo, anonymous read
already enabled cluster-wide — no `imagePullSecrets` needed). Push host and
pull host differ because Nexus is reached from a workstation via its NodePort
but from inside the cluster via its Service DNS name:
```bash
docker build -t localhost:30892/ai-for-dummies-vote-service:latest .
docker push localhost:30892/ai-for-dummies-vote-service:latest
# pods pull the same image as: nexus-service.nexus.svc.cluster.local:8082/ai-for-dummies-vote-service:latest
```
## Deploy (microk8s)
The `ai-for-dummies-vote-data` PVC uses `microk8s-hostpath`, whose PVs carry a
`nodeAffinity` for whichever node first binds them — so scheduling and storage
must agree on one node. This runs on `kubernets` (the control-plane node that
hosts the rest of the cluster's workloads), pinned via `nodeSelector` in
`deployment.yaml`.
kubelet's image pulls run in the *host* network namespace and there is no
`certs.d/hosts.toml` entry for `localhost:30892`, so a plain pull of the Nexus
ref fails (`no basic auth credentials`). Push to Nexus for a durable off-node
copy, then import straight into that node's containerd store and let
`imagePullPolicy: Never` skip the network pull entirely — the same pattern the
`pragent-webhook` image uses in this cluster:
```bash
docker save localhost:30892/ai-for-dummies-vote-service:latest -o /tmp/vote-service.tar
/snap/microk8s/current/bin/ctr --address /var/snap/microk8s/common/run/containerd.sock \
--namespace k8s.io image import /tmp/vote-service.tar
# use microk8s's own bundled ctr, not the host's — different containerd major
# versions speak incompatible client/server protocols (`unknown service
# containerd.services.streaming.v1.Streaming` otherwise)
microk8s kubectl apply -f deploy/deployment.yaml # namespace + Deployment + PVC + Service
microk8s kubectl apply -f deploy/ingress.yaml
microk8s kubectl -n ai-for-dummies rollout restart deploy ai-for-dummies-vote
```
Re-run the `docker save`/`ctr image import` pair after every image rebuild —
`imagePullPolicy: Never` means the cluster never fetches a newer tag on its
own, and a `rollout restart` is what picks the new image up.
## Public exposure
Public traffic reaches the cluster through **Caddy on the Oracle VPS over
Tailscale**, which is how all ~21 public hosts in this account are served
(`langfuse`, `pragent-dashboard`, `vault`, …) — *not* through the cloudflared
tunnel. The tunnel's public-hostname routes are dashboard-managed and the DNS
API token cannot write them, so the Caddy path is also the only one that can
be automated end to end.
```bash
cf-dns add ai-for-dummies-vote A 129.148.56.8 # DNS-only (grey cloud), like every other Caddy host
```
Caddy block (`/etc/caddy/Caddyfile` on the VPS, local copy `~/scripts/Caddyfile`):
```caddyfile
ai-for-dummies-vote.marcospaulo.dev.br {
tls {
dns cloudflare <CF_TOKEN>
}
reverse_proxy 100.74.17.70:80 {
header_up Host {host}
header_up X-Client-IP {remote_host}
}
}
```
It proxies to port `80` (not a NodePort): the cluster's nginx ingress runs on
`hostNetwork` on `kubernets` and routes by `Host`.
### Why `X-Client-IP`
The ingress controller runs with `use-forwarded-headers` **off** (the microk8s
default — `nginx-load-balancer-microk8s-conf` has no `data`). nginx therefore
*overwrites* `X-Forwarded-For` and `X-Real-IP` with its own downstream peer,
which is the VPS's tailnet address `100.67.25.57`. Every visitor would collapse
into one voter, and since one IP holds at most one active vote per skill, each
skill would only ever hold a single vote in total — the anti-abuse rule would
silently become a hard cap.
Rather than flip `use-forwarded-headers` globally (it would change client-IP
handling for every other ingress in the cluster), Caddy stamps the true remote
address into `X-Client-IP`, a non-standard header nginx forwards untouched, and
`clientIP()` reads it first. `header_up` sets it unconditionally, so a public
client cannot spoof it; the trust placed in it is exactly the trust already
placed in `X-Forwarded-For`.
Verified after deploy: requests from two distinct sources are recorded as two
separate votes rather than overwriting one another.
## Frontend wiring
`skills-review/index.html` sets `window.SKILLS_REVIEW_VOTE_API` to
`https://ai-for-dummies-vote.marcospaulo.dev.br`; keep it in sync with
`ALLOWED_ORIGIN` in `deployment.yaml`
(`https://netcracker.pages.marcospaulo.dev.br`), which is the real caller
boundary — CORS restricts which origin's browser code may call the API, not
which network can reach it.
`replicas: 1` and `strategy: Recreate` are deliberate: the store is one file
on one `ReadWriteOnce` PVC, so two pods writing it concurrently would race.
Scale up only after moving the store to something that supports concurrent
writers (e.g. SQLite on a shared volume with proper locking, or Postgres) —
not needed at this traffic scale.
+106
View File
@@ -0,0 +1,106 @@
# Namespace, image ref, and storage class confirmed against this cluster
# (microk8s, 2026-09-04). Image is pushed to Nexus for a durable, off-node
# copy (docker push localhost:30892/... — see README), but the Deployment
# below pulls it from the *node's local containerd image store* instead of
# over the network: kubelet's image pulls run in the host network namespace,
# which uses this node's public DNS resolver, not cluster CoreDNS, so
# `nexus-service.nexus.svc.cluster.local` is NOT resolvable for a plain pull
# (only for in-cluster builders like Kaniko, whose *build* pod runs in pod
# netns). The `microk8s-hostpath` PVC below also pins every pod to whichever
# node created it (`ai-workstation`, confirmed via the PV's nodeAffinity), so
# a single local `ctr image import` of the pushed tar is enough — see
# vote-service/README.md for the import command. `imagePullPolicy: Never`
# enforces that: no accidental network pull attempt, no ImagePullBackOff.
# `ai-for-dummies` did not exist yet as a namespace, so it is created below,
# matching the one-namespace-per-app pattern every other small app in this
# cluster uses (judge0, minio, pragent, …). No storageClassName set:
# microk8s's `hostpath-storage` addon is the default.
apiVersion: v1
kind: Namespace
metadata:
name: ai-for-dummies
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ai-for-dummies-vote
namespace: ai-for-dummies
labels:
app: ai-for-dummies-vote
spec:
replicas: 1 # single replica: the store is one JSON file on one PVC, not a shared DB
selector:
matchLabels:
app: ai-for-dummies-vote
strategy:
type: Recreate # avoid two pods writing the same PVC-backed file at once
template:
metadata:
labels:
app: ai-for-dummies-vote
spec:
# Pinned to `kubernets`: the image is imported straight into that node's
# containerd store (see README) and `microk8s-hostpath` PVs carry a
# nodeAffinity for whichever node first binds them, so scheduling and
# storage must agree on one node. `kubernets` is the control-plane node
# that hosts the rest of this cluster's workloads.
nodeSelector:
kubernetes.io/hostname: kubernets
securityContext:
fsGroup: 65532 # matches distroless "nonroot" uid/gid; without it the PVC mounts root-owned and the container can't write votes.json
containers:
- name: vote-service
image: localhost:30892/ai-for-dummies-vote-service:latest
imagePullPolicy: Never # image is side-loaded via `ctr image import`; never fetch over the network
ports:
- containerPort: 8080
env:
- name: PORT
value: "8080"
- name: VOTE_DB_PATH
value: /data/votes.json
- name: ALLOWED_ORIGIN
value: https://netcracker.pages.marcospaulo.dev.br
resources:
requests: { cpu: 10m, memory: 16Mi }
limits: { cpu: 100m, memory: 64Mi }
readinessProbe:
httpGet: { path: /healthz, port: 8080 }
initialDelaySeconds: 2
livenessProbe:
httpGet: { path: /healthz, port: 8080 }
initialDelaySeconds: 5
volumeMounts:
- name: data
mountPath: /data
securityContext:
runAsNonRoot: true
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
volumes:
- name: data
persistentVolumeClaim:
claimName: ai-for-dummies-vote-data
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ai-for-dummies-vote-data
namespace: ai-for-dummies
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: ai-for-dummies-vote
namespace: ai-for-dummies
spec:
selector:
app: ai-for-dummies-vote
ports:
- port: 80
targetPort: 8080
+31
View File
@@ -0,0 +1,31 @@
# Public exposure is required: the vote widget runs in each visitor's
# browser (client-side JS on a static Pages site), so it calls this API
# straight from the internet — it cannot reach a cluster-internal-only
# Service. CORS (ALLOWED_ORIGIN in deployment.yaml) is the real boundary:
# it restricts which origin's browser code may call the API, not which
# network can reach it.
#
# No `tls:` block here on purpose: TLS is terminated upstream by Caddy on the
# Oracle VPS, which reverse-proxies over Tailscale to this node's port 80
# (the nginx ingress runs on hostNetwork and routes by Host). That is how all
# ~21 public hosts in this account are served. Reaching this host publicly
# needs the DNS record plus the Caddy block — see vote-service/README.md:
# cf-dns add ai-for-dummies-vote A 129.148.56.8
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ai-for-dummies-vote
namespace: ai-for-dummies
spec:
ingressClassName: public
rules:
- host: ai-for-dummies-vote.marcospaulo.dev.br
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: ai-for-dummies-vote
port:
number: 80
+3
View File
@@ -0,0 +1,3 @@
module github.com/marcospaulo/ai-for-dummies/vote-service
go 1.22
+270
View File
@@ -0,0 +1,270 @@
// Command vote-service is a tiny, dependency-free HTTP API that lets the
// skills-review page (a static site) collect "prefer original / prefer
// improved" votes per submitted skill.
//
// It is intentionally minimal: one Go binary, no external dependencies, one
// JSON file on disk as the store. That fits the workshop scale of this
// feature (dozens of voters, not thousands) and keeps the container image
// and the Kubernetes footprint small.
//
// Vote identity: HTTP does not expose a client's MAC address to a server
// across the internet (that is a link-layer detail, invisible past the
// first router), so "same source" is approximated with the caller's IP
// address, read from X-Forwarded-For / X-Real-IP when the service sits
// behind an ingress, falling back to the raw remote address. One IP may
// hold at most one active vote per skill; casting a new choice updates that
// vote instead of adding a second one. A client-supplied X-Voter-Id header
// (a random id the frontend keeps in localStorage) is layered on top only
// to let a browser recognize and display its own prior vote — it is not
// trusted as the sole anti-abuse signal, since it is trivially resettable.
package main
import (
"encoding/json"
"errors"
"log"
"net"
"net/http"
"os"
"strings"
"sync"
)
type voteChoice string
const (
choiceOriginal voteChoice = "original"
choiceImproved voteChoice = "improved"
)
func (c voteChoice) valid() bool { return c == choiceOriginal || c == choiceImproved }
// store is the on-disk vote ledger. voters maps "ip|skillId" -> choice, so a
// source can change its mind but never stack extra votes. counts is kept in
// sync for O(1) tally reads.
type store struct {
mu sync.Mutex
path string
Voters map[string]voteChoice `json:"voters"`
Counts map[string]map[voteChoice]int `json:"counts"`
}
func loadStore(path string) (*store, error) {
s := &store{path: path, Voters: map[string]voteChoice{}, Counts: map[string]map[voteChoice]int{}}
data, err := os.ReadFile(path)
if errors.Is(err, os.ErrNotExist) {
return s, nil
}
if err != nil {
return nil, err
}
if len(data) == 0 {
return s, nil
}
decoded := struct {
Voters map[string]voteChoice `json:"voters"`
}{}
if err := json.Unmarshal(data, &decoded); err != nil {
return nil, err
}
s.Voters = decoded.Voters
for key, choice := range s.Voters {
skillID := key[strings.IndexByte(key, '|')+1:]
s.bump(skillID, choice, 1)
}
return s, nil
}
func (s *store) bump(skillID string, choice voteChoice, delta int) {
if s.Counts[skillID] == nil {
s.Counts[skillID] = map[voteChoice]int{}
}
s.Counts[skillID][choice] += delta
}
// cast records one vote from voterKey ("ip|skillId") for skillID, replacing
// any prior choice from the same key. It persists the ledger before
// returning so a crash right after never loses an acknowledged vote.
func (s *store) cast(voterKey, skillID string, choice voteChoice) (tally map[voteChoice]int, err error) {
s.mu.Lock()
defer s.mu.Unlock()
if prev, ok := s.Voters[voterKey]; ok {
if prev == choice {
return s.snapshot(skillID), nil
}
s.bump(skillID, prev, -1)
}
s.Voters[voterKey] = choice
s.bump(skillID, choice, 1)
if err := s.persist(); err != nil {
return nil, err
}
return s.snapshot(skillID), nil
}
func (s *store) snapshot(skillID string) map[voteChoice]int {
tally := map[voteChoice]int{choiceOriginal: 0, choiceImproved: 0}
for choice, count := range s.Counts[skillID] {
tally[choice] = count
}
return tally
}
func (s *store) all() map[string]map[voteChoice]int {
s.mu.Lock()
defer s.mu.Unlock()
out := make(map[string]map[voteChoice]int, len(s.Counts))
for skillID := range s.Counts {
out[skillID] = s.snapshot(skillID)
}
return out
}
func (s *store) persist() error {
data, err := json.Marshal(struct {
Voters map[string]voteChoice `json:"voters"`
}{s.Voters})
if err != nil {
return err
}
tmp := s.path + ".tmp"
if err := os.WriteFile(tmp, data, 0o600); err != nil {
return err
}
return os.Rename(tmp, s.path)
}
func clientIP(r *http.Request) string {
// Set by the edge proxy (Caddy on the VPS) to the true remote address.
// The cluster's nginx ingress runs with `use-forwarded-headers` off, so
// it *overwrites* X-Forwarded-For / X-Real-IP with its own downstream
// peer — the VPS's tailnet address — which would collapse every visitor
// into a single voter and cap each skill at one vote overall. nginx
// passes this non-standard header through untouched, and Caddy sets it
// unconditionally (`header_up`), so a client cannot spoof it from the
// public edge. Trust here is exactly the trust already placed in
// X-Forwarded-For below.
if edge := r.Header.Get("X-Client-IP"); edge != "" {
return strings.TrimSpace(strings.Split(edge, ",")[0])
}
if forwarded := r.Header.Get("X-Forwarded-For"); forwarded != "" {
return strings.TrimSpace(strings.Split(forwarded, ",")[0])
}
if real := r.Header.Get("X-Real-IP"); real != "" {
return real
}
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return host
}
func withCORS(allowedOrigin string, next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", allowedOrigin)
w.Header().Set("Vary", "Origin")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, X-Voter-Id")
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
next(w, r)
}
}
func main() {
dbPath := envOr("VOTE_DB_PATH", "/data/votes.json")
allowedOrigin := envOr("ALLOWED_ORIGIN", "https://netcracker.pages.marcospaulo.dev.br")
addr := ":" + envOr("PORT", "8080")
s, err := loadStore(dbPath)
if err != nil {
log.Fatalf("load vote store %s: %v", dbPath, err)
}
mux := http.NewServeMux()
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK) })
mux.HandleFunc("/api/votes", withCORS(allowedOrigin, func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
handleResults(w, r, s)
case http.MethodPost:
handleVote(w, r, s)
default:
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
}
}))
log.Printf("vote-service listening on %s (db=%s, origin=%s)", addr, dbPath, allowedOrigin)
log.Fatal(http.ListenAndServe(addr, mux))
}
// handleResults returns the tally for every skill. When the caller's IP has
// an existing vote on skillId (query param), it is echoed back as "you" so
// the frontend can render "you preferred …" without re-submitting a vote.
func handleResults(w http.ResponseWriter, r *http.Request, s *store) {
tallies := s.all()
skillID := strings.TrimSpace(r.URL.Query().Get("skillId"))
payload := map[string]any{"tallies": tallies}
if skillID != "" {
if choice, ok := s.mine(clientIP(r), skillID); ok {
payload["you"] = choice
}
}
writeJSON(w, http.StatusOK, payload)
}
func (s *store) mine(ip, skillID string) (voteChoice, bool) {
s.mu.Lock()
defer s.mu.Unlock()
choice, ok := s.Voters[ip+"|"+skillID]
return choice, ok
}
func handleVote(w http.ResponseWriter, r *http.Request, s *store) {
var body struct {
SkillID string `json:"skillId"`
Choice voteChoice `json:"choice"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<12)).Decode(&body); err != nil {
http.Error(w, "invalid request body", http.StatusBadRequest)
return
}
body.SkillID = strings.TrimSpace(body.SkillID)
if body.SkillID == "" || strings.ContainsAny(body.SkillID, "|") || !body.Choice.valid() {
http.Error(w, "skillId and a valid choice are required", http.StatusBadRequest)
return
}
ip := clientIP(r)
voterKey := ip + "|" + body.SkillID
tally, err := s.cast(voterKey, body.SkillID, body.Choice)
if err != nil {
log.Printf("persist vote: %v", err)
http.Error(w, "could not save vote", http.StatusInternalServerError)
return
}
writeJSON(w, http.StatusOK, map[string]any{
"skillId": body.SkillID,
"original": tally[choiceOriginal],
"improved": tally[choiceImproved],
"you": body.Choice,
})
}
func writeJSON(w http.ResponseWriter, status int, payload any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(payload)
}
func envOr(key, fallback string) string {
if value := os.Getenv(key); value != "" {
return value
}
return fallback
}