zmb-webui/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

**ZMB Webui** is a modern web UI replacing the unstable Cockpit on Raspberry Pi (10.66.120.3 pidata). Built with **Python/FastAPI** (backend) + **Next.js 15** (frontend), deployed on resource-constrained ARM64 systems.

- **Primary Target**: Raspberry Pi (4GB RAM, ARM64) at 10.66.120.3
- **Test Environment**: Debian LXC container at 192.168.1.179
- **Status**: Backend + Frontend complete. Shares management verified. Ready for Phase 3 features.

## Architecture

### Backend (FastAPI + uvicorn)

```
backend/
├── main.py                 # FastAPI app, router mounting, static serving, WebSocket
├── routers/                # API endpoints (auth, pools, datasets, snapshots, files, shares, identities, system)
├── services/               # Business logic (zfs_runner, auth, file_manager, shares, identities, system_info)
├── models/                 # Pydantic models (pool, dataset, snapshot, auth, share)
├── static/                 # Optional static HTML (fallback UI)
├── config/                 # Configuration templates
└── requirements.txt        # FastAPI, pydantic, pam, uvicorn

Key Pattern: Routers define endpoints, Services implement logic, Models define request/response shapes.
ZFS commands run via subprocess wrapper (services/zfs_runner.py) with 5s timeout and caching.
```

**Key Features**:
- JWT authentication with PAM system users (pam v2.0.2 in venv)
- WebSocket broadcast for pool status (30s interval)
- CORS middleware enabled (change allow_origins in production)
- Caching: pools (30s TTL), snapshots/datasets (60s TTL)
- Static file serving for Next.js frontend (if built)

### Frontend (Next.js 15 + TypeScript)

```
frontend/
├── app/                    # Next.js App Router (pages)
│   ├── page.tsx           # Dashboard (pools, auto-refresh, ISR: 30s)
│   ├── login/page.tsx     # JWT login form
│   ├── snapshots/page.tsx # Snapshot table (ISR: 60s)
│   └── files/page.tsx     # File browser placeholder
├── components/             # React components (Header, PoolCard, UI primitives)
├── lib/
│   ├── api.ts             # Axios client for backend (40+ methods, type-safe)
│   └── utils.ts           # Helpers (formatBytes, cn, color mapping)
├── package.json           # 19 dependencies
├── tsconfig.json          # Strict TypeScript
├── tailwind.config.ts     # Dark mode, custom theme
├── next.config.ts         # ISR + compression, static export
└── .env.example           # NEXT_PUBLIC_API_URL template

Key Pattern: Pages use ISR for caching, components are functional/typed, api.ts handles all backend calls.
```

**Key Features**:
- Incremental Static Regeneration (ISR) for fast loads on weak hardware
- JWT localStorage persistence + auto-refresh on 401
- Full TypeScript (zero `any` types)
- Tailwind CSS with dark mode + custom color scheme
- Responsive mobile-first design
- Static export capability (npm run export → nginx-ready HTML)

## Development Commands

### Backend (Python/FastAPI)

```bash
cd backend

# Setup
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Create default admin user
python3 -c "from services.auth import auth_service; auth_service.add_user('admin', 'yourpassword')"

# Run development server (port 8000)
python3 main.py

# Run with uvicorn directly (same as main.py)
uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Check ZFS availability
zpool list
zfs list

# View API docs
# → http://localhost:8000/docs (auto-generated by FastAPI)

# View logs (if systemd service running)
journalctl -u zmb-webui-backend -f
```

### Frontend (Next.js/TypeScript)

```bash
cd frontend

# Setup
npm install

# Development server (port 3000, hot reload)
npm run dev

# Build for production
npm run build

# Start production server
npm start

# Export to static HTML (for nginx)
npm run export
# → Creates ./out/ directory with static HTML

# Lint TypeScript
npm run lint
```

## Common Development Tasks

### Add a new API endpoint

1. Create/edit router in `backend/routers/yourfeature.py`
2. Import in `backend/main.py`: `from routers import yourfeature`
3. Mount: `app.include_router(yourfeature.router)`
4. Add corresponding method to `lib/api.ts` frontend client

**Example**: Adding a new Pools endpoint
```python
# backend/routers/pools.py
@router.post("/refresh")
async def refresh_pools(credentials: HTTPAuthorizationCredentials = Depends(security)):
    # verify token, call service
    return {"pools": zfs_runner.list_pools()}

# frontend/lib/api.ts
async refreshPools(): Promise<Pool[]> {
  return this.authenticated_get("/pools/refresh");
}
```

### Add a new frontend page

1. Create directory and page: `frontend/app/newpage/page.tsx`
2. Use layout from `app/layout.tsx` as template
3. Import Header and API client:
   ```typescript
   import { Header } from "@/components/Header";
   import { api } from "@/lib/api";
   ```
4. Add link to navigation in `components/Header.tsx`

### Test against test container (192.168.1.179)

```bash
# Terminal 1: Backend already running on :8000

# Terminal 2: Frontend dev
cd frontend
npm run dev
# → http://localhost:3000

# Test login: admin / (check container for correct password)
# Watch browser console for API errors
# Check curl:
curl http://192.168.1.179:8000/api/status -H "Authorization: Bearer YOUR_TOKEN"
```

## Deployment

### To Raspberry Pi (10.66.120.3)

```bash
# Build frontend on faster machine
cd frontend && npm run build && npm run export
# Creates ./out/ with static HTML

# Copy to Pi
scp -r backend root@10.66.120.3:/opt/zmb-webui/
scp -r frontend/out root@10.66.120.3:/opt/zmb-webui/frontend/

# SSH to Pi
ssh root@10.66.120.3
cd /opt/zmb-webui/backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Create system user for WebUI login (uses PAM authentication)
useradd -m -s /bin/bash webadmin
echo "webadmin:yourpassword" | chpasswd

# Run via systemd (or nohup for testing)
nohup python3 main.py > /tmp/zfs-backend.log 2>&1 &
```

### To Test Container (192.168.1.179)

```bash
# Backend already deployed
# For frontend, either:

# Option 1: Copy static export
npm run export
scp -r frontend/out root@192.168.1.179:/opt/zmb-webui/frontend/

# Option 2: Run dev server (slower, for testing only)
npm run dev -- -p 3000  # Access via http://192.168.1.179:3000
```

## Data Persistence

All user and configuration data is **persisted on the system** – not stored in the application. This means data survives application restarts, updates, and redeployments:

### System Users
- **Location**: `/etc/passwd`, `/etc/shadow`
- **Creation**: `useradd -m webadmin; echo "webadmin:password" | chpasswd`
- **Persistence**: ✅ Survives application updates, restarts, reinstalls
- **Management**: Via CLI (`useradd`, `userdel`, `passwd`) or WebUI (`/identities` page)
- **Note**: WebUI uses PAM authentication – users must exist as system users

### Samba Users
- **Location**: Samba TDB database (e.g., `/var/lib/samba/private/sam.tdb`) or Registry
- **Creation**: `smbpasswd -a username` or via WebUI (`/identities` → "Set Samba Password")
- **Persistence**: ✅ Survives application updates, restarts, reinstalls
- **Backup tip**: Backup `/var/lib/samba/private/` directory to preserve user database

### Samba Shares & Config
- **Location**: `/etc/samba/smb.conf` or Samba Registry (`net conf`)
- **Creation**: Via WebUI (`/shares` page) or CLI
- **Persistence**: ✅ Survives application updates, restarts, reinstalls
- **Editable**: Changes made in WebUI are written directly to config files

### NFS Exports
- **Location**: `/etc/exports`
- **Creation**: Via WebUI (`/shares` page) or CLI
- **Persistence**: ✅ Survives application updates, restarts, reinstalls
- **Reload**: `exportfs -r` is called automatically after changes

### ZFS Pools & Datasets
- **Location**: ZFS metadata (on-disk, hardware managed)
- **Creation**: Via WebUI (`/pools`, `/datasets` pages)
- **Persistence**: ✅ Persists across all updates and restarts (hardware level)
- **Note**: ZFS data is never modified by application code – only read

### Update Safety
When deploying updates (redeploying backend/frontend code):
1. System users are **NOT** affected – still exist, still work
2. Samba users are **NOT** affected – database persists
3. File shares are **NOT** affected – config files persist
4. All WebUI pages will continue to work with existing data

**No user re-creation required on updates.**

## Architecture Patterns & Key Files

### Router Pattern (Backend)

Each router in `routers/` defines a feature's endpoints:
- Imports Pydantic models from `models/`
- Calls services for business logic
- Uses `HTTPBearer()` for auth (reads JWT token)
- Returns typed Pydantic response models

**File**: `backend/routers/pools.py` — Pool endpoints calling `zfs_runner` service

### Service Pattern (Backend)

Services in `services/` implement core logic:
- `zfs_runner.py`: Subprocess wrapper around `zpool`/`zfs` CLI with caching
- `auth.py`: JWT generation, PAM user authentication
- `file_manager.py`: Directory traversal in `/tank/share`
- `shares.py`: Samba/NFS configuration
- `identities.py`: User/group management

**Key**: All ZFS commands are cached with TTL; mutations clear cache. No direct shell execution.

### WebSocket Broadcasting (Backend)

`main.py` startup task broadcasts pool status every 30s to connected WebSocket clients:
```python
async def pool_status_broadcaster():
    while True:
        await asyncio.sleep(30)
        pools_data = zfs_runner.list_pools()
        await ws_broadcast({"type": "pool_status", "data": pools_data})
```

Frontend can subscribe via `ws://host/ws` for real-time updates (not yet used in UI).

### API Client Pattern (Frontend)

`lib/api.ts` is a singleton TypeScript class wrapping axios with:
- Automatic JWT token management (localStorage)
- Auto-refresh on 401 Unauthorized
- Type-safe method signatures for all endpoints
- Centralized error handling

**Usage in pages**:
```typescript
const { data: pools } = await api.getPools();
```

### ISR Strategy (Frontend)

Pages define revalidation intervals:
```typescript
export const revalidate = 30; // Dashboard: refresh every 30s
```

Useful on weak hardware: ISR pre-computes static pages on rebuild, serving cached HTML + on-demand updates.

## Performance Notes (Raspberry Pi 4GB RAM)

### Backend Optimizations
- Gunicorn: 2 workers (low memory footprint)
- ZFS queries: max depth 2 (avoid deep dataset trees)
- Snapshot limit: 50 by default (adjustable with `?limit=N`)
- Subprocess timeout: 5s (prevents hung processes)
- TTL cache: no external Redis needed

### Frontend Optimizations
- ISR for page caching (static HTML)
- Bundle size: ~120KB gzipped
- No heavy libraries (minimal dependencies)
- Static export mode: nginx serves pure HTML (fastest)

### For Extreme Constraints (2GB RAM)
- Use static export only (`npm run export`)
- Single gunicorn worker
- Aggressive cache TTLs (increase in `zfs_runner.py`)
- Consider disabling WebSocket broadcaster (comment in `main.py`)

## Important Constraints & Gotchas

1. **PAM Authentication**: Uses system PAM (via `pam` package). Must run as root for `/etc/shadow` access. Users must exist as system users (`useradd`) – no local user database. Users created in WebUI are persisted to `/var/lib/samba/private/` (Samba users) or system groups.
2. **User Persistence**: System users (Linux) and Samba users are stored on disk and survive application updates. No user re-creation needed on redeployment.
3. **ZFS Commands**: Require root or proper sudo configuration. Test with `sudo zpool list`.
4. **Frontend Build on Pi**: Node.js build is slow on ARM64 (4-10 min). Build on x86 and copy instead.
5. **CORS in Production**: Configurable via the `ZMB_CORS_ORIGINS` environment variable (comma-separated list of allowed origins). Defaults to `["*"]` when unset (development). Set specific origins before exposing, e.g. `ZMB_CORS_ORIGINS=https://<host>:8090`. Note: when concrete origins are set, `allow_credentials` is automatically enabled; with the `"*"` wildcard it is disabled (CORS spec forbids combining the two). Configure it in the systemd unit (`deploy/zfs-manager-backend.service`).
6. **Static Export Mode**: Cannot use dynamic API routes in Next.js. All data fetched client-side.
7. **Port 8090**: Default for ZMB Webui (HTTPS via nginx). Adjust in nginx/systemd if needed.

## Memory Usage

- **Backend**: ~50-100MB (Python + FastAPI)
- **Frontend (dev)**: ~200-300MB (Node.js)
- **Frontend (static)**: ~5-10MB (just HTML files)
- **Total on Pi**: ~150-200MB with backend + static frontend

If over 300MB, restart backend service or increase swap.

## Debugging

### Backend Issues

```bash
# Check logs
tail -f /tmp/zfs-backend.log
journalctl -u zmb-webui-backend -f

# Check ZFS availability
zpool list
zfs list

# Clear cache manually (if needed)
# Edit services/zfs_runner.py and restart

# Test an endpoint manually
curl -X POST http://localhost:8000/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"testpass123"}'
```

### Frontend Issues

```bash
# Check browser console (F12) for API errors
# Check .env.local points to correct backend

# Verify API connectivity
curl http://localhost:8000/health
curl http://localhost:8000/api/status

# Rebuild if cache issues
rm -rf .next
npm run build
npm start
```

## Useful References

- **PHASE1_SUMMARY.md** — Initial backend setup & architecture
- **PHASE2_SUMMARY.md** — Frontend build & features
- **DEPLOYMENT_MATRIX.md** — Multi-platform deployment guide
- **BACKEND_COMPLETE.md** — Feature status (routers, endpoints, services)
- **backend/README.md** — API endpoint examples (curl commands)
- **frontend/README.md** — Frontend configuration & components

## Next Phase (Phase 3)

Planned features (currently scaffolded):
- Advanced snapshot management (rollback, clone, retention)
- Dataset creation/deletion UI
- Share management UI (NFS/Samba)
- File manager with upload/download
- WebSocket-based real-time alerts
- SMART disk monitoring
- Email/Webhook notifications

See PHASE2_SUMMARY.md for detailed roadmap.