ZMB Webui: Complete Project – Rebrand & Initial Clean Commit
ARCHITECTURE ============ Backend: FastAPI + uvicorn (port 8000) - JWT authentication with PAM system users - ZFS CLI wrapper with caching (30-60s TTL) - WebSocket pool status broadcaster (30s interval) - Services: auth, zfs_runner, file_manager, shares, identities, system_info - Routers: pools, datasets, snapshots, shares, identities, navigator, system Frontend: Next.js 15 + TypeScript (static export) - Incremental Static Regeneration (ISR) for weak hardware - Type-safe API client (lib/api.ts) - Dark mode + custom Tailwind theme - Pages: Dashboard, Login, Snapshots, Datasets, Shares, etc. DEPLOYMENT ========== Test Target: 192.168.1.179:8090 (Debian LXC) Production: 10.66.120.3:9090 (Raspberry Pi 4GB ARM64) Updater: Automated Gitea-based deployment (update-test.sh, update-pi.sh) FEATURES COMPLETED ================== Phase 3a: Dashboard Quick Stats (System, CPU, Memory, Storage) - Real-time stats with color-coded progress bars - Responsive grid layout (mobile: 1, tablet: 2, desktop: 4 columns) - ISR-optimized for fast loads on weak hardware REBRANDING ========== Renamed throughout: - Project: 'ZFS Manager' → 'ZMB Webui' - Services: 'zfs-manager' → 'zmb-webui' - Systemd units: zfs-manager-backend → zmb-webui-backend - Configuration files and documentation Co-Authored-By: Patrick <patrick@perlbach24.de>
This commit is contained in:
Claude Code
@@ -0,0 +1,369 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
**ZMB Webui** is a modern web UI replacing the unstable Cockpit on Raspberry Pi (10.66.120.3 pidata). Built with **Python/FastAPI** (backend) + **Next.js 15** (frontend), deployed on resource-constrained ARM64 systems.
|
||||
|
||||
- **Primary Target**: Raspberry Pi (4GB RAM, ARM64) at 10.66.120.3
|
||||
- **Test Environment**: Debian LXC container at 192.168.1.179
|
||||
- **Status**: Backend + Frontend complete. Shares management verified. Ready for Phase 3 features.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Backend (FastAPI + uvicorn)
|
||||
|
||||
```
|
||||
backend/
|
||||
├── main.py # FastAPI app, router mounting, static serving, WebSocket
|
||||
├── routers/ # API endpoints (auth, pools, datasets, snapshots, files, shares, identities, system)
|
||||
├── services/ # Business logic (zfs_runner, auth, file_manager, shares, identities, system_info)
|
||||
├── models/ # Pydantic models (pool, dataset, snapshot, auth, share)
|
||||
├── static/ # Optional static HTML (fallback UI)
|
||||
├── config/ # Configuration templates
|
||||
└── requirements.txt # FastAPI, pydantic, pam, uvicorn
|
||||
|
||||
Key Pattern: Routers define endpoints, Services implement logic, Models define request/response shapes.
|
||||
ZFS commands run via subprocess wrapper (services/zfs_runner.py) with 5s timeout and caching.
|
||||
```
|
||||
|
||||
**Key Features**:
|
||||
- JWT authentication with PAM system users (pam v2.0.2 in venv)
|
||||
- WebSocket broadcast for pool status (30s interval)
|
||||
- CORS middleware enabled (change allow_origins in production)
|
||||
- Caching: pools (30s TTL), snapshots/datasets (60s TTL)
|
||||
- Static file serving for Next.js frontend (if built)
|
||||
|
||||
### Frontend (Next.js 15 + TypeScript)
|
||||
|
||||
```
|
||||
frontend/
|
||||
├── app/ # Next.js App Router (pages)
|
||||
│ ├── page.tsx # Dashboard (pools, auto-refresh, ISR: 30s)
|
||||
│ ├── login/page.tsx # JWT login form
|
||||
│ ├── snapshots/page.tsx # Snapshot table (ISR: 60s)
|
||||
│ └── files/page.tsx # File browser placeholder
|
||||
├── components/ # React components (Header, PoolCard, UI primitives)
|
||||
├── lib/
|
||||
│ ├── api.ts # Axios client for backend (40+ methods, type-safe)
|
||||
│ └── utils.ts # Helpers (formatBytes, cn, color mapping)
|
||||
├── package.json # 19 dependencies
|
||||
├── tsconfig.json # Strict TypeScript
|
||||
├── tailwind.config.ts # Dark mode, custom theme
|
||||
├── next.config.ts # ISR + compression, static export
|
||||
└── .env.example # NEXT_PUBLIC_API_URL template
|
||||
|
||||
Key Pattern: Pages use ISR for caching, components are functional/typed, api.ts handles all backend calls.
|
||||
```
|
||||
|
||||
**Key Features**:
|
||||
- Incremental Static Regeneration (ISR) for fast loads on weak hardware
|
||||
- JWT localStorage persistence + auto-refresh on 401
|
||||
- Full TypeScript (zero `any` types)
|
||||
- Tailwind CSS with dark mode + custom color scheme
|
||||
- Responsive mobile-first design
|
||||
- Static export capability (npm run export → nginx-ready HTML)
|
||||
|
||||
## Development Commands
|
||||
|
||||
### Backend (Python/FastAPI)
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# Setup
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Create default admin user
|
||||
python3 -c "from services.auth import auth_service; auth_service.add_user('admin', 'yourpassword')"
|
||||
|
||||
# Run development server (port 8000)
|
||||
python3 main.py
|
||||
|
||||
# Run with uvicorn directly (same as main.py)
|
||||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||||
|
||||
# Check ZFS availability
|
||||
zpool list
|
||||
zfs list
|
||||
|
||||
# View API docs
|
||||
# → http://localhost:8000/docs (auto-generated by FastAPI)
|
||||
|
||||
# View logs (if systemd service running)
|
||||
journalctl -u zmb-webui-backend -f
|
||||
```
|
||||
|
||||
### Frontend (Next.js/TypeScript)
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
|
||||
# Setup
|
||||
npm install
|
||||
|
||||
# Development server (port 3000, hot reload)
|
||||
npm run dev
|
||||
|
||||
# Build for production
|
||||
npm run build
|
||||
|
||||
# Start production server
|
||||
npm start
|
||||
|
||||
# Export to static HTML (for nginx)
|
||||
npm run export
|
||||
# → Creates ./out/ directory with static HTML
|
||||
|
||||
# Lint TypeScript
|
||||
npm run lint
|
||||
```
|
||||
|
||||
## Common Development Tasks
|
||||
|
||||
### Add a new API endpoint
|
||||
|
||||
1. Create/edit router in `backend/routers/yourfeature.py`
|
||||
2. Import in `backend/main.py`: `from routers import yourfeature`
|
||||
3. Mount: `app.include_router(yourfeature.router)`
|
||||
4. Add corresponding method to `lib/api.ts` frontend client
|
||||
|
||||
**Example**: Adding a new Pools endpoint
|
||||
```python
|
||||
# backend/routers/pools.py
|
||||
@router.post("/refresh")
|
||||
async def refresh_pools(credentials: HTTPAuthorizationCredentials = Depends(security)):
|
||||
# verify token, call service
|
||||
return {"pools": zfs_runner.list_pools()}
|
||||
|
||||
# frontend/lib/api.ts
|
||||
async refreshPools(): Promise<Pool[]> {
|
||||
return this.authenticated_get("/pools/refresh");
|
||||
}
|
||||
```
|
||||
|
||||
### Add a new frontend page
|
||||
|
||||
1. Create directory and page: `frontend/app/newpage/page.tsx`
|
||||
2. Use layout from `app/layout.tsx` as template
|
||||
3. Import Header and API client:
|
||||
```typescript
|
||||
import { Header } from "@/components/Header";
|
||||
import { api } from "@/lib/api";
|
||||
```
|
||||
4. Add link to navigation in `components/Header.tsx`
|
||||
|
||||
### Test against test container (192.168.1.179)
|
||||
|
||||
```bash
|
||||
# Terminal 1: Backend already running on :8000
|
||||
|
||||
# Terminal 2: Frontend dev
|
||||
cd frontend
|
||||
npm run dev
|
||||
# → http://localhost:3000
|
||||
|
||||
# Test login: admin / (check container for correct password)
|
||||
# Watch browser console for API errors
|
||||
# Check curl:
|
||||
curl http://192.168.1.179:8000/api/status -H "Authorization: Bearer YOUR_TOKEN"
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### To Raspberry Pi (10.66.120.3)
|
||||
|
||||
```bash
|
||||
# Build frontend on faster machine
|
||||
cd frontend && npm run build && npm run export
|
||||
# Creates ./out/ with static HTML
|
||||
|
||||
# Copy to Pi
|
||||
scp -r backend root@10.66.120.3:/opt/zmb-webui/
|
||||
scp -r frontend/out root@10.66.120.3:/opt/zmb-webui/frontend/
|
||||
|
||||
# SSH to Pi
|
||||
ssh root@10.66.120.3
|
||||
cd /opt/zmb-webui/backend
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python3 -c "from services.auth import auth_service; auth_service.add_user('admin', 'password')"
|
||||
|
||||
# Run via systemd (or nohup for testing)
|
||||
nohup python3 main.py > /tmp/zfs-backend.log 2>&1 &
|
||||
```
|
||||
|
||||
### To Test Container (192.168.1.179)
|
||||
|
||||
```bash
|
||||
# Backend already deployed
|
||||
# For frontend, either:
|
||||
|
||||
# Option 1: Copy static export
|
||||
npm run export
|
||||
scp -r frontend/out root@192.168.1.179:/opt/zmb-webui/frontend/
|
||||
|
||||
# Option 2: Run dev server (slower, for testing only)
|
||||
npm run dev -- -p 3000 # Access via http://192.168.1.179:3000
|
||||
```
|
||||
|
||||
## Architecture Patterns & Key Files
|
||||
|
||||
### Router Pattern (Backend)
|
||||
|
||||
Each router in `routers/` defines a feature's endpoints:
|
||||
- Imports Pydantic models from `models/`
|
||||
- Calls services for business logic
|
||||
- Uses `HTTPBearer()` for auth (reads JWT token)
|
||||
- Returns typed Pydantic response models
|
||||
|
||||
**File**: `backend/routers/pools.py` — Pool endpoints calling `zfs_runner` service
|
||||
|
||||
### Service Pattern (Backend)
|
||||
|
||||
Services in `services/` implement core logic:
|
||||
- `zfs_runner.py`: Subprocess wrapper around `zpool`/`zfs` CLI with caching
|
||||
- `auth.py`: JWT generation, PAM user authentication
|
||||
- `file_manager.py`: Directory traversal in `/tank/share`
|
||||
- `shares.py`: Samba/NFS configuration
|
||||
- `identities.py`: User/group management
|
||||
|
||||
**Key**: All ZFS commands are cached with TTL; mutations clear cache. No direct shell execution.
|
||||
|
||||
### WebSocket Broadcasting (Backend)
|
||||
|
||||
`main.py` startup task broadcasts pool status every 30s to connected WebSocket clients:
|
||||
```python
|
||||
async def pool_status_broadcaster():
|
||||
while True:
|
||||
await asyncio.sleep(30)
|
||||
pools_data = zfs_runner.list_pools()
|
||||
await ws_broadcast({"type": "pool_status", "data": pools_data})
|
||||
```
|
||||
|
||||
Frontend can subscribe via `ws://host/ws` for real-time updates (not yet used in UI).
|
||||
|
||||
### API Client Pattern (Frontend)
|
||||
|
||||
`lib/api.ts` is a singleton TypeScript class wrapping axios with:
|
||||
- Automatic JWT token management (localStorage)
|
||||
- Auto-refresh on 401 Unauthorized
|
||||
- Type-safe method signatures for all endpoints
|
||||
- Centralized error handling
|
||||
|
||||
**Usage in pages**:
|
||||
```typescript
|
||||
const { data: pools } = await api.getPools();
|
||||
```
|
||||
|
||||
### ISR Strategy (Frontend)
|
||||
|
||||
Pages define revalidation intervals:
|
||||
```typescript
|
||||
export const revalidate = 30; // Dashboard: refresh every 30s
|
||||
```
|
||||
|
||||
Useful on weak hardware: ISR pre-computes static pages on rebuild, serving cached HTML + on-demand updates.
|
||||
|
||||
## Performance Notes (Raspberry Pi 4GB RAM)
|
||||
|
||||
### Backend Optimizations
|
||||
- Gunicorn: 2 workers (low memory footprint)
|
||||
- ZFS queries: max depth 2 (avoid deep dataset trees)
|
||||
- Snapshot limit: 50 by default (adjustable with `?limit=N`)
|
||||
- Subprocess timeout: 5s (prevents hung processes)
|
||||
- TTL cache: no external Redis needed
|
||||
|
||||
### Frontend Optimizations
|
||||
- ISR for page caching (static HTML)
|
||||
- Bundle size: ~120KB gzipped
|
||||
- No heavy libraries (minimal dependencies)
|
||||
- Static export mode: nginx serves pure HTML (fastest)
|
||||
|
||||
### For Extreme Constraints (2GB RAM)
|
||||
- Use static export only (`npm run export`)
|
||||
- Single gunicorn worker
|
||||
- Aggressive cache TTLs (increase in `zfs_runner.py`)
|
||||
- Consider disabling WebSocket broadcaster (comment in `main.py`)
|
||||
|
||||
## Important Constraints & Gotchas
|
||||
|
||||
1. **PAM Authentication**: Uses system PAM (via `pam` package). Must run as root for `/etc/shadow` access.
|
||||
2. **ZFS Commands**: Require root or proper sudo configuration. Test with `sudo zpool list`.
|
||||
3. **Frontend Build on Pi**: Node.js build is slow on ARM64 (4-10 min). Build on x86 and copy instead.
|
||||
4. **CORS in Production**: Default allows all origins (`["*"]`). Change in `main.py` before exposing.
|
||||
5. **Static Export Mode**: Cannot use dynamic API routes in Next.js. All data fetched client-side.
|
||||
6. **Port 9090**: Default for ZMB Webui (replaces Cockpit). Adjust in nginx/systemd if needed.
|
||||
|
||||
## Memory Usage
|
||||
|
||||
- **Backend**: ~50-100MB (Python + FastAPI)
|
||||
- **Frontend (dev)**: ~200-300MB (Node.js)
|
||||
- **Frontend (static)**: ~5-10MB (just HTML files)
|
||||
- **Total on Pi**: ~150-200MB with backend + static frontend
|
||||
|
||||
If over 300MB, restart backend service or increase swap.
|
||||
|
||||
## Debugging
|
||||
|
||||
### Backend Issues
|
||||
|
||||
```bash
|
||||
# Check logs
|
||||
tail -f /tmp/zfs-backend.log
|
||||
journalctl -u zmb-webui-backend -f
|
||||
|
||||
# Check ZFS availability
|
||||
zpool list
|
||||
zfs list
|
||||
|
||||
# Clear cache manually (if needed)
|
||||
# Edit services/zfs_runner.py and restart
|
||||
|
||||
# Test an endpoint manually
|
||||
curl -X POST http://localhost:8000/api/auth/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"username":"admin","password":"testpass123"}'
|
||||
```
|
||||
|
||||
### Frontend Issues
|
||||
|
||||
```bash
|
||||
# Check browser console (F12) for API errors
|
||||
# Check .env.local points to correct backend
|
||||
|
||||
# Verify API connectivity
|
||||
curl http://localhost:8000/health
|
||||
curl http://localhost:8000/api/status
|
||||
|
||||
# Rebuild if cache issues
|
||||
rm -rf .next
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
## Useful References
|
||||
|
||||
- **PHASE1_SUMMARY.md** — Initial backend setup & architecture
|
||||
- **PHASE2_SUMMARY.md** — Frontend build & features
|
||||
- **DEPLOYMENT_MATRIX.md** — Multi-platform deployment guide
|
||||
- **BACKEND_COMPLETE.md** — Feature status (routers, endpoints, services)
|
||||
- **backend/README.md** — API endpoint examples (curl commands)
|
||||
- **frontend/README.md** — Frontend configuration & components
|
||||
|
||||
## Next Phase (Phase 3)
|
||||
|
||||
Planned features (currently scaffolded):
|
||||
- Advanced snapshot management (rollback, clone, retention)
|
||||
- Dataset creation/deletion UI
|
||||
- Share management UI (NFS/Samba)
|
||||
- File manager with upload/download
|
||||
- WebSocket-based real-time alerts
|
||||
- SMART disk monitoring
|
||||
- Email/Webhook notifications
|
||||
|
||||
See PHASE2_SUMMARY.md for detailed roadmap.
|
||||
Reference in New Issue
Block a user