diff --git a/CLAUDE.md b/CLAUDE.md index 4d2e749..d734483 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -114,6 +114,8 @@ Reihenfolge: Testbasis, erster Lauf im Test-LXC (`pytest`), Korrekturrunde, Phas - Recovery-Meldungen sind in der UI schaltbar: globaler Schalter `alerts.recovery_enabled` (Default an) in der Section "Alarmierung" und pro Regel über die Ereignisarten (`kinds` enthält `recovery` oder nicht). Greift zusätzlich zur Regel "Recovery nur, wenn das Problem vorher gemeldet wurde". Ist Recovery aus, bleibt sie im Verlauf sichtbar, wird aber nicht gesendet. - Verzögerung (Phase 6b): Section "Alarmierung" bekommt `alerts.default_delay_minutes` (Werkseinstellung 15; ersetzt `poll.min_duration`). Jede Regel kann die Verzögerung überschreiben, auch mit 0. Passen mehrere Regeln, gilt die kleinste. Sie zählt ab der letzten Statusänderung, eine Verschlechterung startet sie neu; wird das Problem vorher OK, gibt es weder Alarm noch Recovery. Das Dashboard zeigt wartende Probleme ("wartet noch 8 min"). Mute und Downtime wirken auch während der Wartezeit. Host-Alarme und der Checkmk-Ausfall (Phase 6c) kommen immer sofort, Host-Verzögerung `alerts.host_delay_minutes` Standard 0. Die vorinstallierte Regel "ZFS Zpool" (`*zpool*`) hat Verzögerung 0, damit gilt ab Werk: Zpool sofort, alles andere nach 15 min. - Phase 6d (nach 6c): API-Selbsttest. Lesend beim Start, täglich 04:00 und sofort bei Checkmk-Versionswechsel: `/version`, Service- und Host-Abfrage (alle benötigten Spalten in der Antwort), Downtime-Abruf (GET). Ergebnis auf einer UI-Seite, Teilcheck `checkmk_api` in `/health`, Chat-Meldung bei Fehlschlag oder Versionswechsel (und bei Rückkehr zu ok). Unterscheidet "nicht erreichbar" von "Antwort unerwartet". + - Aufgenommen nach Prüfung des Referenzrepos `jlk/checkmk-llm-server` (MIT, nur als Referenz; es belegt weder Host-Live-Status noch die Downtime-Liste): (a) toleranter Antwort-Parser, der `value` oder `members` akzeptiert und `state_type` als int oder String (`hard`/`soft`) zulässt; (b) Downtime-Fallback: scheitert `GET /domain-types/downtime/collections/all`, ermittelt der Bot Downtimes aus `scheduled_downtime_depth > 0` der Service- und Host-Abfrage und meldet das im Selbsttest als Abweichung; (c) lesbare Fehlerhinweise je Status (401, 403, 404, 422, 5xx) statt nur `HTTP `. Die Spalten `last_state_change`, `is_flapping`, `state_type` und der Host-Endpunkt bleiben bis zum Selbsttest gegen 2.5 ungeprüft. + - Zweite Referenz `chexma/vibeMK` (GPL-3.0, nur Wissen, kein Code): Host-Live-Status dort per **GET** `/domain-types/host/collections/all` mit `columns=name,state,acknowledged,scheduled_downtime_depth`, State 0 clientseitig verworfen, Host-State 0/1/2 = UP/DOWN/UNREACH (bestätigt); Downtime-Liste per GET `/domain-types/downtime/collections/all` mit `query` (`host_name`, `is_service`), `downtime_type` als Listenfilter dort nicht belegt; `state_type` int 0 = soft, 1 = hard. Fallback im Selbsttest: scheitert der POST der Host-Abfrage, GET mit `columns` und clientseitigem Filter; scheitert der Downtime-Abruf mit `downtime_type`, GET mit `query` und `is_service`. - Entschieden nach Phase 6b: Erinnerung pro Regel: leer = Standard, 0 = aus, >0 = Stunden. Keine Recovery ohne vorherigen Alarm bleibt so. Migrationen laufen künftig in einer Transaktion pro Migration mit Rollback. `deploy/first-run.sh` richtet das Dev-venv ein, lässt `pytest` laufen und packt Log, Python-Version, Paketliste und Git-Stand ohne Secrets in ein Archiv. Zugriff auf den Test-LXC per SSH nur dort, eigener Benutzer ohne Root, jeder Befehl mit Freigabe. Nach dem ersten grünen `pytest` und vor der Abnahme: `docs/THREAT_MODEL.md` (STRIDE) und ein einmaliger Sicherheits-Review des gesamten Codes. - Vorgemerkt, noch NICHT entschieden (Phase 10, Idee): Chat mit dem Agenten ("Hermes") in der UI. Heute ist der Agent nur per Telegram/Matrix (`/ask`, `/explain`, Tiefenanalyse) und als Einzelfrage-Testlauf unter `/prompts` erreichbar. Vorschläge zur späteren Entscheidung: nur Admins (Schalter `ui.chat_viewers_allowed`, Standard aus); 6 Runden Kontext, Gespräche mit Gesprächs-ID in `agent_runs`, Nutzer sehen eigene Gespräche, Admins alle unter `/agent`, Aufbewahrung 90 Tage, Rate-Limit 5 pro Minute; Antwort nach Abschluss ohne Streaming ("läuft …" mit Abbrechen, max. 90 s), Tool-Schritte aufklappbar. Das Agentenmodell ist in der Section "Ollama" einstellbar (auch ein Hermes-Modell). Erst nach der Abnahme angehen. - Aktionen: Der Bot führt nie Shell-Befehle aus (weder lokal noch per SSH). Webhook pro Regel (POST JSON an Allowlist-URLs, Audit) kommt erst nach der Abnahme. diff --git a/app/agent/constants.py b/app/agent/constants.py index e10bcaa..42c60da 100644 --- a/app/agent/constants.py +++ b/app/agent/constants.py @@ -1,5 +1,8 @@ MAX_STEPS = 6 TOTAL_TIMEOUT = 90.0 +FINISH_RESERVE = 10.0 +MAX_TOOL_CALLS_PER_STEP = 3 +KEEP_FULL_TOOL_RESULTS = 3 TOOL_RESULT_MAX = 1500 ARG_MAX = 200 HOST_MAX = 128 diff --git a/app/agent/loop.py b/app/agent/loop.py index 6207802..8c75df6 100644 --- a/app/agent/loop.py +++ b/app/agent/loop.py @@ -7,7 +7,8 @@ import time from typing import Callable from .constants import (ABORT_MODEL_ERROR, ABORT_STEP_LIMIT, ABORT_TIMEOUT, ANSWER_MAX, FALLBACK_SNIPPET_MAX, - MAX_STEPS, ROLE_SYSTEM, ROLE_TOOL, ROLE_USER, STEP_ARG_LOG_MAX, TOTAL_TIMEOUT) + FINISH_RESERVE, KEEP_FULL_TOOL_RESULTS, MAX_STEPS, MAX_TOOL_CALLS_PER_STEP, ROLE_SYSTEM, + ROLE_TOOL, ROLE_USER, STEP_ARG_LOG_MAX, TOTAL_TIMEOUT) from .executor import ToolExecutor from .prompts import SUMMARY_REQUEST, SYSTEM_PROMPT, ask_prompt, explain_prompt from .sanitize import sanitize, wrap_data @@ -18,6 +19,8 @@ log = logging.getLogger(__name__) THINK_RE = re.compile(r".*?", re.DOTALL | re.IGNORECASE) NO_ANSWER = "Keine Antwort erhalten." +NOT_EXECUTED = "Nicht ausgeführt." +COMPACTED_RESULT = "[gekürzt] Ergebnis früherer Abfrage" MESSAGES = { ABORT_TIMEOUT: "Abgebrochen: Zeitlimit erreicht.", ABORT_STEP_LIMIT: "Abgebrochen: Schrittlimit erreicht.", @@ -42,6 +45,17 @@ def _clean_answer(text: object) -> str: return sanitize(THINK_RE.sub("", str(text or "")), ANSWER_MAX) or NO_ANSWER +def _not_executed(calls: list[dict]) -> list[dict]: + return [{"role": ROLE_TOOL, "tool_name": _arguments(call)[0], "content": NOT_EXECUTED} for call in calls] + + +def _compact_old_tool_results(messages: list[dict], keep: int = KEEP_FULL_TOOL_RESULTS) -> None: + tool_indexes = [i for i, m in enumerate(messages) if m.get("role") == ROLE_TOOL] + for i in tool_indexes[:-keep] if keep > 0 else tool_indexes: + if messages[i]["content"] not in (COMPACTED_RESULT, NOT_EXECUTED): + messages[i] = {**messages[i], "content": COMPACTED_RESULT} + + class Agent: def __init__(self, chat: ChatModel, model: str, executor: ToolExecutor, recorder: Recorder | None = None, clock: Callable[[], float] = time.monotonic, max_steps: int = MAX_STEPS, @@ -100,40 +114,46 @@ class Agent: return "\n".join(lines) async def _finish(self, reason: str, messages: list[dict], steps: list[AgentStep], - deadline: float) -> AgentResult: - if reason == ABORT_STEP_LIMIT: - try: - messages.append({"role": ROLE_USER, "content": SUMMARY_REQUEST}) - reply = await self._call(messages, deadline, None) - text = _clean_answer(reply.get("content")) - if text != NO_ANSWER: - return AgentResult(f"{text}\n({MESSAGES[reason]})", tuple(steps), reason) - except Exception: - log.warning("agent summary after step limit failed") + final_deadline: float) -> AgentResult: + try: + messages.append({"role": ROLE_USER, "content": SUMMARY_REQUEST}) + reply = await self._call(messages, final_deadline, None) + text = _clean_answer(reply.get("content")) + if text != NO_ANSWER: + return AgentResult(f"{text}\n({MESSAGES[reason]})", tuple(steps), reason) + except Exception: + log.warning("agent summary after %s failed", reason) return AgentResult(self._fallback(reason, steps), tuple(steps), reason) async def run(self, prompt: str) -> AgentResult: - deadline = self._clock() + self._timeout + start = self._clock() + final_deadline = start + self._timeout + deadline = final_deadline - FINISH_RESERVE messages: list[dict] = [{"role": ROLE_SYSTEM, "content": self._system_prompt}, {"role": ROLE_USER, "content": prompt}] steps: list[AgentStep] = [] while True: + last_step = len(steps) >= self._max_steps - 1 + tools = None if last_step else TOOL_SCHEMAS try: - reply = await self._call(messages, deadline, TOOL_SCHEMAS) + reply = await self._call(messages, deadline, tools) except asyncio.TimeoutError: - return AgentResult(self._fallback(ABORT_TIMEOUT, steps), tuple(steps), ABORT_TIMEOUT) + return await self._finish(ABORT_TIMEOUT, messages, steps, final_deadline) except Exception: log.warning("agent model call failed", exc_info=True) - return AgentResult(self._fallback(ABORT_MODEL_ERROR, steps), tuple(steps), ABORT_MODEL_ERROR) + return await self._finish(ABORT_MODEL_ERROR, messages, steps, final_deadline) calls = reply.get("tool_calls") or [] if not calls: return AgentResult(_clean_answer(reply.get("content")), tuple(steps)) + allowed, overflow = calls[:MAX_TOOL_CALLS_PER_STEP], calls[MAX_TOOL_CALLS_PER_STEP:] messages.append({"role": "assistant", "content": reply.get("content") or "", "tool_calls": calls}) - for call in calls: - if len(steps) >= self._max_steps: - return await self._finish(ABORT_STEP_LIMIT, messages, steps, deadline) - if self._clock() >= deadline: - return AgentResult(self._fallback(ABORT_TIMEOUT, steps), tuple(steps), ABORT_TIMEOUT) + for index, call in enumerate(allowed): + if len(steps) >= self._max_steps or self._clock() >= deadline: + messages.extend(_not_executed(allowed[index:] + overflow)) + reason = ABORT_STEP_LIMIT if len(steps) >= self._max_steps else ABORT_TIMEOUT + return await self._finish(reason, messages, steps, final_deadline) step, result = await self._run_tool(len(steps) + 1, call, deadline) steps.append(step) messages.append({"role": ROLE_TOOL, "tool_name": step.tool, "content": wrap_data(result)}) + _compact_old_tool_results(messages) + messages.extend(_not_executed(overflow)) diff --git a/app/agent/sanitize.py b/app/agent/sanitize.py index a2504aa..df0f229 100644 --- a/app/agent/sanitize.py +++ b/app/agent/sanitize.py @@ -5,6 +5,7 @@ from .constants import DATA_CLOSE, DATA_OPEN, TOOL_RESULT_MAX CONTROL_RE = re.compile(r"[\x00-\x08\x0b-\x1f\x7f-\x9f]") DELIMITER_RE = re.compile(r"<\s*/?\s*daten\s*>", re.IGNORECASE) WHITESPACE_RE = re.compile(r"\s+") +TRUNCATION_SUFFIX = " …[gekürzt]" def sanitize(text: object, limit: int = TOOL_RESULT_MAX, single_line: bool = False) -> str: @@ -15,7 +16,13 @@ def sanitize(text: object, limit: int = TOOL_RESULT_MAX, single_line: bool = Fal text = DELIMITER_RE.sub("", text) if single_line: text = WHITESPACE_RE.sub(" ", text) - return text.strip()[:limit] + text = text.strip() + if len(text) <= limit: + return text + cut = limit - len(TRUNCATION_SUFFIX) + if cut <= 0: + return text[:limit] + return text[:cut].rstrip() + TRUNCATION_SUFFIX def wrap_data(text: str) -> str: diff --git a/app/alerts/delivery.py b/app/alerts/delivery.py index 2150af6..88a5fae 100644 --- a/app/alerts/delivery.py +++ b/app/alerts/delivery.py @@ -2,7 +2,7 @@ import asyncio import logging from typing import Awaitable, Callable, Sequence -from ..chat import ChatAdapter, ChatError, MessageRef, Target +from ..chat import BACKEND_TELEGRAM, ChatAdapter, ChatError, MessageRef, Target, TELEGRAM_MESSAGE_MAX, truncate_html from .extras import Extras, ExtrasProvider log = logging.getLogger(__name__) @@ -29,7 +29,7 @@ class Deliverer: async def _send_one(self, target: Target, text: str, extras: ExtrasProvider | None, silent: bool) -> MessageRef | None: - full, options = _apply(text, extras(target.backend) if extras else None, silent) + full, options = _apply(target.backend, text, extras(target.backend) if extras else None, silent) try: return await self._call(target.backend, lambda adapter: adapter.send(target.target, full, **options)) except Exception as exc: @@ -37,7 +37,7 @@ class Deliverer: return None async def _edit_one(self, ref: MessageRef, text: str, extras: ExtrasProvider | None, silent: bool) -> None: - full, options = _apply(text, extras(ref.backend) if extras else None, silent) + full, options = _apply(ref.backend, text, extras(ref.backend) if extras else None, silent) try: await self._call(ref.backend, lambda adapter: adapter.edit(ref, full, **options)) except Exception as exc: @@ -60,13 +60,19 @@ class Deliverer: return await asyncio.wait_for(action(fresh), self._timeout) -def _apply(text: str, extra: Extras | None, silent: bool = False) -> tuple[str, dict]: +def _apply(backend: str, text: str, extra: Extras | None, silent: bool = False) -> tuple[str, dict]: options: dict = {"silent": True} if silent else {} if extra is None: return text, options if extra.buttons: options["buttons"] = extra.buttons - return (f"{text}\n{extra.hint}" if extra.hint else text), options + if not extra.hint: + return text, options + if backend == BACKEND_TELEGRAM: + # Reserviert Platz für den Hinweis, damit die anschließende Kürzung im Adapter ihn nicht abschneidet. + budget = max(TELEGRAM_MESSAGE_MAX - len(extra.hint) - 1, 0) + text = truncate_html(text, budget) + return f"{text}\n{extra.hint}", options def _brief(exc: Exception) -> str: diff --git a/app/chat/__init__.py b/app/chat/__init__.py index d627165..3ffe4fc 100644 --- a/app/chat/__init__.py +++ b/app/chat/__init__.py @@ -3,11 +3,11 @@ from . import callback_data from .matrix import BACKEND_MATRIX, MatrixAdapter, MatrixError from .registry import AdapterRegistry, build_adapters, build_matrix, build_telegram, default_targets from .targets import MATRIX_ROOM_ID_RE, MATRIX_USER_ID_RE, TELEGRAM_CHAT_ID_RE, valid_target -from .telegram import BACKEND_TELEGRAM, TelegramAdapter, TelegramError +from .telegram import BACKEND_TELEGRAM, MESSAGE_MAX as TELEGRAM_MESSAGE_MAX, TelegramAdapter, TelegramError, truncate_html __all__ = [ "AdapterRegistry", "BACKEND_MATRIX", "Button", "Buttons", "callback_data", "BACKEND_TELEGRAM", "ChatAdapter", "ChatError", "MATRIX_ROOM_ID_RE", "MATRIX_USER_ID_RE", "MatrixAdapter", "MatrixError", "MessageRef", "TELEGRAM_CHAT_ID_RE", "Target", - "TelegramAdapter", "TelegramError", "build_adapters", "build_matrix", "build_telegram", - "default_targets", "json_body", "mask_secret", "valid_target", + "TELEGRAM_MESSAGE_MAX", "TelegramAdapter", "TelegramError", "build_adapters", "build_matrix", "build_telegram", + "default_targets", "json_body", "mask_secret", "truncate_html", "valid_target", ] diff --git a/app/chat/matrix.py b/app/chat/matrix.py index 0837de0..20539fd 100644 --- a/app/chat/matrix.py +++ b/app/chat/matrix.py @@ -19,6 +19,8 @@ MAX_RETRY_AFTER = 30.0 DEFAULT_RETRY_AFTER = 1.0 STATUS_TOO_MANY = 429 ERROR_MAX = 300 +MAX_BODY_BYTES = 30000 +TRUNCATION_MARK = "…" class MatrixError(ChatError): @@ -29,9 +31,22 @@ def _enc(value: str) -> str: return quote(value, safe="") +def _truncate_bytes(text: str, limit: int = MAX_BODY_BYTES) -> str: + encoded = text.encode("utf-8") + if len(encoded) <= limit: + return text + mark_bytes = TRUNCATION_MARK.encode("utf-8") + budget = max(limit - len(mark_bytes), 0) + cut = encoded[:budget] + while cut and (cut[-1] & 0xC0) == 0x80: + cut = cut[:-1] + return cut.decode("utf-8") + TRUNCATION_MARK + + def _content(text: str, silent: bool = False) -> dict: body, formatted = to_matrix(text) - return {"msgtype": MSGTYPE_NOTICE if silent else MSGTYPE_TEXT, "body": body, "format": HTML_FORMAT, "formatted_body": formatted} + return {"msgtype": MSGTYPE_NOTICE if silent else MSGTYPE_TEXT, "body": _truncate_bytes(body), + "format": HTML_FORMAT, "formatted_body": _truncate_bytes(formatted)} class MatrixAdapter: diff --git a/app/chat/telegram.py b/app/chat/telegram.py index 1d99ed6..89cf040 100644 --- a/app/chat/telegram.py +++ b/app/chat/telegram.py @@ -1,4 +1,6 @@ import asyncio +import re +from html import unescape import httpx @@ -15,10 +17,61 @@ NOT_MODIFIED = "message is not modified" ALLOWED_UPDATES = ["message", "callback_query"] ANSWER_MAX = 200 ERROR_MAX = 300 +MESSAGE_MAX = 4096 +TRUNCATION_MARK = "…" - +CANT_PARSE_ENTITIES = "can't parse entities" URL_BUTTON_ERRORS = ("button_url_invalid", "keyboard button url") +TAG_OPEN_RE = re.compile(r"<([a-zA-Z][a-zA-Z0-9]*)[^>]*>") +TAG_CLOSE_RE = re.compile(r"") +TAG_OR_ENTITY_RE = re.compile(r"<[^>]*>?$|&[a-zA-Z0-9#]*;?$") +STRIP_TAGS_RE = re.compile(r"<[^>]+>") + + +def _strip_tags(text: str) -> str: + return unescape(STRIP_TAGS_RE.sub("", text)) + + +def _close_open_tags(truncated: str) -> str: + stack: list[str] = [] + pos = 0 + while True: + open_match = TAG_OPEN_RE.search(truncated, pos) + close_match = TAG_CLOSE_RE.search(truncated, pos) + if open_match and (not close_match or open_match.start() < close_match.start()): + stack.append(open_match.group(1)) + pos = open_match.end() + continue + if close_match: + if stack and stack[-1] == close_match.group(1): + stack.pop() + pos = close_match.end() + continue + break + closing = "".join(f"" for tag in reversed(stack)) + return truncated + closing + + +def truncate_html(text: str, limit: int = MESSAGE_MAX) -> str: + if len(text) <= limit: + return text + mark_len = len(TRUNCATION_MARK) + cut = max(limit - mark_len, 0) + while cut > 0: + truncated = text[:cut] + # Nicht mitten in einem Tag oder einer &entity; aufhören, sonst wird das HTML ungültig. + boundary = TAG_OR_ENTITY_RE.search(truncated) + if boundary: + truncated = truncated[:boundary.start()] + closed = _close_open_tags(truncated) + TRUNCATION_MARK + if len(closed) <= limit: + return closed + cut -= 1 + # Fallback: als Plain Text kürzen, wenn die HTML-sichere Kürzung das Limit trotzdem sprengt. + plain = _strip_tags(text) + return plain[:max(limit - mark_len, 0)] + TRUNCATION_MARK + def _button(b: Button) -> dict: return {"text": b.label, "url": b.url} if b.url else {"text": b.label, "callback_data": b.data} @@ -83,8 +136,20 @@ class TelegramAdapter: payload["reply_markup"] = _markup(remaining) return await self._call(method, payload) + async def _send_plain_on_parse_error(self, method: str, payload: dict, buttons: Buttons, + html: bool) -> dict: + try: + return await self._call_buttons(method, payload, buttons) + except TelegramError as exc: + if not html or CANT_PARSE_ENTITIES not in str(exc).lower(): + raise + plain_payload = {k: v for k, v in payload.items() if k != "parse_mode"} + plain_payload["text"] = _strip_tags(payload["text"])[:MESSAGE_MAX] + return await self._call_buttons(method, plain_payload, buttons) + async def send(self, target: str, text: str, buttons: Buttons = (), reply_to: str | None = None, html: bool = True, silent: bool = False) -> MessageRef: + text = truncate_html(text) if html else text[:MESSAGE_MAX] payload: dict = {"chat_id": target, "text": text, "disable_web_page_preview": True} if silent: payload["disable_notification"] = True @@ -94,16 +159,17 @@ class TelegramAdapter: payload["reply_markup"] = _markup(buttons) if reply_to: payload["reply_parameters"] = {"message_id": int(reply_to), "allow_sending_without_reply": True} - result = await self._call_buttons("sendMessage", payload, buttons) + result = await self._send_plain_on_parse_error("sendMessage", payload, buttons, html) return MessageRef(self.name, str(target), str(result["message_id"])) async def edit(self, ref: MessageRef, text: str, buttons: Buttons = (), silent: bool = False) -> None: + text = truncate_html(text) payload: dict = {"chat_id": ref.target, "message_id": int(ref.message_id), "text": text, "parse_mode": PARSE_MODE, "disable_web_page_preview": True} if buttons: payload["reply_markup"] = _markup(buttons) try: - await self._call_buttons("editMessageText", payload, buttons) + await self._send_plain_on_parse_error("editMessageText", payload, buttons, html=True) except TelegramError as exc: if NOT_MODIFIED not in str(exc).lower(): raise diff --git a/app/chatops.py b/app/chatops.py index ca1c6e8..c5d7dc4 100644 --- a/app/chatops.py +++ b/app/chatops.py @@ -4,6 +4,7 @@ from typing import Callable from .agent_runs import AgentRunRepository, AgentService, configured_secrets from .alerts import AlertRepository from .chat import BACKEND_MATRIX, BACKEND_TELEGRAM +from .checkmk import CheckmkClient from .commands import (CallbackHandler, CommandDeps, CommandHandler, SettingsReader, command_audit) from .constants import MATRIX_ALLOWLIST, TELEGRAM_ALLOWLIST from .db import Database @@ -25,7 +26,8 @@ class ChatOps: self.agent_runs = AgentRunRepository(db) self.agent = AgentService(store, db, self.agent_runs, state.recent_changes, lambda: configured_secrets(store), prompt_provider=agent_prompt) - self.downtime = DowntimeService(store, db, PendingStore(time.monotonic), allowlist) + self.downtime = DowntimeService(store, db, PendingStore(time.monotonic), allowlist, + reader_factory=lambda: CheckmkClient.from_settings(store)) deps = CommandDeps(SettingsReader(store), self.agent, mutes=mutes, lookup=alerts, downtime=self.downtime) audit_log = command_audit(db) self.handler = CommandHandler(deps, allowlist, audit_log, bot_name=bot_names.get) diff --git a/app/commands/ai.py b/app/commands/ai.py index 90700c4..1b8d846 100644 --- a/app/commands/ai.py +++ b/app/commands/ai.py @@ -11,9 +11,17 @@ EXPLAIN_USAGE = '{p}explain , z.B. {p}explain web01 "CPU load"' ASK_USAGE = "{p}ask " +def _escape_within_limit(text: str, limit: int) -> str: + escaped = escape(text) + while len(escaped) > limit and text: + text = text[:-1] + escaped = escape(text) + return escaped + + def agent_reply(result) -> Reply: text = result.text[:ANSWER_MAX] - return Reply(text, escape(text)) + return Reply(text, _escape_within_limit(text, ANSWER_MAX)) async def explain_command(ctx: CommandContext) -> Reply: diff --git a/app/downtime/service.py b/app/downtime/service.py index 13a9079..42c4db2 100644 --- a/app/downtime/service.py +++ b/app/downtime/service.py @@ -1,4 +1,5 @@ import asyncio +import logging import re import time from dataclasses import dataclass @@ -13,14 +14,19 @@ from ..constants import (CHECKMK_DT_ENABLED, CHECKMK_DT_MAX_HOURS, DEFAULT_DT_MA from .client import CheckmkDowntimeClient, DowntimeError, is_configured from .pending import DowntimeRequest, Outcome, PendingStore +log = logging.getLogger(__name__) + COMMENT_MAX = 200 DEFAULT_COMMENT = "Downtime per Chat (checkbot)" CONTROL_RE = re.compile(r"[\x00-\x1f\x7f-\x9f]") UNTIL_FORMAT = "%d.%m. %H:%M" +HTTP_STATUS_RE = re.compile(r"^HTTP (\d{3})") +VERIFY_TOLERANCE = timedelta(minutes=5) ACTION_REQUEST = "downtime.request" ACTION_CONFIRM = "downtime.confirm" ACTION_CREATED = "downtime.created" +ACTION_UNCONFIRMED = "downtime.unconfirmed" ACTION_CANCEL = "downtime.cancel" ACTION_EXPIRED = "downtime.expired" ACTION_ERROR = "downtime.error" @@ -32,8 +38,31 @@ MSG_NOT_CONFIGURED = "Für Downtimes sind Checkmk-URL, Downtime-User und Secret MSG_UNKNOWN = "Code unbekannt, abgelaufen oder bereits verwendet." MSG_WRONG_SENDER = "Nur der Absender der Anfrage kann bestätigen oder abbrechen." MSG_CANCELLED = "Abgebrochen. Es wurde keine Downtime gesetzt." +MSG_REJECTED = "Checkmk hat die Downtime nicht angenommen." ClientFactory = Callable[[], CheckmkDowntimeClient | None] +ReaderFactory = Callable[[], object | None] + + +def _rejection_message(error: str) -> str: + match = HTTP_STATUS_RE.match(error) + return f"{MSG_REJECTED} (HTTP {match.group(1)})" if match else MSG_REJECTED + + +def _parse_time(value: str) -> datetime | None: + try: + return datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError: + return None + + +def _matches(downtime, host: str, service: str, start: datetime, end: datetime) -> bool: + if downtime.host.casefold() != host.casefold() or (downtime.service or "").casefold() != service.casefold(): + return False + found_start, found_end = _parse_time(downtime.start), _parse_time(downtime.end) + if found_start is None or found_end is None: + return False + return (abs(found_start - start) <= VERIFY_TOLERANCE and abs(found_end - end) <= VERIFY_TOLERANCE) @dataclass(frozen=True) @@ -57,12 +86,14 @@ def clean_comment(comment: str) -> str: class DowntimeService: def __init__(self, store, db, pending: PendingStore, allowlist: Callable[[str], str | None], - client_factory: ClientFactory | None = None, clock: Callable[[], float] = time.time): + client_factory: ClientFactory | None = None, clock: Callable[[], float] = time.time, + reader_factory: ReaderFactory | None = None): self._store = store self._db = db self._pending = pending self._allowlist = allowlist self._client_factory = client_factory or (lambda: CheckmkDowntimeClient.from_settings(store)) + self._reader_factory = reader_factory or (lambda: None) self._clock = clock def enabled(self) -> bool: @@ -151,14 +182,34 @@ class DowntimeService: except DowntimeError as exc: error = str(exc) except Exception as exc: - error = type(exc).__name__ + error = f"{type(exc).__name__}: {exc}" else: error = None finally: await client.aclose() if error: + log.warning("downtime create failed for %s / %s: %s", request.host, request.service, error) await self._audit(ACTION_ERROR, *who, *target, error) - return f"Checkmk hat die Downtime nicht angenommen: {error}" + return _rejection_message(error) until = end.astimezone().strftime(UNTIL_FORMAT) - await self._audit(ACTION_CREATED, *who, *target, f"{detail} bis {until}") - return f"Downtime gesetzt: {request.host} / {request.service}, {detail}, bis {until}." + confirmed = await self._verify(request.host, request.service, start, end) + if confirmed: + await self._audit(ACTION_CREATED, *who, *target, f"{detail} bis {until}") + return f"Downtime gesetzt: {request.host} / {request.service}, {detail}, bis {until}." + await self._audit(ACTION_UNCONFIRMED, *who, *target, f"{detail} bis {until}") + return f"Downtime gesendet, aber nicht bestätigt: {request.host} / {request.service}, {detail}, bis {until}." + + async def _verify(self, host: str, service: str, start: datetime, end: datetime) -> bool: + reader = self._reader_factory() + if reader is None: + return False + try: + downtimes = await reader.downtimes(host) + except Exception as exc: + log.warning("downtime verification read failed for %s: %s", host, type(exc).__name__) + return False + finally: + aclose = getattr(reader, "aclose", None) + if aclose is not None: + await aclose() + return any(_matches(d, host, service, start, end) for d in downtimes) diff --git a/tests/support.py b/tests/support.py index bb93692..7280f9a 100644 --- a/tests/support.py +++ b/tests/support.py @@ -9,7 +9,9 @@ from app.constants import (CHECKMK_DT_ENABLED, CHECKMK_DT_SECRET, CHECKMK_DT_USE CHECKMK_SITE, CHECKMK_URL, FLAG_ON, MATRIX_ALLOWLIST, TELEGRAM_ALLOWLIST) from app.crypto import SecretBox from app.db import Database +from app.checkmk import Downtime from app.downtime import DowntimeError, DowntimeService, PendingStore +from app.downtime.client import iso_utc from app.models import KIND_PROBLEM, Event from app.mutes import MuteService from app.settings import SettingsStore @@ -47,6 +49,21 @@ class FakeDowntimeClient: self.closed = True +class FakeDowntimeReader: + def __init__(self, client): + self._client = client + self.closed = False + + async def downtimes(self, host=None): + items = [Downtime(h, s, iso_utc(start), iso_utc(end), c, "auto") for h, s, start, end, c in self._client.calls] + if host: + items = [d for d in items if d.host.casefold() == host.casefold()] + return items + + async def aclose(self): + self.closed = True + + class Env: def __init__(self, dt_enabled=True): self.db = Database(":memory:") @@ -66,7 +83,8 @@ class Env: self.pending = PendingStore(self.clock) self.agent = FakeAgent() self.downtime = DowntimeService(self.store, self.db, self.pending, self.allowlist, - lambda: self.client, clock=lambda: 1_800_000_000.0) + lambda: self.client, clock=lambda: 1_800_000_000.0, + reader_factory=lambda: FakeDowntimeReader(self.client)) self.deps = CommandDeps(None, self.agent, mutes=self.mutes, lookup=self.alerts, downtime=self.downtime) self.events = [] self.handler = CommandHandler(self.deps, self.allowlist, self.events.append) diff --git a/tests/test_agent_loop.py b/tests/test_agent_loop.py index 6313cb1..894193d 100644 --- a/tests/test_agent_loop.py +++ b/tests/test_agent_loop.py @@ -169,3 +169,38 @@ async def test_explain_prompt_contains_target(): async def test_empty_answer(): result = await make_agent(ScriptedChat([{"content": " "}])).ask("x") assert result.text == "Keine Antwort erhalten." + + +def multi_tool_call(n): + return {"content": "", "tool_calls": [ + {"function": {"name": "get_service_status", "arguments": {"host": "web01", "service": "CPU load"}}} + for _ in range(n) + ]} + + +async def test_excess_tool_calls_per_step_get_placeholder(): + chat = ScriptedChat([multi_tool_call(5), {"content": "ok"}]) + result = await make_agent(chat).ask("x") + assert len(result.steps) == 3 + placeholders = [m for m in chat.calls[1]["messages"] if m.get("content") == "Nicht ausgeführt."] + assert len(placeholders) == 2 + assert result.text == "ok" + + +async def test_step_limit_mid_batch_adds_placeholders_for_rest(): + chat = ScriptedChat([multi_tool_call(2) for _ in range(4)] + [{"content": "Zusammenfassung"}]) + result = await make_agent(chat).ask("x") + assert result.abort_reason == "step_limit" + assert len(result.steps) == 6 + last_messages = chat.calls[-1]["messages"] + assert any(m.get("content") == "Nicht ausgeführt." for m in last_messages) + + +async def test_old_tool_results_are_compacted(): + chat = ScriptedChat([tool_call() for _ in range(6)] + [{"content": "fertig"}]) + result = await make_agent(chat).ask("x") + assert result.text == "fertig" + tool_messages = [m for m in chat.calls[-1]["messages"] if m.get("role") == "tool"] + compacted = [m for m in tool_messages if m["content"] == "[gekürzt] Ergebnis früherer Abfrage"] + assert len(compacted) == 3 + assert tool_messages[-1]["content"].startswith("") diff --git a/tests/test_agent_sanitize.py b/tests/test_agent_sanitize.py new file mode 100644 index 0000000..093b58a --- /dev/null +++ b/tests/test_agent_sanitize.py @@ -0,0 +1,17 @@ +from app.agent.sanitize import TRUNCATION_SUFFIX, sanitize + + +def test_sanitize_short_text_untouched(): + assert sanitize("hallo", 20) == "hallo" + + +def test_sanitize_truncates_with_suffix_and_respects_limit(): + text = "A" * 50 + result = sanitize(text, 20) + assert len(result) <= 20 + assert result.endswith(TRUNCATION_SUFFIX) + + +def test_sanitize_tiny_limit_still_within_bound(): + result = sanitize("A" * 50, 3) + assert len(result) <= 3 diff --git a/tests/test_commands_ai.py b/tests/test_commands_ai.py new file mode 100644 index 0000000..89d570f --- /dev/null +++ b/tests/test_commands_ai.py @@ -0,0 +1,22 @@ +from app.agent import AgentResult +from app.commands.ai import agent_reply + + +def test_agent_reply_escapes_html(): + result = AgentResult("CPU & RAM ", ()) + reply = agent_reply(result) + assert reply.text == "CPU & RAM " + assert reply.html == "CPU & RAM <hoch>" + + +def test_agent_reply_escaped_length_stays_within_limit(): + result = AgentResult("&" * 3000, ()) + reply = agent_reply(result) + assert len(reply.html) <= 3000 + assert reply.html == "&" * (len(reply.html) // 5) + + +def test_agent_reply_plain_text_truncated_to_answer_max(): + result = AgentResult("x" * 4000, ()) + reply = agent_reply(result) + assert len(reply.text) == 3000 diff --git a/tests/test_delivery.py b/tests/test_delivery.py new file mode 100644 index 0000000..f855c58 --- /dev/null +++ b/tests/test_delivery.py @@ -0,0 +1,33 @@ +from app.alerts.delivery import Deliverer +from app.alerts.extras import Extras +from app.chat import BACKEND_TELEGRAM, Target, TELEGRAM_MESSAGE_MAX + + +class RecordingAdapter: + name = BACKEND_TELEGRAM + + def __init__(self): + self.sent = [] + + async def send(self, target, text, buttons=(), silent=False): + self.sent.append(text) + return None + + async def edit(self, ref, text, buttons=(), silent=False): + self.sent.append(text) + + +async def test_hint_is_kept_intact_when_text_is_long(): + adapter = RecordingAdapter() + deliverer = Deliverer(lambda: _adapters(adapter)) + hint = "Antwort mit !mute 2h zum Stummschalten" + extra = Extras(hint=hint) + await deliverer.send("" + "x" * 5000 + "", [Target(BACKEND_TELEGRAM, "42")], + extras=lambda backend: extra) + sent = adapter.sent[0] + assert sent.endswith(f"\n{hint}") + assert len(sent) <= TELEGRAM_MESSAGE_MAX + + +async def _adapters(adapter): + return [adapter] diff --git a/tests/test_downtime_service_verify.py b/tests/test_downtime_service_verify.py new file mode 100644 index 0000000..12eb12a --- /dev/null +++ b/tests/test_downtime_service_verify.py @@ -0,0 +1,86 @@ +from datetime import timedelta + +from app.checkmk import Downtime +from app.commands import IncomingCallback +from tests.support import CHAT, TG_USER, Env, tg + + +class ReaderStub: + def __init__(self, downtimes=None, fail=False): + self._downtimes = downtimes or [] + self.fail = fail + self.closed = False + + async def downtimes(self, host=None): + if self.fail: + raise RuntimeError("boom") + return self._downtimes + + async def aclose(self): + self.closed = True + + +async def confirm(env): + reply = await env.handler.handle(tg('/downtime web01 "CPU load" 2h Wartung')) + return await env.callbacks.handle(IncomingCallback("telegram", TG_USER, CHAT, "9", reply.buttons[0][0].data)) + + +async def test_downtime_confirmed_when_reader_finds_matching_entry(): + env = Env() + result = await confirm(env) + assert "Downtime gesetzt" in result.reply.text + assert env.downtime_audit()[-1] == "downtime.created" + + +async def test_downtime_unconfirmed_when_reader_finds_nothing(): + env = Env() + env.downtime._reader_factory = lambda: ReaderStub([]) + result = await confirm(env) + assert "gesendet, aber nicht bestätigt" in result.reply.text + assert env.downtime_audit()[-1] == "downtime.unconfirmed" + + +async def test_downtime_unconfirmed_when_reader_fails(): + env = Env() + env.downtime._reader_factory = lambda: ReaderStub(fail=True) + result = await confirm(env) + assert "gesendet, aber nicht bestätigt" in result.reply.text + + +async def test_downtime_unconfirmed_when_no_reader_configured(): + env = Env() + env.downtime._reader_factory = lambda: None + result = await confirm(env) + assert "gesendet, aber nicht bestätigt" in result.reply.text + + +async def test_downtime_not_confirmed_for_mismatching_entry(): + env = Env() + wrong = Downtime("otherhost", "CPU load", "2026-11-06T08:00:00Z", "2026-11-06T10:00:00Z", "x", "a") + env.downtime._reader_factory = lambda: ReaderStub([wrong]) + result = await confirm(env) + assert "gesendet, aber nicht bestätigt" in result.reply.text + + +async def test_rejection_message_hides_detail_but_keeps_http_status(): + env = Env() + env.client.fail = "HTTP 403: nope, denied-secret" + result = await confirm(env) + assert "nicht angenommen" in result.reply.text and "403" in result.reply.text + assert "denied-secret" not in result.reply.text + + +async def test_rejection_message_hides_exception_class_name(): + env = Env() + + class BrokenClient: + async def create_service_downtime(self, *a, **k): + raise ValueError("weirdly broken") + + async def aclose(self): + pass + + env.downtime._client_factory = lambda: BrokenClient() + result = await confirm(env) + assert "ValueError" not in result.reply.text + assert "nicht angenommen" in result.reply.text diff --git a/tests/test_matrix.py b/tests/test_matrix.py index fa64eda..9f89062 100644 --- a/tests/test_matrix.py +++ b/tests/test_matrix.py @@ -4,6 +4,7 @@ import httpx import pytest from app.chat import MatrixAdapter, MatrixError, MessageRef +from app.chat.matrix import MAX_BODY_BYTES, _truncate_bytes from app.chat.matrix_format import to_matrix TOKEN = "syt_SECRET-token.abc" @@ -148,3 +149,44 @@ def test_to_matrix_escapes_foreign_markup(): body, formatted = to_matrix("x <script>") assert body == "x