diff --git a/AGENTS.md b/AGENTS.md index e97e67a..78e7125 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,32 +1,31 @@ # Agent Instructions -## First Step - Read [`INSTALL.md`](./INSTALL.md) first. -If the goal is to set up or use this repository in Codex, treat `INSTALL.md` as the primary operator and handoff guide. - -## Setup Priority +It is the Codex setup and handoff guide. -When working from a fresh clone: +## Set up a cloned checkout -1. Read `INSTALL.md` -2. Follow the Codex/plugin installation flow there -3. Use the plugin-local skills when relevant: +1. Follow the Codex/plugin flow in `INSTALL.md`. +2. Use the relevant local skill: - `plugins/kibana-log-investigation/skills/ensure-node-runtime/SKILL.md` - `plugins/kibana-log-investigation/skills/install-and-configure/SKILL.md` -## Repository Context +## Before editing -This repo provides a read-only Kibana log investigation MCP server plus a repo-scoped Codex plugin for local installation from a cloned checkout. -Documented learnings live in `docs/solutions/`, organized by category with YAML frontmatter such as `module`, `tags`, and `problem_type`, and are relevant when working in already documented areas. +This repo provides a read-only Kibana MCP server and a repo-local Codex plugin. -For product details and MCP behavior, read `README.md`. +Read `README.md` for product behavior. Check relevant learnings in `docs/solutions/`, organized by category and frontmatter (`module`, `tags`, `problem_type`). ## CLI conventions -Keep the human CLI decomposed by concern: the executable lives in `src/bin/`, Commander command registration lives in `src/commands/`, and terminal I/O plus Clack prompt adapters live outside command modules. Commands contain only command-specific validation and orchestration; do not add command logic back to `src/cli.ts`. +- `src/bin/`: executables. +- `src/commands/`: Commander registration, command-specific validation and orchestration only. +- Keep terminal I/O and Clack prompt adapters outside command modules. + +Do not add command logic back to `src/cli.ts`. ## Lessons +- When writing skills or instructions, lead with the action, use short numbered steps, keep warnings beside the affected action, and remove repeated rationale without removing contracts. - When a project format was used only by the owner, remove its obsolete code, callers, tests and documentation instead of adding compatibility or migration paths. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a9eb052..85ad73a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,7 +1,5 @@ # Contributing -Thanks for contributing to Kibana Log Investigation MCP. - ## Quick Start 1. Ensure Node.js 22+ (see `.node-version`). @@ -25,25 +23,20 @@ Thanks for contributing to Kibana Log Investigation MCP. ## Source Catalogs -- Connection bootstrap saves an empty managed catalog with the machine profile; `configure_index` adds only indexes explicitly approved by the user. -- Guided `setup` imports a hand-authored catalog selected by the operator. Use `config/sources.example.json` only as an explicitly chosen template. -- Environment-based startup requires `KIBANA_SOURCE_CATALOG_PATH` unless a configuration-file store supplies the catalog. Saved profiles carry their own catalog path; no checkout file is loaded implicitly. -- Do not commit real environment source catalogs or credentials. - -## Schema Endpoints +1. Use connection bootstrap for an empty managed catalog saved with the machine profile. Add indexes through `configure_index` only with explicit user approval. +2. Use guided `setup` to import an operator-selected hand-authored catalog. Choose `config/sources.example.json` explicitly if needed as a template. +3. Set `KIBANA_SOURCE_CATALOG_PATH` for environment startup unless a configuration-file store supplies it. Saved profiles carry their catalog path; no checkout file loads implicitly. -Some Kibana deployments block schema metadata endpoints. When that happens: +Never commit real environment catalogs or credentials. -- `describe_fields` can return errors -- exact-field promotion and nested validation may be unavailable +## Schema Endpoints -Please include this detail in bug reports when relevant. +Report blocked schema endpoints when relevant: `describe_fields` can fail, and exact-field promotion or nested validation may be unavailable. ## Pull Requests -- Keep changes focused. -- Update docs when behavior changes. -- Ensure `npm run verify` passes before requesting review. +1. Keep changes focused; update docs when behavior changes. +2. Pass `npm run verify` before requesting review. ## Security diff --git a/HANDOFF_timeout-config-bug.md b/HANDOFF_timeout-config-bug.md index e3d3c70..37f4c1b 100644 --- a/HANDOFF_timeout-config-bug.md +++ b/HANDOFF_timeout-config-bug.md @@ -1,14 +1,13 @@ # Handoff: Kibana MCP startup failure on KIBANA_TIMEOUT_MS -## Context -The Codex MCP entries in `~/.codex/config.toml` were failing to start the Kibana MCP server when `KIBANA_TIMEOUT_MS` was present. +## Recorded issue — 2026-04-09 -As a quick workaround, the `KIBANA_TIMEOUT_MS` lines were removed from the local Codex config entries on 2026-04-09. +Codex MCP entries in `~/.codex/config.toml` failed to start with `KIBANA_TIMEOUT_MS`. The local workaround removed those lines on 2026-04-09; this is historical evidence, not a current startup check. ## Reproduction The environment values below are intentionally synthetic and portable. -This fails: +Recorded failing command: ```bash KIBANA_BASE_URL='https://kibana.example.com/logs' \ @@ -24,7 +23,7 @@ ZodError: Invalid input: expected string, received number path: ["KIBANA_TIMEOUT_MS"] ``` -This succeeds: +Recorded successful command: ```bash KIBANA_BASE_URL='https://kibana.example.com/logs' \ @@ -33,20 +32,18 @@ node dist/src/index.js ``` ## Root cause -`loadConfigFromEnvironment()` parses env with `envSchema.parse(envInput)`, where `KIBANA_TIMEOUT_MS` is transformed from string to number. - -Later, after loading the source catalog, it calls `parseAppConfig(env, sourceCatalog)`. But `parseAppConfig()` parses the env again using the same schema, which still expects `KIBANA_TIMEOUT_MS` to be a string input. - -That second parse crashes because the already-normalized value is now a number. +1. `loadConfigFromEnvironment()` calls `envSchema.parse(envInput)`, converting `KIBANA_TIMEOUT_MS` from string to number. +2. After catalog loading, `parseAppConfig(env, sourceCatalog)` parses that env with the same schema. +3. The second parse expects a string and crashes on the normalized number. ## Relevant files - `src/config.ts` - `dist/src/config.js` - optional tests covering `loadConfigFromEnvironment()` with and without `KIBANA_TIMEOUT_MS` -## Requested follow-up -Implement a real fix so startup works when `KIBANA_TIMEOUT_MS` is set, then commit and push the change. +## Original requested follow-up + +1. Fix startup with `KIBANA_TIMEOUT_MS`; keep the fix minimal and add focused tests. Do not revert unrelated untracked dirs such as `.codex/` or `openspec/`. +2. Commit and push the fix. -Constraints: -- Do not revert unrelated untracked dirs like `.codex/` or `openspec/`. -- Keep the fix minimal and add focused tests. +This records the original request; it does not authorize changes or publishing in later documentation tasks. diff --git a/INSTALL.md b/INSTALL.md index 5a928be..7c8ce82 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,201 +1,148 @@ -# Deterministic Installation +# Install the Kibana MCP -This is the canonical install and handoff guide for `@havesomecode/kibana-mcp-server`. +Use Node.js 22+, a reachable Kibana base URL, and basic-auth credentials. Codex registration also needs `codex` on `PATH`. -The supported newcomer path is connection-only bootstrap. It prompts for missing connection values and is prompt-free when all values are supplied. Bootstrap never lists, scans, infers, or configures Kibana indexes. Agents must use the CLI contract below instead of editing Codex or MCP configuration files. +For a selected release or agent handoff, [resolve its commit first](#version-matched-readme-handoff). Agents must use bootstrap, not edit MCP client configuration files. -## Version-Matched README Handoff +## Install in Codex -Before installation or agent handoff, resolve the selected branch, tag, or package metadata to the full commit SHA. This is the **resolved installation commit**. Read `README.md`, `INSTALL.md`, and installation skills from that immutable commit; older commits may have different installation contracts. +1. Run connection bootstrap. It prompts for missing values; never put a password in an argument. -The version-matched README URL is always commit-pinned: + ```bash + npx -y @havesomecode/kibana-mcp-server bootstrap --client codex + ``` -```text -https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md -``` + For automation, provision `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` first. With all three supplied, no prompts appear. -A branch or tag is an input to commit resolution, never the final handoff URL. Do not infer a Git tag from an npm version. For npm installations, resolve the requested package to the exact version being installed, obtain its `gitHead` with `npm view @havesomecode/kibana-mcp-server@ gitHead`, and pin the installation to that same package version. If that metadata is absent or the commit cannot be resolved, report the missing installation ref rather than claiming master matches the package. For a cloned checkout, resolve the selected branch/tag with `git rev-parse ^{commit}`; for a detached checkout, use `git rev-parse HEAD`'s full commit SHA. +2. Clear those variables from the MCP server environment so it uses the saved profile. -From a maintained checkout containing the repository-only helper, verify a handoff with: + Bash: -```bash -node scripts/readme-handoff.mjs refs/heads/master -# Or pass the actual refs/tags/ or full commit SHA selected for installation. -``` + ```bash + unset KIBANA_BASE_URL KIBANA_USERNAME KIBANA_PASSWORD + ``` -The helper does not install anything or change configuration. It resolves locally available Git refs, fetches the immutable commit URL with a timeout, compares it byte-for-byte to the resolved commit's README, and outputs JSON with `ref`, `commit`, and `url` only after verification. Fetch a missing ref explicitly before retrying; the helper does not fetch Git refs or assume a tag. Ambiguous branch/tag names require the qualified `refs/heads/` or `refs/tags/` form. Unknown refs, missing READMEs, HTTP failures, and content mismatches exit nonzero without a handoff. Resolve a moved branch again to create a new commit-pinned handoff; do not silently substitute master. This helper is not a published npm command or a requirement that historical tags contain it. + PowerShell: -Pass the verified URL and resolved commit to the receiving agent. A master preview link is not evidence that master matches a tagged or npm installation. + ```powershell + Remove-Item Env:KIBANA_BASE_URL, Env:KIBANA_USERNAME, Env:KIBANA_PASSWORD -ErrorAction SilentlyContinue + ``` -## Prerequisites +3. Install the investigation skill. -- Node.js 22 or newer -- `codex` on `PATH` when `--client codex` is used -- a Kibana base URL reachable with basic authentication -- credentials supplied through a trusted environment, stdin, or a secret prompt + ```bash + npx skills add Havesomecode/kibana-mcp-server \ + --skill kibana-log-investigation \ + --agent codex --global --yes + ``` -For prompt-free automation, provision these values before installation: +4. Open a fresh agent session and call `discover`. A first install has no sources. +5. Ask the user for the exact index or pattern. Wait for their answer, call `configure_index` with that exact `index`, then call `discover` again. -- `KIBANA_BASE_URL` -- `KIBANA_USERNAME` -- `KIBANA_PASSWORD` +Never guess, enumerate, scan, or auto-select an index. `configure_index` accepts no credentials; it inspects fields and runs one bounded validation search for the selected index only. Failed validation leaves the catalog unchanged. -Without these values, bootstrap prompts for the missing connection inputs. No index is required or accepted during bootstrap. +**Success:** bootstrap verifies `/api/status`, saves the connection, and reads back the requested registration. It never inspects indexes. A non-zero exit means installation is incomplete; index access is verified only by `configure_index`. -## Install The Agent Skill +## Choose a profile or registration mode -```bash -npx skills add Havesomecode/kibana-mcp-server \ - --skill kibana-log-investigation \ - --agent codex \ - --global \ - --yes -``` +| Need | Option | +| --- | --- | +| Additional environment | `--profile prod` | +| Keep the current default | `--no-default` | +| Plugin or externally managed registration | `--client none` | +| Import a hand-authored catalog | bare `setup` | -The skill is stored at `skills/kibana-log-investigation/SKILL.md`. After the repository is indexed, its skills.sh route is: +The first bootstrap creates the default profile. With connection options, `setup` uses the same connection-only bootstrap flow. `config/sources.example.json` is documentation data, never an automatic bootstrap input. -- `https://skills.sh/havesomecode/kibana-mcp-server/kibana-log-investigation` +Codex registers `npx -y @havesomecode/kibana-mcp-server@ serve --profile prod` under `kibana-log-investigation-`. Inspect it with `codex mcp list --json`. -## Prompt-Free Connection Bootstrap +## Supply secrets or overrides -With connection values provisioned: +Password precedence: `--password-stdin`, then `--password-env NAME`, then `KIBANA_PASSWORD`. ```bash -npx -y @havesomecode/kibana-mcp-server bootstrap \ - --client codex +printf '%s\n' "$SECRET_VALUE" | npx -y @havesomecode/kibana-mcp-server bootstrap \ + --password-stdin --client codex ``` -Optional profile selection: - ```bash npx -y @havesomecode/kibana-mcp-server bootstrap \ - --profile prod \ - --client codex + --password-env KIBANA_PROD_PASSWORD --client codex ``` -With all connection values supplied, bootstrap is non-interactive. It is idempotent for profiles it generated and: - -1. validates the connection inputs; -2. checks for a missing or conflicting Codex registration; -3. verifies only the Kibana connection through `/api/status`; -4. acquires an interprocess state lock; -5. writes an empty managed source catalog on first install; -6. preserves sources that were explicitly configured after a prior bootstrap; -7. saves credentials in the platform credential store; -8. saves profile and catalog files atomically; -9. calls `codex mcp add` with the exact running package version and a transport-hashed registration name; -10. reads the registered transport back exactly; -11. restores prior machine state if registration fails. - -A non-zero exit means installation did not complete. A successful exit verifies the connection, not any index. - -After bootstrap, remove `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` from the MCP server environment before reconnecting so startup uses the saved profile. In Bash, run `unset KIBANA_BASE_URL KIBANA_USERNAME KIBANA_PASSWORD`; in PowerShell, run `Remove-Item Env:KIBANA_BASE_URL, Env:KIBANA_USERNAME, Env:KIBANA_PASSWORD -ErrorAction SilentlyContinue`. Leaving direct connection values set overrides profile selection and requires an explicit catalog path or store. - -## Explicit Index Selection - -Start a fresh agent session after bootstrap and call `discover`. An empty catalog responds with instructions to ask the user which exact Kibana index or index pattern they want to use. - -The safe sequence is: - -1. call `discover`; -2. ask the user for the exact index or pattern; -3. wait for their answer—never guess, enumerate, scan, or auto-select; -4. call `configure_index` with exactly the user-provided `index` value; -5. let that tool inspect fields and run one bounded validation search only for that index; -6. call `discover` again and use the returned source id. - -`configure_index` accepts no Kibana credentials. A validation failure leaves the existing catalog unchanged. - -The MCP server starts only with a configured connection. Bootstrap saves one for subsequent starts. Operators who supply the connection directly through environment variables must also set `KIBANA_SOURCE_CATALOG_PATH`, or declare a catalog store in a configuration file. No catalog is loaded implicitly from the checkout, and no MCP tool accepts connection credentials. - -## Secret Input - -Never put a password in a command-line argument. Supported sources, in precedence order: +| Value | Flag | Fallback | +| --- | --- | --- | +| Profile | `--profile NAME` | `KIBANA_PROFILE`, then `default` | +| Base URL | `--url URL` | `KIBANA_BASE_URL` | +| Username | `--username USER` | `KIBANA_USERNAME` | +| Client | `--client codex` or `--client none` | `codex` | -1. `--password-stdin` -2. `--password-env NAME` -3. `KIBANA_PASSWORD` +Credentials go to macOS Keychain, Windows Credential Manager, or Linux Secret Service when available, never profile, catalog, or client configuration files. -```bash -printf '%s\n' "$SECRET_VALUE" | npx -y @havesomecode/kibana-mcp-server bootstrap \ - --password-stdin \ - --client codex -``` - -```bash -npx -y @havesomecode/kibana-mcp-server bootstrap \ - --password-env KIBANA_PROD_PASSWORD \ - --client codex -``` +Bootstrap uses a state lock and atomic writes. It preserves previously configured managed sources, rejects conflicting registrations, and restores prior state if registration fails. -Secrets are stored in macOS Keychain, Windows Credential Manager, or Linux Secret Service when available. They are not written to profiles, generated catalogs, or MCP client configuration. +**Before using `--replace`:** get explicit operator authorization. It replaces the target with an empty managed catalog. Without it, bootstrap refuses hand-authored or modified catalogs; managed catalogs carry a marker and content hash. -## Profiles +## Start without a saved profile -The first bootstrap creates the default profile. Use `--profile NAME` for additional environments and `--no-default` when the new profile must not replace the current default. +Supply the connection and an explicit `KIBANA_SOURCE_CATALOG_PATH`, or a configuration-file catalog store. Direct environment values override profiles. Startup requires a connection; it never searches the checkout for a catalog or accepts credentials through MCP tools. -The registered stdio command selects the profile explicitly: `npx -y @havesomecode/kibana-mcp-server@ serve --profile prod`. Registration names use `kibana-log-investigation-`. Inspect them with `codex mcp list --json`. +## Install the repo-local Codex plugin -## Client Registration Modes +1. Build from the repository root. -Use `--client codex` for a full install. Use `--client none` when registration is already managed, such as the repo-local plugin path or CI: + ```bash + npm install + npm run build + ``` -```bash -npx -y @havesomecode/kibana-mcp-server bootstrap --client none -``` +2. Install the built plugin from the repo-local Codex marketplace. The bundle lives in `plugins/kibana-log-investigation/dist/`; `cwd: "."` means the installed plugin directory. It needs Node.js 22+ on `PATH`, not the original checkout or its `node_modules`. +3. Save the connection without duplicate registration. -## Explicit Overrides + ```bash + node dist/src/bin/kibana-mcp-server.js bootstrap --client none + ``` -| Purpose | Option | Environment fallback | -|---|---|---| -| Profile | `--profile NAME` | `KIBANA_PROFILE`, then `default` | -| Kibana base URL | `--url URL` | `KIBANA_BASE_URL` | -| Username | `--username USER` | `KIBANA_USERNAME` | -| Password | `--password-stdin` / `--password-env NAME` | `KIBANA_PASSWORD` | -| Client | `--client codex|none` | `codex` | -| Replace catalog | `--replace` | preserve managed sources; refuse protected catalogs | +4. Clear connection variables, reconnect, and follow the `discover` → user-selected `configure_index` flow above. -If a profile points to a hand-authored or operator-modified catalog, bootstrap fails without changing it. Generated catalogs carry a tool marker and content hash. `--replace` intentionally converts the target to an empty managed catalog; use it only with explicit operator authorization. +## Fix startup failures -## Example Catalog Policy +1. Inspect Codex MCP startup status and sanitized stderr. Saved configuration does not prove startup. +2. For a missing runtime, rebuild and reinstall the plugin. For Node or module-loading errors, follow stderr repair instructions; stdout is reserved for MCP. +3. After startup succeeds, open one fresh discussion and call `discover`. If absent, check plugin enablement and startup logs instead of repeatedly restarting. -`config/sources.example.json` is documentation data. It never participates in bootstrap precedence. `bootstrap` is always connection-only, including without flags. Bare `setup` guides operators who explicitly want a hand-authored catalog; with connection options, `setup` uses the same connection-only bootstrap flow: +`npm run verify:entrypoint` packs and installs the npm artifact, then tests a copied plugin with `initialize`, `tools/list`, and `discover`. Its connection and empty catalog are synthetic: this proves packaged startup, not native Codex or real backend access. -```bash -npx -y @havesomecode/kibana-mcp-server setup -``` +Before sharing logs, remove credentials, authorization headers, credential-bearing URLs, and operational data. Never share complete environment dumps. -## Repository Contributor Path +## Version-matched README handoff -```bash -npm install -npm run build -node dist/src/bin/kibana-mcp-server.js bootstrap --client none -``` +1. Resolve the full **commit SHA** before installing or handing off. This is the **resolved installation commit**. -The repo-scoped Codex plugin already supplies the MCP entry, so `--client none` avoids a duplicate global registration. + | Source | Resolve with | + | --- | --- | + | Checkout branch/tag | `git rev-parse ^{commit}` | + | Detached checkout | `git rev-parse HEAD` | + | Exact npm version | `npm view @havesomecode/kibana-mcp-server@ gitHead` | -`npm run build` also bundles the MCP server and its runtime dependencies into `plugins/kibana-log-investigation/dist/`. Install the plugin only after that build. The plugin declares `cwd: "."`, which Codex resolves against its installed plugin directory, and its launcher resolves the bundle relative to itself. The installed plugin does not depend on the original checkout, its root `dist/`, or its `node_modules`. Node.js 22+ must still be available on `PATH`. + Do not infer a Git tag from an npm version. Pin installation to the same exact package version. Missing metadata or refs are errors, not permission to substitute master. -Run `npm run verify:entrypoint` to pack and install the actual npm artifact, copy only its plugin into an isolated cache directory, and perform a bounded `initialize`, `tools/list`, and `discover` handshake. This uses an explicitly synthetic connection and empty managed catalog; it does not validate a real Kibana backend or a native Codex discussion. +2. Read `README.md`, `INSTALL.md`, and skills from that same commit. -### Diagnose plugin startup before restarting discussions + ```text + https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md + ``` -1. **Saved configuration:** a plugin entry or successful bootstrap is not proof the process started. -2. **Process startup:** inspect Codex's MCP startup status and sanitized stderr. A missing plugin runtime requires rebuilding the checkout and reinstalling the built plugin, not another discussion. Node/version and module-loading failures print repair instructions on stderr, never MCP stdout. -3. **Current discussion:** after startup succeeds, open one fresh Codex discussion and call the native `discover` tool. If it is absent, inspect plugin enablement and startup logs; do not repeat restarts indefinitely. -4. **Backend access:** an empty `discover` response proves local tools work, not backend reachability. Bootstrap verifies `/api/status`; only an explicitly selected `configure_index` validation establishes access to that index. +3. In a maintained checkout, verify the URL with the repository-only helper. -Before sharing startup logs, remove credentials, authorization headers, credential-bearing URLs, and any operational data. Never paste passwords or complete environment dumps into an issue. + ```bash + node scripts/readme-handoff.mjs refs/heads/master + ``` -## Verification + Pass the actual `refs/heads/`, `refs/tags/`, or full SHA selected for installation. Fetch missing refs explicitly; resolve moved branches again. -Bootstrap success verifies: +4. Pass the verified `url` and `commit` to the receiving agent. -- Kibana returned a valid status envelope through `/api/status` -- no index endpoint called -- empty managed catalog created on first install -- profile and credential saved -- requested client registration read back +The helper makes a bounded fetch and compares README bytes against Git. It returns JSON `{ ref, commit, url }` only on success; no configuration changes or Git fetches. -After a new agent session starts, confirm `discover` reports the empty state. Configure and validate an index only after the user explicitly names it. +Unknown or ambiguous refs, missing READMEs, HTTP errors, and mismatches fail without a handoff. Historical installs without the helper need explicit resolution and retrieval verification. It is not an npm command. diff --git a/README.md b/README.md index 49535ad..58edf90 100644 --- a/README.md +++ b/README.md @@ -1,71 +1,46 @@ # Kibana Log Investigation MCP -Read-only MCP server for agent-driven log investigation against Kibana-backed search endpoints protected with basic auth. - -It gives an agent what a Kibana UI cannot: a short list of logical sources it can trust, the fields it may filter on, and typed results. - -**Documentation: https://havesomecode.github.io/kibana-mcp-server/** - -## What it provides - -- `discover` lists the logical sources configured for the caller -- `describe_fields` exposes the effective fields of a source, including which ones support exact matching -- `query` searches one or more sources over an absolute time window in `hits`, `count`, `histogram`, `terms`, `stats`, or `grouped_top_hits` mode -- `filter` runs an exact-field filter when the field name is already known -- `configure_index` validates and persists exactly one index or pattern after the user explicitly names it - -The server never writes to Elasticsearch or Kibana, and never guesses an index. - -## Requirements - -- Node.js 22+ -- A Kibana-compatible read endpoint reachable with basic auth -- One or more Elasticsearch index patterns +Read Kibana logs with an agent. No Elasticsearch or Kibana writes, no guessed indexes. ## Quick start -Bootstrap the connection; missing values are prompted for: +Use Node.js 22+, `codex` on `PATH`, and a reachable Kibana base URL with basic auth. Run: ```bash npx -y @havesomecode/kibana-mcp-server bootstrap --client codex ``` -For prompt-free automation, provision the connection values in the environment: - -```bash -export KIBANA_BASE_URL='https://kibana.example.com' -export KIBANA_USERNAME='elastic' -export KIBANA_PASSWORD='use-a-secret-provider' - -npx -y @havesomecode/kibana-mcp-server bootstrap --client codex -unset KIBANA_BASE_URL KIBANA_USERNAME KIBANA_PASSWORD -``` +Missing values are prompted for. For automation, provision `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` first. Never pass a password in an argument; use environment, `--password-env NAME`, or `--password-stdin`. -Bootstrap verifies only the Kibana connection, creates an empty managed catalog, stores the credentials securely and registers the server. It does not list, scan, infer, or configure indexes: the agent asks which exact index or pattern to use, then calls `configure_index`. +1. Clear those variables from the server environment after bootstrap so it uses the saved profile. Bash: `unset KIBANA_BASE_URL KIBANA_USERNAME KIBANA_PASSWORD`; PowerShell instructions are in [INSTALL.md](INSTALL.md). +2. Open a fresh agent session and call `discover`. +3. Name the exact index or pattern when asked. The agent calls `configure_index`, then `discover` again. It must not guess, enumerate, scan, or auto-select indexes. -Clear the connection variables from the MCP server environment after bootstrap so startup uses the saved profile. Direct environment connections take precedence over profiles and require an explicit catalog path or store. +Bootstrap saves and verifies the connection, not index access. Startup requires a connection; direct environment startup overrides profiles and needs `KIBANA_SOURCE_CATALOG_PATH` or a configuration-file store. No implicit checkout catalog or MCP credential input is supported. -The server requires a connection at startup. Use bootstrap, a configuration file, or connection environment variables with an explicit `KIBANA_SOURCE_CATALOG_PATH`. It does not accept credentials through MCP tools or search the checkout for a catalog. `bootstrap` always saves only the connection. Bare `setup` guides hand-authored catalog import; `setup` with connection options uses the same bootstrap flow. - -Never pass passwords as command-line arguments. Use `KIBANA_PASSWORD`, `--password-env NAME`, or `--password-stdin`. - -Install the agent skill that teaches the investigation workflow: +## Install the investigation skill ```bash npx skills add Havesomecode/kibana-mcp-server \ --skill kibana-log-investigation \ - --agent codex \ - --global \ - --yes + --agent codex --global --yes ``` -`INSTALL.md` is the deterministic install and handoff guide for agents. +Use [INSTALL.md](INSTALL.md) for profiles, plugin installation, or hand-authored catalogs. Bare `setup` imports a hand-authored catalog; with connection options it runs bootstrap. + +## MCP tools -For a version-matched handoff, resolve the selected branch, tag, or package metadata to the full commit SHA. That SHA is the **resolved installation commit** used in `https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md`. Read `INSTALL.md` and the skills from the same commit. Do not infer a Git tag from an npm version; use the exact package's `gitHead`. A maintained checkout can verify the handoff with `node scripts/readme-handoff.mjs `. +| Tool | Use | +| --- | --- | +| `discover` | List configured sources for the caller. | +| `describe_fields` | Find fields and exact-match support. | +| `query` | Search sources over an absolute time window: `hits`, `count`, `histogram`, `terms`, `stats`, or `grouped_top_hits`. | +| `filter` | Filter a known exact field. | +| `configure_index` | Validate and save the user's exact index or pattern. | ## Run it as a shared service -By default the server speaks stdio and serves one configuration. Over HTTP, one process serves every caller of your gateway, each with their own source catalog: +Use HTTP behind an authenticated gateway to give each caller a separate catalog. Stdio serves one configuration. ```ts // kibana.mcp.config.ts @@ -86,11 +61,21 @@ export default defineKibanaMcpConfig({ kibana-mcp-server serve --http --config ./kibana.mcp.config.ts ``` -Catalogs can live in files, Redis, PostgreSQL, MongoDB, SQLite, or a store you write. The HTTP transport trusts the profile header as sent: only a gateway that has authenticated the caller should reach it. +**Restrict access to the authenticated gateway:** the server trusts its profile header. Stores: files, Redis, PostgreSQL, MongoDB, SQLite, or a custom store. ## Repository Codex plugin -`npm run build` creates a self-contained runtime in `plugins/kibana-log-investigation/dist/`. The repo-local Codex plugin launches that installed bundle through `mcp-launcher.mjs`, without depending on the original checkout or its `node_modules`. Use `npm run verify:entrypoint` for the isolated copied-plugin handshake, and inspect Codex's MCP startup status plus sanitized stderr before restarting a discussion. +Run `npm run build` before installing the plugin. Its bundled runtime needs no original checkout or `node_modules`. Check startup status and sanitized stderr before restarting a discussion; `npm run verify:entrypoint` tests the copied plugin. + +## Version-matched agent handoff + +Resolve the full **commit SHA** before installation: this is the **resolved installation commit**. Read README, INSTALL, and skills from that commit: + +```text +https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md +``` + +Do not infer a Git tag from an npm version. Use the exact package's `gitHead`; pin the same package version. From a maintained checkout, verify with `node scripts/readme-handoff.mjs `. Missing metadata or failed retrieval must be reported, not replaced with master. ## Documentation @@ -104,16 +89,7 @@ Catalogs can live in files, Redis, PostgreSQL, MongoDB, SQLite, or a store you w | Integration examples (gateway, Ts.ED, Docker) | https://havesomecode.github.io/kibana-mcp-server/integrations/ | | Tool, CLI and configuration reference | https://havesomecode.github.io/kibana-mcp-server/reference/cli | -The documentation sources live in `website/`. - -## Project status - -- Published package: `npx -y @havesomecode/kibana-mcp-server` -- Agent skill: `npx skills add Havesomecode/kibana-mcp-server --skill kibana-log-investigation` -- Repo-local Codex plugin install remains a first-class path for cloned checkouts -- GitHub Releases are the canonical release notes and version record; the `version` field in `package.json` inside git is not the release source of truth -- Support posture, compatibility matrix, distribution strategy and release checklist live in `docs/project/` -- Contributing and security reporting live in `CONTRIBUTING.md` and `SECURITY.md` +Documentation sources: `website/`. Support and release policy: `docs/project/`. Version record: GitHub Releases, not the checkout's `package.json`. See [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md) for contributions and private vulnerability reports. ## Development diff --git a/docs/project/distribution-strategy.md b/docs/project/distribution-strategy.md index 65188de..606b997 100644 --- a/docs/project/distribution-strategy.md +++ b/docs/project/distribution-strategy.md @@ -6,87 +6,55 @@ updated: 2026-04-09 # Distribution Strategy -## Goals - -- Make the MCP installable by AI agents without manual explanation. -- Make prompt-free package bootstrap and skills.sh discovery the guaranteed newcomer path. -- Preserve the repo-local Codex plugin workflow for contributors. -- Keep the public package path healthy and aligned with the repo-local workflow. +Keep agent installation self-contained. Make prompt-free package bootstrap and skills.sh discovery the guaranteed newcomer path; preserve repo-local installation for contributors under the same contract. ## Supported Distribution Paths ### 1. Public package plus Agent Skill (guaranteed) -The newcomer path is: - 1. Install `kibana-log-investigation` from `Havesomecode/kibana-mcp-server` with the `skills` CLI. 2. Run the package's prompt-free `bootstrap --client codex` command. -3. Let bootstrap verify only the Kibana connection, save an empty managed catalog, register through `codex mcp add`, and read the registration back. -4. In a fresh agent session, ask the user which exact index or pattern to configure before calling `configure_index`. +3. Confirm bootstrap verified only the connection, saved an empty managed catalog, registered through `codex mcp add`, and read registration back. +4. Start a fresh agent session. Ask for the exact index or pattern before calling `configure_index`. Bootstrap never lists, scans, infers, or configures indexes. Index validation begins only after the user explicitly supplies one. ### 2. Repo-local Codex plugin -This is the primary, always-supported path: - 1. Clone the repo. 2. `npm install` 3. `npm run build` 4. Install the `Kibana Log Investigation` plugin from the repo marketplace in Codex. -This path is required for development and is the baseline for support. +This always-supported path is required for development and is the support baseline. ### 3. Public package without the skill -The package is published for agent-friendly execution via: +Run `npx -y @havesomecode/kibana-mcp-server bootstrap --client codex`, or have the MCP client invoke the published binary instead of a repo-local build. -- `npx -y @havesomecode/kibana-mcp-server bootstrap --client codex` -- MCP clients that invoke the published package binary instead of a repo-local build +Keep this live path aligned with repo-local installation and the hosted homepage. Before publishing: -This path depends on the following remaining true: - -- npm package ownership is under maintainer control for the chosen package name -- Verified `npm pack` contents (runtime entrypoint, plugin metadata, README, LICENSE). -- CI and release workflows green on the supported Node line. -- Clear support policy and compatibility matrix published. -- Trusted publishing enabled (OIDC), no long-lived publish tokens. - -This path is live and should stay aligned with the repo-local install story and the hosted homepage. +1. Confirm maintainers control the npm package identity. +2. Verify `npm pack` contents: runtime entrypoint, plugin metadata, README, LICENSE. +3. Pass CI and release workflows on the supported Node line. +4. Publish the support policy and compatibility matrix. +5. Use OIDC trusted publishing; never keep long-lived publish tokens. ## Artifact Boundary -The release artifact must include: - -- `dist/` runtime build output -- plugin metadata and MCP config -- `skills/kibana-log-investigation/SKILL.md` -- `README.md`, `LICENSE` - -The release artifact must exclude: - -- tests and fixtures -- local operator catalogs and saved machine profiles -- development-only scripts and tooling that do not affect runtime execution +| Include | Exclude | +| --- | --- | +| `dist/` runtime output | Tests and fixtures | +| Plugin metadata and MCP config | Local operator catalogs and saved machine profiles | +| `skills/kibana-log-investigation/SKILL.md` | Development-only scripts and tooling unrelated to runtime | +| `README.md`, `LICENSE` | | ## Versioning -Releases use semantic versioning driven by `semantic-release`. - -The authoritative release record is: - -- Git tags -- npm package versions -- GitHub Releases - -The repository does not commit generated version bumps or release notes back into git. -The `version` field in `package.json` on `master` is therefore a source manifest value, not an authoritative shipped version. +Use `semantic-release` for semantic versioning. Read shipped versions from Git tags, npm, or GitHub Releases—not `package.json` on `master`. Generated version bumps and release notes are not committed back to Git. ## Ongoing Obligations -Keep public publishing healthy by ensuring all of the following remain true: - -- the npm package identity stays under maintainer control -- `npm run verify` continues to enforce the published artifact boundary -- release automation remains green on the supported Node line -- the hosted homepage, repo-local install path, and support policy describe the same install contract +1. Keep npm identity under maintainer control and release automation green on the supported Node line. +2. Keep `npm run verify` enforcing the artifact boundary. +3. Keep the homepage, repo-local instructions, and support policy on the same install contract. diff --git a/docs/project/npm-publishing.md b/docs/project/npm-publishing.md index 89709ac..144c303 100644 --- a/docs/project/npm-publishing.md +++ b/docs/project/npm-publishing.md @@ -6,53 +6,32 @@ updated: 2026-04-09 # npm Publishing Setup -This repo publishes to npm from GitHub Actions using trusted publishing. - ## Package Identity The published package is `@havesomecode/kibana-mcp-server`. -The release workflow assumes both of these remain true: - -- the maintainer account controls the npm scope `@havesomecode` and the published package -- npm trusted publishing points to this repository's `release.yml` workflow - -If the owner or workflow changes, update the trusted publisher before the next release. +Confirm maintainer control of `@havesomecode` and `@havesomecode/kibana-mcp-server`. Point npm trusted publishing to this repository's `release.yml`; update it before releasing if ownership or the workflow changes. ## Repo-Side Publish Workflow -The authoritative workflow file is `.github/workflows/release.yml`. - -It is responsible for: - -- running `npm run verify` on `master` pushes before any publish attempt -- running `semantic-release` on normal `master` pushes -- publishing to npm automatically when merged history contains a releasable semantic change -- creating Git tags and GitHub Releases from the same release event -- using GitHub OIDC instead of npm automation tokens +Use `.github/workflows/release.yml`. On normal `master` pushes it runs `npm run verify` before `semantic-release`. Releasable history produces an npm version, Git tag, and GitHub Release from one event, using GitHub OIDC—not npm automation tokens. ## npm-Side Setup -For future maintenance: - -1. Open the package settings on npmjs.com. -2. Add a trusted publisher. -3. Choose GitHub Actions. -4. Set: +1. Open package settings on npmjs.com; add a **GitHub Actions** trusted publisher. +2. Set: - Organization or user: `Havesomecode` - Repository: `kibana-mcp-server` - Workflow filename: `release.yml` - Environment name: leave blank unless GitHub environments are added later -5. Save the trusted publisher. -6. Merge a PR into `master` with a semantic title such as `feat: ...` or `fix: ...`. -7. Keep token-based publishing disabled in npm package settings after trusted publishing is verified. +3. Save the publisher. +4. Merge a PR into `master` with a semantic title such as `feat: ...` or `fix: ...`. +5. After verifying trusted publishing, keep token-based publishing disabled in npm settings. ## Publish Expectations -- The workflow requires GitHub-hosted runners. -- The workflow requires `id-token: write`. -- The package must remain public for provenance to be generated automatically. -- The package `repository` field must match this public GitHub repository exactly. -- The npm package name in `package.json` must stay aligned with the package configured in npm trusted publishers. -- Squash merge should be enabled so the semantic PR title becomes the final commit title on `master`. -- The published version is determined by `semantic-release` and recorded in npm, tags, and GitHub Releases; the `package.json` version committed in git is not the canonical released version. +1. Use GitHub-hosted runners with `id-token: write`. +2. Keep the package public for automatic provenance. Match its `repository` field exactly to this public GitHub repository. +3. Match the `package.json` name to the npm trusted publisher's package. +4. Enable squash merge so semantic PR titles become commit titles on `master`. +5. Read `semantic-release` versions from npm, tags, or GitHub Releases; the committed `package.json` version is not canonical. diff --git a/docs/project/release-checklist.md b/docs/project/release-checklist.md index d93d7cf..9cee8ed 100644 --- a/docs/project/release-checklist.md +++ b/docs/project/release-checklist.md @@ -6,54 +6,38 @@ updated: 2026-04-09 # Release Checklist -## Preconditions - -- CI is green on `master`. -- `npm run verify` passes locally. -- Build output must correspond to current source: `npm run build` replaces root `dist/`, and every packed `dist/src/**/*.js` or `dist/src/**/*.d.ts` must have its corresponding `src/**/*.ts` file. `npm run verify:packlist` rejects orphaned outputs from removed modules. -- The compatibility matrix is up to date (`docs/project/compatibility-matrix.md`). -- Any new env or config requirements are documented in `README.md` and `INSTALL.md`. -- Startup requires a connection and an explicit catalog path or store; saved profiles use their stored path. Docs must not advertise a credential-taking MCP setup tool or implicit checkout catalog. -- Connection-only bootstrap remains the default install flow; guided setup remains supported for explicitly selected hand-authored catalogs. -- The hosted homepage still matches the supported install paths and support posture. -- The npm scope and package name (`@havesomecode/kibana-mcp-server`) are owned by the maintainers. -- Maintainers understand that released versions are tracked in npm, tags, and GitHub Releases, not by the `package.json` version committed in git. - -## Release Authorization - -- Releases are created via the `Release` GitHub Actions workflow. -- Trusted publishing for npm must point at workflow file `release.yml`. -- Tags and GitHub Releases are created by `semantic-release` after a verified push to `master`. -- PRs that should affect semver must use a Conventional Commit title because squash merge will carry that title onto `master`. -- Only maintainers listed in `CODEOWNERS` should have permission to trigger releases. - -## Security Posture for CI/Release - -- Workflow permissions must be minimal (`contents: read` for CI, `contents: write` + `id-token: write` for publish). -- Publishing to npm must use OIDC trusted publishing. -- Do not store long-lived npm tokens in repo secrets. -- After trusted publishing is verified, disallow token-based publishing in npm package settings. +## Before merging + +1. Require green CI on `master` and local `npm run verify`. +2. Check build freshness: `npm run build` replaces root `dist/`. Every packed `dist/src/**/*.js` or `dist/src/**/*.d.ts` needs its corresponding `src/**/*.ts`; `npm run verify:packlist` rejects orphaned output. +3. Update `docs/project/compatibility-matrix.md`, and document new env/config requirements in `README.md` and `INSTALL.md`. +4. Check the install contract and hosted homepage: connection-only bootstrap is default; guided setup supports explicitly selected hand-authored catalogs. Startup needs a connection and explicit catalog path/store; saved profiles use their stored path. Never advertise credential-taking MCP setup or implicit checkout catalogs. +5. Confirm maintainers own the npm scope and `@havesomecode/kibana-mcp-server`, and the maintainer account can publish it. + +## Authorize publishing + +1. Restrict release triggers to maintainers in `CODEOWNERS`. Use the `Release` GitHub Actions workflow and point npm trusted publishing at `release.yml`. +2. Use minimal permissions: `contents: read` for CI; `contents: write` + `id-token: write` for publishing. +3. Use OIDC trusted publishing. Never store long-lived npm tokens in repo secrets; disallow token-based publishing in npm settings after verifying OIDC. ## Threat Model (Top 3) -1. **Compromised GitHub Actions workflow**: mitigate with minimal permissions, required reviews, and avoiding untrusted actions. -2. **Malicious dependency update**: mitigate with dependency review workflow and human review of automated PRs. -3. **Accidental secrets leakage in artifacts**: mitigate with `verify-packlist` and explicit exclusion of local config files. +| Threat | Required mitigation | +| --- | --- | +| Compromised GitHub Actions workflow | Minimal permissions, required reviews, no untrusted actions | +| Malicious dependency update | Dependency review workflow and human review of automated PRs | +| Secrets leaked in artifacts | `verify-packlist` and explicit exclusion of local config files | ## Steps -1. Open a PR with a semantic title such as `feat: ...`, `fix: ...`, or `feat!: ...`. -2. Run `npm run verify`. -3. Confirm the npm package identity is publishable by this maintainer account. -4. Configure npm trusted publishing for `release.yml`. -5. Merge with squash merge into `master`. -6. Confirm the `Release` workflow either exits with no release or publishes a new version. -7. Validate the Git tag, GitHub Release notes, published package contents, and provenance. -8. Confirm the `Pages` workflow still deploys the homepage surface and that the install copy remains truthful. +1. Open a PR with a Conventional Commit title such as `feat: ...`, `fix: ...`, or `feat!: ...`; squash merge carries it onto `master` for semver. +2. Complete the checks and authorization above, then squash merge into `master`. +3. Confirm `Release` either exits without a release or publishes. After a verified push to `master`, `semantic-release` creates the tags and GitHub Releases. +4. Validate the Git tag, GitHub Release notes, package contents, and provenance. +5. Confirm `Pages` deploys the homepage and its install copy remains truthful. ## Post-Release -- Confirm install instructions still match the shipped artifact. -- Confirm the hosted homepage still matches the shipped artifact and current support posture. -- Update any compatibility claims that changed in the release. -- If docs mention a specific current version, verify it against npm or GitHub Releases rather than reading `package.json` in git. +1. Match install instructions and the homepage to the shipped artifact and support posture. +2. Update changed compatibility claims. +3. Read released versions from npm, tags, or GitHub Releases—not committed `package.json`. Verify any current-version claim against npm or GitHub Releases. diff --git a/docs/project/support-policy.md b/docs/project/support-policy.md index 2bff95d..12b9c07 100644 --- a/docs/project/support-policy.md +++ b/docs/project/support-policy.md @@ -6,56 +6,53 @@ updated: 2026-10-14 # Support Policy -## Audience - -This MCP is designed for operators and AI agents that need read-only access to Kibana-backed logs for investigation workflows. +This MCP supports read-only Kibana log investigation by operators and AI agents. Kibana, Elasticsearch, and data-view writes, and automation beyond log investigation, are out of scope. ## Guaranteed Support -- Node.js 22+ runtime -- Repo-local Codex plugin install -- `kibana_internal_search_es` and `elasticsearch_search` backends -- Connection-only bootstrap with saved profiles and platform credentials -- Guided machine setup for explicitly selected hand-authored source catalogs -- Startup with a saved profile, or a complete explicit connection plus `KIBANA_SOURCE_CATALOG_PATH` or a configuration-file catalog store -- Explicit user-approved source configuration through `configure_index`; no MCP tool accepts connection credentials +| Area | Guaranteed contract | +| --- | --- | +| Runtime | Node.js 22+ | +| Installation | Repo-local Codex plugin | +| Search | `kibana_internal_search_es` and `elasticsearch_search` | +| Connection setup | Connection-only bootstrap, saved profiles, platform credentials | +| Guided setup | Explicitly selected hand-authored source catalogs | +| Startup | Saved profile, or complete explicit connection plus `KIBANA_SOURCE_CATALOG_PATH` or configuration-file catalog store | +| Source configuration | User-approved `configure_index`; no MCP tool accepts connection credentials | ## Connection Setup Boundary -Connection setup is an operator step, not an MCP capability. The credential-taking `configure` tool and unconfigured startup are intentionally retired in favor of `bootstrap` plus `configure_index`. Direct environment startup must select a catalog path or store explicitly; the server no longer searches the working directory for one. This does not change the catalog JSON schema or delete existing catalog or profile files. +1. Run operator-side `bootstrap` for the connection, then user-approved `configure_index` for sources. Credential-taking `configure` and unconfigured startup are intentionally retired. +2. Select a catalog path/store explicitly for direct environment startup. No working-directory catalog is searched. Catalog JSON and existing catalog/profile files are unchanged. +3. Bootstrap before reconnecting if startup reports `No saved Kibana connection is available. Run connection bootstrap first.` For saved profiles, leave direct connection variables unset in the server environment. ## Best-Effort Support -- Fully autonomous repo-local plugin installation by prompting alone across all Codex model variants -- Schema metadata endpoints (`kibana_data_views_fields`, `kibana_index_patterns_fields`, `elasticsearch_field_caps`) -- Nested query features that depend on schema availability -- Linux secure storage when a Secret Service-compatible keyring is missing or locked - -Best-effort means we will investigate issues but cannot guarantee behavior across all deployments. +| Area | Limitation | +| --- | --- | +| Autonomous repo-local installation | Prompt-only completion varies by Codex model | +| Schema metadata | `kibana_data_views_fields`, `kibana_index_patterns_fields`, `elasticsearch_field_caps` may be unavailable | +| Nested queries | Depend on schema availability | +| Linux secure storage | Missing or locked Secret Service-compatible keyring | -Startup without a saved connection fails with `No saved Kibana connection is available. Run connection bootstrap first.` Run connection-only bootstrap before reconnecting the MCP client. For saved-profile use, leave direct connection variables unset in the server environment. +We investigate best-effort issues without guaranteeing behavior across deployments. ## Known Environment Constraints -- Some Codex model variants are less reliable at completing repo-local plugin installation without a manual Codex UI step. The supported fallback is to install the plugin manually from the local marketplace, then continue with MCP configuration. -- Some Kibana deployments proxy or block schema metadata endpoints. When all known Kibana data-view or index-pattern metadata endpoints return `404`, the server can sample fields through the configured search backend. Direct Elasticsearch field-capability errors do not use this fallback. Sampling is heuristic, not authoritative; schema-dependent features can still be unavailable. -- `KIBANA_BASE_URL` must be the Kibana base prefix, not a full search endpoint such as `/internal/search/es`. -- Linux secure storage depends on a Secret Service-compatible keyring being available and unlocked. -- If you run staging and production side by side, prefer saved machine profiles plus `KIBANA_PROFILE` selection. For direct environment startup, name `KIBANA_SOURCE_CATALOG_PATH` explicitly and keep it distinct per entry so managed sources from different environments do not mix. -- Credentials are not persisted to repo-local files or tracked config. Bootstrap and guided setup store them in the platform credential store. +1. If autonomous plugin installation stalls, install manually from the local marketplace, then continue MCP configuration. This is a supported fallback. +2. Treat search-backend field sampling as heuristic, not authoritative. It applies only when all known Kibana data-view/index-pattern metadata endpoints return `404`, never to direct Elasticsearch field-capability errors. Schema-dependent features may remain unavailable. +3. Set `KIBANA_BASE_URL` to the Kibana base prefix, not a search endpoint such as `/internal/search/es`. +4. Keep Linux's Secret Service-compatible keyring available and unlocked. Bootstrap and guided setup store credentials in the platform credential store, never repo-local files or tracked config. +5. Separate staging and production with saved profiles and `KIBANA_PROFILE`. For direct environment startup, set a distinct `KIBANA_SOURCE_CATALOG_PATH` per entry to avoid mixing managed sources. ## Security Reporting -If you find a security issue, report it through the process documented in `SECURITY.md`. +Report security issues through `SECURITY.md`. ## Response Expectations -- Critical security reports: acknowledge within 3 business days. -- Non-security issues: response cadence depends on maintainer availability. -- Release notes and shipped version history are published through GitHub Releases and npm, not through committed changelog updates in the repository. -- Questions about the current shipped version should be answered from npm or GitHub Releases, not from the `package.json` version committed in git. - -## Out of Scope - -- Writes to Kibana, Elasticsearch, or data views -- Workflow-specific automation beyond log investigation +| Request | Expectation | +| --- | --- | +| Critical security report | Acknowledge within 3 business days | +| Non-security issue | Depends on maintainer availability | +| Release notes or shipped versions | Read GitHub Releases or npm, not committed changelogs or the Git `package.json` version | diff --git a/docs/solutions/documentation-gaps/self-contained-codex-install-handoff-2026-04-08.md b/docs/solutions/documentation-gaps/self-contained-codex-install-handoff-2026-04-08.md index 289b46e..5f2e692 100644 --- a/docs/solutions/documentation-gaps/self-contained-codex-install-handoff-2026-04-08.md +++ b/docs/solutions/documentation-gaps/self-contained-codex-install-handoff-2026-04-08.md @@ -25,56 +25,45 @@ tags: [codex, install, plugin, multi-environment, kibana] ## Context -The repo-local install path for this Kibana MCP worked in practice, but the handoff was still fragile. A colleague could not reliably give only the raw `INSTALL.md` link to another agent and expect the full setup flow to succeed without extra human explanation about the repo name, plugin location, build requirements, fallback behavior, or environment-specific configuration rules. +The 2026-04-08 handoff needed extra explanation about repository, build, plugin fallback, and environment isolation. Older Codex model variants stalled on plugin installation; endpoint-shaped `KIBANA_BASE_URL` values caused avoidable `404`s. -The friction showed up most clearly when older Codex model variants failed to complete the repo-local plugin install autonomously, and when one workspace needed both staging and production MCP entries. The setup guidance also left room for a common configuration mistake: setting `KIBANA_BASE_URL` to a full Kibana API endpoint instead of the base prefix that the server should join with source backend paths. +## Repo-local handoff steps -## Guidance +Use `INSTALL.md` as the canonical operator handoff, including for raw-link setup, multiple environments, host-variable mapping, and base-URL troubleshooting. -Treat `INSTALL.md` as the canonical operator handoff for this repo. An agent that receives only the raw file link should still be able to infer all of the following without extra prompting: +1. With Node.js 22+, run `git clone https://github.com/Havesomecode/kibana-mcp-server` and `cd kibana-mcp-server`. Read `INSTALL.md` first. +2. Resolve the full installation commit SHA using the commands below, then `git checkout `. Verify README, INSTALL, and skills from that commit. Stop on missing refs or files; never substitute master. +3. Run `npm install` and `npm run build`. -- the GitHub repository to clone -- the repo-scoped plugin name and location -- the required Node.js major and build steps -- the manual plugin-install fallback path if the model cannot complete the Codex UI step itself -- the MCP configuration and verification flow +### Install and connect -For multi-environment setup, ask the user for a short profile name such as `staging`, `prod`, `preprod`, or `qa`, then run connection-only bootstrap with `--profile `. Bootstrap assigns that profile its machine-local catalog; do not invent profile names or catalog contents. Call `configure_index` only after the user explicitly names an index or pattern. +1. In Codex, install **Kibana Log Investigation** from `.agents/plugins/marketplace.json`; plugin root: `plugins/kibana-log-investigation`. If the model stalls, use the manual fallback below and wait for confirmation. +2. Ask for a short profile name, such as `staging` or `prod`. Supply credentials through operator-side environment input or secure prompts. On Linux, unlock the Secret Service keyring. +3. Run `node dist/src/bin/kibana-mcp-server.js bootstrap --profile --client none`. Do not register a duplicate standalone MCP or invent catalog contents. Bootstrap verifies the connection, stores platform credentials, and creates an empty machine-local catalog without listing, scanning, inferring, or configuring indexes. +4. Restart Codex, load `skills/kibana-log-investigation/SKILL.md` from the same commit, and check MCP tools. Ask for the exact index or pattern before `configure_index`. -For direct environment-based startup, the operator must explicitly provide `KIBANA_SOURCE_CATALOG_PATH` or a configuration-file store. Keep catalogs distinct across environments; no checkout catalog is loaded implicitly. +### Resolve the installation commit -If the operator already keeps target-specific host variables such as `KIBANA_BASE_URL_STAGING` and `KIBANA_BASE_URL_PROD`, preserve that host-side convention if it is useful to them. For direct environment-based MCP entries, map those values into the standard runtime variables expected by the server: +| Input | Resolve the SHA | +| --- | --- | +| Checkout branch/tag | `git rev-parse ^{commit}` | +| Detached checkout | `git rev-parse HEAD` | +| Exact npm version | `npm view @havesomecode/kibana-mcp-server@ gitHead` | -- `KIBANA_BASE_URL` -- `KIBANA_USERNAME` -- `KIBANA_PASSWORD` -- `KIBANA_SOURCE_CATALOG_PATH` +Do not infer a Git tag from an npm version. Pin the same exact package version. Fetch missing selected refs explicitly; absent metadata is an error. -Treat manual plugin installation as a supported continuation path, not as a setup failure. If a model cannot complete the repo-local plugin install from prompting alone, the agent should tell the user exactly which Codex UI steps to click, wait for confirmation, then continue with configuration and verification. +Verify the commit-pinned README with `node scripts/readme-handoff.mjs `. The repository-only helper compares fetched bytes against Git and returns `url` and `commit` without installing or configuring anything. Historical checkouts without it need explicit resolution and retrieval verification. -Keep `KIBANA_BASE_URL` limited to the Kibana base prefix. Endpoint paths such as `/internal/search/es` belong in the source backend configuration, not in `KIBANA_BASE_URL`. +## Keep environments separate -## Why This Matters - -The main benefit is durable handoff quality. A colleague should be able to send only the raw `INSTALL.md` link to another agent and get the same setup flow every time, instead of relying on undocumented tribal knowledge about the repo structure or Codex behavior. - -The profile-name rule matters because saved profiles keep connections and catalog paths distinct. `configure_index` persists sources in the selected managed catalog; sharing one catalog across environments would mix their sources. Guided setup can instead import a hand-authored catalog explicitly selected by the operator. - -The env-var mapping rule matters because operators often already have separate secrets or shell variables for each target environment. Allowing those host-side names while standardizing each MCP entry onto the server's `KIBANA_*` contract keeps the server simple without forcing users to reorganize their environment management. - -The `KIBANA_BASE_URL` clarification matters because the wrong value looks plausible but produces a bad combined URL at runtime. Calling out correct and incorrect shapes removes a common source of confusing 404 debugging. - -## When to Apply - -- When this repo is being handed to another operator or agent for first-time setup from a cloned checkout -- When the handoff must work from the raw `INSTALL.md` link alone -- When one Codex workspace needs both staging and production MCP entries -- When the operator already uses target-specific host variables and needs them mapped into the server's runtime variables -- When a user reports 404s after setup and the configured `KIBANA_BASE_URL` may include a full endpoint path +1. Give each saved profile its own catalog. `configure_index` writes to the selected managed catalog; guided `setup` instead imports an explicitly selected hand-authored catalog. +2. For direct environment startup, provide `KIBANA_SOURCE_CATALOG_PATH` or a configuration-file store. Keep catalogs distinct; no checkout catalog loads implicitly. +3. Preserve useful host-specific variables such as `KIBANA_BASE_URL_STAGING` and `KIBANA_BASE_URL_PROD`; map each entry to standard `KIBANA_BASE_URL`, `KIBANA_USERNAME`, `KIBANA_PASSWORD`, and `KIBANA_SOURCE_CATALOG_PATH` as shown below. +4. Set `KIBANA_BASE_URL` to the Kibana base prefix. Put paths such as `/internal/search/es` in the source backend configuration, not the base URL. ## Examples -Resolve the selected branch, tag, or package metadata to the full installation commit SHA before handing off. Branch and tag names are resolution inputs, not durable README URLs. Do not infer a Git tag from an npm version or silently fall back to master. Read `INSTALL.md` and skills from the same resolved installation commit. The current [installation guide](../../../INSTALL.md#version-matched-readme-handoff) documents the resolver and retrieval checks. +Copy the resolved commit into this handoff; branch/tag names are resolution inputs, not durable README URLs: ```text For the resolved installation commit, read this README: @@ -89,14 +78,14 @@ Manual plugin-install fallback: If you cannot complete the repo-local plugin install yourself, open this repo in Codex, open plugins/kibana-log-investigation, install "Kibana Log Investigation" from the local marketplace, restart Codex if needed, then continue with MCP configuration. ``` -Named saved profiles: +Standalone package registration with named profiles (not alongside the repo-local plugin): ```bash npx -y @havesomecode/kibana-mcp-server bootstrap --profile staging --client codex npx -y @havesomecode/kibana-mcp-server bootstrap --profile prod --no-default --client codex ``` -Saved-profile server processes should not receive direct connection variables: those select environment startup and require an explicit catalog or store. No connection means startup fails with `No saved Kibana connection is available. Run connection bootstrap first.` +Leave direct connection variables unset for saved-profile server processes; they select environment startup and require an explicit catalog/store. Without a saved connection, startup fails with `No saved Kibana connection is available. Run connection bootstrap first.` Mapping target-specific host variables into the standard runtime variables for one MCP entry: diff --git a/docs/solutions/integration-issues/kibana-mcp-schema-endpoints-may-be-unavailable-2026-04-03.md b/docs/solutions/integration-issues/kibana-mcp-schema-endpoints-may-be-unavailable-2026-04-03.md index 9097b52..fed7089 100644 --- a/docs/solutions/integration-issues/kibana-mcp-schema-endpoints-may-be-unavailable-2026-04-03.md +++ b/docs/solutions/integration-issues/kibana-mcp-schema-endpoints-may-be-unavailable-2026-04-03.md @@ -20,42 +20,31 @@ tags: [mcp, kibana, schema, proxy, 404] ## Problem -The Kibana MCP can expose schema-aware features only when the configured source has a working schema backend. In some deployments, all supported schema backend kinds return `404`, which makes `describe_fields` unusable and blocks schema-dependent query validation. +Schema-aware features need a working schema backend. Some deployments return `404` for every supported kind, blocking `describe_fields` and schema-dependent query validation. ## Observed Attempts -Live checks were run against a proxied Kibana deployment and a structured log source: +Live checks against one proxied Kibana deployment and structured log source returned: -- `kibana_data_views_fields` with `/api/data_views/fields_for_wildcard` -- `kibana_index_patterns_fields` with `/api/index_patterns/_fields_for_wildcard` -- `elasticsearch_field_caps` with `//_field_caps` - -All three returned `404 Not Found`. +| Backend | Path | Result | +| --- | --- | --- | +| `kibana_data_views_fields` | `/api/data_views/fields_for_wildcard` | `404 Not Found` | +| `kibana_index_patterns_fields` | `/api/index_patterns/_fields_for_wildcard` | `404 Not Found` | +| `elasticsearch_field_caps` | `//_field_caps` | `404 Not Found` | ## What Works -The search backend itself still works through Kibana internal search: - -- `backend.kind = kibana_internal_search_es` -- `backend.path = /internal/search/es` - -So source discovery and log querying remain usable even when schema metadata is absent. +In that deployment, `backend.kind = kibana_internal_search_es` with `backend.path = /internal/search/es` still supported source discovery and log queries without schema metadata. ## Recommended Guidance -For deployments with this shape: - -- configure the appropriate supported schema backend; Kibana data-view and index-pattern backends try their known metadata endpoints -- for a Kibana data-view or index-pattern schema backend, if all known metadata endpoints return `404`, the server falls back to sampling fields through the configured search backend; direct `elasticsearch_field_caps` errors do not use that fallback -- treat sampled capabilities as heuristic, not authoritative; they can recover `.keyword` promotion and object paths, but cannot prove nested mappings -- if schema loading and sampling both fail, use configured field hints and non-schema-dependent `discover`, `filter`, and `query` operations +1. Configure a supported schema backend. Kibana data-view and index-pattern backends try their known metadata endpoints. +2. If all those endpoints return `404`, use their configured-search-backend sampling fallback. Direct `elasticsearch_field_caps` errors never use it. +3. Treat sampled capabilities as heuristic: they can recover `.keyword` promotion and object paths, but cannot prove nested mappings. +4. If metadata and sampling both fail, use configured field hints and non-schema-dependent `discover`, `filter`, and `query` operations. ## Verification Checklist -Before re-enabling schema-dependent behavior in a deployment, confirm one of the following works end to end: - -- Kibana data views field metadata endpoint -- Kibana legacy index patterns field metadata endpoint -- direct Elasticsearch field capabilities endpoint - -If none respond, select a Kibana data-view or index-pattern schema backend and verify its search-backend sampling fallback. Inspect the returned field capabilities and query advisories; the response does not expose a sampling-provenance field. Do not equate sampled fields with authoritative mapping metadata. +1. Before re-enabling schema-dependent behavior, verify an end-to-end metadata path: Kibana data views, legacy index patterns, or direct Elasticsearch field capabilities. +2. If none respond, select a Kibana data-view/index-pattern backend and verify sampling through its search backend. +3. Inspect field capabilities and query advisories. Responses expose no sampling-provenance field; sampled fields are not authoritative mappings. diff --git a/docs/sources/handoff.md b/docs/sources/handoff.md index bb553ff..3cb5fe4 100644 --- a/docs/sources/handoff.md +++ b/docs/sources/handoff.md @@ -2,156 +2,44 @@ ## Goal -Build a minimal MCP interface that lets another agent investigate runtime behavior in logs without direct Kibana access. - -This handoff is intentionally abstracted from any single business case. The objective is to provide the smallest useful log-investigation surface for operational validation and incident work. +Original design handoff: a minimal MCP for agents investigating runtime logs without direct Kibana access. Its scope was operational validation and incidents, not a specific business case. This is the initial design, not the current tool inventory. ## Minimum toolset -The minimum useful MCP surface is: - -1. `discover` -2. `query` - -That is enough for the intended workflows. +The proposed first version exposed `discover` and `query`; more tools were deferred until that model proved insufficient. ## `discover` -### Purpose - -Let an agent find which log sources are relevant before querying them. - -### Required capabilities - -- list available log sources -- search sources by keyword -- return identifiers and basic metadata for each source +The design required source listing and keyword search so agents could select sources before querying. Each result included a source id, human-readable name, time field, and short description when available. -### Expected output - -For each source, return at least: - -- source id -- human-readable name -- time field -- short description if available - -The result should make source selection straightforward for an agent. +Short service, domain, event, or environment keywords were the intended inputs. ## `query` -### Purpose - -Let an agent query logs in a structured and repeatable way. - -### Required capabilities - -- select one or more sources -- filter by time range -- filter by exact fields -- filter by free-text keywords -- sort by time -- limit result size -- return matching raw documents - -### Optional capabilities - -If easy to add within the same tool, support: - -- count-only queries -- simple aggregations -- grouped counts over time - -These are useful, but not required for the first version. - -## Core agent workflows this MCP must support - -The agent must be able to use these tools to answer generic runtime questions such as: - -### 1. Find the relevant execution window - -Given a timestamp, a rough keyword, or a functional area, find the relevant logs around the event. - -### 2. Correlate multiple sources +The design required repeatable queries with: -Inspect one source for execution logs and another for API or application behavior in the same time window. +| Input | Requirement | +| --- | --- | +| Sources | One or more source ids from `discover` | +| Time | Absolute start/end timestamps | +| Filters | Exact-match fields and free-text keywords | +| Ordering and size | Explicit sort order, time sorting, explicit limits | +| Output | Matching raw documents, clear hits without wrapper noise | -### 3. Narrow the investigation iteratively +Count-only queries, simple aggregations, and grouped counts over time were optional if easy to add within the same tool—not first-version requirements. -Support a workflow like: +## Intended investigation workflow -- start with a broad time window -- find candidate events -- narrow the range -- inspect the exact execution span - -### 4. Extract structured evidence - -Allow the agent to retrieve enough detail to build a report containing: - -- timestamps -- identifiers -- durations or timing-related fields if present -- event names -- relevant field values - -### 5. Compare before / during / after - -Allow the agent to compare log activity and request behavior across several windows around the same event. - -## Agent-facing ergonomics - -The tools should be optimized for repeated use by another agent. - -### `discover` - -Should work well with short search inputs like: - -- service name -- domain keyword -- event keyword -- environment keyword - -### `query` - -Should work well with: - -- source ids from `discover` -- absolute start/end timestamps -- exact-match field filters -- text filters -- explicit sort order -- explicit limits - -The response should expose hits clearly, without unnecessary wrapper noise. +1. Find logs around a timestamp, rough keyword, or functional area. +2. Correlate execution logs with API/application behavior across sources in the same window. +3. Start broad, find candidate events, narrow the range, then inspect the exact execution span. +4. Extract timestamps, identifiers, event names, relevant fields, and durations/timing fields when present. +5. Compare log activity and request behavior before, during, and after the event. ## Success criteria -This MCP is sufficient if another agent can: - -1. find the right log sources -2. query them over precise time ranges -3. correlate activity across sources -4. extract concrete evidence for a runtime investigation -5. produce a compact validation or incident report from those results +Success meant finding the right sources, querying precise ranges, correlating activity, and turning concrete runtime evidence into a compact validation or incident report. ## Non-goals -This MCP does not need to: - -- manage dashboards -- edit visualizations -- mutate saved objects -- expose administration features -- model business-specific workflows directly - -The goal is a focused query surface for agents, not a full Kibana management interface. - -## Suggested final shape - -A good first version can expose exactly: - -- `discover` -- `query` - -Only add more tools later if the two-tool model proves insufficient. +Dashboard management, visualization edits, saved-object mutations, administration, and business-specific workflow modeling were excluded. The goal was a focused agent query surface, not full Kibana management. diff --git a/plugins/kibana-log-investigation/skills/ensure-node-runtime/SKILL.md b/plugins/kibana-log-investigation/skills/ensure-node-runtime/SKILL.md index 1c97ffe..b4b7422 100644 --- a/plugins/kibana-log-investigation/skills/ensure-node-runtime/SKILL.md +++ b/plugins/kibana-log-investigation/skills/ensure-node-runtime/SKILL.md @@ -1,118 +1,73 @@ --- name: ensure-node-runtime -description: Ensure Node.js 22+ is available for this repo using a user-level install path on Windows, macOS, or Linux. +description: Check or install Node.js 22+ at user scope. --- # Ensure Node Runtime -Use this skill when the repository has been cloned locally and Codex needs to make sure Node.js is available before building or running the Kibana MCP server. +Run these before building the cloned repository: -This skill prefers user-level installation paths and avoids system-wide package installs. +```bash +node --version +npm --version +``` -## Goals +If both work and Node is 22+, stop. Otherwise install at user scope with `fnm`; avoid system-wide installs. Choose the section matching the OS and shell. -- detect whether `node` and `npm` are already usable -- install Node.js 22+ at user scope when missing -- choose commands based on the current operating system and shell -- verify the runtime before continuing with repo setup +## Windows: PowerShell -## Detection +1. If `fnm` is missing, run `winget install Schniz.fnm`. If winget is blocked or unavailable, download the official Windows `fnm` release binary, extract it under the user's home, and add that directory to the user PATH. Do not assume Scoop exists. +2. Configure the current session: -1. Check whether `node` and `npm` are already present: - - `node --version` - - `npm --version` -2. If `node` is present and major version is `22` or newer, stop here. -3. Otherwise install a user-level Node version manager, then install Node.js `22`. + ```powershell + fnm env --use-on-cd --shell powershell | Out-String | Invoke-Expression + ``` -## Preferred cross-platform strategy +3. Create `$PROFILE` if needed. Add that same line once for future sessions. +4. [Install and verify Node](#install-and-verify-node). -Use `fnm` as the default version manager because it works on Windows, macOS, and Linux and supports shell auto-configuration. +## macOS or Linux -The repo pins the desired major in `.node-version`, so after `fnm` is installed you can use either: +1. If `fnm` is missing, run its official user-level installer: -- `fnm install 22 && fnm default 22` -- or `fnm use --install-if-missing` + ```bash + curl -fsSL https://fnm.vercel.app/install | bash + ``` -## Windows +2. Configure the active shell; add the same line once to its startup file. -Prefer a PowerShell-first install path. + | Shell | Current session | Startup file | + | --- | --- | --- | + | Bash | `eval "$(fnm env --use-on-cd)"` | `~/.bashrc` | + | Zsh | `eval "$(fnm env --use-on-cd)"` | `~/.zshrc` | + | Fish | `fnm env --use-on-cd \| source` | `~/.config/fish/config.fish` | -Use `winget` when it is available. If it is blocked or unavailable, fall back to the official `fnm` release binary extracted into the user's home directory. +3. [Install and verify Node](#install-and-verify-node). -Reference basis: +## Install and verify Node -- `fnm` officially documents Windows installation via `winget` -- `fnm` also supports installation from a release binary on Windows -- PowerShell profile setup is the standard way to enable `fnm` in Windows shells +1. Install the repository's Node major: -### Recommended flow + ```bash + fnm install 22 + fnm default 22 + ``` -1. If `fnm` already exists, skip to Node installation. -2. If `winget` is available: - - `winget install Schniz.fnm` -3. If `winget` is not available or is blocked: - - download the latest Windows `fnm` release binary from the official GitHub releases page - - extract it into a user-level directory such as `%USERPROFILE%\\.local\\bin\\fnm` or `%USERPROFILE%\\.fnm\\bin` - - add that directory to the user PATH in PowerShell -4. Configure the current PowerShell session: - - `fnm env --use-on-cd --shell powershell | Out-String | Invoke-Expression` -5. Persist shell integration for future PowerShell sessions: - - create `$PROFILE` if needed - - append `fnm env --use-on-cd --shell powershell | Out-String | Invoke-Expression` to `$PROFILE` if it is not already present -6. Install and select Node.js 22: - - `fnm install 22` - - `fnm default 22` + In the checkout, `fnm use --install-if-missing` also reads `.node-version`. -## macOS +2. Verify the selected runtime: -Prefer a user-level `fnm` install using the official install script. + ```bash + fnm --version + node --version + npm --version + ``` -### Recommended flow + Node must report 22+ and npm must work. If PATH has not refreshed, restart the terminal or Codex and check again. -1. If `fnm` is missing: - - `curl -fsSL https://fnm.vercel.app/install | bash` -2. Configure the current shell: - - bash/zsh: `eval "$(fnm env --use-on-cd)"` - - fish: `fnm env --use-on-cd | source` -3. Persist shell integration: - - bash: add `eval "$(fnm env --use-on-cd)"` to `~/.bashrc` - - zsh: add `eval "$(fnm env --use-on-cd)"` to `~/.zshrc` - - fish: add `fnm env --use-on-cd | source` to `~/.config/fish/config.fish` -4. Install and select Node.js 22: - - `fnm install 22` - - `fnm default 22` +3. Continue setup: -## Linux - -Use the same user-level `fnm` flow as macOS. - -### Recommended flow - -1. If `fnm` is missing: - - `curl -fsSL https://fnm.vercel.app/install | bash` -2. Configure and persist shell integration for the active shell: - - bash: `eval "$(fnm env --use-on-cd)"` plus `~/.bashrc` - - zsh: `eval "$(fnm env --use-on-cd)"` plus `~/.zshrc` - - fish: `fnm env --use-on-cd | source` plus `~/.config/fish/config.fish` -3. Install and select Node.js 22: - - `fnm install 22` - - `fnm default 22` - -## Verification - -After installation, verify all of: - -- `fnm --version` -- `node --version` -- `npm --version` - -The resulting `node --version` must report major `22` or higher. - -## Notes - -- Prefer user-level installs over admin installs. -- On Windows, prefer PowerShell-native flows and avoid assuming Scoop is available. -- If the current shell cannot pick up the new PATH cleanly, restart the terminal or restart Codex after installation. -- Once Node is available, continue with: - - `npm install` - - `npm run build` + ```bash + npm install + npm run build + ``` diff --git a/plugins/kibana-log-investigation/skills/install-and-configure/SKILL.md b/plugins/kibana-log-investigation/skills/install-and-configure/SKILL.md index 6ea3e38..999ed66 100644 --- a/plugins/kibana-log-investigation/skills/install-and-configure/SKILL.md +++ b/plugins/kibana-log-investigation/skills/install-and-configure/SKILL.md @@ -5,51 +5,59 @@ description: Install the repo plugin with connection-only bootstrap. # Install and Configure -Use this skill only when the repository has been cloned locally and Codex needs the repo-scoped plugin. For public installs, use the canonical `skills/kibana-log-investigation` skill and the published npm package instead. +Build the cloned checkout before installing its Codex plugin. Public npm installs use `skills/kibana-log-investigation` instead. -## Version-matched README handoff +## Build and install -Resolve the selected branch or tag to the full installation commit SHA before following installation instructions. Read the README at this immutable URL: +1. [Resolve the installation commit](#pin-the-handoff) and verify Node.js 22+. Use `ensure-node-runtime` if missing. +2. Run from the repository root: -```text -https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md -``` + ```bash + npm install + npm run build + ``` -Branch and tag names are resolution inputs, not final handoff URLs. Use `INSTALL.md` and skills from the same resolved installation commit. Do not infer a Git tag from an npm version or silently fall back to master. Detached installs use `git rev-parse HEAD`'s full commit SHA. +3. Confirm `plugins/kibana-log-investigation/dist/mcp_entry.mjs` exists. Install `Kibana Log Investigation` from the repo-local Codex marketplace. The built plugin includes its runtime dependencies; it runs from its installed directory, not the checkout. +4. Restart Codex only if the plugin directory does not refresh. -From a maintained repository root containing the helper, run `node scripts/readme-handoff.mjs `. Include its verified `url` and `commit` in agent handoffs. The helper resolves local refs and verifies the downloaded README against the selected Git object. Unknown refs, missing READMEs, or retrieval/content verification failures must be reported instead of presenting another version's instructions. The helper is repository-only; historical installations without it require explicit ref resolution and README retrieval verification. +## Save the connection -## Workflow +1. Obtain `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD`, or safe CLI alternatives. Use the base URL prefix, without `/internal/search/es`. Never put passwords in arguments or summaries. +2. Bootstrap without duplicate registration; do not edit client files. -1. Ensure Node.js 22+ is available. Use `ensure-node-runtime` when needed. -2. From the repository root: + ```bash + node dist/src/bin/kibana-mcp-server.js bootstrap --client none + ``` -```bash -npm install -npm run build -``` + First install creates an empty catalog without inspecting indexes. Do not import the bundled example unless requested. Use `--profile NAME` for additional environments; outside the plugin, select them with `serve --profile NAME`. -3. Confirm the build produced `plugins/kibana-log-investigation/dist/mcp_entry.mjs`. Install `Kibana Log Investigation` from the repo-local Codex marketplace only after building. The built plugin includes runtime dependencies and launches from its installed directory, not the checkout. -4. Restart Codex if the plugin directory does not refresh automatically. -5. Resolve the Kibana connection through `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD`, or safe CLI alternatives. -6. Bootstrap only the connection, without registering a duplicate global MCP: +3. Clear connection variables from the plugin environment before reconnecting. -```bash -node dist/src/bin/kibana-mcp-server.js bootstrap --client none -``` + Bash: `unset KIBANA_BASE_URL KIBANA_USERNAME KIBANA_PASSWORD` -7. Confirm first-time bootstrap reports an empty source catalog. It must not inspect or configure indexes. Remove `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` from the plugin process environment before reconnecting so startup uses the saved profile. In Bash: `unset KIBANA_BASE_URL KIBANA_USERNAME KIBANA_PASSWORD`; in PowerShell: `Remove-Item Env:KIBANA_BASE_URL, Env:KIBANA_USERNAME, Env:KIBANA_PASSWORD -ErrorAction SilentlyContinue`. -8. In a fresh Codex session, call `discover`. When it reports no sources, ask the operator which exact index or index pattern to configure. -9. Only after the operator answers, call `configure_index` with exactly that value. Then run `discover` again. + PowerShell: `Remove-Item Env:KIBANA_BASE_URL, Env:KIBANA_USERNAME, Env:KIBANA_PASSWORD -ErrorAction SilentlyContinue` -## Rules +## Select an index -- Do not edit client configuration files. -- Separate saved configuration, process startup, tools in the current discussion, and real backend access. If startup fails, inspect sanitized stderr and repair/reinstall the plugin before requesting another discussion. Run `npm run verify:entrypoint` for the isolated packaged-plugin handshake; it is not a native Codex or real-backend check. -- Do not use the bundled example catalog unless the operator explicitly requests example data. -- Do not put passwords in command arguments or summaries. -- Do not guess, enumerate, scan, or auto-select Kibana indexes. -- Do not call `configure_index` until the operator explicitly names the index or pattern. -- Use `--profile NAME` for additional environments. -- Use `serve --profile NAME` when invoking a non-default profile outside the repo plugin. -- `KIBANA_BASE_URL` is a base prefix. Do not append `/internal/search/es`. +1. In a fresh session, call `discover`. +2. If empty, ask for the exact index or pattern and wait. Never guess, enumerate, scan, or auto-select. +3. Call `configure_index` with exactly that value, then `discover` again. + +## Fix startup before restarting again + +Inspect MCP startup status and sanitized stderr. Repair and reinstall a failing plugin first. Saved configuration, process startup, tools in a discussion, and backend access are separate checks. + +`npm run verify:entrypoint` tests the isolated packaged plugin, not native Codex or a real backend. + +## Pin the handoff + +1. Resolve a full **commit SHA** using `git rev-parse ^{commit}`, or `git rev-parse HEAD` for detached installs. Use README, INSTALL, and skills from this **resolved installation commit**. +2. Verify the commit-pinned README: + + ```text + https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md + ``` + + In a maintained checkout: `node scripts/readme-handoff.mjs `. Hand off the verified `url` and `commit`. + +Do not infer a Git tag from an npm version. Never substitute master on missing refs, README, or retrieval failures. The helper compares fetched bytes with Git; historical installs without it need explicit resolution and retrieval verification. It is repository-only, not an npm command. diff --git a/skills/kibana-log-investigation/SKILL.md b/skills/kibana-log-investigation/SKILL.md index e3dae06..2c14aec 100644 --- a/skills/kibana-log-investigation/SKILL.md +++ b/skills/kibana-log-investigation/SKILL.md @@ -5,140 +5,75 @@ description: Bootstrap and use the Kibana log investigation MCP safely. # Kibana Log Investigation -Use this skill when an operator wants to install, configure, or use the read-only Kibana MCP server from `@havesomecode/kibana-mcp-server`. +Call `discover` before investigating. For installation, follow the steps below. -## Version-matched installation instructions +## Install the connection -Before installing or generating a handoff, resolve the selected branch, tag, or package metadata to the full installation commit SHA. Read `README.md` at this immutable URL: +1. Verify Node.js 22+ and [resolve the installation version](#pin-the-installation-version). +2. Obtain `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` from trusted inputs. If unavailable, report what is missing. Never put passwords in arguments, logs, summaries, or client files. Alternatives: `--url URL`, `--username USER`, `--password-env NAME`, or `--password-stdin`. +3. Choose the registration mode. Never edit client configuration yourself. -```text -https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md -``` - -Branch and tag names are resolution inputs, not final handoff URLs. Read `INSTALL.md` and skills from the same resolved installation commit. Do not infer a Git tag from an npm version. Resolve the exact npm package's `gitHead` instead, or report missing metadata; pin the package installation to the same exact version. Never silently fall back to master if the ref or README is missing or retrieval fails. - -In a maintained clone with the repository-only helper, run `node scripts/readme-handoff.mjs ` and use its verified `url` and `commit` in the handoff. The helper checks the retrieved README against that Git commit before output. It is not an npm command and need not exist in historical tags. If unavailable, resolve the selected ref and validate its raw README explicitly; do not invent successful retrieval. See `INSTALL.md`'s version-matched handoff section. - -## Connection-only installation - -Do not edit Codex, Claude Code, or other MCP configuration files by hand. Bootstrap establishes only the Kibana connection and an empty managed source catalog. It must not list, scan, infer, or configure indexes. - -1. Verify Node.js 22 or newer is available. -2. Resolve connection values from `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD`, unless the operator supplied safe alternatives. -3. Choose the client mode for the agent you are running in (see below). -4. Run the prompt-free bootstrap with that mode. - -### Choosing the client mode + | Agent | Registration owner | Mode | + | --- | --- | --- | + | Codex | bootstrap | `--client codex` | + | Claude Code | plugin | `--client none` | + | Other agent | operator | `--client none` | -Bootstrap can register the MCP server itself only in Codex. Everywhere else the server is registered by a plugin or by the operator, and bootstrap must only save the connection. +4. Run bootstrap with the chosen mode; supplied connection values make it prompt-free. -| You are running in | The MCP server is registered by | Client mode | -|---|---|---| -| Codex | bootstrap | `--client codex` | -| Claude Code | the `kibana-log-investigation` plugin | `--client none` | -| Any other agent | the operator's MCP configuration | `--client none` | + ```bash + npx -y @havesomecode/kibana-mcp-server bootstrap --client codex + ``` -Codex: + **Expected:** verified Kibana status, OS-stored credentials, and an empty catalog on first install. No index inspection; reruns preserve managed sources. -```bash -npx -y @havesomecode/kibana-mcp-server bootstrap \ - --client codex -``` - -Claude Code and other agents: - -```bash -npx -y @havesomecode/kibana-mcp-server bootstrap \ - --client none -``` + Codex registration is version-pinned, transport-hashed, and read back exactly. Failed registration restores prior profile state. `--client none` leaves client files untouched. -If you are not running in Codex and the tools of this server (`discover`, `configure_index`) are not available in the session, the server is not registered. Do not write MCP configuration yourself. Ask the operator to register it: + **Do not bypass failures:** never overwrite a conflict, add `--replace` to empty a catalog, or import the bundled example automatically. -- in Claude Code, by installing the plugin: `/plugin marketplace add Havesomecode/kibana-mcp-server`, then `/plugin install kibana-log-investigation@kibana-mcp-server`; -- in any other agent, by declaring the stdio command `npx -y @havesomecode/kibana-mcp-server serve` in its MCP configuration. +5. Remove connection variables from the server environment and reconnect. Saved profiles provide their catalog; direct environment startup overrides profiles and needs `KIBANA_SOURCE_CATALOG_PATH` or a configuration-file store. No implicit checkout catalog or credential-taking MCP tool exists. -With `--client none`, reconnect the MCP server or start a new session after bootstrap. A server without a configured connection fails at startup; it does not wait for an MCP tool to supply credentials. +If non-Codex tools are absent, ask the operator to register the server: -### What bootstrap does - -Bootstrap must: - -- validate the Kibana base URL and verify the connection through Kibana status only -- write an empty managed catalog on first install -- preserve previously and explicitly configured generated sources on idempotent reruns -- save credentials in the operating-system credential store -- with `--client codex`, register the exact, version-pinned stdio command under a transport-hashed Codex name, read the registration back, and reject conflicting entries -- with `--client none`, leave every MCP client configuration untouched -- restore prior profile state if registration fails - -A successful command verifies installation, not an index. Do not claim an index is usable until the explicit configuration flow below succeeds. - -## Supplying secrets safely - -Never put a password directly in a command argument. Prefer a pre-provisioned `KIBANA_PASSWORD` environment variable. The examples below use `--client codex`; substitute the client mode chosen above. When a secret is available on stdin, use: - -```bash -printf '%s\n' "$SECRET_VALUE" | npx -y @havesomecode/kibana-mcp-server bootstrap \ - --password-stdin \ - --client codex -``` - -If the password is stored under another environment variable, reference its name without exposing its value: - -```bash -npx -y @havesomecode/kibana-mcp-server bootstrap \ - --password-env KIBANA_PROD_PASSWORD \ - --client codex -``` +- Claude Code: `/plugin marketplace add Havesomecode/kibana-mcp-server`, then `/plugin install kibana-log-investigation@kibana-mcp-server`. +- Other agents: stdio command `npx -y @havesomecode/kibana-mcp-server serve`. -## Explicit index configuration +For `No saved Kibana connection is available. Run connection bootstrap first.`, bootstrap and reconnect; do not send credentials through MCP. -After bootstrap: +## Configure the selected index -1. Call `discover`. -2. If no source is configured, ask the user which exact Kibana index or index pattern they want to use. -3. Wait for the user's answer. Never guess, enumerate, scan, or auto-select an index. -4. Call `configure_index` exactly once with the user's value in `index`. -5. Report the configured `source_id` and time field. If validation fails, preserve the empty catalog and report the error. -6. Call `discover` again before beginning an investigation. +1. Call `discover`. If empty, ask for the exact index or pattern. +2. Wait for the answer. Never guess, enumerate, scan, or auto-select. +3. Call `configure_index` once with that exact `index`. It accepts no credentials and runs field inspection plus one bounded validation search for that index only. +4. Report `source_id` and time field. On failure, preserve the catalog and report the error; bootstrap success alone does not prove index access. +5. Call `discover` again before investigating. -`configure_index` contains no credential fields. It may inspect fields and perform one bounded validation search only for the exact user-approved index or pattern. +## Investigate -## Profiles and overrides +1. Use the source id returned by `discover`. +2. Call `describe_fields` before unfamiliar filters or aggregations. +3. Use a bounded time range and small result limit. +4. Keep queries read-only and cite returned evidence, not unseen logs. -| Value | Flag | Environment fallback | -|---|---|---| -| Profile | `--profile prod` | `KIBANA_PROFILE`, then `default` | -| Base URL | `--url https://kibana.example.com` | `KIBANA_BASE_URL` | -| Username | `--username elastic` | `KIBANA_USERNAME` | -| Password | `--password-stdin` or `--password-env NAME` | `KIBANA_PASSWORD` | -| Client | `--client codex` or `--client none` | `codex` | +## Choose a profile -To select an existing non-default profile: +Use `--profile NAME`; fallback is `KIBANA_PROFILE`, then `default`. Start an existing profile with: ```bash npx -y @havesomecode/kibana-mcp-server serve --profile prod ``` -For saved-profile startup, do not pass `KIBANA_BASE_URL`, `KIBANA_USERNAME`, or `KIBANA_PASSWORD` to the server process. Supplying all three selects direct environment startup, which requires an explicit `KIBANA_SOURCE_CATALOG_PATH` or a configuration-file catalog store. Saved profiles already carry their catalog path; no checkout catalog is loaded implicitly. Guided `setup` remains available when the operator explicitly wants to import a hand-authored catalog. +Bare `setup` is for operator-requested hand-authored catalog import, not automatic installation. -## Failure rules +## Pin the installation version -- Never silently import the bundled example catalog. -- Never pass an index to bootstrap; bootstrap is connection-only. -- Never call `configure_index` until the user explicitly names the index or pattern. -- Never add `--replace` merely to make a failed install pass. It intentionally replaces an existing catalog with an empty managed catalog. -- Never overwrite a conflicting MCP client registration. -- Never write or edit an MCP client configuration file to register the server; only bootstrap in Codex, a plugin, or the operator does that. -- If startup fails with `No saved Kibana connection is available. Run connection bootstrap first.`, run connection-only bootstrap, then reconnect the server. No MCP tool accepts connection credentials. -- Never expose a password in logs, summaries, shell history, generated catalogs, or MCP client configuration. -- If URL, username, or password is unavailable, report the exact missing prerequisite rather than inventing it. +1. Resolve the full **commit SHA**: checkout branch/tag or detached `HEAD`; for npm, the exact package's `gitHead`. This is the **resolved installation commit**. Do not infer a Git tag from an npm version. Pin the package to that exact version. +2. Read README, INSTALL, and skills from that commit. -## Using the MCP + ```text + https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md + ``` -After a source is explicitly configured: +3. In a maintained clone, run `node scripts/readme-handoff.mjs `. Hand off its verified `url` and `commit`. -1. Use `discover` to obtain the source id. -2. Use field description before constructing unfamiliar filters or aggregations. -3. Start with a bounded time range and a small result limit. -4. Keep investigations read-only and cite returned evidence rather than inferring unseen log content. +The helper checks retrieved README bytes against Git. It is repository-only, not an npm command. If absent in a historical install, resolve and verify retrieval explicitly. Missing refs, metadata, README, or failed retrieval are errors: never invent success or substitute master. diff --git a/website/ai/agent-plugins.md b/website/ai/agent-plugins.md index 25f03f3..2421fa2 100644 --- a/website/ai/agent-plugins.md +++ b/website/ai/agent-plugins.md @@ -4,12 +4,7 @@ description: "Install the Kibana log investigation plugin for Claude Code or Cod # Agent plugins and skills -The repository ships an agent plugin that bundles two things: - -- **the `kibana-log-investigation` skill**, which teaches the agent the investigation workflow and its safety rules; -- **the Kibana MCP server**, registered for you. - -Pick the channel that matches your agent. +Choose your agent below. Plugins install the `kibana-log-investigation` skill and register the MCP server; skills.sh installs only the skill. | Channel | Skill | MCP server | Agents | | --- | --- | --- | --- | @@ -17,25 +12,25 @@ Pick the channel that matches your agent. | [Codex](#codex) | Yes | Yes | Codex CLI and the Codex app | | [skills.sh](#skills-sh-any-agent) | Yes | Manual | Cursor, Gemini CLI, Windsurf and others | -Whatever the channel, the server needs a Kibana connection before it can answer. See [Connect to Kibana](#connect-to-kibana). +Then [Connect to Kibana](#connect-to-kibana). Registration alone does not save a connection. ## Claude Code -Add the marketplace, then install the plugin: +Run: ```bash claude plugin marketplace add Havesomecode/kibana-mcp-server claude plugin install kibana-log-investigation@kibana-mcp-server ``` -Or from inside a Claude Code session: +Or run inside Claude Code: ```text /plugin marketplace add Havesomecode/kibana-mcp-server /plugin install kibana-log-investigation@kibana-mcp-server ``` -To share the plugin with your team, declare it in the `.claude/settings.json` of your project: +For team installation, add this to the project's `.claude/settings.json`: ```json { @@ -53,17 +48,17 @@ To share the plugin with your team, declare it in the `.claude/settings.json` of } ``` -The plugin registers the MCP server as `npx -y @havesomecode/kibana-mcp-server serve`. Claude loads the skill on its own when a task matches it. +Expected: the plugin registers `npx -y @havesomecode/kibana-mcp-server serve`. Claude loads the skill for matching tasks. ## Codex -The recommended path for Codex is the bootstrap command, which verifies the connection and registers the server in one step: +1. Run connection bootstrap. Supply credentials through a trusted environment or its secret prompt, never password arguments: ```bash npx -y @havesomecode/kibana-mcp-server bootstrap --client codex ``` -Then add the skill: +2. Install the skill: ```bash npx skills add Havesomecode/kibana-mcp-server --skill kibana-log-investigation --agent codex --global --yes @@ -71,23 +66,29 @@ npx skills add Havesomecode/kibana-mcp-server --skill kibana-log-investigation - ### Codex plugin marketplace -The repository also declares a Codex marketplace in `.agents/plugins/marketplace.json`. It is meant for contributors working from a cloned checkout. Run `npm install` and `npm run build` before installing it: the build produces a self-contained runtime under `plugins/kibana-log-investigation/dist/`, and the installed plugin launches that local bundle without depending on the checkout or its `node_modules`. +For a cloned checkout, use `.agents/plugins/marketplace.json`: + +1. Run `npm install` and `npm run build` before installing the plugin. The build creates `plugins/kibana-log-investigation/dist/`; the installed bundle needs neither the checkout nor its `node_modules`. +2. Check Codex's MCP startup status before restarting a discussion. On failure, inspect sanitized stderr; rebuild and reinstall a missing runtime. -Check Codex's MCP process startup status before restarting a discussion. If startup fails, inspect sanitized stderr, rebuild, and reinstall the plugin. `npm run verify:entrypoint` exercises a copied installed plugin with a synthetic connection; it does not verify native Codex startup or a real Kibana backend. See [Getting started](../guide/getting-started#use-in-codex). +`npm run verify:entrypoint` checks a copied plugin with a synthetic connection, not native Codex startup or a real Kibana backend. See [Getting started](../guide/getting-started#use-in-codex). ### Version-matched handoff -Before handing a branch, tag, or package installation to another agent, resolve it to the full commit SHA. That SHA is the **resolved installation commit** used in the immutable README URL: +1. Resolve the branch, tag, or package installation to its full commit SHA: the **resolved installation commit**. +2. Hand off this immutable README URL: ```text https://raw.githubusercontent.com/Havesomecode/kibana-mcp-server//README.md ``` -Read `INSTALL.md` and the skills from that same commit. Branch and tag names are resolution inputs, not final handoff URLs. Do not infer a Git tag from an npm version; use the exact package's `gitHead`. In a maintained checkout, `node scripts/readme-handoff.mjs ` verifies the commit-pinned README before emitting the handoff. +3. Read `INSTALL.md` and the skills from that same commit. Branches and tags are resolution inputs, not final URLs. + +Do not infer a Git tag from an npm version; use the exact package's `gitHead`. In a maintained checkout, run `node scripts/readme-handoff.mjs ` to verify the commit-pinned README before handoff. ## skills.sh (any agent) -The [`skills`](https://skills.sh/) CLI copies the skill into the directory your agent reads. +Run the [`skills`](https://skills.sh/) CLI: ```bash # pick the agents interactively @@ -105,7 +106,7 @@ skills.sh installs the skill only. Declare the MCP server yourself with the snip ## Declare the MCP server manually -For any MCP client, the server is a stdio command. It requires Node.js 22 or newer. +With Node.js 22+, add the stdio command to your MCP client: ::: code-group @@ -139,11 +140,11 @@ args = ["-y", "@havesomecode/kibana-mcp-server", "serve"] ::: -A team that runs the server [over HTTP](../deployment/http) behind a gateway declares the URL of that gateway instead. +For a shared server [over HTTP](../deployment/http), declare the gateway URL instead. ## Connect to Kibana -A plugin or a manual declaration only registers the server. Save the connection once per machine, without registering a client: +1. Provision credentials through a secret provider, then save the connection once per machine. `--client none` avoids duplicate registration: ```bash export KIBANA_BASE_URL='https://kibana.example.com' @@ -153,8 +154,11 @@ export KIBANA_PASSWORD='use-a-secret-provider' npx -y @havesomecode/kibana-mcp-server bootstrap --client none ``` -Bootstrap verifies the connection, stores the credentials in the secret store of the operating system and creates an empty source catalog. Start the MCP server without direct connection variables so it uses the saved profile, then reconnect the client or start a new session: the agent asks which exact index or index pattern to use, and calls `configure_index` with your answer. +Expected: a verified connection, credentials in the OS secret store, and an empty source catalog. Bootstrap does not inspect indexes. + +2. Remove direct connection variables from the MCP server environment to use the saved profile. Reconnect the client or start a new session. +3. Call `discover`. If the catalog is empty, ask for the exact index or pattern and wait. Never guess, list, or scan indexes; call `configure_index` only with the user's answer. -Without a saved connection the server fails at startup with `No saved Kibana connection is available. Run connection bootstrap first.` Direct environment-based startup requires an explicit `KIBANA_SOURCE_CATALOG_PATH` or configuration-file store, never an implicit checkout catalog. +Without a saved connection, startup fails: `No saved Kibana connection is available. Run connection bootstrap first.` Direct environment startup requires an explicit `KIBANA_SOURCE_CATALOG_PATH` or configuration-file store; no implicit checkout catalog. -You can also let the agent do it. The skill knows which mode fits the agent it runs in: `--client codex` in Codex, where bootstrap registers the server itself, and `--client none` in Claude Code and elsewhere, where a plugin or your own configuration does. +For agent-run bootstrap, use `--client codex` in Codex; use `--client none` where a plugin or manual configuration already registers the server. diff --git a/website/ai/index.md b/website/ai/index.md index c73bb4b..b6c0315 100644 --- a/website/ai/index.md +++ b/website/ai/index.md @@ -4,7 +4,7 @@ description: "How Kibana MCP Server supports AI agents - LLM-friendly documentat # Built for agents -Kibana MCP Server exists so that an agent can investigate logs. The project treats agents as first-class readers and users, in three ways. +Choose how to give your agent the server, workflow, or documentation: | | What you get | Where | | --- | --- | --- | @@ -14,19 +14,19 @@ Kibana MCP Server exists so that an agent can investigate logs. The project trea ## What the skill teaches -The `kibana-log-investigation` skill tells an agent how to behave around this server: +The `kibana-log-investigation` skill follows this sequence: -- bootstrap the connection with the mode that fits the agent (Codex, Claude Code or another one), without editing configuration files by hand, and never put a password in a command argument; -- call `discover` first, and when no source exists, ask the user which exact Kibana index or index pattern to use; -- call `configure_index` only with the value the user gave, never a guessed or enumerated one; -- inspect fields with `describe_fields` before filtering, then query over an absolute time window. +1. Bootstrap with the agent's client mode. Do not hand-edit configuration files or put passwords in command arguments. +2. Call `discover`. If empty, ask the user for the exact Kibana index or pattern and wait. Never guess, enumerate, or scan indexes. +3. Call `configure_index` only with the user's value. +4. Inspect `describe_fields` before filtering. Query an absolute time window. ## Pointing an agent at this documentation -Give your agent the index of the documentation and let it fetch the pages it needs: +Send your agent this prompt: ```text Read https://havesomecode.github.io/kibana-mcp-server/llms.txt and use it to answer questions about Kibana MCP Server. ``` -Every page also has a Markdown twin: add `.md` to its URL. +For one page, add `.md` to its URL. diff --git a/website/ai/llms.md b/website/ai/llms.md index c2dd7da..e2943c0 100644 --- a/website/ai/llms.md +++ b/website/ai/llms.md @@ -4,7 +4,7 @@ description: "The documentation is published as plain Markdown for language mode # LLM-friendly documentation -This site follows the [`llms.txt`](https://llmstxt.org/) standard: alongside the HTML pages, it publishes the same content as plain Markdown that a language model can read without parsing a web page. +Use the plain Markdown docs below. They follow the [`llms.txt`](https://llmstxt.org/) standard and match the HTML pages. ## What is published @@ -14,7 +14,7 @@ This site follows the [`llms.txt`](https://llmstxt.org/) standard: alongside the | [`/llms-full.txt`](https://havesomecode.github.io/kibana-mcp-server/llms-full.txt) | The whole documentation in a single file | You want to load everything into the context at once | | `.md` | The Markdown version of one page | You need one precise page | -For example, the reference of the `query` tool is available at: +For the `query` reference, fetch: ```text https://havesomecode.github.io/kibana-mcp-server/reference/tools/query.md @@ -22,18 +22,18 @@ https://havesomecode.github.io/kibana-mcp-server/reference/tools/query.md ## Copy a page as Markdown -Every documentation page has two buttons under its title, to copy the page as Markdown or to download it. Paste the result into a conversation when your agent cannot browse. +Click **Copy as Markdown** or **Download** under the page title. If your agent cannot browse, paste the result into its conversation. ## In an agent or an IDE -Most coding agents can fetch a URL. Tell yours where the index is: +Send your agent this prompt: ```text Use https://havesomecode.github.io/kibana-mcp-server/llms.txt as the documentation of Kibana MCP Server. ``` -In a project, add that line to your `AGENTS.md` or `CLAUDE.md` so every session starts with it. +Add that line to `AGENTS.md` or `CLAUDE.md` to reuse it each session. ## How it is generated -The files are produced at build time by [`vitepress-plugin-llms`](https://github.com/okineadev/vitepress-plugin-llms) from the same Markdown sources as the HTML pages, so the two never drift apart. +[`vitepress-plugin-llms`](https://github.com/okineadev/vitepress-plugin-llms) builds these files from the same Markdown sources as the HTML pages. diff --git a/website/deployment/catalog-stores.md b/website/deployment/catalog-stores.md index b83becf..cf1bfe4 100644 --- a/website/deployment/catalog-stores.md +++ b/website/deployment/catalog-stores.md @@ -4,9 +4,7 @@ description: "Where source catalogs can live: files, Redis, PostgreSQL, MongoDB, # Catalog stores -A source catalog is the list of logical sources a profile can query. By default catalogs are files. A **catalog store** keeps them somewhere else, one entry per profile, so that several server processes can share them and a redeploy does not lose them. - -A store is declared in the [configuration file](../guide/configuration-file): +Declare a store in the [configuration file](../guide/configuration-file): ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; @@ -21,9 +19,11 @@ export default defineKibanaMcpConfig({ }); ``` +Catalogs list the logical sources a profile can query. They default to files; a store keeps one entry per profile for shared processes and persistence across redeploys. + ## Bundled stores -Each store is imported from its own entry point, so that only the driver you use is loaded: +Import only the store you use; only its driver loads: | Import | Driver to install | `storeOptions` | | --- | --- | --- | @@ -33,14 +33,14 @@ Each store is imported from its own entry point, so that only the driver you use | `@havesomecode/kibana-mcp-server/stores/mongodb` | `mongodb` | `url`, `database`, `collectionName`, `collection` | | `@havesomecode/kibana-mcp-server/stores/sqlite` | none, uses `node:sqlite` | `path`, `table`, `database` | -- Drivers are optional peer dependencies: install the one you need next to the server. +- Install the required driver next to the server; drivers are optional peer dependencies. - `url` defaults to `KIBANA_STORE_URL`, then to the usual variable of the backend (`REDIS_URL`, `DATABASE_URL`, `MONGODB_URI` or `MONGO_URL`). For SQLite, `path` defaults to `KIBANA_STORE_URL`. - Defaults: key prefix `kibana-mcp:catalog:`, table and collection `kibana_mcp_catalogs`. The SQL stores create their table unless `createTable` is `false`. -- Pass `client`, `collection` or `database` to reuse a connection you configured yourself. The store then never closes it. -- Every store replaces a catalog only if it still holds the content that was read, so concurrent writers cannot overwrite each other. +- Pass `client`, `collection` or `database` to reuse your connection. The store never closes it. +- Stores replace catalogs only if the stored content still matches what was read; concurrent writers cannot overwrite each other. - With Redis, use an instance with persistence and a `noeviction` policy: catalogs are data, not a cache. -A store-backed catalog is read on every tool call, so several server processes sharing one store always serve the same sources. +Every tool call rereads the store-backed catalog; processes sharing a store serve the same sources. ## Choosing a store diff --git a/website/deployment/http.md b/website/deployment/http.md index e4e3ff1..ae2af63 100644 --- a/website/deployment/http.md +++ b/website/deployment/http.md @@ -4,7 +4,7 @@ description: "Run one shared server over Streamable HTTP with a source catalog p # HTTP transport -By default the server speaks stdio and serves one global configuration. To run it as a shared service, for example behind an MCP gateway, start it over Streamable HTTP: +Start a shared Streamable HTTP server behind an authenticated MCP gateway: ```bash KIBANA_BASE_URL=https://kibana.example.com \ @@ -15,17 +15,25 @@ KIBANA_HTTP_AUTH_TOKEN=... \ kibana-mcp-server serve --http --port 3000 --profile-header x-kibana-profile ``` -One process then serves every caller. Each tool call is bound to the profile named by the configured header, so callers never share a source catalog: +Only the gateway should reach the server. The profile header is trusted as sent: the server authenticates the gateway, not end users. Non-loopback listening requires `KIBANA_HTTP_AUTH_TOKEN`. -- with the `{profile}` placeholder in `KIBANA_SOURCE_CATALOG_PATH`, every profile gets its own catalog file and shares the environment connection. -- with a `store` in the configuration file (see below), every profile gets its own entry in that store. -- without bootstrap environment variables, the header selects a saved machine profile by its exact name, not its generated id. CLI profile selection still accepts case-insensitive names and ids. -- without `--profile-header`, the server refuses to start unless a default profile is named (`--profile`, `KIBANA_PROFILE` or `profile` in the configuration file); every caller is then served that one profile. Over stdio, per-profile catalogs fall back to the `default` profile. -- profile names are compared exactly: `Alice` and `alice` are two profiles. +Without `--http`, the default is stdio with one global configuration. -A profile without a catalog starts empty and fills it through `configure_index`. Profile names are turned into filesystem-safe keys before they reach a path or a store: a plain lowercase slug is kept as is, anything else gets a digest suffix. Literal names ending in the reserved `-<12 hex digits>` suffix are hashed too, so they cannot impersonate another profile's generated key. The server never adopts another identity's catalog. +## Select each caller's catalog -HTTP options, as flags or environment variables: +One process serves all callers; each tool call uses the configured profile header: + +- `{profile}` in `KIBANA_SOURCE_CATALOG_PATH`: separate catalog files, shared environment connection. +- Configuration-file [`store`](./catalog-stores): separate store entries per profile. +- Without bootstrap environment variables: selects a saved machine profile by **exact name**, not generated id. CLI selection still accepts case-insensitive names and ids. +- Without `--profile-header`: startup requires an explicit default (`--profile`, `KIBANA_PROFILE` or configuration-file `profile`); all callers use it. Over stdio, per-profile catalogs fall back to `default`. +- HTTP profile names are exact: `Alice` and `alice` are distinct. + +Missing catalogs start empty; fill them with `configure_index`. + +Profile names become filesystem-safe path/store keys: plain lowercase slugs stay unchanged; other names get a digest suffix. Literal names ending in reserved `-<12 hex digits>` are hashed too, preventing impersonation of generated keys. The server never adopts another identity's catalog. + +## HTTP options - `--host` / `KIBANA_HTTP_HOST`, default `127.0.0.1` - `--port` / `KIBANA_HTTP_PORT`, default `3000` @@ -35,10 +43,25 @@ HTTP options, as flags or environment variables: `GET /healthz` answers `200` without authentication. -The profile header is trusted as sent: the server does not authenticate end users itself. Only a gateway that has already authenticated the caller should be able to reach it. The server refuses to listen on a non-loopback host without `KIBANA_HTTP_AUTH_TOKEN`. +Without a token, MCP requests require a loopback `Host` and, if present, loopback `Origin`. Authenticated gateways may forward their own host/origin headers. + +## Legacy sessions + +1. Send `initialize` to receive an `Mcp-Session-Id`. +2. Echo it on subsequent requests, including cancellation notifications. +3. Send `DELETE` with it when finished. + +Sessions stay bound to the original profile: + +- Maximum: 128 per process; a full registry returns `503`. +- Idle expiry: 30 minutes, reclaimed on the next legacy request. +- Retirement: after 1,024 requests, bounding SDK cancellation metadata and closing remaining session exchanges. +- Expired, exhausted, unknown or mismatched session: `404`; reinitialize. + +The `2025-06-18` initialization flow supports session-based GET/SSE streams and authenticated DELETE. Disconnecting GET releases only the stream, not its session. Handshake-free legacy calls remain supported; cross-request cancellation requires a session. -Without a token, MCP requests must also have a loopback `Host` and, when present, a loopback `Origin`. Authenticated gateways may forward their own host and origin headers. +## Modern requests -Legacy clients that send `initialize` receive an `Mcp-Session-Id` and must echo it on subsequent requests, including cancellation notifications and `DELETE` when finished. Sessions are bound to the original profile, limited to 128 per process, and expire after 30 minutes without a request (reclaimed on the next legacy request). They also retire after 1,024 requests to bound cancellation metadata retained by the SDK; retirement closes any remaining exchanges in that session. A full session registry returns `503`; an expired, exhausted, unknown, or mismatched session returns `404` and requires reinitialization. Modern requests and handshake-free legacy calls remain supported, but cross-request cancellation requires a session. +For stateless `2026-07-28` tool calls, send the per-request `_meta` envelope and SDK-required `MCP-Protocol-Version`, `Mcp-Method` and `Mcp-Name` headers. Disconnecting an in-flight modern request cancels backend work; backend deadlines still bound tool calls. -The `2025-06-18` initialization flow supports session-based GET/SSE streams and authenticated DELETE. Disconnecting a GET stream releases that stream without deleting its session. Modern `2026-07-28` tool calls remain stateless: send the per-request `_meta` envelope and `MCP-Protocol-Version`, `Mcp-Method`, and `Mcp-Name` headers as required by the SDK. Disconnecting an in-flight modern request cancels its backend work; backend deadlines still bound tool calls. Authentication and profile selection are checked on every request. +Authentication and profile selection are checked on **every request**, legacy or modern. diff --git a/website/deployment/security.md b/website/deployment/security.md index 82b1ae1..92205a1 100644 --- a/website/deployment/security.md +++ b/website/deployment/security.md @@ -4,7 +4,9 @@ description: "What the HTTP transport trusts, the safeguards built in and how cr # Security model -## What the server trusts +Restrict HTTP access to an authenticated gateway. Bind to loopback or a private network; **overwrite the profile header** at the gateway, never forward a client-supplied value. + +## Gateway trust Over HTTP the server authenticates **the gateway**, not the end user. @@ -13,35 +15,31 @@ Over HTTP the server authenticates **the gateway**, not the end user. | `Authorization: Bearer ` | Proves the request comes from your gateway | It matches `KIBANA_HTTP_AUTH_TOKEN` | | Profile header | Names the caller | The gateway set it after authenticating the caller | -The profile header is taken as sent. Anyone who can reach the port and knows the shared secret can act as any profile. Two consequences: - -- **Only the gateway should reach the server.** Bind it to the loopback interface or a private network. -- **The gateway must overwrite the profile header**, never forward a client-supplied value. +The header is trusted as sent. Anyone with port access and the shared secret can act as **any profile**. ## Safeguards built in -- **No public listening without a secret.** The server refuses to start on a non-loopback host unless `KIBANA_HTTP_AUTH_TOKEN` is set. -- **No accidental shared catalog.** Over HTTP, the server refuses to start with neither a profile header nor an explicitly named default profile. -- **A missing profile is refused.** With a profile header configured, a request without it gets `401`. -- **Profile names never reach a path or key as is.** They are turned into a key of lowercase letters, digits and dashes; anything else gets a digest suffix. `../../etc/passwd` becomes `etc-passwd-`. -- **Profiles are compared exactly.** `Alice` and `alice` are two profiles with two catalogs. -- **MCP tools never accept connection credentials.** Supply the connection at startup, from saved platform credentials or explicit environment/configuration values; `configure_index` changes only the caller’s managed source catalog. +- Non-loopback listening requires `KIBANA_HTTP_AUTH_TOKEN`. +- HTTP startup requires a profile header or explicitly named default profile. With a configured header, requests missing it get `401`. +- Profile path/key values contain lowercase letters, digits and dashes; other names get a digest suffix. `../../etc/passwd` becomes `etc-passwd-`. +- Profile names are exact: `Alice` and `alice` have separate catalogs. +- **MCP tools never accept connection credentials.** Supply saved platform credentials or explicit environment/configuration values at startup. `configure_index` changes only the caller’s managed source catalog. ## Kibana credentials -- With saved profiles, credentials live in the secret store of the operating system: macOS Keychain, Windows Credential Manager, or the Secret Service on Linux. -- With environment variables or a configuration file, they live wherever you put them. Read them from the environment in the configuration file rather than writing them in it. -- Prefer environment variables over command-line flags for secrets: arguments are visible to other users of the machine. +- Saved profiles use macOS Keychain, Windows Credential Manager or Linux Secret Service. +- Environment/configuration credentials live wherever you put them. In configuration files, read secrets from the environment; do not hard-code them. +- Prefer environment variables to secret flags: other machine users can see arguments. ## What a caller can do -Every caller of a shared server queries Kibana with the same service account, so they can all read what that account can read. A profile isolates the **catalog** of a caller, the sources it configured, not the underlying data. Scope the service account to the indexes you are willing to expose, and protect the gateway endpoint with a scope reserved for the right audience. +Scope the shared service account to indexes you will expose. All shared-server callers can read what it can read: profiles isolate **catalogs**, not underlying data. Protect the gateway endpoint with a scope reserved for the intended audience. -`configure_index` lets a caller add any index pattern the service account can read to its own catalog. If that is too broad, exclude the tool at the gateway and manage catalogs yourself. +`configure_index` can add any index pattern the service account can read to the caller's catalog. If too broad, exclude it at the gateway and manage catalogs yourself. ## Read-only -The server issues search and field-metadata requests only. It never writes to Elasticsearch or Kibana. +The server sends only search and field-metadata requests; it never writes to Elasticsearch or Kibana. ## Reporting a vulnerability diff --git a/website/deployment/stdio.md b/website/deployment/stdio.md index d8d8250..481f997 100644 --- a/website/deployment/stdio.md +++ b/website/deployment/stdio.md @@ -4,21 +4,21 @@ description: "The default stdio transport: what it serves and when to move to HT # stdio transport -stdio is the default transport: the MCP client starts the server as a child process and talks to it over standard input and output. It serves **one global configuration** to its single caller. - ```bash npx -y @havesomecode/kibana-mcp-server serve ``` +stdio is the default: the MCP client starts a child process over standard input/output, serving **one global configuration** to one caller. + +Supply a connection before startup. Otherwise it fails with `No saved Kibana connection is available. Run connection bootstrap first.` + ## What it serves - After [guided setup or bootstrap](../guide/getting-started), the default saved profile. - With `--profile ` or `KIBANA_PROFILE`, that saved profile. -- With the connection in [environment variables](../reference/environment), the catalog explicitly named by `KIBANA_SOURCE_CATALOG_PATH`. No checkout file is selected implicitly. +- With an [environment connection](../reference/environment), the explicit `KIBANA_SOURCE_CATALOG_PATH` catalog. No checkout file is selected implicitly. - With a [configuration file](../guide/configuration-file) that declares a store, the profile named by `profile`, `--profile` or `KIBANA_PROFILE`, falling back to `default`. -The server must have a connection before it starts. Without a saved or explicit connection, startup fails with `No saved Kibana connection is available. Run connection bootstrap first.` - ## Running from a checkout ```bash @@ -29,7 +29,7 @@ KIBANA_SOURCE_CATALOG_PATH=config/sources.json \ npm run dev ``` -Or, after guided setup: +After guided setup, omit the environment values: ```bash npm run dev @@ -37,4 +37,4 @@ npm run dev ## When to move to HTTP -stdio needs one process per caller. Behind a gateway that serves many users, each with their own catalog, run a single [HTTP server](./http) instead. +Use one [HTTP server](./http) behind a multi-user gateway with per-user catalogs. stdio needs one process per caller. diff --git a/website/deployment/stores/custom.md b/website/deployment/stores/custom.md index 7708932..bdb3199 100644 --- a/website/deployment/stores/custom.md +++ b/website/deployment/stores/custom.md @@ -4,7 +4,9 @@ description: "Implement the CatalogStore contract to keep source catalogs in any # Custom store -A store is an object with two required methods. Implement them to keep catalogs anywhere: another database, an object bucket, an internal service. +## 1. Implement the store + +Implement `read` and `replace` for your backend: a database, object bucket or internal service. ```ts import { CatalogConflictError, type CatalogStore } from "@havesomecode/kibana-mcp-server/config"; @@ -26,7 +28,7 @@ export default function myStore(options: { table: string }): CatalogStore { } ``` -## The contract +### Contract | Method | Required | Role | | --- | --- | --- | @@ -35,17 +37,17 @@ export default function myStore(options: { table: string }): CatalogStore { | `describe(profileKey)` | no | Human-readable location reported by tools. Must not expose credentials. | | `close()` | no | Releases connections when the server stops. | -`profileKey` is derived by the server from the profile name. It only contains lowercase letters, digits and dashes, so it is safe as a file name, a key or a primary key. +The server derives `profileKey` from the profile name using only lowercase letters, digits and dashes: safe for file names, keys and primary keys. Check the optional cancellation signal after any wait and before issuing a write. Cancellation never rolls back an already-dispatched write. -## Why `replace` compares first +### Require compare-and-set -`configure_index` reads the catalog, validates the index against Kibana, then writes. Two callers, or two server processes, can do that at the same time. Writing only when the content is unchanged means the second writer fails with a conflict instead of silently erasing the first one's source. The caller simply retries. +`configure_index` reads, validates against Kibana, then writes. Concurrent callers/processes must fail with a conflict if content changed, not erase another writer's source. Retry the operation after a conflict. -## Using it +## 2. Configure the store -Create the store with its required options and give it to the configuration file. +Pass the store and its required options to the configuration file: ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; diff --git a/website/deployment/stores/mongodb.md b/website/deployment/stores/mongodb.md index aa7e683..93a926d 100644 --- a/website/deployment/stores/mongodb.md +++ b/website/deployment/stores/mongodb.md @@ -4,12 +4,16 @@ description: "Keep source catalogs in MongoDB: options, document shape and repli # MongoDB store -Keeps one document per profile. +## 1. Install the driver ```bash npm install mongodb ``` +## 2. Configure the store + +MongoDB keeps one document per profile: + ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; import store from "@havesomecode/kibana-mcp-server/stores/mongodb"; @@ -33,7 +37,7 @@ export default defineKibanaMcpConfig({ | `collection` | none | A collection you obtained yourself, used instead of `url`. The store never closes its client. | ::: warning Replica sets -A connection string listing several hosts, such as `mongodb://db1,db2,db3/logs?replicaSet=rs0`, is not accepted by the `url` option yet. Use a `mongodb+srv://` URL, or create the client yourself and pass its `collection`: +The `url` option does not yet accept multi-host strings such as `mongodb://db1,db2,db3/logs?replicaSet=rs0`. Use `mongodb+srv://`, or create a client and pass its `collection`: ```ts import { MongoClient } from "mongodb"; @@ -53,4 +57,4 @@ export default defineKibanaMcpConfig({ /* … */ store, storeOptions: { collecti ## How writes are protected -A new catalog is inserted, and a duplicate key is reported as a conflict. An existing one is updated with a filter on its previous content; an update that matches no document is reported as a conflict. +New catalogs use insert; duplicate keys report a conflict. Existing catalogs update only when previous content matches; no matching document reports a conflict. diff --git a/website/deployment/stores/postgres.md b/website/deployment/stores/postgres.md index 67e0ebf..a6537c4 100644 --- a/website/deployment/stores/postgres.md +++ b/website/deployment/stores/postgres.md @@ -4,12 +4,16 @@ description: "Keep source catalogs in PostgreSQL: options, table definition and # PostgreSQL store -Keeps one row per profile. +## 1. Install the driver ```bash npm install pg ``` +## 2. Configure the store + +PostgreSQL keeps one row per profile: + ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; import store from "@havesomecode/kibana-mcp-server/stores/postgres"; @@ -44,8 +48,8 @@ CREATE TABLE IF NOT EXISTS kibana_mcp_catalogs ( ); ``` -The database user needs `SELECT`, `INSERT` and `UPDATE` on that table, plus `CREATE` on the schema if the store creates it. +Grant the database user `SELECT`, `INSERT` and `UPDATE` on the table, plus schema `CREATE` if the store creates it. ## How writes are protected -A new catalog is inserted with `ON CONFLICT DO NOTHING`; an existing one is updated only `WHERE content = `. A write that changes no row is reported as a conflict. +New catalogs use `ON CONFLICT DO NOTHING`; existing ones update only `WHERE content = `. No changed row reports a conflict. diff --git a/website/deployment/stores/redis.md b/website/deployment/stores/redis.md index a63fc80..6abff28 100644 --- a/website/deployment/stores/redis.md +++ b/website/deployment/stores/redis.md @@ -4,7 +4,9 @@ description: "Keep source catalogs in Redis with the redis or ioredis client: op # Redis store -Keeps one key per profile, without expiry. Two entry points are provided, one per client library; they behave the same. +## 1. Install a driver + +Both entry points behave identically; choose your client: | Entry point | Driver | | --- | --- | @@ -15,6 +17,10 @@ Keeps one key per profile, without expiry. Two entry points are provided, one pe npm install redis ``` +## 2. Configure the store + +Redis keeps one key per profile, without expiry: + ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; import store from "@havesomecode/kibana-mcp-server/stores/redis"; @@ -49,13 +55,13 @@ export default defineKibanaMcpConfig({ /* … */ store, storeOptions: { client } ## Redis must be durable -Catalogs are data, not a cache. Point the store at a Redis that: +Use durable Redis; catalogs are data, not cache: -- persists to disk (AOF or RDB); -- uses the `noeviction` policy, so that a full instance refuses writes instead of dropping catalogs. +- Enable disk persistence (AOF or RDB). +- Set `noeviction`: full instances must refuse writes, not drop catalogs. -A Redis used as a cache elsewhere in your stack is usually configured the opposite way. Use a dedicated instance or database if in doubt. +Cache instances usually have opposite settings. If unsure, use a dedicated instance or database. ## How writes are protected -A catalog is replaced by a short Lua script that sets the key only if it still holds the content that was read. Two server processes sharing one Redis cannot overwrite each other's changes. +A Lua script sets the key only if its content still matches what was read. Shared server processes cannot overwrite each other's changes. diff --git a/website/deployment/stores/sqlite.md b/website/deployment/stores/sqlite.md index 89531c6..d8eefa4 100644 --- a/website/deployment/stores/sqlite.md +++ b/website/deployment/stores/sqlite.md @@ -4,7 +4,7 @@ description: "Keep source catalogs in a local SQLite file with the built-in node # SQLite store -Keeps one row per profile in a local database file. It needs no extra package: it uses the `node:sqlite` module built into Node.js. +Configure a local SQLite file; no extra package is needed (`node:sqlite` is built into Node.js): ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; @@ -19,6 +19,8 @@ export default defineKibanaMcpConfig({ }); ``` +SQLite keeps one row per profile. + ## Options | Option | Default | Description | @@ -29,11 +31,11 @@ export default defineKibanaMcpConfig({ ## When to use it -SQLite suits a single host: one server, or a few processes on the same machine sharing the file. The database is opened in WAL mode so readers do not block. +Use SQLite for one server or a few same-host processes sharing the file. WAL mode keeps readers from blocking. -It is not the right store for several replicas on different machines: use [PostgreSQL](./postgres), [MongoDB](./mongodb) or [Redis](./redis) there. +For replicas on different hosts, use [PostgreSQL](./postgres), [MongoDB](./mongodb) or [Redis](./redis). ## Limits -- Calls are synchronous. A write that waits for another process holding the lock can pause the server for up to five seconds. Writes only happen when a source is configured, so this is rare. -- The file must be on a persistent volume. In a container, mount one at its directory. +- Calls are synchronous. Waiting for another process's write lock can pause the server for up to five seconds. Writes occur only during source configuration, so this is rare. +- Put the file on a persistent volume. In containers, mount that volume at its directory. diff --git a/website/guide/configuration-file.md b/website/guide/configuration-file.md index 767971e..6338f32 100644 --- a/website/guide/configuration-file.md +++ b/website/guide/configuration-file.md @@ -4,11 +4,9 @@ description: "Configure the server with a kibana.mcp.config.ts file: connection, # Configuration file -Instead of environment variables, the server can read a configuration file, in stdio and in HTTP mode alike: +1. Create `kibana.mcp.config.ts`: -```bash -kibana-mcp-server serve --http --config ./kibana.mcp.config.ts -``` +**Use only trusted files:** configuration executes as code. `.ts` needs Node.js 22.18+; `.mjs` works on any supported Node.js version. ```ts // kibana.mcp.config.ts @@ -25,9 +23,17 @@ export default defineKibanaMcpConfig({ }); ``` -- `url`, `user`, `password`, `timeoutMs`, `profile` and `http.*` are defaults: the matching `KIBANA_*` environment variable or `serve` flag wins when set. -- `store` decides where source catalogs live, one per profile. `storeOptions` is handed to the store factory. Without `store`, direct connection startup requires an explicit `KIBANA_SOURCE_CATALOG_PATH`; saved profiles carry their own file path. No catalog is discovered from the checkout. -- The file is executed as code, so it is loaded only when named with `--config` or `KIBANA_MCP_CONFIG`, never discovered from the working directory. A `.ts` file needs Node.js 22.18 or later; `.mjs` works everywhere. +2. Run with the explicit file (stdio or HTTP): + +```bash +kibana-mcp-server serve --http --config ./kibana.mcp.config.ts +``` + +Matching `KIBANA_*` variables or `serve` flags override `url`, `user`, `password`, `timeoutMs`, `profile` and `http.*`. + +`store` holds one catalog per profile; `storeOptions` goes to its factory. Without a store, direct startup requires `KIBANA_SOURCE_CATALOG_PATH`; saved profiles supply their own path. No implicit checkout catalog. + +Select the file with `--config` or `KIBANA_MCP_CONFIG`; it is never discovered from the working directory. ## Next steps diff --git a/website/guide/getting-started.md b/website/guide/getting-started.md index 07558a4..a2e4995 100644 --- a/website/guide/getting-started.md +++ b/website/guide/getting-started.md @@ -8,13 +8,15 @@ description: "Requirements, prompt-free bootstrap, agent skill installation and - Node.js 22+ - A Kibana-compatible read endpoint reachable with basic auth -- No index is needed for bootstrap. The user names the exact Elasticsearch index or pattern only when adding a source through `configure_index`. Hand-authored catalogs remain supported for advanced setups. +- No index for bootstrap. Select the exact index or pattern later with `configure_index`; hand-authored catalogs remain an advanced option. ## Installation ### Prompt-free connection bootstrap -Pre-provision the connection values in the environment: +1. Provision the connection through a trusted environment, then run: + +Never pass passwords as command-line arguments. Use `KIBANA_PASSWORD`, `--password-env NAME`, or `--password-stdin`. ```bash export KIBANA_BASE_URL='https://kibana.example.com' @@ -25,19 +27,16 @@ npx -y @havesomecode/kibana-mcp-server bootstrap \ --client codex ``` -This command performs no prompts and does not edit Codex configuration files directly. It verifies only the Kibana connection, writes an empty managed source catalog on first install, stores credentials securely, and registers the exact running package transport. It does not list, scan, infer, or configure indexes. - -After bootstrap, start the server without `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` in its environment so it uses the saved profile. Direct connection startup instead requires an explicit `KIBANA_SOURCE_CATALOG_PATH` or configuration-file store; no checkout catalog is loaded implicitly. +Expected: no prompts with these values supplied; a verified connection, secure credentials, an empty managed catalog on first install, and registration of the exact running package transport. No direct Codex config edits or index listing, scanning, inference, or configuration. -Reconnect the MCP client, then call `discover`. When no source is configured, the agent must ask the user which exact Kibana index or index pattern to use. Only after the user answers may the agent call `configure_index` with that value. That tool validates only the approved index and persists it without accepting credentials. +2. Start the server without `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` to use the saved profile. Direct startup requires an explicit `KIBANA_SOURCE_CATALOG_PATH` or configuration-file store; no implicit checkout catalog. +3. Reconnect the client and call `discover`. If empty, ask the user for the exact Kibana index or pattern and wait. Never guess, list, or scan indexes. Call `configure_index` only with their answer; it validates and saves only that index and accepts no credentials. -Bootstrap reruns preserve sources that were configured through this explicit flow. It refuses to replace a hand-authored or modified catalog unless `--replace` is supplied explicitly; `--replace` creates an empty managed catalog. - -Never pass passwords as command-line arguments. Use `KIBANA_PASSWORD`, `--password-env NAME`, or `--password-stdin`. +Reruns preserve explicitly configured sources. Bootstrap refuses to replace hand-authored or modified catalogs without `--replace`. Use that flag only with operator approval: it creates an empty managed catalog. ### Install the agent skill -The repository contains a standard Agent Skill discoverable by the `skills` CLI and skills.sh: +Run: ```bash npx skills add Havesomecode/kibana-mcp-server \ @@ -47,18 +46,16 @@ npx skills add Havesomecode/kibana-mcp-server \ --yes ``` -The skill teaches compatible agents to run the same verified bootstrap instead of editing MCP config files. Its source is `skills/kibana-log-investigation/SKILL.md`. +Expected: the skill at `skills/kibana-log-investigation/SKILL.md` teaches bootstrap instead of manual MCP config edits. ## Use in Codex -The prompt-free package bootstrap above is the primary Codex path. The repo-scoped plugin remains available for contributors working from a cloned checkout. +Use package bootstrap above for Codex. Use the repo plugin below only from a cloned checkout. ### Quick path 1. Clone the repository. -2. Make sure Node.js 22+ is available. - - if it is missing, ask Codex to use the plugin skill `ensure-node-runtime` - - the repo pins Node major `22` in `.node-version` +2. Check Node.js 22+ (pinned in `.node-version`). If missing, ask Codex to use `ensure-node-runtime`. 3. From the repo root, run: ```bash @@ -66,12 +63,13 @@ npm install npm run build ``` -4. Open the cloned repo in Codex. -5. Open the plugin directory in Codex and install `Kibana Log Investigation` from the repo marketplace. - - if the current model cannot complete that install itself, do the Codex UI click manually and let the agent continue with configuration afterward -6. Run `node dist/src/bin/kibana-mcp-server.js bootstrap --client none` to verify the connection and save an empty profile. The repo plugin already supplies the client registration. -7. Reconnect the plugin's MCP process and check its startup status. It requires a saved connection before it can start. Inspect sanitized stderr on failure; rebuild and reinstall only if the plugin runtime is missing. Restart once after startup succeeds if the tools still need refreshing. -8. In a fresh thread, call `discover`, ask the user for the exact index or pattern, then call `configure_index` only after they answer. +### Install the built plugin + +1. Open the cloned repo in Codex. +2. Install `Kibana Log Investigation` from the repo marketplace in Codex's plugin directory. If the model cannot install it, click in the UI yourself, then let the agent continue. +3. Run `node dist/src/bin/kibana-mcp-server.js bootstrap --client none`. This verifies the connection and creates an empty catalog on first install; the plugin already registers the server. +4. Reconnect the MCP process and check startup status. On failure, inspect sanitized stderr; rebuild and reinstall a missing runtime. After startup succeeds, restart once if tools need refreshing. +5. In a fresh thread, call `discover`. If empty, ask for the exact index or pattern and wait; only then call `configure_index`. Repo-scoped plugin files: @@ -83,7 +81,7 @@ Repo-scoped plugin files: ### Minimal handoff prompt -If you are handing only the repo link to another Codex agent, this usually works: +Send this prompt with the repo link: ```text Install the `kibana-log-investigation` skill from Havesomecode/kibana-mcp-server, then run its connection-only bootstrap with the provisioned KIBANA_BASE_URL, KIBANA_USERNAME, and KIBANA_PASSWORD values. Do not edit Codex config files, import the example catalog, or inspect indexes. After install, ask me which exact index or pattern to configure before calling configure_index. diff --git a/website/guide/investigating.md b/website/guide/investigating.md index 9a6e7fd..850cfb6 100644 --- a/website/guide/investigating.md +++ b/website/guide/investigating.md @@ -4,41 +4,41 @@ description: "The investigation workflow step by step: configure a source, disco # Investigating with an agent -An investigation follows the same three moves every time: find the source, check its fields, then query. The agent drives them through MCP tools; this page shows what happens at each step. +Ask the agent to discover a source, inspect its fields, then query. Configure a source only if none exists. ## 1. Configure a source, once -A fresh profile has no source. The agent must not guess one: it asks which exact Kibana index or index pattern to use, then calls [`configure_index`](../reference/tools/configure-index) with that value. +For a fresh profile, call `discover` first. If empty, ask for the exact Kibana index or pattern and wait. Never guess, list, or scan indexes. Call [`configure_index`](../reference/tools/configure-index) only with the user's answer; this example assumes they selected `app-logs-*`: ```json { "index": "app-logs-*", "source_name": "Application logs", "source_id": "app-logs" } ``` -The server checks that the pattern resolves to fields, picks a time field, runs one probe search and saves the source in the profile's catalog. Later conversations reuse it. +Expected: fields checked, a time field selected, one probe search, and a saved source for later conversations. ## 2. Discover the sources -[`discover`](../reference/tools/discover) lists the logical sources of the current profile, with their time field and field hints. +Call [`discover`](../reference/tools/discover): ```json { "query": "application" } ``` +Expected: this profile's logical sources, time fields, and field hints. + ## 3. Inspect the fields -Before filtering, the agent asks [`describe_fields`](../reference/tools/describe-fields) which fields exist and which ones support exact matching. +Before filtering, call [`describe_fields`](../reference/tools/describe-fields): ```json { "source_id": "app-logs", "query": "status" } ``` -A text field with a keyword sub-field is reported with a preferred exact field, for example `event` and `event.keyword`. That is what makes later filters reliable. +Use the returned preferred exact field for matching, such as `event.keyword` for `event`. ## 4. Query -[`query`](../reference/tools/query) searches one or more sources over an absolute time window. Start broad, then narrow. - -Count first: +Call [`query`](../reference/tools/query) over an absolute time window. Count first: ```json { @@ -50,7 +50,7 @@ Count first: } ``` -Then group to see where the errors come from: +Group to locate the errors: ```json { @@ -63,7 +63,7 @@ Then group to see where the errors come from: } ``` -Then read the hits of the service that stands out: +Read hits for the service that stands out: ```json { @@ -76,7 +76,7 @@ Then read the hits of the service that stands out: } ``` -When the exact field name is already known, [`filter`](../reference/tools/filter) skips alias resolution and filters on it directly. +For a known exact field, use [`filter`](../reference/tools/filter) without alias resolution. ## Query modes @@ -91,4 +91,4 @@ When the exact field name is already known, [`filter`](../reference/tools/filter ## When something fails -Backend failures come back with a stable code such as `KIBANA_TIMEOUT` or `KIBANA_AUTHENTICATION`. See [Errors](../reference/errors). +Look up stable failure codes such as `KIBANA_TIMEOUT` or `KIBANA_AUTHENTICATION` in [Errors](../reference/errors). diff --git a/website/guide/profiles.md b/website/guide/profiles.md index 53e9f23..2b994a2 100644 --- a/website/guide/profiles.md +++ b/website/guide/profiles.md @@ -4,39 +4,25 @@ description: "Saved machine profiles and how to work with several Kibana environ # Profiles and environments -The preferred path is deterministic connection-only bootstrap: +Run `npx -y @havesomecode/kibana-mcp-server bootstrap --client codex`. -- `npx -y @havesomecode/kibana-mcp-server bootstrap --client codex` -- verifies `/api/status` without touching an index endpoint -- creates an empty managed source catalog on first install -- rejects conflicting MCP registrations instead of overwriting them -- saves non-secret profile metadata and credentials for later threads -- exposes `configure_index` for one user-approved index or pattern at a time +Expected: `/api/status` verified without index access, an empty managed catalog on first install, saved profile metadata and secure credentials. Conflicting MCP registrations are rejected, not overwritten. -Guided setup remains available for hand-authored source catalogs: +Call `discover`; if empty, ask for the exact index or pattern and wait. Never guess, list, or scan indexes. Use `configure_index` only for the user's selected value. -- `npm run setup` or `node dist/src/bin/kibana-mcp-server.js setup` -- saves non-secret profile metadata in the user’s machine-level app config directory -- saves credentials in the platform credential store - - macOS Keychain - - Windows Credential Manager - - Linux Secret Service when available -- imports the selected source catalog into machine-local state so later threads do not depend on the repo checkout -- requires an explicit catalog path; type `example` to opt into the bundled example deliberately +For hand-authored catalogs, run `npm run setup` or `node dist/src/bin/kibana-mcp-server.js setup`. Supply an explicit catalog path; type `example` only to opt into the bundled example. Setup imports the catalog into machine-local state, independent of the checkout. -For direct process startup, supply the connection through environment variables together with an explicit `KIBANA_SOURCE_CATALOG_PATH`, or declare a catalog store in a [configuration file](./configuration-file). There is no implicit checkout catalog and no MCP tool accepts connection credentials. +Metadata lives in the machine-level app config directory. Credentials use macOS Keychain, Windows Credential Manager, or Linux Secret Service when available. Never put passwords in command arguments. -Without a saved or explicit connection, startup fails: `No saved Kibana connection is available. Run connection bootstrap first.` +For direct startup, supply environment credentials and an explicit `KIBANA_SOURCE_CATALOG_PATH`, or a store in a [configuration file](./configuration-file). No implicit checkout catalog; no MCP tool accepts credentials. -## Machine profiles - -After connection bootstrap or guided setup, the default saved profile loads automatically on startup. Do not leave `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` set in the server environment when you intend to use saved credentials: supplying all three selects direct environment startup and requires an explicit catalog path or store. +Without a saved or explicit connection: `No saved Kibana connection is available. Run connection bootstrap first.` -If you saved more than one environment, select a non-default profile with: +## Machine profiles -- `KIBANA_PROFILE=` +Remove `KIBANA_BASE_URL`, `KIBANA_USERNAME`, and `KIBANA_PASSWORD` from the server environment to use saved profiles. All three together select direct startup and require an explicit catalog path or store. -Example: +The default saved profile loads automatically. To select another, set `KIBANA_PROFILE=`: ```bash KIBANA_PROFILE=staging npm run dev @@ -44,26 +30,19 @@ KIBANA_PROFILE=staging npm run dev ## Multiple environments -Save a separate profile per environment with `bootstrap --profile --client codex`, or import hand-authored catalogs through guided setup. Use `--no-default` during bootstrap when the new profile must not replace the current default. - -Then: - -- the default profile loads automatically -- extra MCP entries can select another saved environment with `KIBANA_PROFILE` -- you do not need to redefine `KIBANA_BASE_URL`, credentials, or source-catalog paths per thread - -Example: +1. Save each environment with `bootstrap --profile --client codex`, or guided setup for hand-authored catalogs. Use `--no-default` to keep the current default. +2. Set `KIBANA_PROFILE` in extra MCP entries; no per-thread connection values or catalog paths are needed: ```bash KIBANA_PROFILE=staging npm run dev KIBANA_PROFILE=prod npm run dev ``` -For direct environment-based entries, keep these distinct per environment: +For direct startup instead, keep these distinct per environment: - `KIBANA_BASE_URL` - `KIBANA_USERNAME` - `KIBANA_PASSWORD` - `KIBANA_SOURCE_CATALOG_PATH` -Use separate catalog paths because `configure_index` persists managed sources. Sharing a file would mix sources from different environments. Hand-authored catalogs remain operator-managed and are not overwritten by that tool. +Use separate catalog paths to avoid mixing persisted sources. `configure_index` does not overwrite operator-managed, hand-authored catalogs. diff --git a/website/guide/what-is-it.md b/website/guide/what-is-it.md index b8a7dc2..bcc9a8c 100644 --- a/website/guide/what-is-it.md +++ b/website/guide/what-is-it.md @@ -6,36 +6,25 @@ description: "What Kibana MCP Server is, why it exists and what it deliberately Read-only MCP server for agent-driven log investigation against Kibana-backed search endpoints protected with basic auth. -The server is intentionally small: +## Start an investigation -- guided `setup` imports an explicitly selected hand-authored catalog and saves machine-level Kibana profiles for later threads -- `bootstrap` verifies only the Kibana connection, creates an empty managed catalog, saves credentials, and registers Codex without config-file edits; it is prompt-free when all connection values are supplied -- `configure_index` validates and persists exactly one index or pattern after the user explicitly names it -- `describe_fields` exposes effective field capabilities for one configured source -- `discover` lists configured logical sources and field hints -- `filter` runs an exact-field filter when the field name is already known -- `query` searches one or more sources over an absolute time window in `hits`, `count`, `histogram`, `terms`, `stats`, or `grouped_top_hits` mode - -It is designed as a general-purpose log investigation MCP for environments where operators can expose a small set of useful logical sources. +1. Run `bootstrap`: verify only the connection, save credentials, create an empty managed catalog, and register Codex without config-file edits. Supply all connection values for prompt-free use. For hand-authored catalogs, guided `setup` imports your explicit selection and saves machine profiles. +2. Call `discover` for configured logical sources and field hints. If empty, ask the user for an exact index or pattern and wait; `configure_index` validates and saves only that selection. +3. Call `describe_fields` for a source's effective field capabilities before filtering. +4. Use `query` over an absolute time window: `hits`, `count`, `histogram`, `terms`, `stats`, or `grouped_top_hits`. Use `filter` when the exact field name is known. ## Why it exists -Kibana is built for people: dashboards, saved searches, a query bar. An agent investigating an incident needs something else: a short list of sources it can trust, the fields it may filter on, and results it can read without scraping a UI. - -This server gives an agent exactly that, over the Model Context Protocol: - -- **Logical sources instead of raw indexes.** Operators expose a small catalog of named sources. The agent starts from those, never from a list of every index in the cluster. -- **Fields before filters.** The agent can inspect the effective fields of a source, including which ones support exact matching, before it tightens a query. -- **Structured output.** Hits, counts, histograms, term buckets and statistics come back as typed results. +Agents use a small operator-selected source catalog, field capabilities, and typed results over the Model Context Protocol—no UI scraping or cluster-wide index listing. ## Boundaries - **Read-only by design.** The server searches and describes. It never writes to Elasticsearch or Kibana. -- **No index guessing.** A source is configured only after a person has named the exact index or index pattern. +- **No index guessing.** Never guess, list, or scan indexes before a person names the exact index or pattern. - **Schema-aware features depend on the deployment.** When Kibana does not expose field metadata, the server falls back to sampling fields through search, which is a heuristic. ## Where to go next -- [Getting started](./getting-started) installs the server and runs a first investigation. -- [HTTP transport](../deployment/http) covers running one shared server for many callers. -- The [tool reference](../reference/tools/query) documents every input and output. +- [Getting started](./getting-started): install and investigate. +- [HTTP transport](../deployment/http): share a server across callers. +- [Tool reference](../reference/tools/query): inputs and outputs. diff --git a/website/integrations/docker.md b/website/integrations/docker.md index 7e5f92c..1fe8828 100644 --- a/website/integrations/docker.md +++ b/website/integrations/docker.md @@ -4,10 +4,10 @@ description: "A Docker image and compose file that run the server over HTTP with # Docker image -This example builds an image that runs the server over HTTP, with catalogs in PostgreSQL. Swap the store for any other one by changing two lines. +Create these files to run HTTP with PostgreSQL catalogs. ::: warning Version -The HTTP transport, the configuration file and the catalog stores are not part of the releases published before this documentation. Use a version of the package that includes them. +Use a package version with HTTP transport, configuration files, and catalog stores. Releases predating this documentation do not include them. ::: ## Files @@ -40,11 +40,11 @@ export default defineKibanaMcpConfig({ }); ``` -A `.mjs` file loads on every supported Node.js version. A `.ts` file works too from Node.js 22.18. +Use `.mjs` on any supported Node.js version; `.ts` needs Node.js 22.18+. ### `Dockerfile` -The server and the driver are installed in the same directory as the configuration file, so that its imports resolve. +Install the server and driver beside the config file so imports resolve: ```dockerfile FROM node:22-alpine @@ -66,6 +66,10 @@ CMD ["npx", "kibana-mcp-server", "serve", "--http", "--config", "kibana.mcp.conf ### `compose.yaml` +**Keep the port private:** only your [gateway](./gateway) should reach it. This file binds to host loopback. + +**Keep catalogs persistent:** the `catalogs` volume below outlives the container. For file or SQLite stores, mount a volume at the catalog or database path. + ```yaml services: kibana-mcp: @@ -101,14 +105,20 @@ volumes: ## Run and check +Supply `KIBANA_PASSWORD`, `KIBANA_MCP_TOKEN`, and `POSTGRES_PASSWORD` through your secret provider. Listening on `0.0.0.0` requires `KIBANA_MCP_TOKEN`; the server refuses non-loopback startup without a shared secret. + ```bash docker compose up -d --build ``` +Check health: + ```bash curl -s http://127.0.0.1:3000/healthz ``` +Check tool discovery for profile `alice`: + ```bash curl -s http://127.0.0.1:3000/mcp \ -H "Authorization: Bearer $KIBANA_MCP_TOKEN" \ @@ -118,9 +128,8 @@ curl -s http://127.0.0.1:3000/mcp \ -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' ``` +Expected: a health response and MCP tool list, not proof of index access. + ## Notes -- **Listening on `0.0.0.0` requires `KIBANA_MCP_TOKEN`.** The server refuses to start on a non-loopback host without a shared secret. -- **Catalogs must outlive the container.** They live in the `catalogs` volume here. With the file store, mount a volume at the catalog path; with SQLite, at the database path. -- **Do not publish the port publicly.** Only your [gateway](./gateway) should reach it. The compose file binds it to the loopback interface of the host. -- **SQLite instead of PostgreSQL.** Import `stores/sqlite`, set `storeOptions: { path: "/data/catalogs.sqlite" }`, drop the `pg` install and mount a volume at `/data`. +**SQLite instead of PostgreSQL:** import `stores/sqlite`, set `storeOptions: { path: "/data/catalogs.sqlite" }`, drop the `pg` install, and mount a volume at `/data`. diff --git a/website/integrations/gateway.md b/website/integrations/gateway.md index b280ef8..b1079a1 100644 --- a/website/integrations/gateway.md +++ b/website/integrations/gateway.md @@ -4,16 +4,16 @@ description: "What an MCP gateway or reverse proxy must do in front of the HTTP # Behind an MCP gateway -The HTTP transport does not authenticate end users. It expects to sit behind something that does: an MCP gateway, an API gateway or a reverse proxy. This page describes what that component has to do, whatever it is built with. +Put HTTP behind an MCP/API gateway or reverse proxy. The server does not authenticate end users; the gateway must. ## The contract -For every request it forwards, the gateway must: +For every forwarded request: -1. **Authenticate the caller** with whatever your organization uses: OIDC, an API key, mutual TLS. -2. **Set the profile header** to a stable identifier of that caller, typically the `sub` claim of its token. The server uses it to select the caller's source catalog. -3. **Overwrite, never forward, that header.** A value sent by the client must not reach the server, or a caller could read another caller's catalog. -4. **Send the shared secret** as `Authorization: Bearer `, so the server knows the request comes from the gateway. +1. **Authenticate the caller:** OIDC, API key, or mutual TLS. +2. **Set the profile header** from a stable verified identity, typically token `sub`. It selects the caller's catalog. +3. **Overwrite, never forward, that header.** Client-supplied values could expose another caller's catalog. +4. **Send the shared secret** as `Authorization: Bearer ` to authenticate the gateway to the server. ``` caller ──(OIDC token)──▶ gateway ──(Authorization: Bearer @@ -22,6 +22,8 @@ caller ──(OIDC token)──▶ gateway ──(Authorization: Bearer ` or `KIBANA_MCP_CONFIG`. It is executed as code, so it is never discovered from the working directory. +Load a file explicitly with `serve --config ` or `KIBANA_MCP_CONFIG`. It executes as code; working-directory discovery is disabled. ```ts import { defineKibanaMcpConfig } from "@havesomecode/kibana-mcp-server/config"; @@ -14,7 +14,7 @@ export default defineKibanaMcpConfig({ }); ``` -The default export may be the configuration object, or a function, possibly asynchronous, returning it. `defineKibanaMcpConfig` only provides types. +Default-export an object or a function (sync or async) returning it. `defineKibanaMcpConfig` provides types only. ## Options @@ -35,15 +35,15 @@ The default export may be the configuration object, or a function, possibly asyn ## Precedence -Command-line flag, then environment variable, then configuration file, then built-in default. +Command-line flag → environment variable → configuration file → built-in default. ## Constraints - `store` requires `url`, `user` and `password`, from the file or from the environment. - `store` cannot be combined with `KIBANA_SOURCE_CATALOG_PATH`. -- Without `store`, a direct connection requires an explicit `KIBANA_SOURCE_CATALOG_PATH`. Saved-profile startup uses the path stored with the profile. There is no implicit catalog path. +- Without `store`, direct connections require explicit `KIBANA_SOURCE_CATALOG_PATH`; saved profiles use their saved path. No implicit catalog path. - A `.ts` file needs Node.js 22.18 or later. A `.mjs` file works on every supported version. -- The package must be resolvable from the directory of the file, since the file imports it. +- The package must resolve from the file's directory for its import to work. ## Types diff --git a/website/reference/environment.md b/website/reference/environment.md index b48239f..7de60d9 100644 --- a/website/reference/environment.md +++ b/website/reference/environment.md @@ -4,7 +4,7 @@ description: "Every KIBANA_* environment variable read by the server." # Environment variables -The server reads its connection from environment variables when they are set. A [configuration file](./config-file) supplies the same values as defaults: a variable that is set always wins. +Set environment variables to override [configuration-file](./config-file) defaults. ## Connection @@ -15,11 +15,19 @@ The server reads its connection from environment variables when they are set. A - `KIBANA_TIMEOUT_MS` optional, default `10000` - `KIBANA_SOURCE_CATALOG_PATH` required for direct environment-based startup unless a configuration-file store supplies the catalog; saved profiles carry their own path. No implicit default. -`KIBANA_TIMEOUT_MS` bounds each Kibana HTTP request end to end: it includes waiting for a request slot, connecting, and reading the response body. The server allows at most eight concurrent Kibana requests and 32 queued requests per configured client; excess work fails immediately with `KIBANA_OVERLOADED`. Cancelled MCP tool calls abort their queued or in-flight backend work, except shared schema discovery: cancelling one caller stops only its wait while other callers remain; cancelling the last caller aborts the shared backend request. +### Deadlines and cancellation -Backend failures use stable diagnostic codes such as `KIBANA_TIMEOUT`, `KIBANA_CANCELLED`, `KIBANA_OVERLOADED`, `KIBANA_AUTHENTICATION`, `KIBANA_DNS`, `KIBANA_TLS`, `KIBANA_CONNECTION`, and `KIBANA_RESPONSE`. Failed backend tool results also expose the diagnostic fields in `_meta.kibana_request_error`. +`KIBANA_TIMEOUT_MS` covers each Kibana HTTP request end to end: queue wait, connection and response-body read. -`KIBANA_BASE_URL` is the Kibana base prefix that the server joins with each configured backend or schema path. +- Per configured client: eight concurrent requests, 32 queued. Excess work fails immediately with `KIBANA_OVERLOADED`. +- Cancelled MCP calls abort queued/in-flight backend work. +- Shared schema discovery: cancellation stops only that caller's wait while others remain; the last caller's cancellation aborts the shared backend request. + +Backend failures use stable codes: `KIBANA_TIMEOUT`, `KIBANA_CANCELLED`, `KIBANA_OVERLOADED`, `KIBANA_AUTHENTICATION`, `KIBANA_DNS`, `KIBANA_TLS`, `KIBANA_CONNECTION`, `KIBANA_RESPONSE` and [others](./errors). Failed backend tool results expose diagnostics in `_meta.kibana_request_error`. + +### Choose the base URL + +Set `KIBANA_BASE_URL` to the base prefix; the server joins it with configured backend/schema paths. Use: @@ -31,11 +39,12 @@ Do not use: - `https://kibana.example.com/internal/search/es` - `https://gateway.example.com/logs/internal/search/es` -The source definition already carries endpoint paths such as `/internal/search/es`, so including a full API path in `KIBANA_BASE_URL` usually produces a bad combined URL and a `404`. +Sources already carry paths such as `/internal/search/es`; a full API path here usually produces a bad combined URL and `404`. -Direct environment-based startup requires all three connection values together and an explicit `KIBANA_SOURCE_CATALOG_PATH`, or a [configuration-file store](../deployment/catalog-stores). The server reads only the named catalog; it never searches the checkout for one. Hand-authored catalogs may use `config/sources.example.json` as a deliberate template, not as an automatic setup input. +### Select startup mode -For saved-profile startup, leave the connection variables unset and select the profile with `KIBANA_PROFILE` if needed. Its catalog path and credentials are loaded from saved state. Without a saved connection, startup fails with `No saved Kibana connection is available. Run connection bootstrap first.` +- **Direct environment:** supply all three connection values plus explicit `KIBANA_SOURCE_CATALOG_PATH` or a [configuration-file store](../deployment/catalog-stores). Only the named catalog is read; the checkout is never searched. Use `config/sources.example.json` only as a deliberate hand-authored template, never automatic setup input. +- **Saved profile:** unset connection variables; optionally select `KIBANA_PROFILE`. Credentials and catalog path come from saved state. Without a saved connection, startup fails with `No saved Kibana connection is available. Run connection bootstrap first.` ## Configuration file diff --git a/website/reference/errors.md b/website/reference/errors.md index 7c2f7cf..7a389ad 100644 --- a/website/reference/errors.md +++ b/website/reference/errors.md @@ -6,7 +6,7 @@ description: "Kibana request error codes, catalog errors and HTTP transport stat ## Kibana request errors -A tool call that fails while talking to Kibana returns an error result whose `_meta.kibana_request_error` carries diagnostic fields: `code`, `phase` (`queue`, `request` or `response`), `source_id`, `timeout_ms`, and `status` or `cause_code` when available. +Read `_meta.kibana_request_error` on failed Kibana tool calls: `code`, `phase` (`queue`, `request` or `response`), `source_id`, `timeout_ms`, plus `status` or `cause_code` when available. | Code | Meaning | | --- | --- | @@ -23,7 +23,7 @@ A tool call that fails while talking to Kibana returns an error result whose `_m ## Concurrency limits -Each Kibana connection allows eight concurrent requests and thirty-two queued ones. Beyond that, calls fail immediately with `KIBANA_OVERLOADED`. Profiles that share a connection share that budget. +Each Kibana connection allows eight concurrent requests and thirty-two queued; excess calls fail immediately with `KIBANA_OVERLOADED`. Profiles sharing a connection share its budget. ## Startup errors diff --git a/website/reference/source-catalog.md b/website/reference/source-catalog.md index 6620992..c2fbf18 100644 --- a/website/reference/source-catalog.md +++ b/website/reference/source-catalog.md @@ -4,11 +4,11 @@ description: "The format of a source definition, the supported backends and sche # Source catalog -A source catalog is a JSON document listing the logical sources a profile can query. `configure_index` generates and maintains it; hand-authored catalogs remain supported for advanced setups. +Use `configure_index` to maintain a profile's JSON source catalog. Hand-authored catalogs remain supported for advanced setups. ## Source definition -Each source definition should provide: +Define each logical source with: - stable `id` - human-readable `name` @@ -20,7 +20,7 @@ Each source definition should provide: - `backend` describing how the source is queried - optional `schema` describing how field introspection is performed for `describe_fields` and other schema-dependent features -Supported backend kinds: +## Search backends - `kibana_internal_search_es` - POSTs to a Kibana endpoint like `/internal/search/es` @@ -29,27 +29,19 @@ Supported backend kinds: - `elasticsearch_search` - POSTs the compiled Elasticsearch body directly to the configured path -Supported schema backend kinds: +## Schema backends -- `kibana_data_views_fields` - - tries Kibana wildcard field-metadata endpoints, preferring `/internal/data_views/_fields_for_wildcard` - - adds `pattern=...` from `schema.index` or `backend.index` - - if all known field-metadata endpoints return `404`, falls back to sampling fields through the configured search backend -- `kibana_index_patterns_fields` - - tries the legacy wildcard field-metadata endpoint `/api/index_patterns/_fields_for_wildcard` and related fallbacks - - adds `pattern=...` from `schema.index` or `backend.index` - - if all known field-metadata endpoints return `404`, falls back to sampling fields through the configured search backend +- `kibana_data_views_fields`: tries wildcard field-metadata endpoints, preferring `/internal/data_views/_fields_for_wildcard`. +- `kibana_index_patterns_fields`: tries legacy `/api/index_patterns/_fields_for_wildcard` and related fallbacks. - `elasticsearch_field_caps` - GETs an Elasticsearch field capabilities endpoint such as `//_field_caps?fields=*` - uses `schema.path` when provided or derives the default from `schema.index` -Deployment note: +Both Kibana schema backends add `pattern=...` from `schema.index` or `backend.index`. If all known metadata endpoints return `404`, they sample fields through the configured search backend. Some deployments expose only search transport; sampling is heuristic, not authoritative, but often recovers `.keyword` promotion and object-path discovery. -- some Kibana deployments expose search transport but not field-metadata endpoints -- when all known metadata endpoints for a Kibana data-view or index-pattern schema backend return `404`, the server falls back to sampling fields through the configured search backend; direct `elasticsearch_field_caps` failures are returned without that fallback -- this fallback is heuristic rather than authoritative, but it is often sufficient to recover `.keyword` promotion and object-path discovery in search-only environments +Direct `elasticsearch_field_caps` failures return without that fallback. -Example source shape for an application log stream: +## Application log example ```json { @@ -68,4 +60,4 @@ Example source shape for an application log stream: } ``` -Keep `schema.path` omitted for this setup. The server will try the known Kibana metadata endpoints first and then fall back to search-transport sampling if those endpoints still return `404`. +Omit `schema.path` here: try known Kibana metadata endpoints, then sample through search transport if all return `404`. diff --git a/website/reference/tools/configure-index.md b/website/reference/tools/configure-index.md index e6e99f8..801c07c 100644 --- a/website/reference/tools/configure-index.md +++ b/website/reference/tools/configure-index.md @@ -4,9 +4,7 @@ description: "Reference of the configure_index tool: configure one index or patt # `configure_index` -Configures exactly one index or index pattern as a logical source, after the user has named it. - -Use this tool only after the user explicitly names the exact index or index pattern to configure. It accepts no credentials and never lists or scans for candidate indexes. +Configure **exactly the index or pattern explicitly named by the user**. Never guess, list or scan candidates; this tool accepts no credentials. Input: @@ -17,4 +15,4 @@ Input: } ``` -The tool inspects fields and runs one bounded validation search only for that index. It then persists the generated source in the managed catalog. Optional `source_id`, `time_field`, and `replace` fields support explicit collision or schema handling. +The tool inspects fields, runs one bounded validation search for that index only, then persists the generated logical source in the managed catalog. Use optional `source_id`, `time_field` and `replace` for explicit collision or schema handling. diff --git a/website/reference/tools/describe-fields.md b/website/reference/tools/describe-fields.md index cbd3326..3d36078 100644 --- a/website/reference/tools/describe-fields.md +++ b/website/reference/tools/describe-fields.md @@ -4,11 +4,9 @@ description: "Reference of the describe_fields tool: inspect the effective field # `describe_fields` -Describes the effective fields of one source, including which ones support exact matching. +Inspect one source's effective fields for safe exact matching, grouping or sorting. -Use `describe_fields` when the agent needs to know whether a field is safe for exact term matching, grouping, or sorting. -If the source does not configure a schema backend, `describe_fields` fails clearly instead of guessing. -When direct schema endpoints are unavailable, the server will infer fields from sample hits via the configured search backend. +Requires a configured schema backend; missing configuration fails clearly, not by guessing. When direct schema endpoints are unavailable, fields are inferred from sample hits through the configured search backend. Input: diff --git a/website/reference/tools/discover.md b/website/reference/tools/discover.md index b760855..0c06d58 100644 --- a/website/reference/tools/discover.md +++ b/website/reference/tools/discover.md @@ -4,7 +4,7 @@ description: "Reference of the discover tool: list the logical sources of the cu # `discover` -Lists the logical sources configured for the current profile, with their field hints. +List the current profile's configured logical sources and field hints. Input: diff --git a/website/reference/tools/filter.md b/website/reference/tools/filter.md index 2f38894..2294f99 100644 --- a/website/reference/tools/filter.md +++ b/website/reference/tools/filter.md @@ -4,9 +4,7 @@ description: "Reference of the filter tool: exact-field filtering without alias # `filter` -Runs an exact-field filter when the field name is already known, bypassing alias resolution. - -Use `filter` when you already know the backend field you want to target and want to avoid alias resolution. +Filter a known backend field directly, without field-hint alias resolution. Input: @@ -31,6 +29,6 @@ Input: } ``` -`filter` returns the same structured output envelope as `query`, but it does not apply field-hint alias rewrites. -If schema metadata is available and you pass a logical text field such as `event`, the server may still promote it to a preferred exact field such as `event.keyword` for exact term matching. If you pass an exact backend field like `event.keyword`, it is used as-is. -`sort_by` is supported for `hits` and `grouped_top_hits`. +- Returns the same structured output envelope as `query`, without field-hint alias rewrites. +- With schema metadata, a logical text field such as `event` may still promote to preferred exact field `event.keyword`. An exact backend field such as `event.keyword` is used as-is. +- `sort_by` supports `hits` and `grouped_top_hits`. diff --git a/website/reference/tools/query.md b/website/reference/tools/query.md index 5649026..e592683 100644 --- a/website/reference/tools/query.md +++ b/website/reference/tools/query.md @@ -4,7 +4,7 @@ description: "Reference of the query tool: search sources over a time window in # `query` -Searches one or more sources over an absolute time window, in hits, count, histogram, terms, stats or grouped mode. +Search one or more sources over an absolute time window. Input: @@ -33,26 +33,29 @@ Input: } ``` -Supported modes: +## Modes -- `hits` -- `count` -- `histogram` - - requires `histogram_interval` -- `terms` - - requires `group_by` -- `stats` - - requires `stats_field` - - can also take `group_by` -- `grouped_top_hits` - - requires `group_by` - - requires `sort_by` - - accepts `top_hits_size` +| Mode | Required fields | Optional fields | +| --- | --- | --- | +| `hits` | | `sort_by` (sort on a chosen field) | +| `count` | | | +| `histogram` | `histogram_interval` | | +| `terms` | `group_by` | | +| `stats` | `stats_field` | `group_by` | +| `grouped_top_hits` | `group_by`, `sort_by` | `top_hits_size` | -Every response includes a `query_echo` section so the investigating agent can explain which sources, bounds, filters, and mode were actually used. -For `hits` mode, you can also pass `sort_by` to sort on a chosen field. In `query`, exact filters and grouped bucket fields use field-hint alias resolution plus schema-aware exact-field preference when metadata is available. When the server rewrites a logical field such as `event` to a safer exact field such as `event.keyword`, the response includes an advisory. If schema metadata is unavailable, the query proceeds with the requested field and returns a `schema_unavailable` advisory instead of silently pretending exact-safe resolution happened. +## Field resolution -For very large single-source hit sets, `query` also supports cursor pagination: +- Exact filters and grouped bucket fields use field-hint aliases and, with metadata, schema-aware exact-field preference. +- Rewriting `event` to safer exact field `event.keyword` returns an advisory. +- Without metadata, queries use the requested field and return `schema_unavailable`; exact-safe resolution is not claimed. +- Nested filters require working `schema` metadata; otherwise they fail early with a configuration error, not a backend 404. + +Every response includes `query_echo`: sources, bounds, filters and mode actually used. + +## Cursor pagination + +For large hit sets, pass `cursor`. Supported only for **single-source `hits`** queries: ```json { @@ -65,11 +68,7 @@ For very large single-source hit sets, `query` also supports cursor pagination: } ``` -Cursor pagination is currently supported only for single-source `hits` queries. - -Nested filtering also depends on schema metadata. If a source does not expose a working `schema` backend, nested requests fail early with a clear configuration error instead of reaching Kibana and failing with a backend 404. - -Example `stats` query: +## `stats` example ```json { @@ -83,7 +82,7 @@ Example `stats` query: } ``` -Example `grouped_top_hits` query: +## `grouped_top_hits` example ```json {