Порівняти коміти

..
Автор SHA1 Повідомлення Дата
arookieofc 606aff1613 fix(remote): sync outside-root classpath entries for SSH Java launches (#2641) 2026-11-11 14:58:49 +08:00
1447 змінених файлів з 4951 додано та 98171 видалено

No files matched your search

-83
Переглянути файл
@@ -1,83 +0,0 @@
---
name: eval
description: Run the code-intelligence evals (`pnpm eval`), pick the worst failing scenario, diagnose it from the attached timeline, fix it, prove the fix with before / after numbers measured on this machine without regressing other scenarios, and open one PR. Use for "跑 eval", "优化 Java 代码智能", "eval 失败了", an issue labelled `eval`, or any change to Java navigation / usages / gutter markers that needs numbers.
---
# Eval loop: measure → fix → prove
`pnpm eval` drives the production workbench in Chromium against a release `openidea-server`
with real language servers (Java / jdtls for now) and judges every scenario on what the user
sees: time to feedback, to the first result, to the final one, and the results themselves vs
`evals/<language>/golden/`. One machine, before and after: numbers from different machines or
from the nightly host don't judge a fix.
## 0. Preconditions
- `jdtls` on PATH and `JAVA_HOME` pointing at a JDK of the pinned `JAVA` version (projects
compile against it, and the JDK scenarios' results are in its sources; jdtls itself runs on
a newer installed JDK when it needs one). Pinned versions: `.github/tool-versions.env`. A
mismatch prints a warning and the report records both: say so in the PR when it applies.
- First run builds `cargo build --release -p ide-host --bin openidea-server` and the perf
bundle; later iterations of a frontend-only change can use `--no-build` after a
`pnpm exec vite build --mode perf --outDir target/eval/dist`, a backend-only one after the
cargo build alone.
- Run evals one at a time on this machine, with nothing heavy (builds, other evals) beside them.
## 1. Baseline on the base commit
```sh
git switch --detach origin/main # or the fix branch's merge base
pnpm eval --save-baseline # target/eval/baseline.json, this machine only
```
Read `target/eval/report.json`. Each verdict has `status`, `failures` (`kind`: `error`,
`timeout`, `degraded`, `correctness`, `budget`, `golden`), `regressions`, `repro` (the command
that reruns it) and, for failures, `diagnostics`: every command sent to the server in the
slowest run's window (`cmd`, `start`, `ms`, `ok`, `bytes`, `items`), background tasks, server
status changes and the server's log tail.
## 2. Pick one problem
Worst first: wrong or missing results and silent degradation, then timeouts, then the
feedback budget (100 ms), then the slowest `completeMs`. One problem per PR. Say which
scenario and metric, with its value vs budget.
## 3. Diagnose from the timeline
- Commands in the window ordered by `start`: what ran, what waited on what, what's serial
that could overlap, which request returned many `items`/`bytes`, which failed.
- `feedbackMs` far above 16 ms with a fast `firstResultMs`: the UI shows nothing while it
waits (a hint or popup that only appears after a delay).
- A task still running across the window: background work competing with the request.
- For the frontend side rerun with `--trace` and open `target/eval/traces/<id>@<cond>.json`
in Chrome's Performance panel.
- Name the cause at class level (as in `issue-fix`): it usually affects sibling scenarios.
## 4. Fix, then rerun that scenario until it passes
```sh
pnpm eval --only <scenario> --condition <warm|cold> # the verdict's `repro`
```
Correctness is enforced, so a faster answer that drops, truncates or serves stale results
fails. Don't trade a budget miss for a degraded answer.
## 5. Full suite, then PR
```sh
pnpm eval # vs the baseline from step 1: no new failures or regressions
```
PR per `issue-fix` §6, plus in **How it was verified** the before / after table from
`target/eval/report.md` (both on this machine, commits named) and the tool versions line.
## Guard rails
- Goldens (`evals/*/golden/`) and budgets (`evals/budgets.json`) are human-owned. Don't edit
them, and don't run `--update-golden`, to make a scenario pass. When a golden looks wrong
(a tool bump, a deliberate behaviour change), stop and say so: a PR that touches them gets
`needs-decision`.
- Don't weaken a scenario's checks or timeouts in `evals/<language>/scenarios.ts` to pass.
Adding a scenario (for a case the sweep or a user found) is fine; record its golden with
`--update-golden --only <id>` and ask a person to review the file.
- A flaky timing (passes on rerun) is not a regression: rerun once before acting on one.
-1
Переглянути файл
@@ -1 +0,0 @@
../../.agents/skills/eval
-2
Переглянути файл
@@ -1,2 +0,0 @@
custom:
- https://github.com/l0ng-ai/openidea#sponsor
Бінарний файл не відображається.

До

Ширина:  |  Висота:  |  Розмір: 70 KiB

Бінарний файл не відображається.

До

Ширина:  |  Висота:  |  Розмір: 97 KiB

-14
Переглянути файл
@@ -1,14 +0,0 @@
# Language server and tool versions for Language smoke (.github/workflows/language-smoke.yml)
# and the code-intelligence evals (pnpm eval). Pinned so a release can't fail an unrelated
# PR or move a golden result; bump them deliberately (and re-record goldens: pnpm eval --update-golden).
VTSLS=0.3.0
TYPESCRIPT=6.0.3
BASEDPYRIGHT=1.40.2
RUFF=0.17.3
GOPLS=v0.24.0
JDTLS=1.63.0-202610191322
KOTLIN_LSP=263.7084.0
GRADLE=8.14.3
INTELEPHENSE=1.18.5
# The JDK projects compile against (JAVA_HOME): library navigation opens its sources.
JAVA=21
+11 -4
Переглянути файл
@@ -24,7 +24,6 @@ on:
- "package.json"
- "pnpm-lock.yaml"
- ".github/workflows/language-smoke.yml"
- ".github/tool-versions.env"
pull_request:
branches: [main]
types: [opened, synchronize, reopened]
@@ -39,6 +38,16 @@ concurrency:
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
# Pinned so a server release can't fail an unrelated PR; bump them deliberately.
VTSLS: 0.3.0
TYPESCRIPT: 6.0.3
BASEDPYRIGHT: 1.40.2
RUFF: 0.17.3
GOPLS: v0.24.0
JDTLS: 1.63.0-202610191322
KOTLIN_LSP: 263.7084.0
GRADLE: 8.14.3
INTELEPHENSE: 1.18.5
jobs:
smoke:
@@ -52,8 +61,6 @@ jobs:
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
# Tool versions, shared with the evals (pnpm eval).
- run: grep -E '^[A-Z0-9_]+=' .github/tool-versions.env >> "$GITHUB_ENV"
- uses: dtolnay/rust-toolchain@stable
with: { components: "rust-analyzer, rustfmt" }
# One build cache for every language; main saves it.
@@ -89,7 +96,7 @@ jobs:
- if: matrix.language == 'java' || matrix.language == 'kotlin'
uses: actions/setup-java@v4
with: { distribution: temurin, java-version: "${{ env.JAVA }}" }
with: { distribution: temurin, java-version: "21" }
- if: matrix.language == 'java'
run: |
mkdir -p "$RUNNER_TEMP/jdtls"
+1 -22
Переглянути файл
@@ -251,28 +251,7 @@ jobs:
core.setFailed(`latest.json would miss ${missing.join(", ")}`);
return;
}
// The update's notes: the release body without its Download section. That
// section only matters on the release page, and What's new follows it.
const { data: release } = await github.rest.repos.getRelease({ owner, repo, release_id });
const withoutDownloadSection = (body) => {
const kept = [];
let skip = 0;
for (const line of body.split(/\r?\n/)) {
const heading = /^(#{1,6})\s+(.*)$/.exec(line);
if (heading) {
const level = heading[1].length;
if (!skip && level <= 3 && /^download$/i.test(heading[2].trim())) {
skip = level;
continue;
}
if (skip && level <= skip) skip = 0;
}
if (!skip) kept.push(line);
}
return kept.join("\n").trim();
};
const notes = withoutDownloadSection(String(release.body ?? ""));
const manifest = { version: tag.replace(/^v/, ""), notes, pub_date: new Date().toISOString(), platforms };
const manifest = { version: tag.replace(/^v/, ""), notes: "", pub_date: new Date().toISOString(), platforms };
const old = assets.find((a) => a.name === "latest.json");
if (old) await github.rest.repos.deleteReleaseAsset({ owner, repo, asset_id: old.id });
await github.rest.repos.uploadReleaseAsset({
-1
Переглянути файл
@@ -40,7 +40,6 @@ CLAUDE.local.md
.claude/skills/*
!.claude/skills/verify
!.claude/skills/issue-fix
!.claude/skills/eval
# Local reference screenshots
pycharm-screenshots/
-2
Переглянути файл
@@ -9,11 +9,9 @@
"dist",
"target",
"src/bindings.ts",
"evals/*/golden",
"src/editor/core/cm/angular/parser*.ts",
"src/editor/core/cm/jsp/parser*.ts",
"src/editor/core/cm/astro/parser*.ts",
"src/editor/core/cm/django/parser*.ts",
"plugins/themes.md",
"pnpm-lock.yaml",
"**/*.toml",
+3 -11
Переглянути файл
@@ -24,8 +24,8 @@ Visual style: dark, low-contrast greys, one cool accent
Plugin docs: `plugins/README.md`, `plugins/themes.md`.
- `java/` — JDT LS extension bundles (`java-usages`, `kotlin-bridge`) and the patched
Lombok agent, embedded into the app by `ide-sdk` / `ide-tools`.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `evals/` — code-intelligence evals (`pnpm eval`).
`fixtures/` — sample projects per language for manual and automated testing.
- `e2e/` — Playwright UI tests (`pnpm e2e`). `fixtures/` — sample projects per language
for manual and automated testing.
- `docs/` — the Mintlify documentation site (`docs.json` navigation, `en/` and `zh/` pages);
preview with `npx mint dev` in `docs/`.
@@ -59,12 +59,9 @@ Unclear product decisions → `needs-decision` and stop.
## Skills
Do not use Impeccable or its related skills in this repository, including globally installed copies.
Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`verify` drives the running app through Tauri MCP and captures screenshot evidence;
`issue-fix` takes an issue from reproduction to one PR;
`eval` takes a failing code-intelligence eval to a fix with before / after numbers.
`issue-fix` takes an issue from reproduction to one PR.
## Commands
@@ -84,11 +81,6 @@ Shared agent skills live in `.agents/skills/` (`.claude/skills/` links to them):
`pnpm gen:bindings`.
- `pnpm perf` (UI interaction latency / frames / leaks), `pnpm perf:rust` (criterion
benches vs budgets). Budgets: `perf/budgets.json`.
- `pnpm eval [--only <id>] [--runs n] [--save-baseline]` — Java code intelligence (navigation,
usages, gutter markers) in the production workbench against a release `openidea-server` and
jdtls: time to feedback / first result / complete, results vs `evals/<language>/golden/`.
Report: `target/eval/report.json`. Goldens and `evals/budgets.json` are changed by people only.
Tool versions: `.github/tool-versions.env` (shared with Language smoke).
- `pnpm release:prepare <version>` — the release commit's changes: app version everywhere,
`Cargo.lock`, and the plugin API snapshot when `wit/` moved to a new version.
Створено
+46 -214
Переглянути файл
@@ -2637,6 +2637,7 @@ version = "1.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "337d46834ee672ab3e48caca2cb0c78cc174fb12b3a68d0d88f99a0519a5e36e"
dependencies = [
"generic-array 0.14.7",
"rustversion",
"typenum",
]
@@ -4144,7 +4145,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "ide-agent"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dunce",
"encoding_rs",
@@ -4168,7 +4169,7 @@ dependencies = [
[[package]]
name = "ide-analysis"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ast-grep-core",
"ide-core",
@@ -4182,7 +4183,7 @@ dependencies = [
[[package]]
name = "ide-android"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"serde",
@@ -4193,7 +4194,7 @@ dependencies = [
[[package]]
name = "ide-build"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"blake3",
"chardetng",
@@ -4211,7 +4212,7 @@ dependencies = [
[[package]]
name = "ide-cli"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dirs 6.0.0",
"serde",
@@ -4225,7 +4226,7 @@ dependencies = [
[[package]]
name = "ide-clipboard"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"gtk",
"objc2 0.6.5",
@@ -4235,7 +4236,7 @@ dependencies = [
[[package]]
name = "ide-core"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dunce",
"encoding_rs",
@@ -4252,7 +4253,7 @@ dependencies = [
[[package]]
name = "ide-coverage"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"ide-run",
@@ -4266,20 +4267,15 @@ dependencies = [
[[package]]
name = "ide-dap"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"futures",
"ide-core",
"ide-heap",
"ide-jdwp",
"ide-jsonrpc",
"ide-jvm-attach",
"ide-jvm-inspect",
"ide-language",
"libc",
"memchr",
"parking_lot",
"percent-encoding",
"portable-pty",
"serde",
"serde_json",
@@ -4292,24 +4288,19 @@ dependencies = [
[[package]]
name = "ide-db"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"base64 0.22.1",
"bb8",
"blake3",
"dirs 6.0.0",
"dunce",
"flate2",
"futures",
"ide-core",
"ide-http",
"ide-ssh",
"notify",
"open",
"oracledb",
"parking_lot",
"reqwest",
"rsa",
"rustls",
"serde",
"serde_json",
@@ -4323,12 +4314,11 @@ dependencies = [
"toml_edit 0.25.17+spec-1.1.0",
"tracing",
"uuid",
"zeroize",
]
[[package]]
name = "ide-docker"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"serde",
@@ -4341,7 +4331,7 @@ dependencies = [
[[package]]
name = "ide-extension"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4362,7 +4352,7 @@ dependencies = [
[[package]]
name = "ide-fonts"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"objc2-core-foundation",
"objc2-core-text",
@@ -4374,7 +4364,7 @@ dependencies = [
[[package]]
name = "ide-github"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"ide-vcs",
@@ -4385,27 +4375,9 @@ dependencies = [
"which",
]
[[package]]
name = "ide-heap"
version = "0.1.43"
dependencies = [
"criterion",
"ide-core",
"libc",
"memmap2",
"proptest",
"serde",
"serde_json",
"specta",
"tempfile",
"thiserror 2.0.21",
"tracing",
"uuid",
]
[[package]]
name = "ide-host"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4425,11 +4397,9 @@ dependencies = [
"ide-db",
"ide-docker",
"ide-extension",
"ide-heap",
"ide-http",
"ide-index",
"ide-jsonrpc",
"ide-jvm-attach",
"ide-language",
"ide-lsp",
"ide-mcp",
@@ -4437,7 +4407,6 @@ dependencies = [
"ide-plugin",
"ide-remote",
"ide-run",
"ide-scaffold",
"ide-sdk",
"ide-search",
"ide-spell",
@@ -4448,7 +4417,6 @@ dependencies = [
"ide-tools",
"ide-vcs",
"ide-vfs",
"ide-web",
"ide-workspace",
"ignore",
"jsonschema",
@@ -4465,7 +4433,6 @@ dependencies = [
"specta-typescript",
"tempfile",
"tokio",
"tokio-tungstenite",
"toml 0.9.12+spec-1.1.0",
"tracing",
"tracing-subscriber",
@@ -4476,7 +4443,7 @@ dependencies = [
[[package]]
name = "ide-http"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"ide-vfs",
@@ -4495,7 +4462,7 @@ dependencies = [
[[package]]
name = "ide-index"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"blake3",
"criterion",
@@ -4515,21 +4482,9 @@ dependencies = [
"tracing",
]
[[package]]
name = "ide-jdwp"
version = "0.1.43"
dependencies = [
"ide-jvm-inspect",
"proptest",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tracing",
]
[[package]]
name = "ide-jsonrpc"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"encoding_rs",
"futures",
@@ -4544,29 +4499,9 @@ dependencies = [
"tracing",
]
[[package]]
name = "ide-jvm-attach"
version = "0.1.43"
dependencies = [
"libc",
"tempfile",
"thiserror 2.0.21",
]
[[package]]
name = "ide-jvm-inspect"
version = "0.1.43"
dependencies = [
"futures",
"serde",
"serde_json",
"specta",
"thiserror 2.0.21",
]
[[package]]
name = "ide-language"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"blake3",
"criterion",
@@ -4607,7 +4542,6 @@ dependencies = [
"tree-sitter-nix",
"tree-sitter-ocaml",
"tree-sitter-php",
"tree-sitter-prisma-io",
"tree-sitter-proto",
"tree-sitter-python",
"tree-sitter-ruby",
@@ -4623,15 +4557,13 @@ dependencies = [
[[package]]
name = "ide-lsp"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dunce",
"encoding_rs",
"futures",
"globset",
"ide-core",
"ide-jsonrpc",
"ide-jvm-inspect",
"ide-language",
"ide-text",
"ide-vfs",
@@ -4651,7 +4583,7 @@ dependencies = [
[[package]]
name = "ide-mcp"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"bytes",
"dunce",
@@ -4670,7 +4602,7 @@ dependencies = [
[[package]]
name = "ide-notebook"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"futures",
"ide-core",
@@ -4693,7 +4625,7 @@ dependencies = [
[[package]]
name = "ide-plugin"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"blake3",
"dunce",
@@ -4716,10 +4648,9 @@ dependencies = [
[[package]]
name = "ide-profiler"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"ide-jvm-attach",
"ide-run",
"serde",
"serde_json",
@@ -4730,10 +4661,9 @@ dependencies = [
[[package]]
name = "ide-remote"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dunce",
"futures",
"globset",
"ide-core",
"ide-run",
@@ -4750,7 +4680,7 @@ dependencies = [
[[package]]
name = "ide-run"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dunce",
"encoding_rs",
@@ -4772,25 +4702,9 @@ dependencies = [
"zip",
]
[[package]]
name = "ide-scaffold"
version = "0.1.43"
dependencies = [
"ide-core",
"ide-tools",
"ide-vcs",
"insta",
"serde",
"serde_json",
"specta",
"tempfile",
"toml 0.9.12+spec-1.1.0",
"tracing",
]
[[package]]
name = "ide-sdk"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dunce",
"ide-core",
@@ -4805,16 +4719,14 @@ dependencies = [
"tempfile",
"toml 0.9.12+spec-1.1.0",
"tracing",
"url",
"zip",
]
[[package]]
name = "ide-search"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"criterion",
"encoding_rs",
"globset",
"grep-matcher",
"grep-regex",
@@ -4832,7 +4744,7 @@ dependencies = [
[[package]]
name = "ide-spell"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"parking_lot",
@@ -4845,7 +4757,7 @@ dependencies = [
[[package]]
name = "ide-ssh"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dirs 6.0.0",
"flate2",
@@ -4855,22 +4767,18 @@ dependencies = [
"parking_lot",
"russh",
"russh-sftp",
"rustls",
"rustls-platform-verifier",
"serde",
"serde_json",
"specta",
"suppaftp",
"tempfile",
"tokio",
"tokio-rustls",
"tracing",
"uuid",
]
[[package]]
name = "ide-syntax"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"ide-language",
@@ -4885,7 +4793,7 @@ dependencies = [
[[package]]
name = "ide-terminal"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"ide-core",
"libc",
@@ -4898,7 +4806,7 @@ dependencies = [
[[package]]
name = "ide-text"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"criterion",
"ide-core",
@@ -4911,7 +4819,7 @@ dependencies = [
[[package]]
name = "ide-tools"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"flate2",
"hex",
@@ -4932,7 +4840,7 @@ dependencies = [
[[package]]
name = "ide-vcs"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"blake3",
"dunce",
@@ -4953,7 +4861,7 @@ dependencies = [
[[package]]
name = "ide-vfs"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"chardetng",
"dunce",
@@ -4977,11 +4885,11 @@ dependencies = [
[[package]]
name = "ide-web"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"dirs 6.0.0",
"ide-core",
"open",
"parking_lot",
"percent-encoding",
"serde",
"specta",
@@ -4993,7 +4901,7 @@ dependencies = [
[[package]]
name = "ide-workspace"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"blake3",
"dunce",
@@ -5562,29 +5470,6 @@ dependencies = [
"libc",
]
[[package]]
name = "lazy-regex"
version = "3.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0a2cce8a22920a29fdcc8c9170aa5d29edd9fbcf8bea856e24c2cb1c2419f67"
dependencies = [
"lazy-regex-proc_macros",
"once_cell",
"regex",
]
[[package]]
name = "lazy-regex-proc_macros"
version = "3.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f1e4b1f1d8c71edb4ff2b2150ec4606445fd2ddf67ddc097655daf59286baa6"
dependencies = [
"proc-macro2",
"quote",
"regex",
"syn 3.0.9",
]
[[package]]
name = "lazy_static"
version = "1.5.1"
@@ -6594,7 +6479,7 @@ dependencies = [
[[package]]
name = "openidea"
version = "0.1.43"
version = "0.1.41"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -6628,7 +6513,6 @@ dependencies = [
"ide-profiler",
"ide-remote",
"ide-run",
"ide-scaffold",
"ide-sdk",
"ide-search",
"ide-spell",
@@ -6951,26 +6835,6 @@ dependencies = [
"siphasher",
]
[[package]]
name = "pin-project"
version = "1.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2466b2336ed02bcdca6b294417127b90ec92038d1d5c4fbeac971a922e0e0924"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c96395f0a926bc13b1c17622aaddda1ecb55d49c8f1bf9777e4d877800a43f8b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -7719,9 +7583,9 @@ dependencies = [
[[package]]
name = "russh"
version = "0.65.1"
version = "0.64.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94be3d86c71b9ed220a675c405e35c7a02565a009bf8da2e9cc13a700320a5c5"
checksum = "ba61e87b9ec9a39a59a6bbed4c0b8ff7fb07073405b24a767a1f2c8560ac8f58"
dependencies = [
"aes",
"bitflags 2.13.2",
@@ -7855,9 +7719,9 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.1.6"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5e76742e5294153d4dde1207dbec5bdd81eef6fabdc0d2118babadabfd858aa"
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
dependencies = [
"bitflags 2.13.2",
"errno",
@@ -9019,23 +8883,6 @@ version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "suppaftp"
version = "12.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b58fcf921745cca6f0127c9f822069947da8c6dbeff52e9004750bcc448171f"
dependencies = [
"async-trait",
"chrono",
"lazy-regex",
"log",
"pin-project",
"rustls-pki-types",
"thiserror 2.0.21",
"tokio",
"tokio-rustls",
]
[[package]]
name = "swift-rs"
version = "1.0.9"
@@ -9644,7 +9491,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.3",
"getrandom 0.3.4",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -9876,10 +9723,7 @@ checksum = "d25a406cddcc431a75d3d9afc6a7c0f7428d4891dd973e4d54c56b46127bf857"
dependencies = [
"futures-util",
"log",
"rustls",
"rustls-pki-types",
"tokio",
"tokio-rustls",
"tungstenite",
]
@@ -10445,16 +10289,6 @@ dependencies = [
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-prisma-io"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c35912d65a6adb07f781db46d843bf0464935e267da6ed192705516adafc7c24"
dependencies = [
"cc",
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-proto"
version = "0.6.0"
@@ -10583,8 +10417,6 @@ dependencies = [
"httparse",
"log",
"rand 0.9.5",
"rustls",
"rustls-pki-types",
"sha1 0.10.7",
"thiserror 2.0.21",
"utf-8",
+1 -8
Переглянути файл
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.43"
version = "0.1.41"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
@@ -43,15 +43,10 @@ ide-ssh = { path = "crates/ide-ssh" }
ide-remote = { path = "crates/ide-remote" }
ide-host = { path = "crates/ide-host" }
ide-mcp = { path = "crates/ide-mcp" }
ide-scaffold = { path = "crates/ide-scaffold" }
ide-notebook = { path = "crates/ide-notebook" }
ide-extension = { path = "crates/ide-extension" }
ide-plugin = { path = "crates/ide-plugin" }
ide-cli = { path = "crates/ide-cli" }
ide-jvm-attach = { path = "crates/ide-jvm-attach" }
ide-jvm-inspect = { path = "crates/ide-jvm-inspect" }
ide-jdwp = { path = "crates/ide-jdwp" }
ide-heap = { path = "crates/ide-heap" }
# serialization & IPC typing
serde = { version = "1", features = ["derive"] }
@@ -68,7 +63,6 @@ parking_lot = "0.12"
rayon = "1"
dashmap = "6"
futures = "0.3"
tokio-tungstenite = { version = "0.28", default-features = false, features = ["connect", "__rustls-tls"] }
# errors & diagnostics
thiserror = "2"
@@ -88,7 +82,6 @@ trash = "5"
blake3 = "1"
dunce = "1"
dirs = "6"
memmap2 = "0.9"
zip = { version = "9.2.3", default-features = false, features = ["deflate"] }
# search, index, language
-13
Переглянути файл
@@ -19,8 +19,6 @@
<a href="#plugins">Plugins</a>
&nbsp;·&nbsp;
<a href="#community">Community</a>
&nbsp;·&nbsp;
<a href="#sponsor">Sponsor</a>
</p>
https://github.com/user-attachments/assets/234dd54c-1255-4b3a-b35c-533d78d737c0
@@ -70,17 +68,6 @@ the [official plugins](plugins/official). The plugin API is still a preview.
- Contributing: see [CONTRIBUTING.md](CONTRIBUTING.md) and the [contributor docs](https://openidea.sh/en/contribute/setup).
## Sponsor
If OpenIDEA saves you time, you can support its development with Alipay or WeChat Pay.
<table>
<tr>
<td align="center"><img src=".github/assets/sponsor-alipay.png" alt="Alipay QR code" width="200"><br>Alipay</td>
<td align="center"><img src=".github/assets/sponsor-wechat.png" alt="WeChat Pay QR code" width="200"><br>WeChat Pay</td>
</tr>
</table>
## License
[Apache 2.0](LICENSE). Bundled third-party assets are listed in [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md).
+4 -3
Переглянути файл
@@ -1,10 +1,11 @@
# Third-party licenses
## Project type icons
## Flutter and Vite project icons
`src/ui/brands/*.svg` use the unmodified logo paths from
`src/ui/brands/flutter.svg` and `vite.svg` use the unmodified logo paths from
[Simple Icons](https://github.com/simple-icons/simple-icons), retrieved through
Iconify (`simple-icons:angular`, `simple-icons:astro`, `simple-icons:cplusplus`, `simple-icons:dart`, `simple-icons:django`, `simple-icons:dotnet`, `simple-icons:elixir`, `simple-icons:express`, `simple-icons:fastapi`, `simple-icons:flask`, `simple-icons:flutter`, `simple-icons:go`, `simple-icons:haskell`, `simple-icons:html5`, `simple-icons:kotlin`, `simple-icons:ktor`, `simple-icons:laravel`, `simple-icons:nestjs`, `simple-icons:nextdotjs`, `simple-icons:nodedotjs`, `simple-icons:nuxt`, `simple-icons:openjdk`, `simple-icons:phoenixframework`, `simple-icons:php`, `simple-icons:python`, `simple-icons:quarkus`, `simple-icons:ruby`, `simple-icons:rubyonrails`, `simple-icons:rust`, `simple-icons:scala`, `simple-icons:springboot`, `simple-icons:svelte`, `simple-icons:swift`, `simple-icons:symfony`, `simple-icons:vite`, `simple-icons:zig`) and filled with the brand colors (the standard icon grey for black logos). Licensed under [CC0 1.0 Universal](https://github.com/simple-icons/simple-icons/blob/develop/LICENSE.md).
Iconify (`simple-icons:flutter`, `simple-icons:vite`) and filled with the project
icon colors. Licensed under [CC0 1.0 Universal](https://github.com/simple-icons/simple-icons/blob/develop/LICENSE.md).
## Project Lombok
-102
Переглянути файл
@@ -1,102 +0,0 @@
//! A short-lived ACP session to learn models, effort levels and other config options
//! before the user's first task (Cursor and other ACP agents do not expose these in
//! `initialize` alone).
use super::{Conn, Dialect};
use crate::config;
use crate::model::AgentConfigOption;
use crate::provider::ProviderSpec;
use crate::{Shared, client_info};
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::sync::Arc;
use std::time::Duration;
const DISCOVER_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(3) } else { Duration::from_secs(120) };
/// Opens `session/new`, reads `configOptions`, and records per-model `thought_level` choices.
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(), String> {
let root = shared.paths.project_root.clone();
let launch = spec.clone();
let launch = tokio::task::spawn_blocking({
let root = root.clone();
move || launch.resolve_for_launch(&root)
})
.await
.map_err(|e| e.to_string())?;
let cwd = dunce::canonicalize(&root).unwrap_or(root);
let stderr: Arc<Mutex<Vec<String>>> = Arc::default();
let lines = stderr.clone();
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(move |l| lines.lock().push(l));
let client = Dialect::wrap(Arc::new(ide_jsonrpc::NoopHandler));
let peer = (shared.connector)(&launch, &cwd, client.clone(), on_stderr)
.map_err(|e| format!("Couldn't start {} (`{}`): {e}", launch.name, launch.command))?;
let conn = Conn::new(peer.clone(), client);
let work = async {
conn.initialize(&client_info()).await.map_err(|e| e.to_string())?;
let cwd_s = cwd.to_string_lossy().into_owned();
// No project MCP during discovery: Cursor can spend a long time spawning servers.
let new: Value = conn
.request("session/new", json!({ "cwd": cwd_s, "mcpServers": [] }), None)
.await
.map_err(|e| e.to_string())?;
let mut options = config::options_of(&new);
config::add_client_options(&launch.id, &mut options);
let model_option = options.iter().find(|o| o.category.as_deref() == Some("model"));
let effort = levels_for_model_from_slice(&options);
let levels: Vec<(String, Vec<AgentConfigOption>)> = match (model_option, effort) {
(Some(model), Some(effort)) if !model.values.is_empty() => {
model.values.iter().map(|v| (v.value.clone(), effort.clone())).collect()
}
(Some(model), Some(effort)) => model.current.clone().map(|m| vec![(m, effort)]).unwrap_or_default(),
_ => Vec::new(),
};
let option_ids: Vec<_> = options.iter().map(|o| o.id.as_str()).collect();
tracing::info!(
"agent {}: settings discovery got {} config option(s) {:?}, {} model effort list(s)",
launch.id,
options.len(),
option_ids,
levels.len()
);
shared.remember_discovered(&launch.id, &options, &[], &levels);
Ok::<(), String>(())
};
let result = tokio::time::timeout(DISCOVER_TIMEOUT, work).await;
peer.kill();
match result {
Ok(Ok(())) => Ok(()),
Ok(Err(e)) => Err(match stderr.lock().last() {
Some(line) => format!("{e}: {line}"),
None => e,
}),
Err(_) => {
let tail = stderr.lock().join("\n");
if tail.is_empty() {
Err("the agent did not answer in time".into())
} else {
Err(format!("the agent did not answer in time:\n{tail}"))
}
}
}
}
fn is_thought_level(o: &AgentConfigOption) -> bool {
o.category.as_deref() == Some("thought_level")
|| matches!(o.id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
}
fn levels_for_model_from_slice(options: &[AgentConfigOption]) -> Option<Vec<AgentConfigOption>> {
let levels: Vec<_> = options
.iter()
.filter(|o| is_thought_level(o))
.cloned()
.map(|mut o| {
if o.category.as_deref() != Some("thought_level") {
o.category = Some("thought_level".into());
}
o
})
.collect();
(!levels.is_empty()).then_some(levels)
}
-4
Переглянути файл
@@ -14,7 +14,6 @@
//! - Agent → client ([`Dialect`]): `session/update`s are normalized ([`normalize`]) and
//! permission requests get the v1 `toolCall`.
pub(crate) mod discover;
mod normalize;
use ide_jsonrpc::{Handler, Peer, RpcError, codes};
@@ -58,9 +57,6 @@ fn client_capabilities() -> Value {
"terminal": false,
// Subagents as sessions of their own (ACP subagents RFD; v2 baseline).
"subagents": {},
// Cursor exposes model parameters (e.g. Composer `fast`) as separate config options
// only when the client advertises this flag (undocumented Cursor ACP extension).
"_meta": { "parameterizedModelPicker": true },
})
}
+2 -74
Переглянути файл
@@ -93,81 +93,9 @@ fn the_negotiating_request_carries_both_versions() {
assert!(p["capabilities"].is_object());
assert_eq!(p["clientCapabilities"]["subagents"], json!({}));
assert_eq!(p["clientCapabilities"]["terminal"], false);
assert_eq!(
p["clientCapabilities"]["_meta"]["parameterizedModelPicker"], true,
"Cursor and similar agents need parameterized model options (e.g. fast on/off)"
);
// What we sent in the capture is what claude-agent-acp negotiated v2 and v1 from.
let sent = frames(TURN, "out")[0]["params"]["clientCapabilities"].clone();
let mut expected = sent;
expected["_meta"] = p["clientCapabilities"]["_meta"].clone();
assert_eq!(expected, p["clientCapabilities"]);
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent() {
#[derive(Default)]
struct DiscoveryAgent {
peer: std::sync::OnceLock<Peer>,
calls: parking_lot::Mutex<Vec<(String, Value)>>,
}
impl Handler for DiscoveryAgent {
fn on_notification(&self, _: &str, _: Value) {}
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push((method.to_owned(), params));
match method {
"initialize" => Ok(json!({
"protocolVersion": 1,
"agentCapabilities": { "loadSession": true }
})),
"session/new" => Ok(json!({
"sessionId": "discover",
"configOptions": [
{"id": "model", "category": "model", "type": "select", "currentValue": "m1",
"options": [{"value": "m1", "name": "One"}, {"value": "m2", "name": "Two"}]},
{"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
})),
_ => Ok(json!({})),
}
}
}
let agent = Arc::new(DiscoveryAgent::default());
let tmp = tempfile::tempdir().unwrap();
let agent_for_connector = agent.clone();
let connector: crate::Connector = Arc::new(move |_, _, client, _| {
let (ca, cb) = tokio::io::duplex(1 << 20);
let (car, caw) = tokio::io::split(ca);
let (cbr, cbw) = tokio::io::split(cb);
let agent_peer = Peer::connect_framed(cbr, cbw, agent_for_connector.clone(), Framing::Lines);
let _ = agent_for_connector.peer.set(agent_peer);
Ok(Peer::connect_framed(car, caw, client, Framing::Lines))
});
let mgr = crate::AgentManager::with_connector(
crate::AgentPaths {
project_root: tmp.path().to_path_buf(),
store_dir: tmp.path().join("store"),
worktrees_dir: tmp.path().join("worktrees"),
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert_eq!(calls.iter().map(|(m, _)| m.as_str()).collect::<Vec<_>>(), ["initialize", "session/new"]);
assert_eq!(calls[1].1["mcpServers"], json!([]));
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
let fast = cursor.config.iter().find(|o| o.id == "fast").expect("fast option");
assert_eq!(fast.category.as_deref(), Some("model_config"));
assert_eq!(fast.current.as_deref(), Some("false"));
assert!(cursor.thought_levels.contains_key("m1") && cursor.thought_levels.contains_key("m2"));
assert!(cursor.thought_levels["m1"].iter().any(|o| o.category.as_deref() == Some("thought_level")));
let sent = &frames(TURN, "out")[0]["params"];
assert_eq!(sent["clientCapabilities"], p["clientCapabilities"]);
}
#[test]
+1 -24
Переглянути файл
@@ -107,15 +107,7 @@ pub(crate) fn parse_options(options: &Value) -> Vec<AgentConfigOption> {
Some(AgentConfigOption {
name: str_of(o, "name").unwrap_or_else(|| id.clone()),
description: str_of(o, "description"),
category: str_of(o, "category").or_else(|| {
matches!(id.as_str(), "model" | "mode")
.then(|| id.clone())
.or_else(|| {
matches!(id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
.then(|| "thought_level".into())
})
.or_else(|| matches!(id.as_str(), "fast").then(|| "model_config".into()))
}),
category: str_of(o, "category").or_else(|| matches!(id.as_str(), "model" | "mode").then(|| id.clone())),
current: str_of(o, "currentValue"),
values,
via: AgentConfigVia::Option,
@@ -341,21 +333,6 @@ mod tests {
assert!(bypass_approvals(&url_mode));
}
#[test]
fn effort_and_fast_categories_are_inferred_from_ids() {
let c = options_of(&json!({
"configOptions": [
{"id": "effort", "type": "select", "currentValue": "high",
"options": [{"value": "low"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
}));
assert_eq!(c.len(), 2);
assert_eq!(c[0].category.as_deref(), Some("thought_level"));
assert_eq!(c[1].category.as_deref(), Some("model_config"));
}
#[test]
fn command_sources_in_updates_and_stored_tasks() {
let update = json!({"availableCommands": [
+15 -70
Переглянути файл
@@ -8,7 +8,6 @@
use super::native::claude::history;
use serde::Serialize;
use specta::Type;
use std::io::BufRead;
use std::path::{Path, PathBuf};
/// A session on this machine that no task holds yet.
@@ -63,28 +62,11 @@ fn message_text(e: &serde_json::Value) -> Option<String> {
}
}
/// Whether a session that ran in `cwd` belongs to the project at `root` (in it or below it).
fn in_project(cwd: Option<&str>, root: &Path) -> bool {
cwd.is_some_and(|cwd| Path::new(cwd).starts_with(root))
}
/// Claude Code transcripts of the project at `root`: `<config>/projects/<escaped cwd>/<id>.jsonl`.
/// Only the folders named after `root` or a folder below it are read, so other projects'
/// transcripts (often gigabytes) are never opened.
pub fn claude_sessions(config_dir: &Path, root: &Path) -> Vec<ExternalSession> {
/// Claude Code transcripts: `<config>/projects/*/<id>.jsonl`.
pub fn claude_sessions(config_dir: &Path) -> Vec<ExternalSession> {
let mut out = Vec::new();
let projects = config_dir.join("projects");
let escaped = history::escape(root);
let dirs: Vec<PathBuf> = std::fs::read_dir(&projects)
.into_iter()
.flatten()
.flatten()
.filter(|d| {
let name = d.file_name().to_string_lossy().into_owned();
name == escaped || name.starts_with(&format!("{escaped}-"))
})
.map(|d| d.path())
.collect();
let dirs: Vec<PathBuf> = std::fs::read_dir(&projects).into_iter().flatten().flatten().map(|d| d.path()).collect();
for dir in dirs {
let files: Vec<PathBuf> = std::fs::read_dir(&dir)
.into_iter()
@@ -111,10 +93,6 @@ pub fn claude_sessions(config_dir: &Path, root: &Path) -> Vec<ExternalSession> {
.filter_map(|e| e.get("cwd").and_then(serde_json::Value::as_str))
.next()
.map(str::to_owned);
// Escaping is lossy (`a/b-c` and `a/b/c` share a folder): the transcript decides.
if !in_project(cwd.as_deref(), root) {
continue;
}
out.push(ExternalSession {
provider: "claude".to_owned(),
session_id: id,
@@ -160,13 +138,12 @@ fn codex_user_text(line: &serde_json::Value) -> Option<String> {
/// One rollout file's session, or `None` when it names no `session_meta`.
fn codex_file(path: &Path) -> Option<ExternalSession> {
let file = std::fs::File::open(path).ok()?;
let text = std::fs::read_to_string(path).ok()?;
let mut id: Option<(String, Option<String>)> = None;
let mut title: Option<String> = None;
// `session_meta` is first, but cap the scan: giant rollouts must not stall listing.
for line in std::io::BufReader::new(file).lines().take(400) {
let Ok(line) = line else { break };
let Ok(v) = serde_json::from_str::<serde_json::Value>(&line) else { continue };
for line in text.lines().take(400) {
let Ok(v) = serde_json::from_str::<serde_json::Value>(line) else { continue };
if id.is_none()
&& let Some(meta) = codex_meta(&v)
{
@@ -206,11 +183,11 @@ fn rollout_files(dir: &Path, depth: usize, out: &mut Vec<PathBuf>) {
}
}
/// Codex sessions of the project at `root`: `<home>/.codex/sessions/**`.
pub fn codex_sessions(codex_home: &Path, root: &Path) -> Vec<ExternalSession> {
/// Codex sessions: `<home>/.codex/sessions/**`.
pub fn codex_sessions(codex_home: &Path) -> Vec<ExternalSession> {
let mut files = Vec::new();
rollout_files(&codex_home.join("sessions"), 4, &mut files);
files.iter().filter_map(|p| codex_file(p)).filter(|s| in_project(s.cwd.as_deref(), root)).collect()
files.iter().filter_map(|p| codex_file(p)).collect()
}
#[cfg(test)]
@@ -245,7 +222,7 @@ mod tests {
],
);
write(&tmp.path().join("projects/-work-app/empty.jsonl"), &[]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
let sessions = claude_sessions(tmp.path());
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("claude", "abc123"));
@@ -257,8 +234,8 @@ mod tests {
#[test]
fn claude_title_falls_back_to_the_first_prompt() {
let tmp = tempfile::tempdir().unwrap();
write(&tmp.path().join("projects/-work-app/def456.jsonl"), &[claude_user("u1", "hello there")]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
write(&tmp.path().join("projects/-x/def456.jsonl"), &[claude_user("u1", "hello there")]);
let sessions = claude_sessions(tmp.path());
assert_eq!(sessions.len(), 1);
assert_eq!(sessions[0].title, "hello there");
}
@@ -276,7 +253,7 @@ mod tests {
],
);
write(&tmp.path().join("sessions/notes.jsonl"), &[json!({"nope": true})]);
let sessions = codex_sessions(tmp.path(), Path::new("/work"));
let sessions = codex_sessions(tmp.path());
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("codex", "thread-1"));
@@ -287,39 +264,7 @@ mod tests {
#[test]
fn missing_directories_list_nothing() {
let tmp = tempfile::tempdir().unwrap();
assert!(claude_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
assert!(codex_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
}
#[test]
fn only_sessions_of_the_project_are_listed() {
let tmp = tempfile::tempdir().unwrap();
let user = |cwd: &str| {
json!({"type": "user", "uuid": "u1", "parentUuid": null,
"message": {"role": "user", "content": "hi"}, "cwd": cwd})
};
write(&tmp.path().join("projects/-work-app/in-root.jsonl"), &[user("/work/app")]);
write(&tmp.path().join("projects/-work-app-web/below.jsonl"), &[user("/work/app/web")]);
// `/work/app-old` escapes like a folder below `/work/app`; its transcript says otherwise.
write(&tmp.path().join("projects/-work-app-old/sibling.jsonl"), &[user("/work/app-old")]);
write(&tmp.path().join("projects/-other/elsewhere.jsonl"), &[user("/other")]);
let mut ids: Vec<String> =
claude_sessions(tmp.path(), Path::new("/work/app")).into_iter().map(|s| s.session_id).collect();
ids.sort();
assert_eq!(ids, ["below", "in-root"]);
let rollout = |name: &str, cwd: &str| {
write(
&tmp.path().join(format!("codex/sessions/2026/01/02/{name}.jsonl")),
&[json!({"payload": {"type": "session_meta", "payload": {"id": name, "cwd": cwd}}})],
)
};
rollout("mine", "/work/app/api");
rollout("theirs", "/other");
let codex: Vec<String> = codex_sessions(&tmp.path().join("codex"), Path::new("/work/app"))
.into_iter()
.map(|s| s.session_id)
.collect();
assert_eq!(codex, ["mine"]);
assert!(claude_sessions(&tmp.path().join("nope")).is_empty());
assert!(codex_sessions(&tmp.path().join("nope")).is_empty());
}
}
+49 -242
Переглянути файл
@@ -16,7 +16,6 @@ mod model;
mod native;
mod provider;
pub mod registry;
pub mod security;
mod skills;
mod timeline;
mod worktree;
@@ -40,9 +39,6 @@ use std::time::Duration;
/// Receives the events of one project's tasks.
pub type AgentSink = Arc<dyn Fn(AgentEvent) + Send + Sync>;
/// Returns the active command execution security policy for the project.
pub type SecurityHook = Arc<dyn Fn() -> ide_core::settings::AgentSecuritySettings + Send + Sync>;
/// Entry changes are batched for this long before they are sent.
const FLUSH_DELAY: Duration = Duration::from_millis(40);
/// Updates that arrive this late after `session/load` returns are still replayed history.
@@ -106,7 +102,6 @@ struct Shared {
runtime: tokio::runtime::Handle,
connector: Connector,
sink: RwLock<Option<AgentSink>>,
security: RwLock<Option<SecurityHook>>,
known: Mutex<HashMap<String, Known>>,
providers: RwLock<Vec<provider::ProviderSpec>>,
/// Last probe of each provider.
@@ -182,15 +177,9 @@ impl Shared {
let shared = self.clone();
// Keep the lookup alive if a composer unmounts while it is waiting.
self.runtime.spawn(async move {
tracing::info!("agent {}: settings discovery started", spec.id);
let discovered = if spec.driver == provider::Driver::Acp {
acp::discover::run(&shared, &spec).await.map(|_| None)
} else if spec.driver == provider::Driver::Claude {
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await.map(Some)
} else {
return;
};
if let Ok(Some(found)) = discovered {
if let Ok(found) =
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await
{
let commands =
config::commands_of(&json!({ "availableCommands": found.commands }), &spec.id);
let options = config::options_of(&json!({ "configOptions": found.options }));
@@ -209,8 +198,6 @@ impl Shared {
shared.remember_discovered(&spec.id, &options, &commands, &levels);
})
.await;
} else if let Err(e) = discovered {
tracing::info!("agent {}: settings discovery failed: {e}", spec.id);
}
let _ = done.send(true);
});
@@ -233,17 +220,12 @@ impl Shared {
fn knows_all_levels(&self, provider: &str) -> bool {
let cache = self.known.lock();
let Some(known) = cache.get(provider) else { return false };
let models: Vec<&str> = known
known
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
.collect();
// No advertised models yet: nothing to attach effort levels to.
if models.is_empty() {
return false;
}
models.iter().all(|id| known.thought_levels.contains_key(*id))
.flat_map(|o| &o.values)
.all(|v| known.thought_levels.contains_key(&v.value))
}
fn root_key(&self) -> String {
@@ -452,7 +434,6 @@ impl AgentManager {
runtime,
connector,
sink: RwLock::new(None),
security: RwLock::new(None),
known: Mutex::new(known),
providers,
probes: Mutex::new(HashMap::new()),
@@ -496,11 +477,6 @@ impl AgentManager {
tasks
}
/// Sets the callback used to read the active command execution security policy.
pub fn set_security_hook(&self, hook: SecurityHook) {
*self.shared.security.write() = Some(hook);
}
/// The agents (custom ones re-read), whether each is installed, and what they offered last time.
pub fn providers(&self) -> Vec<AgentProvider> {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -525,9 +501,7 @@ impl AgentManager {
let new = specs.iter().find(|p| &p.id == id);
matches!((old, new), (Some(a), Some(b)) if a.command == b.command && a.args == b.args && a.env == b.env)
});
*prev = specs.clone();
drop(probes);
drop(prev);
*prev = specs;
list
}
@@ -548,30 +522,6 @@ impl AgentManager {
provider::set_builtin_env(&dir, id, env)
}
/// Saves a new custom ACP agent through its agent file.
pub fn save_custom_agent(&self, input: CustomAgentInput) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::save_custom(&dir, &input)
}
/// Removes a custom agent's file (registry installs go through `registry_remove`).
pub fn remove_custom_agent(&self, id: &str) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::remove_custom(&dir, id)
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom agents.
pub fn import_acp_agents(&self, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::import_acp(&dir, text)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -602,16 +552,10 @@ impl AgentManager {
let mut offered = offered;
if task.is_none()
&& (offered.is_empty() || !self.shared.has_config(&provider) || !self.shared.knows_all_levels(&provider))
&& let Some(spec) = self
.shared
.spec(&provider)
.filter(|s| matches!(s.driver, provider::Driver::Claude | provider::Driver::Acp))
&& let Some(spec) = self.shared.spec(&provider).filter(|s| s.driver == provider::Driver::Claude)
{
let claude = spec.driver == provider::Driver::Claude;
self.shared.fetch_settings(spec).await;
if claude {
offered = self.shared.known_commands(&provider);
}
offered = self.shared.known_commands(&provider);
}
for &(name, description) in builtin_commands(&provider) {
if !offered.iter().any(|c| c.name == name) {
@@ -648,19 +592,17 @@ impl AgentManager {
self.tasks.read().values().filter_map(|t| t.state.lock().session_id.clone()).collect()
}
/// Sessions of this project on this machine that no task holds yet, newest first. Reads
/// transcripts from disk: call it off the UI and command threads.
/// Sessions on this machine that no task holds yet, newest first (#312).
pub fn external_sessions(&self) -> Vec<ExternalSession> {
let mut out = Vec::new();
let root = self.shared.paths.project_root.clone();
if let Some(spec) = self.shared.spec("claude")
&& let Some(dir) = native::claude::cli::config_dir(&spec.env)
{
out.extend(external::claude_sessions(&dir, &root));
out.extend(external::claude_sessions(&dir));
}
let home = self.shared.paths.home_dir.clone().or_else(home_dir);
if let Some(home) = home {
out.extend(external::codex_sessions(&home.join(".codex"), &root));
out.extend(external::codex_sessions(&home.join(".codex")));
}
let known = self.imported_session_ids();
out.retain(|s| !known.contains(&s.session_id));
@@ -944,38 +886,14 @@ impl AgentManager {
}
o.clone()
};
let previous = target.current.clone();
// Cursor's `session/set_config_option` (Fast, effort) can take seconds. Show the
// choice first so the switch and slider move immediately.
apply_config_choice(&task, option, &value);
task.persist();
task.emit_task();
let live = task.live.lock().clone().filter(|l| !l.peer.is_closed());
let mut model_options =
target.category.as_deref() != Some("model") || previous.as_deref() == Some(value.as_str());
target.category.as_deref() != Some("model") || target.current.as_deref() == Some(value.as_str());
if let Some(live) = live.filter(|_| option != config::CURSOR_APPROVAL) {
let (method, mut params) = config::set_request(&target, &value);
params["sessionId"] = json!(live.session);
let reply: Value = match live.peer.request(method, params, None).await {
Ok(reply) => reply,
Err(e) => {
{
let mut s = task.state.lock();
s.wanted.retain(|c| c.id != option);
if let Some(prev) = previous.clone() {
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: prev });
}
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = previous.clone();
}
s.task.model = model_of(&s.task.config).or(s.task.model.take());
}
task.persist();
task.emit_task();
return Err(IdeError::internal(task.describe(e)));
}
};
let reply: Value =
live.peer.request(method, params, None).await.map_err(|e| IdeError::internal(task.describe(e)))?;
if let Some(options) = reply.get("configOptions") {
config::merge_update(&mut task.state.lock().task.config, options);
model_options = true;
@@ -986,6 +904,8 @@ impl AgentManager {
if !model_options {
s.task.config.retain(|o| o.category.as_deref() != Some("thought_level"));
}
s.wanted.retain(|c| c.id != option);
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: value.clone() });
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = Some(value);
}
@@ -1312,17 +1232,6 @@ impl AgentManager {
};
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
self.tasks.write().remove(id);
*task.deleted.lock() = true;
let _ = std::fs::remove_file(self.shared.paths.store_dir.join(format!("{id}.json")));
let saved = task.state.lock().saved.clone();
if let Some(saved) = saved
&& !self.tasks.read().values().any(|t| t.state.lock().saved.as_ref() == Some(&saved))
{
let _ = std::fs::remove_dir_all(self.shared.paths.store_dir.join(saved));
}
// The list drops the task now; removing a large worktree's folder can take a while.
self.shared.emit(AgentEvent::Removed { id: id.to_owned() });
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
@@ -1332,72 +1241,53 @@ impl AgentManager {
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
self.tasks.write().remove(id);
*task.deleted.lock() = true;
let _ = std::fs::remove_file(self.shared.paths.store_dir.join(format!("{id}.json")));
let saved = task.state.lock().saved.clone();
if let Some(saved) = saved
&& !self.tasks.read().values().any(|t| t.state.lock().saved.as_ref() == Some(&saved))
{
let _ = std::fs::remove_dir_all(self.shared.paths.store_dir.join(saved));
}
self.shared.emit(AgentEvent::Removed { id: id.to_owned() });
Ok(())
}
/// Files the task changed since it started.
pub async fn changes(&self, id: &str) -> IdeResult<Vec<AgentChange>> {
let task = self.task(id)?;
let at = task.base();
let (cwd, base) = task.cwd_and_base();
let before = task.state.lock().before.clone();
blocking(move || at.changes(&before)).await
}
/// The task's uncommitted changes and, for a worktree task, the commits made since it
/// started (newest first). Both are read against the same `HEAD`.
pub async fn grouped_changes(&self, id: &str) -> IdeResult<AgentChanges> {
let task = self.task(id)?;
let at = task.base();
let before = task.state.lock().before.clone();
blocking(move || at.grouped_changes(&before)).await
blocking(move || Ok(worktree::without_untouched(worktree::changes(&cwd, &base)?, &before))).await
}
/// Content of `path` (in the task's directory) when the task started.
pub async fn base_text(&self, id: &str, path: PathBuf) -> IdeResult<Option<String>> {
let at = self.task(id)?.base();
blocking(move || match at.rev()? {
Some(rev) => worktree::file_at(at.cwd(), &path, &rev),
None => Ok(None),
})
.await
let (cwd, base) = self.task(id)?.cwd_and_base();
blocking(move || worktree::file_at(&cwd, &path, &base)).await
}
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as
/// `view` compares them: as they were when it started, or as they are in `HEAD`.
pub async fn rollback_changes(
&self,
id: &str,
rel_paths: Option<Vec<String>>,
view: AgentChangesView,
) -> IdeResult<()> {
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as they
/// were when it started (files the user had changed then, as the user left them).
pub async fn rollback_changes(&self, id: &str, rel_paths: Option<Vec<String>>) -> IdeResult<()> {
let task = self.task(id)?;
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before rolling back its changes"));
}
let at = task.base();
let (cwd, base) = task.cwd_and_base();
let (before, saved) = {
let s = task.state.lock();
(s.before.clone(), s.saved.as_ref().map(|name| self.shared.paths.store_dir.join(name)))
};
let restamped = blocking(move || {
let (changes, rev) = match view {
AgentChangesView::Group => (at.grouped_changes(&before)?.uncommitted, worktree::head_commit(at.cwd())?),
AgentChangesView::SinceStart => (at.changes(&before)?, at.rev()?),
};
let sources = match rev.as_deref() {
Some(rev) => worktree::rename_sources(at.cwd(), rev)?,
None => Default::default(),
};
let changes = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before);
let mut restamped = Vec::new();
for change in changes.iter().filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path))) {
// A rename goes back to its old path too; the new one is taken away first, so a
// case-only rename on a case-insensitive disk ends as the old file.
for change in std::iter::once(change).chain(sources.get(&change.rel_path)) {
worktree::restore(at.cwd(), rev.as_deref(), change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
}
worktree::restore(&cwd, &base, change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
}
}
Ok(restamped)
@@ -1413,41 +1303,6 @@ impl AgentManager {
Ok(())
}
/// Accept: copies the task's changed files among `rel_paths` (all with `None`) from its
/// worktree into the local checkout, uncommitted, for the user to review and commit.
/// Files the checkout changed itself are left alone and reported. Only worktree tasks
/// have anything to accept.
pub async fn accept_changes(&self, id: &str, rel_paths: Option<Vec<String>>) -> IdeResult<AgentAcceptOutcome> {
let task = self.task(id)?;
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before accepting its changes"));
}
let (isolation, cwd, base, root, before) = {
let s = task.state.lock();
(
s.task.isolation,
s.task.cwd.0.clone(),
s.task.base_commit.clone(),
s.task.project_root.clone().map(|p| p.0).unwrap_or_else(|| self.shared.paths.project_root.clone()),
(*s.before).clone(),
)
};
if isolation != Isolation::Worktree {
return Err(IdeError::invalid("the task works in the local checkout: there is nothing to accept"));
}
let Some(base) = base else {
return Err(IdeError::invalid("the task has no base revision to accept from"));
};
blocking(move || {
let wanted: Vec<AgentChange> = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before)
.into_iter()
.filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path)))
.collect();
worktree::accept(&root, &cwd, &base, &wanted)
})
.await
}
/// Stops every agent (project closed).
pub fn shutdown(&self) {
for task in self.tasks.read().values() {
@@ -1498,15 +1353,15 @@ impl Task {
})
}
fn base(&self) -> worktree::TaskBase {
fn cwd_and_base(&self) -> (PathBuf, String) {
let s = self.state.lock();
let cwd = s.task.cwd.0.clone();
match s.task.isolation {
Isolation::Local => worktree::TaskBase::Local { cwd, start: s.task.base_commit.clone() },
Isolation::Worktree => {
worktree::TaskBase::Worktree { cwd, base: s.task.base_commit.clone().unwrap_or_else(|| "HEAD".into()) }
}
}
// In the project's own checkout the user commits too: compare against the current HEAD,
// as the Commit tool window does, so commits made since the task started drop out.
let base = match s.task.isolation {
Isolation::Local => None,
Isolation::Worktree => s.task.base_commit.clone(),
};
(s.task.cwd.0.clone(), base.unwrap_or_else(|| "HEAD".into()))
}
fn emit_task(&self) {
@@ -2266,44 +2121,6 @@ impl Task {
.iter()
.map(|question| question.answer_key.clone().unwrap_or_else(|| question.question.clone()))
.collect();
if let Some(hook) = self.shared.security.read().clone()
&& let Some(command) = security::extract_command(params)
{
match security::evaluate_command(&command, &hook()) {
security::PolicyDecision::Blocked(pattern) => {
let reject = options.iter().find(|o| o.kind.starts_with("reject")).cloned();
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
options,
questions: Vec::new(),
chosen: Some(format!("Blocked by policy ({pattern})")),
});
return match reject {
Some(option) => json!({ "outcome": { "outcome": "selected", "optionId": option.id } }),
None => json!({ "outcome": { "outcome": "cancelled" } }),
};
}
security::PolicyDecision::Allowed => {
if let Some(allow) = options
.iter()
.find(|o| o.kind == "allow_once")
.or_else(|| options.iter().find(|o| o.kind.starts_with("allow")))
.cloned()
{
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
options,
questions: Vec::new(),
chosen: Some(format!("{} (whitelisted)", allow.name)),
});
return json!({ "outcome": { "outcome": "selected", "optionId": allow.id } });
}
}
security::PolicyDecision::AskUser => {}
}
}
let (tx, rx) = std::sync::mpsc::channel();
let entry =
self.push_entry(AgentEntry::Permission { id, title, options: options.clone(), questions, chosen: None });
@@ -2617,7 +2434,7 @@ fn with_details(message: String, data: Option<&Value>) -> String {
}
}
pub(crate) fn client_info() -> Value {
fn client_info() -> Value {
json!({ "name": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") })
}
@@ -2676,16 +2493,6 @@ fn retry_of(update: &Value, now: f64) -> Option<AgentRetry> {
})
}
fn apply_config_choice(task: &Task, option: &str, value: &str) {
let mut s = task.state.lock();
s.wanted.retain(|c| c.id != option);
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: value.to_owned() });
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = Some(value.to_owned());
}
s.task.model = model_of(&s.task.config).or(s.task.model.take());
}
fn model_of(config: &[AgentConfigOption]) -> Option<String> {
config.iter().find(|o| o.category.as_deref() == Some("model")).and_then(|o| o.current.clone())
}
+1 -86
Переглянути файл
@@ -484,8 +484,7 @@ pub struct NewAgentTask {
pub base: Option<String>,
}
/// A file the task changed: against [`AgentTask::base_commit`], or against `HEAD` in the
/// grouped view.
/// A file the task changed, relative to [`AgentTask::base_commit`].
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentChange {
@@ -494,44 +493,6 @@ pub struct AgentChange {
pub kind: ide_vcs::ChangeKind,
pub added: u32,
pub removed: u32,
/// Path before a rename; the left side of its diff reads this path.
pub old_path: Option<String>,
}
/// A commit made while the task ran, with the files it changed against its first parent.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentCommit {
pub id: String,
pub short_id: String,
pub summary: String,
pub message: String,
pub time_ms: f64,
/// First parent, or `None` for a root commit.
pub parent: Option<String>,
pub files: Vec<AgentChange>,
}
/// Which comparison a rollback of the task's changes uses.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum AgentChangesView {
/// Everything since the commit the task started from.
#[default]
SinceStart,
/// The uncommitted files against `HEAD`; commits made since the start stay.
Group,
}
/// A task's uncommitted files and the commits made since it started.
#[derive(Clone, Debug, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentChanges {
/// Files changed against `HEAD`, less those changed before the task started and untouched since.
pub uncommitted: Vec<AgentChange>,
/// Commits from the task's start (exclusive) to `HEAD` (inclusive), newest first. Empty
/// for tasks working in the project's own checkout.
pub commits: Vec<AgentCommit>,
}
/// Streamed to the window that subscribed (`AgentManager::subscribe`).
@@ -641,49 +602,3 @@ pub struct McpServerInput {
fn default_true() -> bool {
true
}
/// Payload for creating a custom ACP agent (always the ACP driver); the id
/// becomes the `<config>/agents/<id>.toml` file name.
#[derive(Clone, Debug, Deserialize, Type)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CustomAgentInput {
pub id: String,
pub name: String,
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: BTreeMap<String, String>,
}
/// What importing one entry of a JetBrains `acp.json` file did.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AcpImportOutcome {
/// The `agent_servers` key the entry was imported from.
pub name: String,
/// The id its agent file would get (also when skipped).
pub id: String,
pub saved: bool,
/// Why the entry was skipped; empty when it was saved.
#[serde(default)]
pub reason: String,
}
/// What accepting a worktree task's changes copied into the local checkout.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptOutcome {
/// Relative paths copied into the checkout (uncommitted), including ones already identical.
pub accepted: Vec<String>,
/// Files left alone because the checkout has its own edits to them.
pub skipped: Vec<AgentAcceptSkipped>,
}
/// A file accept left alone, with why.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptSkipped {
pub path: String,
pub reason: String,
}
+1 -1
Переглянути файл
@@ -31,7 +31,7 @@ pub(super) fn transcript(config_dir: &Path, cwd: &Path, id: &str) -> PathBuf {
}
/// The project directory name Claude Code uses for `cwd`.
pub(crate) fn escape(cwd: &Path) -> String {
pub(super) fn escape(cwd: &Path) -> String {
cwd.to_string_lossy().chars().map(|c| if c.is_ascii_alphanumeric() { c } else { '-' }).collect()
}
+1 -239
Переглянути файл
@@ -2,7 +2,7 @@
//! `<config>/agents/<id>.toml` (written by hand, or by installing one from the ACP
//! Registry, see `registry`).
use crate::{AcpImportOutcome, AgentCommandReadOnlyReason, AgentProvider, AgentSource, CustomAgentInput};
use crate::{AgentCommandReadOnlyReason, AgentProvider, AgentSource};
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -59,8 +59,6 @@ const BUILTIN: &[(&str, &str, Driver, &[&str], &str)] = &[
("cursor", "Cursor", Driver::Acp, &["cursor-agent", "acp"], "Run `cursor-agent login` in a terminal."),
("opencode", "OpenCode", Driver::Acp, &["opencode", "acp"], "Run `opencode auth login` in a terminal."),
("qwen", "Qwen Code", Driver::Acp, &["qwen", "--experimental-acp"], "Run `qwen` in a terminal and sign in."),
("qoder", "Qoder", Driver::Acp, &["qodercli", "--acp"], "Run `qodercli login` in a terminal."),
("qodercn", "Qoder CN", Driver::Acp, &["qoderclicn", "--acp"], "Run `qoderclicn login` in a terminal."),
("kimi", "Kimi CLI", Driver::Acp, &["kimi", "acp"], "Run `kimi` in a terminal and sign in."),
("goose", "Goose", Driver::Acp, &["goose", "acp"], "Run `goose configure` in a terminal."),
("auggie", "Auggie", Driver::Acp, &["auggie", "--acp"], "Run `auggie login` in a terminal."),
@@ -278,127 +276,6 @@ fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAg
Ok(())
}
/// Saves a new custom agent; refuses built-in and ACP Registry ids, taken ids
/// and an empty command. The id is also the agent file's name, so it must be
/// a safe file stem.
pub(crate) fn save_custom(custom_dir: &Path, input: &CustomAgentInput) -> IdeResult<()> {
let id = validated_custom_id(&input.id)?;
if builtin_spec(&id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent; change its command instead")));
}
if input.command.trim().is_empty() {
return Err(IdeError::invalid("the agent command must not be empty"));
}
let file = custom_dir.join(format!("{id}.toml"));
if file.try_exists()? {
if CustomAgent::read(&file).is_ok_and(|a| a.registry.is_some()) {
return Err(IdeError::invalid(format!(
"{id} is installed from the ACP Registry; uninstall it there to replace it"
)));
}
return Err(IdeError::invalid(format!("there is already a custom agent '{id}'")));
}
CustomAgent {
name: Some(input.name.clone()),
driver: Driver::Acp,
command: input.command.clone(),
args: input.args.clone(),
env: input.env.clone(),
registry: None,
}
.write(&file)?;
Ok(())
}
/// Removes a custom agent's file; refuses built-in, ACP Registry and missing ids.
pub(crate) fn remove_custom(custom_dir: &Path, id: &str) -> IdeResult<()> {
let id = validated_custom_id(id)?;
let id = id.as_str();
if builtin_spec(id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent and cannot be removed")));
}
let file = custom_dir.join(format!("{id}.toml"));
let agent =
CustomAgent::read(&file).map_err(|_| IdeError::not_found(format!("there is no custom agent '{id}'")))?;
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry; uninstall it there")));
}
match std::fs::remove_file(&file) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e.into()),
_ => Ok(()),
}
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom
/// agents, one outcome per entry; entries that collide, lack a command or
/// have no usable name are skipped with a reason.
pub(crate) fn import_acp(custom_dir: &Path, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
#[derive(Deserialize)]
struct AcpJson {
#[serde(default)]
agent_servers: BTreeMap<String, AcpServer>,
}
#[derive(Deserialize)]
struct AcpServer {
#[serde(default)]
command: String,
#[serde(default)]
args: Vec<String>,
#[serde(default)]
env: BTreeMap<String, String>,
}
let parsed: AcpJson =
serde_json::from_str(text).map_err(|e| IdeError::invalid(format!("not an acp.json file: {e}")))?;
let mut outcomes = Vec::with_capacity(parsed.agent_servers.len());
for (name, server) in parsed.agent_servers {
let id = slugify_id(&name);
let mut outcome = AcpImportOutcome { name, id: id.clone(), saved: false, reason: String::new() };
if id.is_empty() {
outcome.reason = "the name has no usable characters for an id".to_owned();
} else if builtin_spec(&id).is_some() {
outcome.reason = format!("{id} is a built-in agent");
} else if server.command.trim().is_empty() {
outcome.reason = "the entry has no command".to_owned();
} else if custom_dir.join(format!("{id}.toml")).try_exists()? {
outcome.reason = format!("there is already a custom agent '{id}'");
} else {
CustomAgent {
name: Some(outcome.name.clone()),
driver: Driver::Acp,
command: server.command,
args: server.args,
env: server.env,
registry: None,
}
.write(&custom_dir.join(format!("{id}.toml")))?;
outcome.saved = true;
}
outcomes.push(outcome);
}
Ok(outcomes)
}
/// A custom agent id doubles as a file stem: letters, digits, `-` and `_` only.
fn validated_custom_id(id: &str) -> IdeResult<String> {
if !id.is_empty() && id.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_') {
return Ok(id.to_owned());
}
Err(IdeError::invalid(format!("'{id}' is not a usable agent id (letters, digits, '-' and '_' only)")))
}
/// Turns an `acp.json` server name into a file-stem id (`"My Agent!"` → `"my-agent"`).
fn slugify_id(name: &str) -> String {
let mut id = String::with_capacity(name.len());
for c in name.chars() {
if c.is_ascii_alphanumeric() {
id.push(c.to_ascii_lowercase());
} else if !id.ends_with('-') {
id.push('-');
}
}
id.trim_matches('-').to_owned()
}
/// An agent file, `<config>/agents/<id>.toml`.
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
@@ -532,45 +409,6 @@ mod tests {
use super::*;
use serde_json::json;
#[test]
fn qoder_global_and_cn_editions_are_both_built_in() {
let info = |id: &str| describe(&load(None).into_iter().find(|s| s.id == id).unwrap());
let (global, cn) = (info("qoder"), info("qodercn"));
assert_eq!((global.name.as_str(), global.command.as_str()), ("Qoder", "qodercli --acp"));
assert_eq!((cn.name.as_str(), cn.command.as_str()), ("Qoder CN", "qoderclicn --acp"));
assert_eq!(global.login_hint.as_deref(), Some("Run `qodercli login` in a terminal."));
}
#[test]
fn qodercn_is_available_without_a_file_and_preserves_existing_customizations() {
let dir = tempfile::tempdir().unwrap();
let qodercn = load(None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(qodercn.driver, Driver::Acp);
assert!(!is_unlisted(&qodercn));
let info = describe(&qodercn);
assert_eq!(info.name, "Qoder CN");
assert_eq!(info.command, "qoderclicn --acp");
assert_eq!(info.source, AgentSource::Builtin);
assert!(info.command_editable);
assert_eq!(info.login_hint.as_deref(), Some("Run `qoderclicn login` in a terminal."));
std::fs::write(
dir.path().join("qodercn.toml"),
"command = \"/custom/qoderclicn\"\nargs = [\"--acp\"]\n[env]\nK = \"v\"\n",
)
.unwrap();
let specs = load(Some(dir.path()));
assert_eq!(specs.iter().filter(|s| s.id == "qodercn").count(), 1);
let custom = specs.iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(custom.command, "/custom/qoderclicn");
assert_eq!(custom.env, [("K".into(), "v".into())]);
set_builtin_command(dir.path(), "qodercn", None).unwrap();
let reset = load(Some(dir.path())).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(reset.command, "qoderclicn");
assert_eq!(reset.args, ["--acp"]);
assert_eq!(reset.env, custom.env);
}
#[test]
fn login_hints_cover_all_builtin_commands_and_preserve_separate_login_tools() {
for spec in builtins() {
@@ -874,80 +712,4 @@ mod tests {
assert_eq!(spec.source, AgentSource::Registry { version: "0.10.0".into(), update: None });
assert!(toml::from_str::<CustomAgent>("command = \"x\"\n[registry]\nid = \"x\"\n").is_err());
}
fn custom_input(id: &str) -> CustomAgentInput {
CustomAgentInput {
id: id.into(),
name: "Mine".into(),
command: "/opt/mine".into(),
args: vec!["acp".into()],
env: BTreeMap::from([("K".to_owned(), "v".to_owned())]),
}
}
#[test]
fn custom_agents_save_list_and_remove() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap();
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
assert_eq!(mine.source, AgentSource::Custom);
remove_custom(dir.path(), "mine").unwrap();
assert!(!load(Some(dir.path())).iter().any(|s| s.id == "mine"));
remove_custom(dir.path(), "mine").unwrap_err();
}
#[test]
fn custom_agents_refuse_builtin_registry_and_bad_ids() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("claude")).unwrap_err();
save_custom(dir.path(), &custom_input("../evil")).unwrap_err();
save_custom(dir.path(), &custom_input("")).unwrap_err();
let mut empty_command = custom_input("mine");
empty_command.command = " ".into();
save_custom(dir.path(), &empty_command).unwrap_err();
save_custom(dir.path(), &custom_input("mine")).unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap_err();
remove_custom(dir.path(), "claude").unwrap_err();
// An id that isn't a plain file stem never reaches outside the agents directory.
let outside = dir.path().join("outside.toml");
std::fs::write(&outside, "command = \"/bin/sh\"\n").unwrap();
let agents = dir.path().join("agents");
std::fs::create_dir(&agents).unwrap();
remove_custom(&agents, "../outside").unwrap_err();
assert!(outside.exists());
std::fs::write(
dir.path().join("amp-acp.toml"),
"command = \"/registry/amp\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n",
)
.unwrap();
save_custom(dir.path(), &custom_input("amp-acp")).unwrap_err();
remove_custom(dir.path(), "amp-acp").unwrap_err();
}
#[test]
fn acp_json_imports_servers_and_skips_collisions() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("taken")).unwrap();
let outcomes = import_acp(
dir.path(),
r#"{"agent_servers": {
"My Agent!": {"command": "/opt/agent", "args": ["--acp"], "env": {"K": "v"}},
"taken": {"command": "/opt/taken"},
"claude": {"command": "/opt/claude"},
"!!!": {"command": "/opt/noname"},
"silent": {"args": []}
}}"#,
)
.unwrap();
let saved: Vec<&str> = outcomes.iter().filter(|o| o.saved).map(|o| o.id.as_str()).collect();
assert_eq!(saved, ["my-agent"]);
assert!(outcomes.iter().filter(|o| !o.saved).all(|o| !o.reason.is_empty()));
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "my-agent").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("My Agent!", "/opt/agent"));
assert_eq!(mine.source, AgentSource::Custom);
import_acp(dir.path(), "not json").unwrap_err();
assert!(import_acp(dir.path(), "{}").unwrap().is_empty());
}
}
-487
Переглянути файл
@@ -1,487 +0,0 @@
//! Command execution security policy (Whitelist / Blacklist) for coding agents.
use ide_core::settings::AgentSecuritySettings;
use serde_json::Value;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum PolicyDecision {
/// Command matched a blacklist pattern (`pattern`).
Blocked(String),
/// Command matched the whitelist and no blacklist pattern.
Allowed,
/// Neither matched; prompt the user as usual.
AskUser,
}
/// Extracts the shell command string from an ACP `session/request_permission` payload
/// when the request is for executing a command.
pub fn extract_command(params: &Value) -> Option<String> {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"))?;
let raw_cmd = tool.pointer("/rawInput/command").or_else(|| tool.pointer("/rawInput/cmd"));
if let Some(cmd) = raw_cmd.and_then(command_from_value) {
return Some(cmd);
}
let kind = tool.get("kind").and_then(Value::as_str).unwrap_or_default();
let name = tool.get("name").and_then(Value::as_str).unwrap_or_default();
let title =
tool.get("title").and_then(Value::as_str).or_else(|| params.get("title").and_then(Value::as_str))?.trim();
if let Some(cmd) = title.strip_prefix("Run ").map(str::trim).filter(|s| !s.is_empty()) {
return Some(cmd.to_owned());
}
if (kind == "execute" || matches!(name, "Bash" | "PowerShell" | "Execute" | "Run" | "terminal"))
&& !title.is_empty()
&& title != "Terminal"
&& title != "Run a command"
{
return Some(title.to_owned());
}
None
}
fn command_from_value(val: &Value) -> Option<String> {
match val {
Value::String(s) => {
let trimmed = s.trim();
(!trimmed.is_empty()).then(|| trimmed.to_owned())
}
Value::Array(items) => {
let words: Vec<&str> = items.iter().filter_map(Value::as_str).collect();
if words.is_empty() {
return None;
}
let cmd = match words.as_slice() {
[_, flag, script] if flag.starts_with('-') && flag.ends_with('c') => (*script).trim().to_owned(),
_ => words.join(" ").trim().to_owned(),
};
(!cmd.is_empty()).then_some(cmd)
}
_ => None,
}
}
/// Checks whether a single pattern matches `command` using prefix + glob rules:
/// - If `pattern` contains `*` or `?`, matches using glob wildcards (where `*` matches
/// any sequence of characters). A trailing ` *` in the pattern also matches the bare
/// command without arguments (e.g. `git status *` matches `git status`).
/// - Otherwise, matches when `command` equals `pattern` or starts with `pattern` followed
/// by whitespace (prefix match at word boundary).
pub fn matches_command(pattern: &str, command: &str) -> bool {
let pat = pattern.trim();
let cmd = command.trim();
if pat.is_empty() || cmd.is_empty() {
return false;
}
if pat.contains('*') || pat.contains('?') {
if let Some(stem) = pat.strip_suffix(" *")
&& cmd == stem.trim()
{
return true;
}
wildcard_match(pat.as_bytes(), cmd.as_bytes())
} else {
cmd == pat || cmd.strip_prefix(pat).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
}
fn wildcard_match(pat: &[u8], text: &[u8]) -> bool {
let (mut p, mut t) = (0, 0);
let (mut star_p, mut star_t) = (None, 0);
while t < text.len() {
if p < pat.len() && (pat[p] == b'?' || pat[p] == text[t]) {
p += 1;
t += 1;
} else if p < pat.len() && pat[p] == b'*' {
star_p = Some(p);
p += 1;
star_t = t;
} else if let Some(sp) = star_p {
p = sp + 1;
star_t += 1;
t = star_t;
} else {
return false;
}
}
while p < pat.len() && pat[p] == b'*' {
p += 1;
}
p == pat.len()
}
/// Matches an allowed command entry against a sub-command.
/// An allow entry is a whole command or a prefix only up to a word boundary:
/// e.g. `git status` matches `git status` and `git status -s`, but NOT `git statusx`.
/// Any accidental trailing `*` in the allow entry is stripped so `git status*` behaves
/// cleanly as a prefix at word boundaries instead of a glob wildcard.
pub fn matches_allow_entry(entry: &str, command: &str) -> bool {
let mut entry = entry.trim();
if let Some(stripped) = entry.strip_suffix('*') {
entry = stripped.trim_end();
}
let cmd = command.trim();
if entry.is_empty() || cmd.is_empty() {
return false;
}
cmd == entry || cmd.strip_prefix(entry).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
/// Checks whether a word is an environment variable assignment (`IDENTIFIER=...`).
fn is_env_assignment_word(word: &str) -> bool {
let Some((name, _)) = word.split_once('=') else {
return false;
};
if name.is_empty() {
return false;
}
let mut chars = name.chars();
match chars.next() {
Some(c) if c.is_ascii_alphabetic() || c == '_' => {}
_ => return false,
}
chars.all(|c| c.is_ascii_alphanumeric() || c == '_')
}
/// Returns true if the sub-command starts with an environment variable assignment,
/// e.g. `FOO=bar cmd` or `PORT=3000`.
fn has_leading_env_assignment(cmd: &str) -> bool {
let trimmed = cmd.trim_start();
let bytes = trimmed.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
}
i += 1;
continue;
}
if b == b'\'' || b == b'"' {
quote = Some(b);
i += 1;
continue;
}
if b.is_ascii_whitespace() {
break;
}
i += 1;
}
let first_word = &trimmed[..i];
is_env_assignment_word(first_word)
}
/// Checks whether `command` contains shell constructs that must never be auto-approved:
/// - Command substitution: `$(...)` or backticks
/// - Redirection: `>`, `>>`, `<`
/// - Here-docs: `<<`
/// - Subshells: `(...)`
/// - Leading environment assignments: `VAR=val cmd`
pub fn has_unsafe_shell_syntax(command: &str) -> bool {
let cmd = command.trim();
if cmd.is_empty() {
return false;
}
if has_leading_env_assignment(cmd) {
return true;
}
let bytes = cmd.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
i += 1;
continue;
}
// Inside double quotes: $(...) and `...` are evaluated by the shell.
if q == b'"' {
if b == b'`' {
return true;
}
if b == b'$' && i + 1 < bytes.len() && bytes[i + 1] == b'(' {
return true;
}
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'`' => return true,
b'$' if i + 1 < bytes.len() && bytes[i + 1] == b'(' => return true,
b'<' | b'>' => return true,
b'(' | b')' => return true,
_ => i += 1,
}
}
false
}
/// Splits a shell command into sub-commands separated by `;`, `&&`, `||`, `|`, or `\n`
/// (outside single/double quotes) so compound commands cannot bypass policy checks.
fn split_subcommands(command: &str) -> Vec<&str> {
let bytes = command.as_bytes();
let mut out = Vec::new();
let mut start = 0;
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if let Some(q) = quote {
if b == b'\\' && q == b'"' && i + 1 < bytes.len() {
i += 2;
continue;
}
if b == q {
quote = None;
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'\\' if i + 1 < bytes.len() => {
i += 2;
}
b';' | b'\n' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
i += 1;
start = i;
}
b'&' | b'|' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
if i + 1 < bytes.len() && bytes[i + 1] == b {
i += 2;
} else {
i += 1;
}
start = i;
}
_ => i += 1,
}
}
let tail = command[start..].trim();
if !tail.is_empty() {
out.push(tail);
}
out
}
/// Evaluates `command` against `settings`:
/// 1. Blacklist takes precedence: if the full command or any sub-command matches a
/// `blocked_commands` entry, returns [`PolicyDecision::Blocked`].
/// 2. Whitelist: allowed_commands cannot auto-approve when empty.
/// 3. Every sub-command in a chain must be safe (no command substitution, redirection,
/// subshell, or env assignments) and match at least one `allowed_commands` entry
/// exactly or up to word boundary; otherwise falls back to [`PolicyDecision::AskUser`].
pub fn evaluate_command(command: &str, settings: &AgentSecuritySettings) -> PolicyDecision {
let cmd = command.trim();
if cmd.is_empty() {
return PolicyDecision::AskUser;
}
let subcommands = split_subcommands(cmd);
for pat in &settings.blocked_commands {
let pat = pat.trim();
if pat.is_empty() {
continue;
}
if matches_command(pat, cmd) || subcommands.iter().any(|sub| matches_command(pat, sub)) {
return PolicyDecision::Blocked(pat.to_owned());
}
}
if settings.allowed_commands.is_empty() || subcommands.is_empty() {
return PolicyDecision::AskUser;
}
let all_allowed = subcommands.iter().all(|sub| {
let s = sub.trim();
if s.is_empty() || has_unsafe_shell_syntax(s) {
return false;
}
settings.allowed_commands.iter().any(|entry| matches_allow_entry(entry, s))
});
if all_allowed {
return PolicyDecision::Allowed;
}
PolicyDecision::AskUser
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn prefix_and_glob_matching() {
assert!(matches_command("git status", "git status"));
assert!(matches_command("git status", "git status -s"));
assert!(!matches_command("git status", "git statusbar"));
assert!(matches_command("git status*", "git status"));
assert!(matches_command("git status*", "git status --short"));
assert!(matches_command("npm test *", "npm test"));
assert!(matches_command("npm test *", "npm test unit"));
assert!(matches_command("rm -rf *", "rm -rf /tmp/foo"));
assert!(matches_command("rm -rf *", "rm -rf /"));
}
#[test]
fn blacklist_takes_precedence_and_catches_chained_commands() {
let settings = AgentSecuritySettings::default();
assert_eq!(evaluate_command("rm -rf /tmp/build", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command(":(){ :|:& };:", &settings), PolicyDecision::Blocked(":(){ :|:& };:".into()));
}
#[test]
fn allow_entry_matching_word_boundary() {
assert!(matches_allow_entry("git status", "git status"));
assert!(matches_allow_entry("git status", "git status -s"));
assert!(!matches_allow_entry("git status", "git statusbar"));
assert!(!matches_allow_entry("git status", "git statusx"));
// Trailing asterisk in allow entry normalizes cleanly to word boundary prefix
assert!(matches_allow_entry("git status*", "git status"));
assert!(matches_allow_entry("git status*", "git status --short"));
assert!(!matches_allow_entry("git status*", "git statusbar"));
assert!(matches_allow_entry("ls", "ls"));
assert!(matches_allow_entry("ls", "ls -la"));
assert!(!matches_allow_entry("ls", "lsof"));
assert!(matches_allow_entry("cargo check", "cargo check"));
assert!(matches_allow_entry("cargo check", "cargo check --workspace"));
assert!(!matches_allow_entry("cargo check", "cargo checkmate"));
}
#[test]
fn unsafe_shell_syntax_detection() {
// Command substitutions: $(...) and `...`
assert!(has_unsafe_shell_syntax("echo $(whoami)"));
assert!(has_unsafe_shell_syntax("echo \"user: $(whoami)\""));
assert!(has_unsafe_shell_syntax("echo `whoami`"));
assert!(has_unsafe_shell_syntax("echo \"`whoami`\""));
// Redirections: >, >>, <
assert!(has_unsafe_shell_syntax("git status > /tmp/out"));
assert!(has_unsafe_shell_syntax("git status >> /tmp/out"));
assert!(has_unsafe_shell_syntax("cat < /tmp/in"));
assert!(has_unsafe_shell_syntax("cat << 'EOF'"));
// Subshells: (...)
assert!(has_unsafe_shell_syntax("(cd /tmp && ls)"));
assert!(has_unsafe_shell_syntax("git status && (exit 1)"));
// Environment assignments: VAR=val ...
assert!(has_unsafe_shell_syntax("FOO=bar git status"));
assert!(has_unsafe_shell_syntax("PORT=3000 npm test"));
assert!(has_unsafe_shell_syntax("_SECRET=123 cargo test"));
assert!(has_unsafe_shell_syntax("A=\"b c\" git status"));
// Safe constructs with characters inside quotes or command flags
assert!(!has_unsafe_shell_syntax("git commit -m \"fix <bug> and (issue)\""));
assert!(!has_unsafe_shell_syntax("git commit -m 'fix <bug>'"));
assert!(!has_unsafe_shell_syntax("cargo test --test=unit"));
assert!(!has_unsafe_shell_syntax("git status -s"));
}
#[test]
fn default_settings_has_no_auto_approval_and_blocks_dangerous_defaults() {
let settings = AgentSecuritySettings::default();
assert!(settings.allowed_commands.is_empty(), "allowed_commands must start empty");
// Safe commands ask user by default
assert_eq!(evaluate_command("git status", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("ls", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo check", &settings), PolicyDecision::AskUser);
// Blacklist patterns are still blocked by default
assert_eq!(evaluate_command("rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("rm -rf /tmp/test", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn whitelist_allows_safe_single_and_chained_commands_only() {
let settings = AgentSecuritySettings {
allowed_commands: vec!["git status".into(), "git diff".into(), "cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
};
assert_eq!(evaluate_command("git status -s", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git diff HEAD~1", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git status && cargo test", &settings), PolicyDecision::Allowed);
// Word boundary: git statusx must not match git status
assert_eq!(evaluate_command("git statusx", &settings), PolicyDecision::AskUser);
// Unsafe syntax (redirection, command substitution, env assignment) falls back to AskUser
assert_eq!(evaluate_command("git status > out.txt", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("git status $(whoami)", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("FOO=bar git status", &settings), PolicyDecision::AskUser);
// Chaining an unapproved command must still ask the user.
assert_eq!(evaluate_command("git status && curl https://example.com", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo publish", &settings), PolicyDecision::AskUser);
// Blacklist still takes precedence over whitelist
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn extracts_command_from_v1_v2_and_codex_payloads() {
let claude = json!({
"toolCall": {
"title": "git status",
"kind": "execute",
"rawInput": { "command": "git status --short" }
}
});
assert_eq!(extract_command(&claude).as_deref(), Some("git status --short"));
let codex = json!({
"toolCall": {
"title": "Run cargo test",
"kind": "execute",
"rawInput": { "command": ["/bin/zsh", "-lc", "cargo test --workspace"] }
}
});
assert_eq!(extract_command(&codex).as_deref(), Some("cargo test --workspace"));
let write = json!({
"toolCall": {
"title": "Write foo.txt",
"kind": "edit",
"rawInput": { "file_path": "foo.txt" }
}
});
assert_eq!(extract_command(&write), None);
}
}
+4 -421
Переглянути файл
@@ -43,11 +43,6 @@ struct FakeAgent {
/// Hold the late mode reply until the prompt's permission request is already waiting.
hold_late_mode: bool,
empty_config: AtomicBool,
/// Blocks `session/new` until discovery times out (see `acp::discover` test timeout).
hold_session_new: AtomicBool,
/// Sleeps this long at the start of `session/set_config_option`, after noting the time.
delay_config: Duration,
config_entered: Mutex<Option<std::time::Instant>>,
/// Config options `session/new` reports after the model one (per-model efforts, say).
extra_options: Mutex<Vec<Value>>,
model_options: Mutex<HashMap<String, Vec<Value>>>,
@@ -162,9 +157,6 @@ impl Handler for FakeAgent {
)
}
"session/new" => {
if self.hold_session_new.load(Ordering::Acquire) {
std::thread::sleep(Duration::from_secs(5));
}
if self.empty_config.load(Ordering::Acquire) {
return Ok(json!({"sessionId": "s1"}));
}
@@ -198,10 +190,6 @@ impl Handler for FakeAgent {
}
}
"session/set_config_option" | "session/set_mode" => {
if method == "session/set_config_option" && !self.delay_config.is_zero() {
*self.config_entered.lock() = Some(std::time::Instant::now());
std::thread::sleep(self.delay_config);
}
self.calls.lock().push(format!("{method} {}", params.get("value").or(params.get("modeId")).unwrap()));
if params["configId"] == "model" && !self.model_options.lock().is_empty() {
return Ok(json!({"configOptions": self.options_for(params["value"].as_str().unwrap())}));
@@ -492,7 +480,7 @@ for line in sys.stdin:
}
let concurrent = async {
if outcome == "session_report" {
tokio::time::timeout(Duration::from_secs(10), async {
tokio::time::timeout(Duration::from_secs(2), async {
while !root.join("initialized").exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
@@ -564,7 +552,7 @@ for line in sys.stdin:
};
assert_eq!(subtypes, expected, "{outcome}");
let pid = std::fs::read_to_string(root.join("pid")).unwrap();
tokio::time::timeout(Duration::from_secs(10), async {
tokio::time::timeout(Duration::from_secs(2), async {
while std::process::Command::new("kill").args(["-0", pid.trim()]).output().unwrap().status.success() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
@@ -2643,92 +2631,6 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn provider_reload_does_not_start_acp_discovery() {
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.reload_providers();
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(agent.calls.lock().is_empty(), "reload must not spawn cursor-agent");
}
#[tokio::test(flavor = "multi_thread")]
async fn composer_commands_discover_cursor_effort_and_fast() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![
json!({"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]}),
json!({"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}),
]),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert!(calls.iter().any(|m| m.as_str() == "session/new"));
assert_eq!(calls.iter().filter(|m| m.as_str() == "initialize").count(), 1);
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert_eq!(cursor.config.iter().find(|o| o.id == "fast").unwrap().current.as_deref(), Some("false"));
assert!(
cursor
.thought_levels
.values()
.next()
.is_some_and(|levels| { levels.iter().any(|o| o.category.as_deref() == Some("thought_level")) })
);
}
#[tokio::test(flavor = "multi_thread")]
async fn config_choice_shows_before_the_agent_answers() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![json!({
"id": "fast", "category": "model_config", "type": "select", "currentValue": "true",
"options": [{"value": "true", "name": "On"}, {"value": "false", "name": "Off"}]
})]),
delay_config: Duration::from_millis(300),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let shown: Arc<Mutex<Option<std::time::Instant>>> = Arc::default();
let mark = shown.clone();
mgr.subscribe(Arc::new(move |e| {
if let AgentEvent::Task { task } = e
&& task.config.iter().any(|o| o.id == "fast" && o.current.as_deref() == Some("false"))
&& mark.lock().is_none()
{
*mark.lock() = Some(std::time::Instant::now());
}
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
mgr.set_config(&task.id, "fast", "false".into()).await.unwrap();
let shown = (*shown.lock()).expect("the switch moved");
let entered = (*agent.config_entered.lock()).expect("the agent received the change");
assert!(shown < entered, "the composer updates before the agent answers");
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_times_out_when_session_new_hangs() {
let agent = Arc::new(FakeAgent { hold_session_new: AtomicBool::new(true), ..Default::default() });
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let started = std::time::Instant::now();
mgr.commands("cursor", None).await.unwrap();
assert!(started.elapsed() < Duration::from_secs(10), "discovery should time out quickly in tests");
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert!(!cursor.config.iter().any(|o| o.category.as_deref() == Some("model")));
}
#[tokio::test(flavor = "multi_thread")]
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
let tmp = tempfile::tempdir().unwrap();
@@ -2853,14 +2755,14 @@ async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edi
}
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt", "other.txt"]);
mgr.rollback_changes(&task.id, Some(vec!["other.txt".into()]), AgentChangesView::SinceStart).await.unwrap();
mgr.rollback_changes(&task.id, Some(vec!["other.txt".into()])).await.unwrap();
assert_eq!(read("other.txt"), "base\n");
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt"]);
// After a restart, from the saved copy.
mgr.shutdown();
let mgr = manager(&dir, Arc::new(FakeAgent::default()));
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(read("mine.txt"), "my edit\n", "the user's edit stays");
assert!(!dir.join("new.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
@@ -2868,322 +2770,3 @@ async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edi
mgr.delete(&task.id, false).await.unwrap();
assert!(!dir.join("store").join(format!("{}.before", task.id)).exists());
}
/// A git repository in a temporary folder, for local tasks whose checkout moves under them.
struct Checkout {
_tmp: tempfile::TempDir,
dir: PathBuf,
eol: Eol,
}
/// How the checkout keeps line endings: as committed, or CRLF on disk with `core.autocrlf`
/// (Git for Windows' default), where the files' bytes never equal the committed blobs.
#[derive(Debug, Clone, Copy)]
enum Eol {
Lf,
Crlf,
}
const EOLS: [Eol; 2] = [Eol::Lf, Eol::Crlf];
impl Checkout {
fn new(eol: Eol) -> Self {
let tmp = tempfile::tempdir().unwrap();
let dir = dunce::canonicalize(tmp.path()).unwrap();
let checkout = Self { _tmp: tmp, dir, eol };
checkout.git(&["init", "-q", "-b", "main"]);
let autocrlf = match eol {
Eol::Lf => "false",
Eol::Crlf => "true",
};
checkout.git(&["config", "core.autocrlf", autocrlf]);
checkout.write(".gitignore", "store/\nworktrees/\nagents/\nmodels.json\n");
checkout
}
fn git(&self, args: &[&str]) {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&self.dir)
.status()
.unwrap()
.success();
assert!(ok, "{args:?}");
}
fn commit(&self, message: &str) {
self.git(&["add", "."]);
self.git(&["commit", "-q", "-m", message]);
}
fn write(&self, rel: &str, text: &str) {
let text = match self.eol {
Eol::Lf => text.to_owned(),
Eol::Crlf => text.replace('\n', "\r\n"),
};
std::fs::write(self.dir.join(rel), text).unwrap();
}
/// The file's text with LF line endings (a rollback writes the committed bytes back).
fn read(&self, rel: &str) -> String {
std::fs::read_to_string(self.dir.join(rel)).unwrap().replace("\r\n", "\n")
}
/// A local task, stopped so it can be rolled back.
async fn task(&self) -> (AgentManager, AgentTask) {
let mgr = manager(&self.dir, Arc::new(FakeAgent::default()));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Hello"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.cancel(&task.id, None).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
(mgr, task)
}
}
async fn changed(mgr: &AgentManager, id: &str) -> Vec<String> {
mgr.changes(id).await.unwrap().into_iter().map(|c| c.rel_path).collect()
}
#[tokio::test(flavor = "multi_thread")]
async fn work_uncommitted_by_a_reset_during_a_local_task_is_not_rolled_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.write("mine.txt", "base\n");
repo.commit("init");
repo.write("mine.txt", "my work\n");
repo.commit("my work");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The user takes the last commit back into the working tree (`git reset --soft`, an
// undone commit): its changes are the user's, not the agent's.
repo.git(&["reset", "-q", "--soft", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
let base = mgr.base_text(&task.id, repo.dir.join("a.txt")).await.unwrap();
assert_eq!(base.map(|b| b.replace("\r\n", "\n")).as_deref(), Some("base\n"), "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("mine.txt"), "my work\n", "{eol:?}: the user's work survives the rollback");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn files_a_mixed_reset_during_a_local_task_leaves_behind_are_not_rolled_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.write("mine.txt", "base\n");
repo.commit("init");
repo.write("mine.txt", "my work\n");
repo.write("mine-new.txt", "my new file\n");
repo.commit("my work");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// `git reset HEAD~1`: the commit's new file is untracked now, its edit unstaged.
repo.git(&["reset", "-q", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("mine.txt"), "my work\n", "{eol:?}");
assert_eq!(repo.read("mine-new.txt"), "my new file\n", "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn a_branch_switched_to_during_a_local_task_is_left_alone() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.write("other.txt", "base\n");
repo.commit("init");
repo.git(&["checkout", "-q", "-b", "feature"]);
repo.write("other.txt", "feature\n");
repo.write("only-feature.txt", "feature\n");
repo.commit("feature");
repo.git(&["checkout", "-q", "main"]);
repo.write("only-main.txt", "main\n");
repo.commit("main");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The agent's uncommitted edit comes along to a branch that doesn't contain the task's start.
repo.git(&["checkout", "-q", "feature"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("other.txt"), "feature\n", "{eol:?}");
assert_eq!(repo.read("only-feature.txt"), "feature\n", "{eol:?}");
assert!(!repo.dir.join("only-main.txt").exists(), "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn commits_made_during_a_local_task_are_kept_by_its_rollback() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.write("b.txt", "agent\n");
// The user commits what the agent did to `a.txt` so far: it drops out of the task's changes.
repo.git(&["add", "a.txt"]);
repo.git(&["commit", "-q", "-m", "keep"]);
assert_eq!(changed(&mgr, &task.id).await, ["b.txt"], "{eol:?}");
repo.write("a.txt", "agent\nagain\n");
assert_eq!(changed(&mgr, &task.id).await, ["a.txt", "b.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "{eol:?}: back to the commit, not to before it");
assert!(!repo.dir.join("b.txt").exists(), "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn rolling_back_a_rename_brings_the_old_file_back() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("Old.java", "class Old {}\n");
repo.write("Case.java", "class Case {}\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.git(&["mv", "Old.java", "New.java"]);
repo.git(&["mv", "Case.java", "CASE.java"]);
// Renames list their new path only.
let listed = changed(&mgr, &task.id).await;
assert_eq!(listed, ["CASE.java", "New.java"], "{eol:?}");
mgr.rollback_changes(&task.id, Some(listed), AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("Old.java"), "class Old {}\n", "{eol:?}");
assert!(!repo.dir.join("New.java").exists(), "{eol:?}");
assert_eq!(repo.read("Case.java"), "class Case {}\n", "{eol:?}");
let names: Vec<String> = std::fs::read_dir(&repo.dir)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.filter(|n| n.ends_with(".java"))
.collect();
assert!(!names.contains(&"CASE.java".to_owned()), "{eol:?}: {names:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn the_group_view_rolls_back_only_uncommitted_files() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.commit("agent work");
repo.write("a.txt", "agent\nagain\n");
let grouped = mgr.grouped_changes(&task.id).await.unwrap();
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"], "{eol:?}");
assert!(grouped.commits.is_empty(), "{eol:?}: local tasks have no commits to group");
mgr.rollback_changes(&task.id, None, AgentChangesView::Group).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "{eol:?}: back to the commit, not to the start");
assert!(mgr.grouped_changes(&task.id).await.unwrap().uncommitted.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_changes() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("mine.txt", "mine\n");
let (mgr, task) = repo.task().await;
repo.write("new.txt", "agent\n");
assert_eq!(changed(&mgr, &task.id).await, ["new.txt"], "{eol:?}");
assert_eq!(mgr.base_text(&task.id, repo.dir.join("new.txt")).await.unwrap(), None, "{eol:?}");
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert!(!repo.dir.join("new.txt").exists(), "{eol:?}");
assert_eq!(repo.read("mine.txt"), "mine\n", "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn security_policy_auto_approves_whitelist_and_rejects_blacklist() {
use ide_core::settings::AgentSecuritySettings;
// 1. Whitelist auto-approves "cargo test" without waiting for manual respond().
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent);
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run tests"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
let entries = mgr.entries(&task.id, None).unwrap();
assert!(
entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose yes"))),
"{entries:?}"
);
assert!(
entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("whitelisted")
)),
"{entries:?}"
);
// 2. Blacklist auto-rejects "cargo test" when matched by blocked_commands.
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["cargo test*".into()],
}));
let blocked_task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run blocked"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &blocked_task.id, AgentStatus::Ready).await;
let blocked_entries = mgr.entries(&blocked_task.id, None).unwrap();
assert!(
blocked_entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose no"))),
"{blocked_entries:?}"
);
assert!(
blocked_entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("Blocked by policy")
)),
"{blocked_entries:?}"
);
}
+20 -611
Переглянути файл
@@ -1,8 +1,8 @@
//! Task worktrees and the changes made in them.
use crate::model::{AgentChange, AgentChanges, AgentCommit};
use crate::model::AgentChange;
use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, LogQuery, Repo};
use ide_vcs::{ChangeKind, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::{Duration, UNIX_EPOCH};
@@ -14,18 +14,8 @@ pub(crate) struct Created {
pub base_commit: String,
}
/// The repository of `path`, free to execute: the host runs every agent command that
/// reaches a repository only in a trusted project.
fn discover(path: &Path) -> IdeResult<Option<Repo>> {
let repo = Repo::discover(path)?;
if let Some(repo) = &repo {
repo.set_execution_allowed(true);
}
Ok(repo)
}
fn repo_at(path: &Path) -> IdeResult<Repo> {
discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
Repo::discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
}
/// The current branch (or commit when detached) and its commit.
@@ -35,14 +25,6 @@ pub(crate) fn head_of(path: &Path) -> IdeResult<(String, String)> {
Ok((info.head.unwrap_or_else(|| commit.clone()), commit))
}
/// `HEAD`'s commit in `cwd`, `None` outside a repository or before its first commit.
pub(crate) fn head_commit(cwd: &Path) -> IdeResult<Option<String>> {
match discover(cwd)? {
Some(repo) => Ok(repo.head()?.1),
None => Ok(None),
}
}
/// `git worktree add -b agent/<slug> <dir>/<slug> <base>`.
pub(crate) fn create(project_root: &Path, worktrees_dir: &Path, title: &str, base: Option<&str>) -> IdeResult<Created> {
let repo = repo_at(project_root)?;
@@ -109,358 +91,28 @@ fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Res
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
/// Outside a repository there is nothing to compare against: no changes.
pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let Some(repo) = Repo::discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let stats: std::collections::HashMap<String, (u32, u32)> =
repo.numstat_with_worktree(base)?.into_iter().map(|(p, a, r)| (p, (a, r))).collect();
let stat = |rel: &str| stats.get(rel).copied().unwrap_or((0, 0));
let stats = repo.numstat_with_worktree(base)?;
let stat = |rel: &str| stats.iter().find(|(p, ..)| p == rel).map_or((0, 0), |&(_, a, r)| (a, r));
let mut out: Vec<AgentChange> = repo
.diff_with_worktree(base)?
.into_iter()
.map(|f| {
let (added, removed) = stat(&f.path);
AgentChange {
path: IpcPath(root.join(&f.path)),
rel_path: f.path,
kind: f.kind,
added,
removed,
old_path: f.old_path,
}
AgentChange { path: IpcPath(root.join(&f.path)), rel_path: f.path, kind: f.kind, added, removed }
})
.collect();
let tracked: std::collections::HashSet<String> = out.iter().map(|c| c.rel_path.clone()).collect();
for s in repo.status()? {
if s.unstaged == Some(ChangeKind::Untracked) && !tracked.contains(&s.rel_path) {
out.push(added(s.path, s.rel_path));
if s.unstaged == Some(ChangeKind::Untracked) && !out.iter().any(|c| c.rel_path == s.rel_path) {
let added = std::fs::read_to_string(&*s.path).map_or(0, |t| t.lines().count() as u32);
out.push(AgentChange { path: s.path, rel_path: s.rel_path, kind: ChangeKind::Added, added, removed: 0 });
}
}
out.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
Ok(out)
}
/// Files renamed since `rev`, by their new repository-relative path: the old path, as a deletion
/// to roll back along with the rename.
pub(crate) fn rename_sources(cwd: &Path, rev: &str) -> IdeResult<BTreeMap<String, AgentChange>> {
let Some(repo) = discover(cwd)? else { return Ok(BTreeMap::new()) };
let root = repo.root().to_path_buf();
Ok(repo
.diff_with_worktree(rev)?
.into_iter()
.filter_map(|f| {
let old = f.old_path.filter(|old| *old != f.path)?;
let source = AgentChange {
path: IpcPath(root.join(&old)),
rel_path: old,
kind: ChangeKind::Deleted,
added: 0,
removed: 0,
old_path: None,
};
Some((f.path, source))
})
.collect())
}
fn added(path: IpcPath, rel_path: String) -> AgentChange {
let added = std::fs::read_to_string(&*path).map_or(0, |t| t.lines().count() as u32);
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0, old_path: None }
}
/// The commits from `base` (exclusive) to `head` (inclusive) in `cwd`, newest first, each
/// with the files it changed against its first parent and their line counts.
pub(crate) fn commits(cwd: &Path, base: &str, head: &str) -> IdeResult<Vec<AgentCommit>> {
if base == head {
return Ok(Vec::new());
}
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let query = LogQuery { range: Some(format!("{base}..{head}")), ..LogQuery::default() };
let mut out = Vec::new();
for commit in repo.log(&query)? {
let parent = commit.parents.first().cloned();
// A root commit has no parent to count lines against; its files still show.
let stats: std::collections::HashMap<String, (u32, u32)> = match &parent {
Some(parent) => repo
.diff_numstat(parent, &commit.id, &[])?
.into_iter()
.map(|s| (s.path, (s.added, s.removed)))
.collect(),
None => std::collections::HashMap::new(),
};
let mut files: Vec<AgentChange> = repo
.commit_files(&commit.id)?
.into_iter()
.map(|f| {
let (added, removed) = stats.get(&f.path).copied().unwrap_or((0, 0));
AgentChange {
path: IpcPath(root.join(&f.path)),
rel_path: f.path,
kind: f.kind,
added,
removed,
old_path: f.old_path,
}
})
.collect();
files.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
out.push(AgentCommit {
id: commit.id,
short_id: commit.short_id,
summary: commit.summary,
message: commit.message,
time_ms: commit.time_ms,
parent,
files,
});
}
Ok(out)
}
/// Whether two paths name one file (a case-insensitive disk resolving both spellings).
fn same_file(a: &Path, b: &Path) -> bool {
match (std::fs::canonicalize(a), std::fs::canonicalize(b)) {
(Ok(a), Ok(b)) => a == b,
_ => false,
}
}
fn crlf_to_lf(bytes: &[u8]) -> Vec<u8> {
let mut out = Vec::with_capacity(bytes.len());
for (i, &b) in bytes.iter().enumerate() {
if !(b == b'\r' && bytes.get(i + 1) == Some(&b'\n')) {
out.push(b);
}
}
out
}
/// Where a task works and what its changes are measured against.
pub(crate) enum TaskBase {
/// A worktree of its own: everything since the commit it was created from, the agent's
/// own commits included.
Worktree { cwd: PathBuf, base: String },
/// The project's checkout, where the user works and commits too. `start` is the commit it
/// started from (`None` in a repository without commits, or an imported session).
Local { cwd: PathBuf, start: Option<String> },
}
impl TaskBase {
pub(crate) fn cwd(&self) -> &Path {
match self {
Self::Worktree { cwd, .. } | Self::Local { cwd, .. } => cwd,
}
}
/// The files the task changed, less those changed before it started and untouched since.
pub(crate) fn changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<Vec<AgentChange>> {
let (cwd, start) = match self {
Self::Worktree { cwd, base } => return Ok(without_untouched(changes(cwd, base)?, before)),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
// What is uncommitted, as the Commit tool window lists it: commits made since the task
// started drop out.
let head = repo.branch_info()?.head_commit;
let listed = match &head {
Some(head) => changes(cwd, head)?,
None => {
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
all
}
};
let listed = without_untouched(listed, before);
let Some(start) = start.filter(|s| head.as_deref() != Some(*s)) else { return Ok(listed) };
// HEAD moved (a reset, a checkout, a pull): a file that is as it was when the task
// started is the user's doing, not the task's. Git compares tracked files, with its
// line-ending and other filters: the bytes on disk and in the commit differ under
// `core.autocrlf`.
let since_start: std::collections::HashSet<String> = repo
.diff_with_worktree(start)?
.into_iter()
.flat_map(|f| [Some(f.path), f.old_path].into_iter().flatten())
.collect();
let untracked: std::collections::HashSet<String> = repo
.status()?
.into_iter()
.filter(|s| s.unstaged == Some(ChangeKind::Untracked))
.map(|s| s.rel_path)
.collect();
Ok(listed
.into_iter()
.filter(|c| {
if before.contains_key(&c.rel_path) {
return true;
}
let Ok(Some(at_start)) = repo.file_bytes_at(&c.path.0, Some(start)) else { return true };
if !untracked.contains(&c.rel_path) {
return since_start.contains(&c.rel_path);
}
// Untracked since (a mixed reset): git doesn't diff it, so compare the bytes,
// CRLF on disk matching LF committed.
!std::fs::read(&c.path.0).is_ok_and(|now| now == at_start || crlf_to_lf(&now) == at_start)
})
.collect())
}
/// The task's uncommitted changes and, for a worktree task, the commits made since it
/// started (newest first). Both read the same `HEAD`, so a commit landing between the two
/// reads cannot put a file in both lists.
pub(crate) fn grouped_changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<AgentChanges> {
let cwd = self.cwd();
let Some(repo) = discover(cwd)? else { return Ok(AgentChanges::default()) };
let head = repo.head()?.1;
let uncommitted = match &head {
Some(head) => without_untouched(changes(cwd, head)?, before),
None => {
// No commits yet: every file is new, as in `changes` for a local task.
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
without_untouched(all, before)
}
};
let commits = match (self, &head) {
(Self::Worktree { cwd, base }, Some(head)) => commits(cwd, base, head)?,
_ => Vec::new(),
};
Ok(AgentChanges { uncommitted, commits })
}
/// The revision the task's changed files go back to on rollback, and are compared with:
/// the commit the task started from, or the current `HEAD` once the user committed on top
/// of it, so those commits stay. `None` while the repository has no commits.
pub(crate) fn rev(&self) -> IdeResult<Option<String>> {
let (cwd, start) = match self {
Self::Worktree { base, .. } => return Ok(Some(base.clone())),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(None) };
let head = repo.branch_info()?.head_commit;
// A start commit git no longer knows (garbage-collected) can't be gone back to.
let start = start.filter(|s| repo.merge_base(s, s).ok().flatten().is_some());
Ok(match (start, head) {
(Some(start), Some(head)) if start != head && repo.merge_base(start, &head)?.as_deref() != Some(start) => {
Some(start.to_owned())
}
(start, head) => head.or_else(|| start.map(str::to_owned)),
})
}
}
/// Copies the task's `changes` (against `base`) from the worktree `cwd` into the
/// local checkout `root`, uncommitted: the user reviews and commits them with the
/// VCS tools. Files the checkout changed itself are left alone and reported, so
/// accepting never destroys the user's own edits. Returns accepted and skipped paths.
pub(crate) fn accept(
root: &Path,
cwd: &Path,
base: &str,
changes: &[AgentChange],
) -> IdeResult<crate::AgentAcceptOutcome> {
let repo = repo_at(cwd)?;
// Change paths are repository-relative, and the task's project may be a folder inside it.
let local_root = discover(root)?.map_or_else(|| root.to_path_buf(), |r| r.root().to_path_buf());
let renamed_from: BTreeMap<String, String> =
repo.diff_with_worktree(base)?.into_iter().filter_map(|f| Some((f.path, f.old_path?))).collect();
// Brings one path of the checkout to the task's version, unless the checkout changed it itself.
let apply = |rel: &str| -> IdeResult<bool> {
let local = local_root.join(rel);
let in_task = repo.root().join(rel);
let base_bytes = repo.file_bytes_at(&in_task, Some(base))?;
let current = std::fs::read(&in_task).ok();
let local_bytes = std::fs::read(&local).ok();
if local_bytes == current {
return Ok(true);
}
// The checkout may hold the committed LF text with CRLF line ends (`core.autocrlf`).
let unchanged = match (&local_bytes, &base_bytes) {
(Some(now), Some(at_base)) => now == at_base || crlf_to_lf(now) == *at_base,
(now, at_base) => now == at_base,
};
if !unchanged {
return Ok(false);
}
match current {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&local, bytes)?;
}
None => match std::fs::remove_file(&local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
Ok(true)
};
let mut accepted = Vec::with_capacity(changes.len());
let mut skipped = Vec::new();
let skip = |path: &str| crate::AgentAcceptSkipped {
path: path.to_owned(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
};
for change in changes {
// A case-only rename (`a.txt` → `A.txt`). Where the disk resolves both
// spellings to one file it is renamed in place and `apply` then refreshes
// the content. On a case-sensitive disk the two names are different files:
// the old one's unchanged-ness (it is the file the checkout had) decides —
// then the task's version lands at the new name and the old one goes.
let old = renamed_from.get(&change.rel_path);
let case_only =
old.is_some_and(|o| o != &change.rel_path && o.to_lowercase() == change.rel_path.to_lowercase());
if case_only {
let old = old.expect("case_only implies a rename source");
let (old_path, new_path) = (local_root.join(old), local_root.join(&change.rel_path));
if same_file(&old_path, &new_path) {
std::fs::rename(&old_path, &new_path)?;
} else {
if !apply(old)? {
skipped.push(skip(old));
continue;
}
// `apply(old)` removed the old file (the task's tree has it only at
// the new name); now write that version under the new name. `apply`
// can't do it — it reads the new name's own base/absence — so the
// write is open-coded from the same task bytes `apply` would use.
if let Ok(bytes) = std::fs::read(repo.root().join(&change.rel_path)) {
if let Some(parent) = new_path.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&new_path, bytes)?;
}
accepted.push(change.rel_path.clone());
continue;
}
}
if !apply(&change.rel_path)? {
skipped.push(skip(&change.rel_path));
continue;
}
accepted.push(change.rel_path.clone());
// A rename also takes the file away from its old path (a case-only rename
// is handled above).
if !case_only
&& let Some(old) = old
&& !apply(old)?
{
skipped.push(skip(old));
}
}
Ok(crate::AgentAcceptOutcome { accepted, skipped })
}
/// Size and modification time of a file, `None` once it is gone.
pub(crate) type Stamp = Option<(u64, u128)>;
@@ -472,10 +124,7 @@ pub(crate) fn stamp(path: &Path) -> Stamp {
/// The files already changed in `cwd` against `HEAD`, stamped so later edits to them show.
pub(crate) fn snapshot(cwd: &Path) -> IdeResult<BTreeMap<String, Stamp>> {
// Paths and stamps only: no diff and no line counts, which read every untracked file
// and held up starting a task in a checkout with many of them.
let Some(repo) = Repo::discover(cwd)? else { return Ok(BTreeMap::new()) };
Ok(repo.status()?.into_iter().map(|s| (s.rel_path, stamp(&s.path))).collect())
Ok(changes(cwd, "HEAD")?.into_iter().map(|c| (c.rel_path, stamp(&c.path))).collect())
}
/// Drops the changes that were there before the task and haven't been touched since.
@@ -512,11 +161,11 @@ pub(crate) fn save_before(cwd: &Path, before: &BTreeMap<String, Stamp>, dir: &Pa
}
/// Rolls `change` back: to the copy saved when the task started when the file had changes
/// then (`before`, `saved` holding the copies), otherwise to `rev` (removed when `rev` lacks
/// it, or when there is no `rev`: the repository had no commits).
/// then (`before`, `saved` holding the copies), otherwise to `base` (removed when `base`
/// lacks it).
pub(crate) fn restore(
cwd: &Path,
rev: Option<&str>,
base: &str,
change: &AgentChange,
before: &BTreeMap<String, Stamp>,
saved: Option<&Path>,
@@ -534,13 +183,10 @@ pub(crate) fn restore(
})?;
Some(std::fs::read(copy)?)
}
None => match rev {
Some(rev) => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(rev))?
}
None => None,
},
None => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(base))?
}
};
match content {
Some(bytes) => {
@@ -558,7 +204,7 @@ pub(crate) fn restore(
}
pub(crate) fn file_at(cwd: &Path, path: &Path, rev: &str) -> IdeResult<Option<String>> {
let Some(repo) = discover(cwd)? else { return Ok(None) };
let Some(repo) = Repo::discover(cwd)? else { return Ok(None) };
let encoding = match ide_vfs::load_text(path) {
Ok(loaded) => encoding_rs::Encoding::for_label(loaded.encoding.as_bytes()).unwrap_or(encoding_rs::UTF_8),
Err(_) => ide_vfs::detect_encoding(&repo.file_bytes_at(path, Some(rev))?.unwrap_or_default(), None),
@@ -667,18 +313,7 @@ mod tests {
fn git(dir: &Path, args: &[&str]) {
let ok = std::process::Command::new("git")
.args([
"-c",
"user.name=t",
"-c",
"user.email=t@t",
"-c",
"commit.gpgsign=false",
// Hermetic line endings: a developer or CI runner with
// `core.autocrlf=true` would otherwise check the fixtures out as CRLF.
"-c",
"core.autocrlf=false",
])
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(dir)
.status()
@@ -726,88 +361,6 @@ mod tests {
assert!(!created.path.exists());
}
#[test]
fn commits_since_the_base_are_listed_with_their_files() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
assert!(commits(&created.path, &created.base_commit, &created.base_commit).unwrap().is_empty());
std::fs::write(created.path.join("a.txt"), "one\ntwo\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["commit", "-q", "-m", "change a"]);
std::fs::write(created.path.join("b.txt"), "b\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["commit", "-q", "-m", "add b"]);
let head = head_of(&created.path).unwrap().1;
let log = commits(&created.path, &created.base_commit, &head).unwrap();
assert_eq!(log.iter().map(|c| c.summary.as_str()).collect::<Vec<_>>(), ["add b", "change a"]);
assert_eq!(log[0].files.iter().map(|f| f.rel_path.as_str()).collect::<Vec<_>>(), ["b.txt"]);
assert_eq!((log[0].files[0].added, log[0].files[0].removed), (1, 0));
assert_eq!(log[1].files.iter().map(|f| f.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
assert_eq!((log[1].files[0].added, log[1].files[0].removed), (1, 0));
assert!(log.iter().all(|c| c.parent.is_some() && !c.message.is_empty()));
// Nothing is uncommitted after the commits; editing a committed file brings it back.
let at = TaskBase::Worktree { cwd: created.path.clone(), base: created.base_commit.clone() };
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert!(grouped.uncommitted.is_empty());
assert_eq!(grouped.commits.len(), 2);
std::fs::write(created.path.join("a.txt"), "one\ntwo\nthree\n").unwrap();
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
assert_eq!(grouped.commits.len(), 2);
}
#[test]
fn a_rename_in_a_commit_keeps_its_old_path() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\nthree\nfour\nfive\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let base = head_of(&repo).unwrap().1;
std::fs::rename(repo.join("a.txt"), repo.join("b.txt")).unwrap();
std::fs::write(repo.join("b.txt"), "one\ntwo\nthree\nfour\nFIVE\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "rename a"]);
let head = head_of(&repo).unwrap().1;
let log = commits(&repo, &base, &head).unwrap();
assert_eq!(log.len(), 1);
assert_eq!(log[0].files.len(), 1);
assert_eq!(log[0].files[0].rel_path, "b.txt");
assert_eq!(log[0].files[0].old_path.as_deref(), Some("a.txt"));
assert_eq!(log[0].files[0].kind, ChangeKind::Renamed);
}
#[test]
fn a_local_task_has_no_commits() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
let at = TaskBase::Local { cwd: repo.clone(), start: None };
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert!(grouped.commits.is_empty());
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
}
#[test]
fn a_broken_worktree_registration_still_leaves_a_deletable_folder() {
let tmp = tempfile::tempdir().unwrap();
@@ -838,148 +391,4 @@ mod tests {
assert!(remove(&repo, &tmp.path().join("wt"), &elsewhere).is_err());
assert!(elsewhere.join("keep.txt").exists());
}
fn accept_repo() -> (tempfile::TempDir, PathBuf, Created) {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
// Hermetic line endings: a machine with `core.autocrlf=true` would check
// the fixtures and the worktree out as CRLF.
git(&repo, &["config", "core.autocrlf", "false"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
std::fs::write(repo.join("gone.txt"), "bye\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
(tmp, repo, created)
}
fn listed(cwd: &Path, base: &str) -> Vec<AgentChange> {
without_untouched(changes(cwd, base).unwrap(), &BTreeMap::new())
}
#[test]
fn accept_moves_a_renamed_file_and_maps_paths_from_a_project_folder() {
let (_tmp, repo, created) = accept_repo();
std::fs::create_dir(created.path.join("app")).unwrap();
git(&created.path, &["mv", "a.txt", "app/moved.txt"]);
std::fs::write(created.path.join("app/new.txt"), "n\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "-m", "move"]);
// The task's project is the repository's `app` folder.
let project = repo.join("app");
std::fs::create_dir(&project).unwrap();
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&project, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(std::fs::read_to_string(repo.join("app/moved.txt")).unwrap(), "one\ntwo\n");
assert_eq!(std::fs::read_to_string(repo.join("app/new.txt")).unwrap(), "n\n");
assert!(!repo.join("a.txt").exists(), "the rename's old path is gone");
assert!(!repo.join("app/app").exists());
}
#[test]
fn accept_treats_a_crlf_checkout_of_the_base_as_unchanged() {
let (_tmp, repo, created) = accept_repo();
// The checkout holds the committed text with CRLF line ends, as core.autocrlf leaves it.
std::fs::write(repo.join("a.txt"), "one\r\ntwo\r\n").unwrap();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
git(&created.path, &["mv", "gone.txt", "kept.txt"]);
std::fs::write(repo.join("gone.txt"), "bye\r\n").unwrap();
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert!(repo.join("kept.txt").exists() && !repo.join("gone.txt").exists());
}
#[test]
fn accept_applies_a_case_only_rename() {
let (_tmp, repo, created) = accept_repo();
git(&created.path, &["mv", "a.txt", "A.txt"]);
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
let names: Vec<_> = std::fs::read_dir(&repo)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.filter(|n| n.eq_ignore_ascii_case("a.txt"))
.collect();
assert_eq!(names, ["A.txt"]);
}
#[test]
fn accept_copies_new_modified_and_deleted_files_into_the_checkout() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("new.txt"), "x\n").unwrap();
std::fs::remove_file(created.path.join("gone.txt")).unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(outcome.skipped.is_empty());
assert_eq!(outcome.accepted, ["a.txt", "gone.txt", "new.txt"]);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert_eq!(std::fs::read_to_string(repo.join("new.txt")).unwrap(), "x\n");
assert!(!repo.join("gone.txt").exists());
// Accepting again changes nothing: the checkout already matches.
let again =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(again.skipped.is_empty());
assert_eq!(again.accepted, ["a.txt", "gone.txt", "new.txt"]);
}
#[test]
fn a_local_task_goes_back_to_its_start_unless_the_user_committed_on_top_of_it() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
let local = |start: Option<&str>| TaskBase::Local { cwd: repo.clone(), start: start.map(str::to_owned) };
let head = || head_of(&repo).unwrap().1;
let commit = |name: &str| {
std::fs::write(repo.join(name), "x\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", name]);
head_of(&repo).unwrap().1
};
assert_eq!(local(None).rev().unwrap(), None, "no commits yet");
let first = commit("a.txt");
assert_eq!(local(None).rev().unwrap(), Some(first.clone()), "an imported session: HEAD");
let start = commit("b.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "HEAD where it started");
let ahead = commit("c.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(ahead), "committed on top: HEAD");
git(&repo, &["reset", "-q", "--soft", &first]);
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "moved back: the start");
git(&repo, &["checkout", "-q", "-b", "side"]);
commit("d.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "another branch: the start");
let gone = "0123456789abcdef0123456789abcdef01234567";
assert_eq!(local(Some(gone)).rev().unwrap(), Some(head()), "an unknown start: HEAD");
let worktree = TaskBase::Worktree { cwd: repo.clone(), base: start.clone() };
assert_eq!(worktree.rev().unwrap(), Some(start));
}
#[test]
fn accept_leaves_files_the_checkout_changed_itself() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("gone.txt"), "bye\nbye\n").unwrap();
std::fs::write(repo.join("a.txt"), "one\nMINE\n").unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert_eq!(outcome.accepted, ["gone.txt"]);
assert_eq!(outcome.skipped.len(), 1);
assert_eq!(outcome.skipped[0].path, "a.txt");
assert!(!outcome.skipped[0].reason.is_empty());
// The user's own edit survives; the other file comes over.
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nMINE\n");
assert_eq!(std::fs::read_to_string(repo.join("gone.txt")).unwrap(), "bye\nbye\n");
}
}
+22 -110
Переглянути файл
@@ -79,26 +79,23 @@ fn scan(
Ok(())
}
/// One planned step as fingerprinted: resolved program, args, cwd and env.
type StepFingerprint = (PathBuf, Vec<String>, PathBuf, BTreeMap<String, String>);
/// Environment, planned steps and input paths shared by every Maven build
/// fingerprint: the filtered process environment plus the wrapper, settings
/// and JDK files the planned invocation reads.
struct Collected {
inputs: Vec<PathBuf>,
environment: BTreeMap<String, String>,
steps: Vec<StepFingerprint>,
inherited_hash: u64,
}
fn collect(
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
root: &Path,
projects: &[&maven::MavenProject],
plan: &BuildPlan,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
) -> IdeResult<Option<Collected>> {
let Some(mut inputs) = maven::launch_inputs(root, projects, local_repo) else { return Ok(None) };
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let Some(mut inputs) = maven::launch_inputs(root, &projects, local_repo) else { return Ok(None) };
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
let mut environment: BTreeMap<String, String> = std::env::vars()
.map(|(key, value)| (key.to_ascii_uppercase(), value))
.filter(|(key, _)| {
@@ -168,111 +165,26 @@ fn collect(
Path::new(home).join(if cfg!(windows) { "bin/java.exe" } else { "bin/java" }),
]);
}
let steps = plan
.steps
.iter()
.map(|s| (process::program_path(&s.program, &s.env), s.args.clone(), s.cwd.clone(), s.env.clone()))
.collect();
let steps: Vec<_> =
plan.steps.iter().map(|s| (process::program_path(&s.program, &s.env), &s.args, &s.cwd, &s.env)).collect();
// Profiles and plugins can read arbitrary environment variables. Keep a signature
// of the inherited environment without persisting unrelated credentials.
let inherited: BTreeMap<_, _> = std::env::vars_os().collect();
let mut inherited_hash = std::hash::DefaultHasher::new();
inherited.hash(&mut inherited_hash);
Ok(Some(Collected { inputs, environment, steps, inherited_hash: inherited_hash.finish() }))
}
/// Scans every input, storing project inputs relative to the project.
fn scan_all(
root: &Path,
inputs: Vec<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<BTreeMap<PathBuf, Option<Stamp>>> {
let parameters =
serde_json::to_string(&(root, module, test, steps, environment, inherited_hash.finish(), projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
let mut files = BTreeMap::new();
for path in inputs {
scan(root, &path, &mut files, &mut HashSet::new(), cancel)?;
}
// Store project inputs relative to the project, including each file's relative path.
Ok(files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect())
}
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
let Some(collected) = collect(root, &projects, &plan, local_repo)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let parameters = serde_json::to_string(&(root, module, test, steps, environment, inherited_hash, projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
/// Inputs fingerprint of a menu build (`Build Project` / `Build Module`): the
/// built modules' sources, resources and POM chain plus the exact planned
/// steps, so an unchanged project skips Maven entirely (#2476). Only `Build`
/// callers reuse it, never `Rebuild` or `Artifacts`; `None` disables reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn menu_fingerprint(
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let plan = plan(root, model, BuildKind::Build, file, None)?;
if plan.system != BuildSystem::Maven {
return Ok(None);
}
let Some(m) = &model.maven else { return Ok(None) };
let projects: Vec<&maven::MavenProject> = match file {
Some(file) => {
// The innermost linked project owning the file, as in single-file compilation.
let rel = rel_str(root, file);
let module = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len());
match module {
Some(project) if project.packaging != "pom" => {
maven_launch_projects(root, m, &subdir(root, &project.dir), true)
}
_ => return Ok(None),
}
}
// A project build walks every linked project.
None => m.projects.iter().collect(),
};
if projects.is_empty() {
return Ok(None);
}
let Some(collected) = collect(root, &projects, &plan, None)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let scope = file.map(|f| rel_str(root, f));
let parameters = serde_json::to_string(&("menu-build", scope, steps, environment, inherited_hash))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
let files = files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect();
Ok(Some(LaunchFingerprint { version: 2, parameters, files }))
}
impl LaunchFingerprint {
/// Folders (relative to the project) of the Maven projects whose POM is an input.
pub(super) fn project_dirs(&self) -> impl Iterator<Item = &Path> {
self.files
.keys()
.filter(|p| p.is_relative() && p.file_name().is_some_and(|n| n == "pom.xml"))
.map(|pom| pom.parent().unwrap_or(Path::new("")))
}
/// Relative paths whose content changed while `(length, mtime)` stayed the same
/// compared to `previous` (timestamp-preserving external edits, coarse-granularity
/// filesystems). Timestamp-incremental builders (Maven, the Kotlin plugin) would
-253
Переглянути файл
@@ -1,253 +0,0 @@
//! Resources a Gradle build changes while copying them (filters, `expand`, renames, extra
//! sources). The Java language server copies them as they are into Buildship's
//! `bin/<source set>` folders, which launches run from; IntelliJ IDEA runs Gradle's copy.
//! Before a launch, Gradle processes them and its output replaces those plain copies, so
//! each resource is on the classpath once, as the build produces it.
use super::{BuildPlan, BuildStep, BuildSystem, Format, run, subdir};
use crate::BuildModel;
use crate::gradle::{self, GradleModel, ReportedProject};
use ide_core::{CancellationToken, IdeError, IdeResult, Severity};
use std::fs;
use std::path::{Path, PathBuf};
/// A source set whose resources Gradle processes, with the folder the launch reads them from.
#[derive(Debug, PartialEq)]
struct Target {
/// Task path from the root build (`:app:processResources`).
task: String,
/// Where the task writes the resources.
output: PathBuf,
/// The Java language server's output folder of the source set, on the launch classpath.
jdtls: PathBuf,
}
fn targets(reported: &[ReportedProject], class_paths: &[String]) -> Vec<Target> {
// buildSrc is built before the build, never addressable as `:buildSrc:<task>`.
let addressable = |p: &&ReportedProject| !p.build.as_deref().is_some_and(|b| b.ends_with(":buildSrc"));
let sets: Vec<_> = reported
.iter()
.filter(addressable)
.flat_map(|p| p.source_sets.iter().filter_map(move |s| Some((p, s, s.processed.as_ref()?))))
.collect();
if sets.is_empty() {
return Vec::new();
}
// Canonical, as Gradle reports its folders: jdtls may spell them differently.
let entries: Vec<PathBuf> = class_paths.iter().filter_map(|p| dunce::canonicalize(p).ok()).collect();
sets.into_iter()
.filter_map(|(p, set, resources)| {
// Buildship's output folder of the source set.
let jdtls = dunce::canonicalize(Path::new(&p.dir).join("bin").join(&set.name)).ok()?;
let project = if p.path == ":" { "" } else { p.path.as_str() };
entries.contains(&jdtls).then(|| Target {
task: format!("{project}:{}", resources.task),
output: PathBuf::from(&resources.dir),
jdtls,
})
})
.collect()
}
/// Before a JVM launch from the Java language server's output folders on `class_paths`:
/// runs the Gradle tasks that process resources of those folders' source sets (see the
/// module docs) and copies their output over the plain copies. `cache_dir` holds the
/// linked builds' last syncs. Returns the tasks run, none for builds that copy their
/// resources as they are.
pub fn process_launch_resources(
root: &Path,
model: &BuildModel,
cache_dir: &Path,
class_paths: &[String],
cancel: &CancellationToken,
) -> IdeResult<Vec<String>> {
// A composite also reports the builds it includes: the deepest build owning a folder runs its task.
let mut builds: Vec<&GradleModel> = model.gradle.iter().collect();
builds.sort_by_key(|g| std::cmp::Reverse(subdir(root, &g.dir).components().count()));
let mut done: Vec<PathBuf> = Vec::new();
let mut tasks = Vec::new();
for g in builds {
let reported = crate::sync::reported_gradle_projects(&gradle::cache_dir(cache_dir, &g.dir));
let targets: Vec<Target> =
targets(&reported, class_paths).into_iter().filter(|t| !done.contains(&t.jdtls)).collect();
if targets.is_empty() {
continue;
}
let mut step = BuildStep::new(&g.command, &["--console=plain"], &subdir(root, &g.dir), Format::Gradle);
step.args.extend(targets.iter().map(|t| t.task.clone()));
step.env = g.env.clone();
let plan = BuildPlan { system: BuildSystem::Gradle, title: "Process resources".into(), steps: vec![step] };
let result = run(root, &plan, cancel, &|_| {})?;
if !result.success {
let errors: Vec<&str> = result
.messages
.iter()
.filter(|m| m.severity == Severity::Error)
.take(5)
.map(|m| m.text.as_str())
.collect();
return Err(IdeError::internal(errors.join("\n")));
}
for t in targets {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
overlay(&t.output, &t.output, &t.jdtls)?;
done.push(t.jdtls);
tasks.push(t.task);
}
}
Ok(tasks)
}
/// Copies the files under `dir` to the same place under `to` (relative to `base`) where
/// they differ, so unchanged resources are not rewritten on every launch.
fn overlay(base: &Path, dir: &Path, to: &Path) -> IdeResult<()> {
let entries = match fs::read_dir(dir) {
// No resources: the task had nothing to copy.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(()),
entries => entries?,
};
for entry in entries {
let entry = entry?;
let path = entry.path();
let kind = entry.file_type()?;
if kind.is_dir() {
overlay(base, &path, to)?;
} else if kind.is_file() {
let dest = to.join(path.strip_prefix(base).map_err(|e| IdeError::internal(e.to_string()))?);
if !same_content(&path, &dest) {
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent)?;
}
fs::copy(&path, &dest)?;
}
}
}
Ok(())
}
fn same_content(a: &Path, b: &Path) -> bool {
match (fs::metadata(a), fs::metadata(b)) {
(Ok(m), Ok(n)) if m.len() == n.len() => matches!((fs::read(a), fs::read(b)), (Ok(x), Ok(y)) if x == y),
_ => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gradle::{ProcessedResources, ReportedSourceSet};
fn write(path: &Path, text: &str) {
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, text).unwrap();
}
fn set(name: &str, processed: Option<&Path>) -> ReportedSourceSet {
ReportedSourceSet {
name: name.into(),
test: name == "test",
processed: processed.map(|dir| ProcessedResources {
task: if name == "main" { "processResources".into() } else { "processTestResources".into() },
dir: dir.to_string_lossy().into_owned(),
}),
..Default::default()
}
}
fn project(path: &str, dir: &Path, sets: Vec<ReportedSourceSet>) -> ReportedProject {
ReportedProject {
path: path.into(),
dir: dir.to_string_lossy().into_owned(),
source_sets: sets,
..Default::default()
}
}
/// Only processed source sets whose jdtls output is on the classpath: the launched
/// module, upstream modules of the build, tests only for a test classpath; never buildSrc.
#[test]
fn targets_are_processed_sets_on_the_classpath() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let (app, lib, plain, src) = (root.join("app"), root.join("lib"), root.join("plain"), root.join("buildSrc"));
for dir in [&root, &app, &lib, &plain, &src] {
for set in ["main", "test"] {
fs::create_dir_all(dir.join("bin").join(set)).unwrap();
}
}
let out = |dir: &Path, set: &str| dir.join("build/resources").join(set);
let mut build_src = project(":buildSrc", &src, vec![set("main", Some(&out(&src, "main")))]);
build_src.build = Some(":buildSrc".into());
let reported = [
project(":", &root, vec![set("main", Some(&out(&root, "main")))]),
project(":app", &app, vec![set("main", Some(&out(&app, "main"))), set("test", Some(&out(&app, "test")))]),
project(":lib", &lib, vec![set("main", Some(&out(&lib, "main")))]),
project(":plain", &plain, vec![set("main", None)]),
build_src,
];
let entry = |dir: &Path, set: &str| dir.join("bin").join(set).to_string_lossy().into_owned();
let runtime = [
entry(&app, "main"),
// Spelled differently from Gradle's canonical folders.
format!("{}/", entry(&lib, "main")),
entry(&plain, "main"),
entry(&src, "main"),
"/m2/spring-core.jar".into(),
];
let tasks = |cp: &[String]| targets(&reported, cp).into_iter().map(|t| t.task).collect::<Vec<_>>();
assert_eq!(tasks(&runtime), [":app:processResources", ":lib:processResources"]);
let mut test = runtime.to_vec();
test.insert(0, entry(&app, "test"));
assert_eq!(tasks(&test), [":app:processResources", ":app:processTestResources", ":lib:processResources"]);
assert_eq!(tasks(&[entry(&root, "main")]), [":processResources"], "the root project");
let app_targets = targets(&reported, &runtime);
let t = &app_targets[0];
assert_eq!(
(t.output.as_path(), t.jdtls.as_path()),
(out(&app, "main").as_path(), app.join("bin/main").as_path())
);
let unprocessed = [project(":app", &app, vec![set("main", None)])];
assert!(targets(&unprocessed, &runtime).is_empty());
}
/// The processed copy replaces jdtls's plain one (the reported `@version@` case), new
/// files are added, identical ones are left alone and files Gradle doesn't produce stay.
#[test]
fn overlay_replaces_plain_copies_with_gradle_output() {
let tmp = tempfile::tempdir().unwrap();
let (output, jdtls) = (tmp.path().join("build/resources/main"), tmp.path().join("bin/main"));
write(&output.join("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
write(&output.join("static/extra.txt"), "from an extra source");
write(&output.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("application.yaml"), "spring:\n application:\n version: @version@\n");
write(&jdtls.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("com/example/App.class"), "bytecode");
let unchanged = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
std::thread::sleep(std::time::Duration::from_millis(20));
overlay(&output, &output, &jdtls).unwrap();
let read = |rel: &str| fs::read_to_string(jdtls.join(rel)).unwrap();
assert_eq!(read("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
assert_eq!(read("static/extra.txt"), "from an extra source");
assert_eq!(read("com/example/App.class"), "bytecode");
let modified = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
assert_eq!(modified, unchanged, "identical files are not rewritten");
overlay(&tmp.path().join("missing"), &tmp.path().join("missing"), &jdtls).unwrap();
}
#[test]
fn builds_without_processed_resources_run_nothing() {
let tmp = tempfile::tempdir().unwrap();
// Running this command would fail the call.
let missing = tmp.path().join("missing-gradle").to_string_lossy().into_owned();
let model =
BuildModel { gradle: vec![GradleModel { command: missing, ..Default::default() }], ..Default::default() };
let class_paths = [tmp.path().join("bin/main").to_string_lossy().into_owned()];
let tasks =
process_launch_resources(tmp.path(), &model, tmp.path(), &class_paths, &CancellationToken::new()).unwrap();
assert!(tasks.is_empty());
}
}
+2 -40
Переглянути файл
@@ -4,17 +4,9 @@
//! for the Build tool window, the Problems view and the editor.
mod fingerprint;
mod launch_resources;
mod parse;
mod single;
mod up_to_date;
pub use fingerprint::{LaunchFingerprint, launch_fingerprint, menu_fingerprint};
pub use launch_resources::process_launch_resources;
pub use single::{
SingleFileTarget, compile_file_plan, find_javac, javac_step, single_file_classpath, single_file_target,
};
pub use up_to_date::{menu_up_to_date, record_menu_build};
pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
@@ -266,11 +258,6 @@ pub(super) fn maven_steps(
BuildKind::Rebuild => &["clean", "test-compile"],
BuildKind::Artifacts => &["package", "-DskipTests"],
};
// The IDE's compile builds are not coverage runs. JaCoCo's offline `instrument`
// rewrites target/classes in `process-classes`; `jacoco.skip` prevents that in this
// build. It does not restore classes instrumented by an earlier build: Rebuild's
// `clean` removes those, while an incremental Build can still reuse them.
let skip_jacoco = matches!(kind, BuildKind::Build | BuildKind::Rebuild);
let tops: Vec<&maven::MavenProject> = m.projects.iter().filter(|p| !p.aggregated).collect();
if tops.is_empty() {
return Err(IdeError::not_found("no Maven project found"));
@@ -283,9 +270,6 @@ pub(super) fn maven_steps(
Format::Maven,
);
s.args.extend(p.options.iter().cloned());
if skip_jacoco {
s.args.push("-Djacoco.skip=true".into());
}
s.args.extend(extra.iter().cloned());
s.args.extend(goals.iter().map(|g| (*g).to_owned()));
s.env = p.env.clone();
@@ -1049,22 +1033,6 @@ mod tests {
std::fs::write(p, text).unwrap();
}
#[test]
fn maven_compile_steps_skip_jacoco_but_artifacts_do_not() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("demo");
write(&root, "pom.xml", "<project><artifactId>demo</artifactId></project>");
let model = crate::load(&root, &tmp.path().join("cache"), None);
for (kind, expected) in [(BuildKind::Build, true), (BuildKind::Rebuild, true), (BuildKind::Artifacts, false)] {
let plan = plan(&root, &model, kind, None, None).unwrap();
assert_eq!(
plan.steps[0].args.iter().any(|arg| arg == "-Djacoco.skip=true"),
expected,
"{kind:?}: {plan:?}"
);
}
}
#[test]
fn plans_per_build_system() {
let tmp = tempfile::tempdir().unwrap();
@@ -1081,10 +1049,7 @@ mod tests {
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(
p.steps[0].args,
["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "-Djacoco.skip=true", "test-compile"]
);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "test-compile"]);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(
p.steps[0].args,
@@ -1092,7 +1057,6 @@ mod tests {
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-Djacoco.skip=true",
"-pl",
"core",
"-am",
@@ -1381,7 +1345,6 @@ mod tests {
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
@@ -1398,7 +1361,6 @@ mod tests {
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
+26 -1
Переглянути файл
@@ -7,7 +7,6 @@
//! [`super::resolve`] turns them into IDE coordinates.
use ide_core::Severity;
pub(crate) use ide_core::output_encoding::strip_ansi;
use serde_json::Value;
/// Output dialect of a build step.
@@ -99,6 +98,32 @@ pub(crate) struct OutputParser {
pending: Option<Pending>,
}
/// Removes ANSI escape sequences (colour, cursor movement).
pub(crate) fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
fn num(s: &str) -> Option<u32> {
s.trim().parse().ok().filter(|n| *n > 0)
}
-331
Переглянути файл
@@ -1,331 +0,0 @@
//! Single-file Java compilation with `javac` for the Compile File action (#2476).
//!
//! A module build (`-pl … -am`) still walks every upstream module, and packaging
//! walks the whole reactor; compiling one file only needs that file's module
//! outputs plus its dependency classpath, which is much faster to assemble.
use super::*;
use std::fs;
use std::process::Command as ProcCommand;
/// Output file of the `dependency:build-classpath` probe, relative to each module.
const CP_FILE: &str = "target/ide-compile-cp.txt";
/// A `.java` source and the module build that compiles it.
pub struct SingleFileTarget {
/// Absolute module directory.
pub module: PathBuf,
/// The source is under `src/test`: compile to test classes with the test classpath.
pub test: bool,
/// Absolute `target/classes` (or `target/test-classes`).
pub out_dir: PathBuf,
/// Reactor root building the module (owns the Maven command and options).
pub top: maven::MavenProject,
/// `-pl` value of the module relative to the reactor root; empty when the
/// module is the root itself.
pub pl: String,
/// `maven.compiler.release` (or equivalent) of the module's project.
pub java_version: Option<u32>,
}
/// Locates the linked Maven module owning a `.java` source file.
pub fn single_file_target(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<SingleFileTarget> {
if file.extension().is_none_or(|ext| ext != "java") {
return Err(IdeError::invalid("only Java sources can be compiled as a single file"));
}
if !file.is_file() {
return Err(IdeError::not_found("no Java source found for single-file compilation"));
}
let Some(m) = &model.maven else {
return Err(IdeError::not_found("no linked Maven project found for module"));
};
// The innermost project whose directory contains the file owns it; the
// aggregator root (`""`) is a prefix of everything, so take the longest match.
let rel = rel_str(root, file);
let project = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len())
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
if project.packaging == "pom" {
// An aggregator owns no sources itself: a file that only matches it
// belongs to no linked module.
return Err(IdeError::not_found("no linked Maven project found for module"));
}
let module = subdir(root, &project.dir);
let top = maven_reactor(m, &project.dir)
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
let top_dir = subdir(root, &top.dir);
let pl = module.strip_prefix(&top_dir).map(|r| r.to_string_lossy().replace('\\', "/")).unwrap_or_default();
// Test sources see the test classpath and compile into test classes.
let parts: Vec<&str> = rel.split('/').collect();
let test = parts.windows(2).any(|w| w == ["src", "test"]);
let out_dir = module.join("target").join(if test { "test-classes" } else { "classes" });
Ok(SingleFileTarget { module, test, out_dir, top: top.clone(), pl, java_version: project.java_version })
}
/// `javac` for the module: its `JAVA_HOME`, else `PATH`.
pub fn find_javac(env: &BTreeMap<String, String>) -> String {
let home = env.get("JAVA_HOME").map(Path::new);
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
if let Some(javac) = home.map(|h| h.join("bin").join(exe)).filter(|p| p.is_file()) {
return javac.to_string_lossy().into_owned();
}
exe.to_owned()
}
/// Dependency classpath of the module: existing reactor class directories plus
/// `mvn dependency:build-classpath` (which resolves without compiling anything,
/// so `-am` stays cheap). Entries are platform-separated (`;` on Windows).
pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFileTarget) -> IdeResult<String> {
let mut entries: Vec<String> = Vec::new();
// The module's own classes first: the file almost always uses classes next to it.
let own: &[&str] = if target.test { &["target/test-classes", "target/classes"] } else { &["target/classes"] };
for out in own {
let path = target.module.join(out);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
if let Some(m) = &model.maven {
// Upstream reactor modules first: their freshly built classes shadow
// installed snapshots of the same artifacts.
for project in maven_launch_projects(root, m, &target.module, target.test) {
let dir = subdir(root, &project.dir);
if dir == target.module {
continue;
}
for classes in ["target/classes", "target/test-classes"] {
let path = dir.join(classes);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
}
}
let probe = CP_FILE;
let mut args = vec!["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG];
args.extend(target.top.options.iter().map(String::as_str));
if !target.pl.is_empty() {
args.extend(["-pl", target.pl.as_str(), "-am"]);
}
let output_arg = format!("-Dmdep.outputFile={probe}");
args.extend(["dependency:build-classpath", output_arg.as_str(), "-Dmdep.includeScope=test", "-q"]);
let top_dir = subdir(root, &target.top.dir);
let status = ProcCommand::new(&target.top.command)
.args(&args)
.envs(&target.top.env)
.current_dir(&top_dir)
.output()
.map_err(|e| IdeError::internal(format!("cannot resolve the compile classpath: {e}")))?;
if !status.status.success() {
let stderr = String::from_utf8_lossy(&status.stderr);
let tail: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
let tail = tail.iter().rev().take(5).rev().cloned().collect::<Vec<_>>().join("\n");
return Err(IdeError::internal(format!("cannot resolve the compile classpath:\n{tail}")));
}
// With `-am` every visited module writes its own file; only this module's counts.
let cp_file = target.module.join(probe);
let content = fs::read_to_string(&cp_file).map_err(|e| {
IdeError::internal(format!("cannot resolve the compile classpath: {} ({e})", cp_file.display()))
})?;
let separator = if cfg!(windows) { ';' } else { ':' };
entries.extend(content.split([separator, '\n', '\r']).map(str::trim).filter(|e| !e.is_empty()).map(str::to_owned));
entries.dedup();
Ok(entries.join(&separator.to_string()))
}
/// The `javac` step compiling `file` into `target`'s output directory.
pub fn javac_step(target: &SingleFileTarget, file: &Path, classpath: &str, javac: &str) -> IdeResult<BuildStep> {
fs::create_dir_all(&target.out_dir)
.map_err(|e| IdeError::internal(format!("cannot create {} ({e})", target.out_dir.display())))?;
let mut step = BuildStep::new(javac, &[], &target.module, Format::Gradle);
// Raw `javac` prints `path:line: error: …`, the same form Gradle forwards.
step.args.push("-encoding".into());
step.args.push("UTF-8".into());
step.args.push("-d".into());
step.args.push(target.out_dir.to_string_lossy().into_owned());
step.args.push("-cp".into());
step.args.push(classpath.to_owned());
// Classes of the module that were never built (or are older than their source) are read
// from its sources; only `file` is written, so a test compile never puts main classes
// into test-classes.
let sources = source_roots(target);
if !sources.is_empty() {
let separator = if cfg!(windows) { ";" } else { ":" };
step.args.push("-sourcepath".into());
step.args.push(sources.iter().map(|p| p.to_string_lossy()).collect::<Vec<_>>().join(separator));
step.args.push("-implicit:none".into());
}
if let Some(version) = target.java_version {
step.args.push("--release".into());
step.args.push(version.to_string());
}
step.args.push(file.to_string_lossy().into_owned());
step.env = target.top.env.clone();
Ok(step)
}
/// The module's Java source folders that exist, generated ones included; test sources see
/// the main ones too.
fn source_roots(target: &SingleFileTarget) -> Vec<PathBuf> {
let scopes: &[(&str, &str)] = if target.test {
&[("src/test/java", "target/generated-test-sources"), ("src/main/java", "target/generated-sources")]
} else {
&[("src/main/java", "target/generated-sources")]
};
let mut roots = Vec::new();
for (src, generated) in scopes {
roots.push(target.module.join(src));
let generated = target.module.join(generated);
let mut dirs: Vec<PathBuf> =
fs::read_dir(&generated).into_iter().flatten().flatten().map(|d| d.path()).filter(|p| p.is_dir()).collect();
dirs.sort();
roots.extend(dirs);
}
roots.retain(|p| p.is_dir());
roots
}
/// Resolves the classpath and plans the `javac` invocation for one Java source.
pub fn compile_file_plan(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<BuildPlan> {
let target = single_file_target(root, model, file)?;
let classpath = single_file_classpath(root, model, &target)?;
let javac = find_javac(&target.top.env);
let name = file.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok(BuildPlan {
system: BuildSystem::Maven,
title: format!("Compile {name}"),
steps: vec![javac_step(&target, file, &classpath, &javac)?],
})
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version>
<packaging>pom</packaging><modules><module>lib</module><module>app</module></modules></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>lib</artifactId></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>app</artifactId><dependencies><dependency><groupId>demo</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(root, "app/src/main/java/demo/Main.java", "class Main {}");
write(root, "app/src/test/java/demo/MainTest.java", "class MainTest {}");
write(root, "lib/target/classes/Lib.class", "built");
crate::load(root, &root.join("cache"), None)
}
#[test]
fn target_finds_the_owning_module_and_the_test_scope() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let main = single_file_target(root, &model, &root.join("app/src/main/java/demo/Main.java")).unwrap();
assert_eq!(main.module, root.join("app"));
assert!(!main.test);
assert_eq!(main.out_dir, root.join("app/target/classes"));
let test = single_file_target(root, &model, &root.join("app/src/test/java/demo/MainTest.java")).unwrap();
assert!(test.test);
assert_eq!(test.out_dir, root.join("app/target/test-classes"));
}
#[test]
fn target_rejects_non_java_missing_and_unlinked_files() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "app/pom.xml", "<project/>");
assert!(single_file_target(root, &model, &root.join("app/pom.xml")).is_err());
assert!(single_file_target(root, &model, &root.join("app/src/main/java/demo/Missing.java")).is_err());
write(root, "other/Main.java", "class Main {}");
assert!(single_file_target(root, &model, &root.join("other/Main.java")).is_err());
}
#[test]
fn javac_prefers_the_module_jdk_and_scopes_test_sources() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let file = root.join("app/src/test/java/demo/MainTest.java");
let target = single_file_target(root, &model, &file).unwrap();
let jdk = root.join("jdk");
fs::create_dir_all(jdk.join("bin")).unwrap();
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
fs::write(jdk.join("bin").join(exe), "fake").unwrap();
let mut target = target;
target.top.env.insert("JAVA_HOME".into(), jdk.to_string_lossy().into_owned());
let step = javac_step(&target, &file, "/repo/a.jar", &find_javac(&target.top.env)).unwrap();
assert!(step.program.ends_with(exe));
assert!(target.out_dir.is_dir());
let args = step.args.join(" ");
assert!(args.contains("-encoding UTF-8"), "{args}");
assert!(args.contains("-d ") && args.contains("test-classes"), "{args}");
assert!(args.contains("-cp /repo/a.jar"), "{args}");
assert!(args.contains("MainTest.java"), "{args}");
// Test sources resolve their own and the main sources, without writing them.
let sourcepath = step.args.iter().position(|a| a == "-sourcepath").map(|i| step.args[i + 1].clone());
let roots = sourcepath.expect("a -sourcepath");
assert!(roots.contains("src/test/java") && roots.contains("src/main/java"), "{roots}");
assert!(step.args.iter().any(|a| a == "-implicit:none"), "{args}");
// No `JAVA_HOME`: plain `javac` from `PATH`.
assert_eq!(find_javac(&BTreeMap::new()), exe);
}
/// The classpath probe runs the project's Maven command with `-pl … -am`
/// `dependency:build-classpath`; the fake `mvnw` below only records its
/// arguments and writes a canned classpath file for the `-pl` module.
#[cfg(unix)]
#[test]
fn classpath_probe_writes_one_file_per_module_and_reads_the_compiled_one() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
fixture(root);
let script = root.join("mvnw");
fs::write(
&script,
"#!/bin/sh\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/a.jar:/repo/b.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
let model = crate::load(root, &root.join("cache"), None);
let file = root.join("app/src/main/java/demo/Main.java");
let target = single_file_target(root, &model, &file).unwrap();
assert!(target.top.command.ends_with("mvnw"));
assert_eq!(target.pl, "app");
let cp = single_file_classpath(root, &model, &target).unwrap();
let separator = ':';
// Reactor class directories first, so fresh outputs shadow installed snapshots.
let lib = root.join("lib/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&lib), "{cp}");
// The module's own classes come first once they exist.
fs::create_dir_all(root.join("app/target/classes")).unwrap();
let cp = single_file_classpath(root, &model, &target).unwrap();
let own = root.join("app/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&format!("{own}:{lib}")), "{cp}");
assert!(cp.contains("/repo/a.jar"), "{cp}");
assert!(cp.contains("/repo/b.jar"), "{cp}");
assert!(cp.split(separator).count() >= 3, "{cp}");
}
}
-253
Переглянути файл
@@ -1,253 +0,0 @@
//! Up-to-date checks for menu builds (`Build Project` / `Build Module`, #2476).
//!
//! Menu builds have no fingerprint record on the launch path, so a second
//! click always pays a full offline Maven run. This module stores one JSON
//! record per scope (project, or one module file's build) after a successful
//! build and compares it before the next build; on a match the build is
//! skipped entirely. Like launch reuse, timestamp-preserving external edits
//! are re-stamped first so Maven actually recompiles them on the next run.
use super::*;
use std::fs;
/// Record of the last successful menu build of one scope.
#[derive(serde::Serialize, serde::Deserialize)]
struct Record {
fingerprint: LaunchFingerprint,
/// Output folders of the built projects that existed after the build, relative to the
/// project. Deleting one outside the IDE (`mvn clean` in a terminal) leaves every input
/// unchanged, so a build is skipped only while they are all still there.
#[serde(default)]
outputs: Vec<PathBuf>,
}
/// The build's output folders that exist now, relative to `root`.
fn existing_outputs(root: &Path, fingerprint: &LaunchFingerprint) -> Vec<PathBuf> {
fingerprint
.project_dirs()
.flat_map(|dir| ["target/classes", "target/test-classes"].map(|out| dir.join(out)))
.filter(|rel| root.join(rel).is_dir())
.collect()
}
fn record_path(cache: &Path, root: &Path, file: Option<&Path>) -> PathBuf {
let base = cache.join("menu-build").join("records");
match file {
Some(f) => base.join(f.strip_prefix(root).unwrap_or(f)).with_extension("json"),
None => base.join("project.json"),
}
}
/// Re-stamps files whose content changed without advancing mtime (git
/// checkout, external sync): timestamp-incremental Maven would otherwise skip
/// them as "unchanged" and the rebuild would compile nothing.
fn restamp(root: &Path, edits: &[PathBuf]) {
let now = std::time::SystemTime::now();
for rel in edits {
let path = root.join(rel);
let _ = fs::File::options().write(true).open(&path).and_then(|f| f.set_modified(now)).or_else(|_| {
fs::OpenOptions::new()
.create(true)
.truncate(false)
.write(true)
.open(&path)
.and_then(|f| f.set_modified(now))
});
}
}
/// Whether the last successful menu build of this scope is still fresh: its
/// record exists and the fingerprint matches. On a mismatch caused by
/// timestamp-preserving external edits the files are re-stamped first, so the
/// rebuild Maven runs actually recompiles them.
pub fn menu_up_to_date(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> bool {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return false,
};
let bytes = match fs::read(record_path(cache, root, file)) {
Ok(bytes) => bytes,
_ => return false,
};
let record: Record = match serde_json::from_slice(&bytes) {
Ok(record) => record,
_ => return false,
};
if record.fingerprint != current {
restamp(root, &record.fingerprint.mtime_preserved_edits(&current));
return false;
}
record.outputs.iter().all(|out| root.join(out).is_dir())
}
/// Records a successful menu build as the baseline for future up-to-date
/// checks. Failures are swallowed: a missing record only costs one Maven run.
pub fn record_menu_build(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return,
};
// A torn write fails to parse later and is treated as "no record", so a
// plain write is enough.
let path = record_path(cache, root, file);
if path.parent().is_some_and(|p| fs::create_dir_all(p).is_err()) {
return;
}
let outputs = existing_outputs(root, &current);
let bytes = match serde_json::to_vec(&Record { fingerprint: current, outputs }) {
Ok(bytes) => bytes,
_ => return,
};
let _ = fs::write(&path, bytes);
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
maven_two_module(root, "int x;");
BuildModel { maven: Some(maven::load(root)), ..Default::default() }
}
fn maven_two_module(root: &Path, extra_app: &str) {
write(
root,
"pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><groupId>t</groupId><artifactId>root</artifactId><version>1</version><packaging>pom</packaging><modules><module>app</module><module>lib</module></modules></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>app</artifactId><dependencies><dependency><groupId>t</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>lib</artifactId></project>"#,
);
write(root, "app/src/main/java/t/A.java", &format!("package t; public class A {{ {extra_app} }}"));
write(root, "lib/src/main/java/t/L.java", "package t; public class L {}");
}
#[test]
fn record_then_skip_without_changes() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
// The project scope is independent of the module scope.
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn deleted_outputs_invalidate_the_record() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let cancel = CancellationToken::new();
for out in ["app/target/classes/t", "lib/target/classes/t", "app/target/test-classes"] {
fs::create_dir_all(root.join(out)).unwrap();
}
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
// `mvn clean` outside the IDE: no input changed, but the classes are gone.
fs::remove_dir_all(root.join("lib/target")).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
fs::create_dir_all(root.join("lib/target/classes")).unwrap();
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn edit_invalidates_and_rerecord_restores_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
write(root, "app/src/main/java/t/A.java", "package t; public class A { int y; }");
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn mtime_preserved_edit_forces_rebuild_and_restamps() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let rel = "app/src/main/java/t/A.java";
let file = root.join(rel);
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
let before = fs::metadata(&file).unwrap().modified().unwrap();
// Same length, same mtime, different content: an external edit that
// preserved the timestamp.
write(root, rel, "package t; public class A { int z; }");
fs::File::options().write(true).open(&file).and_then(|f| f.set_modified(before)).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
assert!(fs::metadata(&file).unwrap().modified().unwrap() > before);
}
#[test]
fn corrupt_record_never_skips() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
fs::create_dir_all(cache.join("menu-build").join("records").join("app/src/main/java/t")).unwrap();
fs::write(cache.join("menu-build").join("records").join("app/src/main/java/t/A.json"), b"{not json").unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn unsupported_builds_never_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "other/Main.java", "public class Main {}");
let cache = root.join("cache");
let cancel = CancellationToken::new();
// A file outside any non-aggregator module has no fingerprint.
let stray = root.join("other/Main.java");
record_menu_build(&cache, root, &model, Some(&stray), &cancel);
assert!(!menu_up_to_date(&cache, root, &model, Some(&stray), &cancel));
}
}
+1 -22
Переглянути файл
@@ -8,24 +8,6 @@ def canon = { File f -> f.canonicalPath }
// the identity path (`:lib:core`, `:buildSrc`) tells them apart and addresses their tasks
// from the root build. Internal API, hence the fallback.
def identity = { Project p -> try { p.identityPath.toString() } catch (ignored) { p.path } }
// The task copying a source set's resources and its output folder, when it changes them on the
// way (filters, `expand`, renames, includes / excludes, extra sources): jdtls copies resources as
// they are, so launches take them from Gradle. Internal API, hence the fallback.
def processedResources = { Project p, s ->
try {
def task = p.tasks.findByName(s.processResourcesTaskName)
if (task == null) return null
def specs = 0
def changes = task.mainSpec.sourcePaths.size() > 1
task.rootSpec.buildRootResolver().walk { r ->
specs++
changes |= !r.allCopyActions.isEmpty() || !r.allIncludes.isEmpty() || !r.allExcludes.isEmpty() ||
!r.allIncludeSpecs.isEmpty() || !r.allExcludeSpecs.isEmpty() || r.destPath.segments.length > 0
}
// The root and main specs exist for every task; more come from `from(…) { … }`.
(changes || specs > 2) ? [task: task.name, dir: canon(task.destinationDir)] : null
} catch (ignored) { null }
}
def describe = { Project p ->
def m = [path: identity(p), dir: canon(p.projectDir), buildDir: canon(p.layout.buildDirectory.get().asFile)]
@@ -44,10 +26,7 @@ def describe = { Project p ->
def test = s.name == 'test' || s.name == 'testFixtures' || suites.contains(s.name) ||
s.name.endsWith('Test') || s.name.endsWith('Tests') ||
(!src.isEmpty() && src.every { ideaTests.contains(it) })
def set = [name: s.name, test: test, src: src, res: res]
def processed = processedResources(p, s)
if (processed != null) set.processed = processed
set
[name: s.name, test: test, src: src, res: res]
}
}
try {
+1 -14
Переглянути файл
@@ -204,7 +204,7 @@ pub fn parse_settings(text: &str) -> Settings {
}
/// `a/b/../c` → `a/c`, without touching the filesystem.
pub(crate) fn normalize(p: &Path) -> PathBuf {
fn normalize(p: &Path) -> PathBuf {
let mut out = PathBuf::new();
for c in p.components() {
match c {
@@ -335,19 +335,6 @@ pub struct ReportedSourceSet {
/// Absolute resource folders.
#[serde(default)]
pub res: Vec<String>,
/// Set when Gradle changes the resources while copying them (filters, `expand`, renames,
/// includes / excludes, extra sources), which the Java language server's copy does not.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub processed: Option<ProcessedResources>,
}
/// The task that copies a source set's resources, and where it writes them.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProcessedResources {
/// Task name in its project (`processResources`).
pub task: String,
/// Absolute output folder.
pub dir: String,
}
const MODEL_LINE: &str = "@openidea-model\t";
-1
Переглянути файл
@@ -536,7 +536,6 @@ mod tests {
crate::maven::LOCAL_REPO_TRACKING_ARG,
"-o",
"-Peditor",
"-Djacoco.skip=true",
"test-compile"
]
);
-8
Переглянути файл
@@ -122,14 +122,6 @@ pub fn gradle_external_roots(
gradle::external_roots(root, dirs.iter().map(String::as_str))
}
/// Directories of Maven POMs linked from outside `root`, outermost only.
/// Reads just those explicit paths (no POM parse and no directory walk).
pub fn maven_external_roots(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let dirs = maven::external_pom_dirs(root, linked);
let owned: Vec<String> = dirs.iter().map(|dir| dir.to_string_lossy().into_owned()).collect();
gradle::external_roots(root, owned.iter().map(String::as_str))
}
/// Source, resource and build folders of the last Gradle sync, absolute.
pub fn gradle_source_roots(
root: &Path,
+31 -337
Переглянути файл
@@ -106,9 +106,7 @@ pub struct MavenProject {
pub profiles: Vec<String>,
/// Java feature version the project compiles for (`maven.compiler.release`, `java.version`…).
pub java_version: Option<u32>,
/// Maven launcher: the nearest `mvnw` (absolute), else `mvn`. Inside the opened
/// project the search stops at its root; outside it, the search stops before
/// that root so a linked project keeps its own wrapper.
/// Maven launcher: the nearest `mvnw` up to the project root (absolute), else `mvn`.
pub command: String,
/// Options before the goals (`-s`, `-Dmaven.repo.local=`); filled in by the shell from settings.
pub options: Vec<String>,
@@ -1034,6 +1032,10 @@ pub(crate) fn plugin(
const MAX_PROJECTS: usize = 1000;
fn rel_path(root: &Path, p: &Path) -> String {
p.strip_prefix(root).unwrap_or(p).to_string_lossy().replace('\\', "/")
}
/// The POMs that are not modules of another one: the root `pom.xml`, else the
/// shallowest `pom.xml` of each branch (a folder holding several builds). An
/// aggregate POM directly inside `src/` counts too (issue #1001: `src/pom.xml`
@@ -1047,29 +1049,18 @@ pub fn is_maven_project(root: &Path) -> bool {
!top_level_poms(root).is_empty()
}
/// The `mvnw` nearest to `dir`. Inside `root` the walk stops at `root`. Outside it,
/// the walk stops before entering `root` or any ancestor of `root`: a sibling project
/// keeps its own wrapper, and the search does not continue to the filesystem root.
/// The `mvnw` nearest to `dir`, looking up to `root`.
fn wrapper(root: &Path, dir: &Path) -> String {
let name = if cfg!(windows) { "mvnw.cmd" } else { "mvnw" };
let root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let dir = dunce::canonicalize(dir).unwrap_or_else(|_| dir.to_path_buf());
let external = !dir.starts_with(&root);
let mut cur = Some(dir.as_path());
let mut cur = Some(dir);
while let Some(d) = cur {
if d.join(name).is_file() {
return d.join(name).to_string_lossy().into_owned();
}
if !external && d == root {
if d == root {
break;
}
let Some(parent) = d.parent() else { break };
// `parent` inside the opened project, or an ancestor of it (the common parent,
// then the filesystem root). Stop before inspecting those.
if external && (parent.starts_with(&root) || root.starts_with(parent)) {
break;
}
cur = Some(parent);
cur = d.parent();
}
if cfg!(windows) { "mvn.cmd".into() } else { "mvn".into() }
}
@@ -1113,118 +1104,24 @@ pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[St
load_with_linked(root, local_repo, profiles, &[])
}
/// The `pom.xml` files `linked` names. Each entry is used as written (project-relative,
/// `../sibling/pom.xml`, or absolute) and canonicalized; a path that is not an existing
/// `pom.xml` is skipped. Nothing here lists a directory or walks parents looking for builds.
/// The POMs of `linked` (project-relative, from Settings) that exist inside `root`.
fn linked_poms(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out: Vec<PathBuf> = Vec::new();
for pom in linked {
let Some(path) = dunce::canonicalize(base.join(pom.trim())).ok() else { continue };
if is_pom_file(&path) && !out.contains(&path) {
let pom = normalized_pom(pom);
if pom.is_empty() || pom.starts_with("../") || pom == ".." {
continue;
}
let path = root.join(&pom);
let inside = dunce::canonicalize(&path).is_ok_and(|p| p.starts_with(&base));
if path.is_file() && inside && !out.contains(&path) {
out.push(path);
}
}
out
}
fn is_pom_file(path: &Path) -> bool {
path.is_file()
&& path
.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| if cfg!(windows) { name.eq_ignore_ascii_case("pom.xml") } else { name == "pom.xml" })
}
/// Identity of `path` (both canonical) within `root`: project-relative inside it,
/// otherwise the absolute path with forward slashes.
fn project_rel(root: &Path, path: &Path) -> String {
path.strip_prefix(root).unwrap_or(path).to_string_lossy().replace('\\', "/")
}
/// Canonical `pom.xml` paths named by `linked`, keyed by the spelling to store and to
/// show so unlink removes the same entry settings hold. The first spelling wins.
fn linked_identities(root: &Path, linked: &[String]) -> HashMap<PathBuf, String> {
let mut out = HashMap::new();
for entry in linked {
let entry = entry.trim();
if entry.is_empty() {
continue;
}
let Some(path) = dunce::canonicalize(root.join(entry)).ok() else { continue };
if is_pom_file(&path) {
out.entry(path).or_insert_with(|| entry.replace('\\', "/"));
}
}
out
}
/// Stable, deduplicated form of `poms` for `languages.java.maven.linkedProjects`.
/// An existing `pom.xml` inside `root` is project-relative; one outside it is the
/// canonical absolute path. Spellings of the same file (`../B/pom.xml`, an absolute
/// path, a symlink) collapse to one entry. A path that exists and is not a `pom.xml`
/// is dropped. A `pom.xml` that cannot be resolved yet is kept, so a link is not
/// discarded while its volume is offline. No directory is scanned.
pub fn normalize_linked_poms(root: &Path, poms: &[String]) -> Vec<String> {
let Some(root) = dunce::canonicalize(root).ok() else {
let mut out = Vec::new();
for pom in poms {
let pom = normalized_pom(pom);
if !pom.is_empty() && pom_name_ok(&pom) && !out.contains(&pom) {
out.push(pom);
}
}
return out;
};
let mut out = Vec::new();
let mut seen = HashSet::new();
for pom in poms {
let pom = pom.trim();
if pom.is_empty() {
continue;
}
let key = match dunce::canonicalize(root.join(pom)) {
Ok(path) if is_pom_file(&path) => project_rel(&root, &path),
Ok(_) => continue,
Err(_) => {
let fallback = normalized_pom(pom);
let joined = root.join(pom);
if !pom_name_ok(&fallback) || (joined.exists() && !is_pom_file(&joined)) {
continue;
}
fallback
}
};
if key.is_empty() || !seen.insert(key.clone()) {
continue;
}
out.push(key);
}
out
}
fn pom_name_ok(path: &str) -> bool {
let name = path.rsplit('/').next().unwrap_or(path);
if cfg!(windows) { name.eq_ignore_ascii_case("pom.xml") } else { name == "pom.xml" }
}
/// Directories of linked `pom.xml` files that sit outside `root` (the opened project
/// stays the root). Callers keep the outermost directory only. Modules a POM declares
/// outside its own directory are not included: a linked file cannot widen the indexed
/// tree to an arbitrary path.
pub fn external_pom_dirs(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out = Vec::new();
for pom in linked_poms(root, linked) {
let Some(dir) = pom.parent() else { continue };
if dir.starts_with(&base) || out.iter().any(|have| have == dir) {
continue;
}
out.push(dir.to_path_buf());
}
out
}
/// Whether `root` holds a Maven build or the settings link one inside it.
pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -> bool {
is_maven_project(root) || !linked_poms(root, &settings.linked_projects).is_empty()
@@ -1233,9 +1130,7 @@ pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -
/// The model with the chosen `profiles` (`-P` entries; `!id` deactivates: their modules,
/// dependencies and properties are part of the model, those turned off are not) and the
/// `linked` POMs (Add Maven Projects): builds that no reactor lists, shown as independent
/// top-level projects, including a `pom.xml` outside the opened root. A linked POM that
/// turns out to be a module of the root reactor stays that module. The same canonical file
/// is loaded once, so a cycle or a second spelling does not duplicate it or edit either POM.
/// top-level projects. A linked POM that turns out to be a module of the root reactor stays that module.
pub fn load_with_linked(
root: &Path,
local_repo: Option<PathBuf>,
@@ -1244,10 +1139,7 @@ pub fn load_with_linked(
) -> MavenModel {
let mut resolver = Resolver::new(local_repo, profiles);
let mut model = MavenModel::default();
// Canonical POM → its project, for a module listed again by a later aggregator.
let mut seen: HashMap<PathBuf, Option<usize>> = HashMap::new();
let canonical_root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let linked_ids = linked_identities(&canonical_root, linked);
let mut seen: HashSet<PathBuf> = HashSet::new();
let mut tops = top_level_poms(root);
for pom in linked_poms(root, linked) {
if !tops.contains(&pom) {
@@ -1255,31 +1147,17 @@ pub fn load_with_linked(
}
}
// (pom, aggregated); the root reactor and its modules come before linked POMs.
// (pom, listed by): a module carries the top-level project its chain started from and
// whether the aggregator listing it lies inside the opened root.
let mut stack: Vec<(PathBuf, Option<(usize, bool)>)> = tops.into_iter().rev().map(|p| (p, None)).collect();
while let Some((path, listed_by)) = stack.pop() {
let aggregated = listed_by.is_some();
let mut stack: Vec<(PathBuf, bool)> = tops.into_iter().rev().map(|p| (p, false)).collect();
while let Some((path, aggregated)) = stack.pop() {
if model.projects.len() >= MAX_PROJECTS {
break;
}
let canonical = dunce::canonicalize(&path).unwrap_or_else(|_| path.clone());
if let Some(known) = seen.get(&canonical) {
// Found as a top-level POM first, then listed by an aggregator elsewhere
// (`<module>../</module>` of `parent/pom.xml`): it is that aggregator's module.
// Not when the chain leads back to where it started (a cycle), and not for an
// aggregator outside the opened root: a linked sibling does not take it over.
if let (Some(known), Some((origin, true))) = (*known, listed_by)
&& known != origin
{
model.projects[known].aggregated = true;
}
if !seen.insert(canonical) {
continue;
}
seen.insert(canonical.clone(), None);
let Some(pom) = resolver.pom(&path) else { continue };
let dir = path.parent().unwrap_or(root).to_path_buf();
let dir_canon = dunce::canonicalize(&dir).unwrap_or_else(|_| dir.clone());
let ctx = resolver.context(&path, 0);
let props = resolver.properties_for(&pom, &ctx, &dir);
let i = |s: &str| interpolate(s, &props);
@@ -1290,7 +1168,7 @@ pub fn load_with_linked(
}
let mut modules: Vec<PathBuf> = Vec::new();
for m in module_names {
let mut mp = crate::gradle::normalize(&dir.join(i(&m)));
let mut mp = dir.join(i(&m));
if mp.is_dir() || mp.extension().is_none_or(|e| e != "xml") {
mp.push("pom.xml");
}
@@ -1354,39 +1232,28 @@ pub fn load_with_linked(
}
}
let artifact_id = i(&pom.artifact_id);
let module_ids: Vec<String> = modules
.iter()
.map(|m| {
let canonical = dunce::canonicalize(m).unwrap_or_else(|_| m.clone());
linked_ids.get(&canonical).cloned().unwrap_or_else(|| project_rel(&canonical_root, &canonical))
})
.collect();
model.projects.push(MavenProject {
project_root: None,
pom: linked_ids.get(&canonical).cloned().unwrap_or_else(|| project_rel(&canonical_root, &canonical)),
dir: project_rel(&canonical_root, &dir_canon),
pom: rel_path(root, &path),
dir: rel_path(root, &dir),
group_id: i(props.get("project.groupId").map(String::as_str).unwrap_or_default()),
version: i(props.get("project.version").map(String::as_str).unwrap_or_default()),
display_name: pom.name.as_deref().map(i).filter(|n| !n.is_empty()).unwrap_or_else(|| artifact_id.clone()),
artifact_id,
packaging,
parent: pom.parent.as_ref().map(|p| format!("{}:{}:{}", p.group_id, p.artifact_id, i(&p.version))),
modules: module_ids,
modules: modules.iter().map(|m| rel_path(root, m)).collect(),
aggregated,
plugins,
dependencies,
profiles: ctx.profiles.iter().map(|(id, _)| id.clone()).collect(),
java_version: jv,
command: wrapper(&canonical_root, &dir_canon),
command: wrapper(root, &dir),
options: Vec::new(),
env: BTreeMap::new(),
});
let index = model.projects.len() - 1;
seen.insert(canonical, Some(index));
let origin = listed_by.map_or(index, |(origin, _)| origin);
let inside = dir_canon.starts_with(&canonical_root);
for m in modules.into_iter().rev() {
stack.push((m, Some((origin, inside))));
stack.push((m, true));
}
}
model
@@ -1474,27 +1341,17 @@ impl UnlinkedProjects {
fn normalized_pom(pom: &str) -> String {
let pom = pom.replace('\\', "/");
let unc = pom.starts_with("//");
let absolute = pom.starts_with('/');
let mut parts: Vec<&str> = Vec::new();
let mut parts = Vec::new();
for part in pom.split('/') {
match part {
"" | "." => {}
// Keep a drive letter (`C:`) and a leading `..` that has nothing to climb.
".." if parts.last().is_some_and(|p| *p != ".." && !p.ends_with(':')) => {
".." if parts.last().is_some_and(|p| *p != "..") => {
parts.pop();
}
part => parts.push(part),
}
}
let joined = parts.join("/");
if unc {
format!("//{joined}")
} else if absolute {
format!("/{joined}")
} else {
joined
}
parts.join("/")
}
/// Every POM of the model (for change detection).
@@ -2290,169 +2147,6 @@ mod tests {
assert_eq!(top(&load_linked(deep.path(), &settings)), [("a/b/c/tool".into(), false)]);
}
/// A sibling Maven project is linked beside the opened root. Its reactor, including a
/// module that points back at the opened project, is read once. Neither POM is rewritten,
/// and only the linked project's directory is an extra content root — not the common
/// parent and not a module that lives outside that directory.
#[test]
fn external_sibling_keeps_its_directory_and_does_not_edit_poms() {
let tmp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(tmp.path()).unwrap();
let a = base.join("A");
let b = base.join("B");
let c = base.join("C");
std::fs::create_dir_all(a.join("core")).unwrap();
std::fs::create_dir_all(b.join("mod")).unwrap();
std::fs::create_dir_all(&c).unwrap();
let a_pom = "<project><groupId>demo</groupId><artifactId>a</artifactId><version>1</version><packaging>pom</packaging><modules><module>core</module></modules></project>";
let b_pom = "<project><groupId>demo</groupId><artifactId>b</artifactId><version>1</version><packaging>pom</packaging><modules><module>mod</module><module>../A</module><module>../C</module><module>.</module></modules></project>";
let c_pom = "<project><groupId>demo</groupId><artifactId>c</artifactId><version>1</version><modules><module>../B</module></modules></project>";
std::fs::write(a.join("pom.xml"), a_pom).unwrap();
std::fs::write(a.join("core/pom.xml"), "<project><artifactId>core</artifactId></project>").unwrap();
std::fs::write(b.join("pom.xml"), b_pom).unwrap();
std::fs::write(b.join("mod/pom.xml"), "<project><artifactId>bmod</artifactId></project>").unwrap();
std::fs::write(c.join("pom.xml"), c_pom).unwrap();
let wrapper = if cfg!(windows) { "mvnw.cmd" } else { "mvnw" };
std::fs::write(a.join(wrapper), "#!/bin/sh\n").unwrap();
std::fs::write(b.join(wrapper), "#!/bin/sh\n").unwrap();
std::fs::write(base.join(wrapper), "#!/bin/sh\n").unwrap();
let discovered = load_with_linked(&a, None, &[], &[]);
assert_eq!(discovered.projects.iter().map(|p| p.artifact_id.as_str()).collect::<Vec<_>>(), ["a", "core"]);
let stored = vec!["../B/pom.xml".into()];
let linked = load_with_linked(&a, None, &[], &stored);
let b_project = linked.projects.iter().find(|p| p.artifact_id == "b").unwrap();
assert!(!b_project.aggregated);
assert_eq!(b_project.pom, "../B/pom.xml");
assert_eq!(dunce::canonicalize(a.join(&b_project.dir)).unwrap(), b);
assert_eq!(dunce::canonicalize(&b_project.command).unwrap(), dunce::canonicalize(b.join(wrapper)).unwrap());
let a_project = linked.projects.iter().find(|p| p.artifact_id == "a").unwrap();
assert!(!a_project.aggregated);
assert_eq!(a_project.modules, ["core/pom.xml"]);
assert_eq!(dunce::canonicalize(&a_project.command).unwrap(), dunce::canonicalize(a.join(wrapper)).unwrap());
assert_eq!(linked.projects.iter().filter(|p| p.artifact_id == "a").count(), 1);
assert_eq!(linked.projects.iter().filter(|p| p.artifact_id == "b").count(), 1);
assert!(linked.projects.iter().any(|p| p.artifact_id == "c" && p.aggregated));
assert!(linked.projects.iter().any(|p| p.artifact_id == "bmod" && p.aggregated));
let again = load_with_linked(&a, None, &[], &stored);
assert_eq!(
again.projects.iter().map(|p| &p.pom).collect::<Vec<_>>(),
linked.projects.iter().map(|p| &p.pom).collect::<Vec<_>>()
);
let absolute = b.join("pom.xml").to_string_lossy().replace('\\', "/");
let via_a = a.join("../B/pom.xml").to_string_lossy().replace('\\', "/");
let normalized = normalize_linked_poms(
&a,
&[
"../B/pom.xml".into(),
absolute,
via_a,
a.join("pom.xml").to_string_lossy().into_owned(),
b.to_string_lossy().into_owned(),
"stub/pom.xml".into(),
"notes.xml".into(),
],
);
assert_eq!(normalized.len(), 3, "{normalized:?}");
assert_eq!(normalized[0], dunce::canonicalize(b.join("pom.xml")).unwrap().to_string_lossy().replace('\\', "/"));
assert_eq!(normalized[1], "pom.xml");
assert_eq!(normalized[2], "stub/pom.xml");
assert_eq!(crate::maven_external_roots(&a, &normalized), vec![b.clone()]);
assert_eq!(crate::maven_external_roots(&a, &stored), vec![b.clone()]);
let removed = load_with_linked(&a, None, &[], &[]);
assert!(removed.projects.iter().all(|p| p.artifact_id != "b"));
assert_eq!(std::fs::read_to_string(a.join("pom.xml")).unwrap(), a_pom);
assert_eq!(std::fs::read_to_string(b.join("pom.xml")).unwrap(), b_pom);
assert_eq!(std::fs::read_to_string(c.join("pom.xml")).unwrap(), c_pom);
assert!(b.join("mod/pom.xml").is_file());
}
/// The launcher recorded on an external project is that project's wrapper, and
/// running it uses the external directory as the working directory.
#[cfg(unix)]
#[test]
fn external_project_wrapper_runs_in_its_own_directory() {
let tmp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(tmp.path()).unwrap();
let a = base.join("A");
let b = base.join("B");
std::fs::create_dir_all(b.join("src/main/java")).unwrap();
std::fs::create_dir_all(&a).unwrap();
std::fs::write(a.join("pom.xml"), "<project><artifactId>a</artifactId></project>").unwrap();
std::fs::write(
b.join("pom.xml"),
"<project><artifactId>b</artifactId><build><sourceDirectory>src/main/java</sourceDirectory></build></project>",
)
.unwrap();
std::fs::write(b.join("src/main/java/B.java"), "class B {}").unwrap();
let script = b.join("mvnw");
std::fs::write(&script, "#!/bin/sh\npwd > \"$1\"\nprintf 'BUILD SUCCESS\\n'\n").unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap();
}
let model = load_with_linked(&a, None, &[], &["../B/pom.xml".into()]);
let project = model.projects.iter().find(|p| p.artifact_id == "b").unwrap();
let cwd = a.join(&project.dir);
assert_eq!(dunce::canonicalize(&cwd).unwrap(), b);
let marker = tmp.path().join("cwd");
let out = crate::process::run(
&project.command,
&[marker.to_string_lossy().into_owned()],
&cwd,
&std::collections::BTreeMap::new(),
&ide_core::CancellationToken::new(),
|_, _| {},
)
.unwrap();
assert!(out.success, "{}", out.text);
let recorded = dunce::canonicalize(std::fs::read_to_string(&marker).unwrap().trim()).unwrap();
assert_eq!(recorded, b);
}
/// #2877: an aggregator in a subfolder lists the root POM as `../`; linked, it holds every
/// module, the root one included, under the same POM paths the projects have.
#[test]
fn an_aggregator_in_a_subfolder_holds_the_root_pom() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let child = |id: &str, rel: &str| {
format!(
"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><relativePath>{rel}</relativePath></parent><artifactId>{id}</artifactId></project>"
)
};
write(root, "pom.xml", &child("app", "parent/pom.xml"));
write(
root,
"parent/pom.xml",
"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>../</module><module>../infra</module></modules></project>",
);
write(root, "infra/pom.xml", &child("infra", "../parent/pom.xml"));
let model = load_with_linked(root, None, &[], &["parent/pom.xml".into()]);
let tops: Vec<&str> = model.projects.iter().filter(|p| !p.aggregated).map(|p| p.artifact_id.as_str()).collect();
assert_eq!(tops, ["parent"]);
let parent = model.projects.iter().find(|p| p.artifact_id == "parent").unwrap();
assert_eq!(parent.modules, ["pom.xml", "infra/pom.xml"]);
for pom in &parent.modules {
assert!(model.projects.iter().any(|p| &p.pom == pom), "{pom}");
}
// A cycle (the root lists the aggregator, which lists the root) keeps the root on top.
write(
root,
"pom.xml",
"<project><groupId>demo</groupId><artifactId>app</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>parent</module></modules></project>",
);
let model = load_with_linked(root, None, &[], &[]);
let tops: Vec<&str> = model.projects.iter().filter(|p| !p.aggregated).map(|p| p.artifact_id.as_str()).collect();
assert_eq!(tops, ["app"]);
}
#[test]
fn chosen_profiles_bring_their_modules_and_dependencies() {
let tmp = tempfile::tempdir().unwrap();
+3 -25
Переглянути файл
@@ -73,9 +73,6 @@ struct GradleCache {
/// The projects as Gradle evaluated them; empty in caches from before they were read.
#[serde(default)]
projects: Vec<ReportedProject>,
/// The projects report how their resources are processed; older caches are stale.
#[serde(default)]
resources: bool,
}
/// Init script printing every project, its source sets and Java version once configured.
@@ -247,12 +244,11 @@ pub fn apply_gradle_cache(root: &Path, cache_dir: &Path, model: &mut GradleModel
}
}
}
// A successful sync from before projects were cached has not seen computed includes,
// nor, before resources were read, which launches need Gradle to process them.
let outdated = cache.error.is_none() && (cache.projects.is_empty() || !cache.resources);
// A successful sync from before projects were cached has not seen computed includes.
let without_projects = cache.error.is_none() && cache.projects.is_empty();
model.synced_at = Some(cache.at_ms);
model.sync_error = cache.error;
model.stale = outdated || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
model.stale = without_projects || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
}
// ---------------------------------------------------------------- processes
@@ -466,7 +462,6 @@ pub fn sync_gradle_model(
error: (!out.success).then(|| error_summary(&out.text)),
tasks: if out.success { gradle::parse_tasks(&out.text) } else { Vec::new() },
projects,
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache)
}
@@ -718,7 +713,6 @@ mod tests {
test,
src: src.iter().map(|p| core_src(p)).collect(),
res: res.iter().map(|p| core_src(p)).collect(),
processed: None,
};
// A settings script's own println comes first; task listing lines follow.
let out = format!(
@@ -743,15 +737,6 @@ mod tests {
);
let projects = parse_reported_projects(&out);
assert_eq!(projects.len(), 3, "the malformed line is skipped");
// As the init script prints a set whose resources Gradle filters or renames.
let processed = parse_reported_projects(&format!(
"@openidea-model\t{}",
r#"{"path":":","dir":"/p","sourceSets":[{"name":"main","test":false,"src":[],"res":[],"processed":{"task":"processResources","dir":"/p/build/resources/main"}}]}"#
));
assert_eq!(
processed[0].source_sets[0].processed,
Some(gradle::ProcessedResources { task: "processResources".into(), dir: "/p/build/resources/main".into() })
);
let src = |p: &str| PathBuf::from(core_src(p));
assert_eq!(
gradle::reported_source_roots(&projects),
@@ -817,18 +802,12 @@ mod tests {
..Default::default()
})
.collect(),
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut model);
assert_eq!(model.projects[1].tasks.len(), 1);
assert!(model.projects[0].tasks.iter().any(|t| t.name == "wrapper"), "root keeps its defaults");
assert!(!model.stale);
write_cache(&cache_dir, GRADLE_CACHE, &GradleCache { resources: false, ..cache.clone() }).unwrap();
let mut m = gradle::load(&root).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut m);
assert!(m.stale, "synced before resource processing was read");
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
let stale_after = |script: &str| {
std::fs::write(root.join("app/build.gradle"), script).unwrap();
@@ -862,7 +841,6 @@ mod tests {
dir: root.to_string_lossy().into_owned(),
..Default::default()
}],
resources: true,
..Default::default()
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
-25
Переглянути файл
@@ -78,10 +78,6 @@ pub struct EditorAppearanceSettings {
/// Right margin and visual guides from Code Style.
pub hard_wrap_guides: bool,
pub intention_bulb: bool,
/// Draw doc comments (Javadoc, KDoc, JSDoc, rustdoc) as rendered documentation
/// instead of their source, as IDEA's option of the same name. View › Active
/// Editor › Render Documentation Comments overrides this for one editor.
pub render_doc_comments: bool,
/// Distraction Free Mode: width of the centred text column, in characters.
pub distraction_free_width: u32,
}
@@ -104,7 +100,6 @@ impl Default for EditorAppearanceSettings {
rainbow_indent_guides: false,
hard_wrap_guides: true,
intention_bulb: true,
render_doc_comments: false,
distraction_free_width: 80,
}
}
@@ -338,23 +333,6 @@ pub struct InlayHintsSettings {
pub code_vision_usages: bool,
/// Code vision: last author of a declaration (VCS blame) beside its usages.
pub code_vision_author: bool,
/// Code vision: "no usages" on declarations nothing uses (IDEA shows it; clicking searches everywhere).
pub code_vision_no_usages: bool,
/// Code vision: where the usages go, above the declaration or at the end of its line.
pub code_vision_usages_position: CodeVisionPosition,
/// Code vision: where the author goes, above the declaration or at the end of its line.
pub code_vision_author_position: CodeVisionPosition,
}
/// Editor › Inlay Hints › Code Vision position (IDEA's "Default position for metrics").
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum CodeVisionPosition {
/// A row of its own above the declaration.
#[default]
Top,
/// After the end of the declaration's first line.
Right,
}
impl Default for InlayHintsSettings {
@@ -366,9 +344,6 @@ impl Default for InlayHintsSettings {
disabled_languages: Vec::new(),
code_vision_usages: true,
code_vision_author: true,
code_vision_no_usages: true,
code_vision_usages_position: CodeVisionPosition::Top,
code_vision_author_position: CodeVisionPosition::Top,
}
}
}
-50
Переглянути файл
@@ -273,32 +273,6 @@ pub enum DebugEvent {
change_type: String,
message: String,
},
/// The session's capabilities changed after it started: fetch them again.
Capabilities,
/// The adapter's `invalidated` event: what it shows for `areas` (`all`, `stacks`, `threads`,
/// `variables`; empty means all) is stale and has to be fetched again.
Invalidated {
areas: Vec<String>,
},
JvmBackend {
state: JvmBackendState,
},
}
/// The backend that reads a debugged JVM's objects (instances, fields, referrers).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmBackendState {
Ready,
/// Turned off for the rest of the session: the debug adapter lacks what it needs.
Unsupported {
detail: String,
},
/// The Kotlin JDWP link only forwards now; inspecting stops.
Degraded {
detail: String,
},
Closed,
}
/// Where backend crates publish [`IdeEvent`]s. Cheap to clone via `Arc`.
@@ -320,27 +294,3 @@ pub struct NullSink;
impl EventSink for NullSink {
fn emit(&self, _: IdeEvent) {}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn jvm_backend_events_json_shape() {
let degraded = DebugEvent::JvmBackend { state: JvmBackendState::Degraded { detail: "x".into() } };
assert_eq!(
serde_json::to_value(&degraded).unwrap(),
json!({"kind": "jvmBackend", "state": {"kind": "degraded", "detail": "x"}})
);
assert_eq!(serde_json::to_value(DebugEvent::Capabilities).unwrap(), json!({"kind": "capabilities"}));
}
#[test]
fn invalidated_event_json_shape() {
assert_eq!(
serde_json::to_value(DebugEvent::Invalidated { areas: vec!["variables".into()] }).unwrap(),
json!({"kind": "invalidated", "areas": ["variables"]})
);
}
}
+2 -10
Переглянути файл
@@ -12,16 +12,11 @@ use std::path::{Path, PathBuf};
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// `p` as the UI sees it: `C:\a\b`, `c:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
/// `p` as the UI sees it: `C:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
pub fn to_ipc(p: &Path) -> String {
#[cfg(windows)]
{
let mut s = dunce::simplified(p).to_string_lossy().replace('\\', "/");
// The UI compares path strings; a drive letter is case-insensitive.
if s.as_bytes().get(1) == Some(&b':') {
s[..1].make_ascii_uppercase();
}
s
dunce::simplified(p).to_string_lossy().replace('\\', "/")
}
#[cfg(not(windows))]
{
@@ -149,8 +144,5 @@ mod tests {
assert_eq!(to_ipc(Path::new(r"\\?\C:\a\b")), "C:/a/b");
assert_eq!(to_ipc(&Path::new("C:/a").join("b")), "C:/a/b");
assert_eq!(serde_json::to_string(&IpcPath::from(r"C:\a\b")).unwrap(), r#""C:/a/b""#);
assert_eq!(to_ipc(Path::new(r"c:\a\b")), "C:/a/b");
assert_eq!(to_ipc(Path::new(r"\\?\d:\a")), "D:/a");
assert_eq!(to_ipc(Path::new(r"\\server\share\a")), "//server/share/a");
}
}
-26
Переглянути файл
@@ -21,32 +21,6 @@
use std::collections::BTreeMap;
/// Removes ANSI escape sequences (colour, cursor movement).
pub fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
/// Decodes complete lines or collected output from IDE-managed tools, preferring UTF-8.
/// Windows launcher/native diagnostics fall back to the system ANSI code page.
/// Callers must buffer incomplete lines so multibyte characters are never split.
-112
Переглянути файл
@@ -72,75 +72,6 @@ pub fn kill_tree_of(pid: u32) {
let _ = pid;
}
/// Whether a process with this id exists. PID reuse makes this conservative: a reused id keeps
/// whatever was tied to the old process alive, a live process is never reported dead.
pub fn process_alive(pid: u32) -> bool {
// `kill(0, 0)` would signal the caller's own process group.
if pid == 0 {
return false;
}
#[cfg(unix)]
{
// Above `i32::MAX` the id would wrap to a negative value, which `kill` reads as a group.
let Ok(pid) = libc::pid_t::try_from(pid) else { return false };
// SAFETY: signal 0 delivers nothing; it only checks that the process exists and is reachable.
unsafe { libc::kill(pid, 0) == 0 || std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM) }
}
#[cfg(windows)]
{
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle};
use windows_sys::Win32::Foundation::{ERROR_ACCESS_DENIED, GetLastError, STILL_ACTIVE};
use windows_sys::Win32::System::Threading::{
GetExitCodeProcess, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION,
};
// SAFETY: plain call; a non-null result is a process handle owned by this function.
let handle = unsafe { OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
if handle.is_null() {
// A process we may not open still exists.
// SAFETY: reads the thread's last error, set by the call above.
return unsafe { GetLastError() } == ERROR_ACCESS_DENIED;
}
// SAFETY: `handle` is valid and nobody else owns it.
let handle = unsafe { OwnedHandle::from_raw_handle(handle) };
let mut code = 0u32;
// SAFETY: a live process handle opened with query access; one out-parameter.
let ok = unsafe { GetExitCodeProcess(handle.as_raw_handle(), &mut code) };
// A handle that cannot be queried is not evidence that the process is gone.
ok == 0 || code == STILL_ACTIVE as u32
}
#[cfg(not(any(unix, windows)))]
{
true
}
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
pub fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// [`kill_tree`] that first asks the group to exit: `SIGTERM`, then `grace`, then `SIGKILL`.
/// Git removes its `index.lock` on `SIGTERM`, where `SIGKILL` leaves it behind and the next
/// git command fails. Windows has no such signal to send from outside the child's console,
@@ -799,30 +730,6 @@ fn append_path_entries(path: &OsStr, extra: &OsStr) -> OsString {
mod tests {
use super::*;
#[test]
fn process_alive_tells_live_from_dead_pids() {
assert!(process_alive(std::process::id()));
assert!(!process_alive(0));
let mut cmd = if cfg!(windows) {
let mut cmd = Command::new("cmd");
cmd.args(["/C", "exit 0"]);
cmd
} else {
Command::new("true")
};
let mut child = cmd.spawn().unwrap();
let pid = child.id();
child.wait().unwrap();
drop(child);
assert!(!process_alive(pid));
}
#[cfg(unix)]
#[test]
fn process_alive_rejects_ids_that_are_not_a_single_process() {
assert!(!process_alive(u32::MAX));
}
#[cfg(unix)]
#[test]
fn kill_tree_of_reaches_what_the_child_started() {
@@ -1143,25 +1050,6 @@ mod tests {
}
assert_eq!(find_executable("nope", &search), None);
}
#[cfg(unix)]
#[test]
fn process_tree_lists_the_child_of_a_shell() {
let mut cmd = Command::new("sh");
own_group(&mut cmd).args(["-c", "sleep 5 & wait"]);
let mut child = cmd.spawn().unwrap();
let sh_pid = child.id() as i32;
// The shell forks `sleep` after it starts, so the tree grows a moment later.
let deadline = std::time::Instant::now() + Duration::from_secs(2);
let mut tree = process_tree(sh_pid);
while tree.len() < 2 && std::time::Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(20));
tree = process_tree(sh_pid);
}
kill_tree(&mut child);
assert_eq!(tree[0], sh_pid, "the pid itself comes first");
assert!(tree.len() >= 2 && tree[1..].iter().all(|p| *p != sh_pid), "the shell's child follows: {tree:?}");
}
}
#[cfg(all(test, unix))]
+2 -30
Переглянути файл
@@ -108,32 +108,9 @@ impl Default for DeploymentSettings {
pub enum DeploymentType {
#[default]
Sftp,
Ftp,
/// FTP secured with `AUTH TLS` (explicit FTPS).
Ftps,
Local,
}
/// Host and login of an FTP / FTPS deployment server. The password is kept with
/// the saved passwords, under the server's id.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct FtpConnection {
pub host: String,
/// `None`: 21.
pub port: Option<u16>,
/// Empty: anonymous login.
pub user: String,
/// Passive mode (the client opens data connections); off: active mode.
pub passive: bool,
}
impl Default for FtpConnection {
fn default() -> Self {
Self { host: String::new(), port: None, user: String::new(), passive: true }
}
}
/// A deployment filesystem and how project folders map onto it.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
@@ -143,8 +120,6 @@ pub struct DeploymentServer {
pub name: String,
/// [`SshConfiguration::id`].
pub ssh_config: String,
/// FTP / FTPS servers.
pub ftp: FtpConnection,
/// Absolute remote folder the mappings are relative to; empty = the login directory.
pub root_path: String,
/// Web server URL of the root (Open in Browser on the deployed file).
@@ -250,11 +225,8 @@ mod deployment_tests {
serde_json::from_str(r#"{"id":"existing","sshConfig":"ssh","rootPath":"/srv"}"#).unwrap();
assert_eq!(legacy.kind, DeploymentType::Sftp);
assert_eq!(legacy.ssh_config, "ssh");
assert!(legacy.ftp.passive);
let ftp =
FtpConnection { host: "ftp.example.com".into(), port: Some(2121), user: "web".into(), passive: false };
for kind in [DeploymentType::Sftp, DeploymentType::Ftp, DeploymentType::Ftps, DeploymentType::Local] {
let server = DeploymentServer { kind, ftp: ftp.clone(), ..legacy.clone() };
for kind in [DeploymentType::Sftp, DeploymentType::Local] {
let server = DeploymentServer { kind, ..legacy.clone() };
assert_eq!(
serde_json::from_str::<DeploymentServer>(&serde_json::to_string(&server).unwrap()).unwrap(),
server
+3 -65
Переглянути файл
@@ -63,8 +63,6 @@ pub struct Settings {
pub agents: AgentSettings,
/// Agents › MCP Server (global only).
pub mcp_server: McpServerSettings,
/// Tools › Agents: Command Execution Policy.
pub agent_security: AgentSecuritySettings,
/// Plugin settings (`[contributes.settings]`): plugin id → its values by key.
#[specta(type = BTreeMap<String, BTreeMap<String, specta_typescript::Unknown>>)]
pub plugins: BTreeMap<String, serde_json::Map<String, serde_json::Value>>,
@@ -194,19 +192,6 @@ where
Ok(serde_json::from_value(serde_json::Value::deserialize(d)?).unwrap_or_default())
}
/// A theme id, with the built-in themes that were removed read as the Islands theme of their kind.
fn theme_id<'de, D>(d: D) -> Result<String, D::Error>
where
D: serde::Deserializer<'de>,
{
let id = String::deserialize(d)?;
Ok(match id.as_str() {
"one-light" => "light".into(),
"one-dark" => "dark".into(),
_ => id,
})
}
/// [`AgentSettings::defaults`], keeping only the text values of each agent's table.
fn agent_defaults<'de, D>(d: D) -> Result<BTreeMap<String, BTreeMap<String, String>>, D::Error>
where
@@ -276,9 +261,6 @@ pub enum WhileRunning {
pub struct McpServerSettings {
/// Serve the tools on 127.0.0.1 while the IDE runs.
pub enabled: bool,
/// Run the tools that change the project or start processes without asking first
/// (IDEA's brave mode). Read from the IDE's own settings only, never a project's.
pub run_without_confirmation: bool,
}
/// Absent profiles retain automatic discovery for existing projects.
@@ -426,7 +408,6 @@ impl Default for Settings {
cmake: CMakeSettings::default(),
agents: AgentSettings::default(),
mcp_server: McpServerSettings::default(),
agent_security: AgentSecuritySettings::default(),
plugins: BTreeMap::new(),
}
}
@@ -472,10 +453,6 @@ pub struct GeneralSettings {
/// Where updates come from: `auto` (the CNB mirror for the Chinese UI, else GitHub) | `github` | `cnb`.
pub update_source: String,
pub auto_check_updates: bool,
/// System Settings › Updates: open the What's New page after the app starts a new version.
pub show_whats_new_after_update: bool,
/// The version the app last started with, so the first start after an update is recognisable.
pub last_run_version: String,
}
impl Default for GeneralSettings {
@@ -494,8 +471,6 @@ impl Default for GeneralSettings {
project_name: String::new(),
update_source: "auto".into(),
auto_check_updates: true,
show_whats_new_after_update: true,
last_run_version: String::new(),
}
}
}
@@ -570,19 +545,13 @@ pub enum NavigationBarPosition {
pub struct AppearanceSettings {
/// A built-in theme id (`light`, `dark`, `dark-hc`, …), `system` (legacy
/// "Sync with OS"), or a plugin's theme: `plugin:<plugin-id>/<theme-id>`.
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub theme: String,
pub accent: String,
pub ui_font_size: f32,
pub locale: String,
/// Follow the OS light/dark mode with the preferred themes below.
pub sync_with_os: bool,
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub preferred_light_theme: String,
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub preferred_dark_theme: String,
/// A plugin's file icon theme (`<plugin-id>/<theme-id>`); empty = the built-in icons.
pub file_icon_theme: String,
@@ -605,7 +574,7 @@ pub struct AppearanceSettings {
pub project_colors_in_toolbar: bool,
pub dnd_with_alt_only: bool,
pub menu_icons: bool,
/// `hamburger` | `merged` | `toolbar` (a separate row): main menu presentation in the frameless title bar.
/// `hamburger` | `merged`: main menu presentation in the frameless title bar.
pub main_menu_style: String,
/// A lone Alt press focuses or opens the main menu (IDEA `ide.windows.main.menu.focus.on.alt`).
pub main_menu_alt: bool,
@@ -824,9 +793,6 @@ pub struct FileSettings {
pub file_types: BTreeMap<String, FileTypeAssociation>,
/// Override File Type: file (project-relative and `/`-separated, or absolute) → language id.
pub file_type_overrides: BTreeMap<String, String>,
/// File Types › Files opened in associated applications: file-name patterns
/// (`*.xlsx`) opened with the system application instead of an editor.
pub associated_applications: Vec<String>,
/// Compose detection choices, keyed by project-relative file path.
pub compose_file_overrides: BTreeMap<String, ComposeFileOverride>,
}
@@ -863,15 +829,6 @@ impl Default for FileSettings {
local_history_max_mb: 512,
file_types: BTreeMap::new(),
file_type_overrides: BTreeMap::new(),
associated_applications: vec![
"*.xlsx".into(),
"*.xls".into(),
"*.docx".into(),
"*.doc".into(),
"*.pptx".into(),
"*.ppt".into(),
"*.pdf".into(),
],
compose_file_overrides: BTreeMap::new(),
}
}
@@ -1201,9 +1158,7 @@ pub struct MavenSettings {
pub runner_jre: String,
/// Project-relative root POMs removed from the Maven project model.
pub unlinked_projects: Vec<String>,
/// POMs added as independent top-level projects. Inside the opened project the path is
/// project-relative (`tools/pom.xml`); outside it, the canonical absolute path. Linking
/// does not edit either POM.
/// Project-relative POMs outside the root reactor added as independent top-level projects.
pub linked_projects: Vec<String>,
/// Maven home (or its `mvn` executable); empty = the project's `mvnw`, else `mvn` on `PATH`.
pub home: String,
@@ -1224,8 +1179,6 @@ pub struct MavenSettings {
pub threads: String,
/// Active Maven profiles (`-P`), comma-separated; empty leaves profiles to the POM/settings.
pub active_profiles: String,
/// Delegate IDE build/run actions to Maven: launches build with Maven (`javac`) instead of jdtls (ECJ).
pub delegate_build: bool,
}
impl MavenSettings {
@@ -1443,19 +1396,6 @@ impl Settings {
mod tests {
use super::*;
#[test]
fn removed_one_themes_read_as_islands_themes() {
let settings: Settings = serde_json::from_str(
r#"{"appearance":{"theme":"one-dark","preferredLightTheme":"one-light","preferredDarkTheme":"one-dark"}}"#,
)
.unwrap();
let a = settings.appearance;
assert_eq!((a.theme.as_str(), a.preferred_light_theme.as_str()), ("dark", "light"));
assert_eq!(a.preferred_dark_theme, "dark");
let kept: Settings = serde_json::from_str(r#"{"appearance":{"theme":"solarized-dark"}}"#).unwrap();
assert_eq!(kept.appearance.theme, "solarized-dark");
}
#[test]
fn window_close_settings_default_off_and_round_trip() {
for old in [r#"{}"#, r#"{"general":{"confirmExit":false}}"#] {
@@ -1743,12 +1683,10 @@ mod maven_linkage_tests {
fn maven_runner_jre_defaults_and_round_trips() {
let old: MavenSettings = serde_json::from_str(r#"{"home":"/opt/maven"}"#).unwrap();
assert!(old.runner_jre.is_empty());
assert!(!old.delegate_build);
assert!(MavenSettings::default().runner_jre.is_empty());
let settings = MavenSettings { runner_jre: "/opt/jdk-21".into(), delegate_build: true, ..old };
let settings = MavenSettings { runner_jre: "/opt/jdk-21".into(), ..old };
let encoded = serde_json::to_string(&settings).unwrap();
assert!(encoded.contains(r#""runnerJre":"/opt/jdk-21""#));
assert!(encoded.contains(r#""delegateBuild":true"#));
assert_eq!(serde_json::from_str::<MavenSettings>(&encoded).unwrap(), settings);
}
+2 -35
Переглянути файл
@@ -138,11 +138,8 @@ impl WorkspaceEdit {
continue;
}
*expected = expected.or(version);
// Repeats within one entry are intended (LSP inserts at one position
// apply in order); only an alias entry repeats what is already merged.
let merged = own.len();
for edit in edits {
if !own[..merged].contains(&edit) {
if !own.contains(&edit) {
own.push(edit);
}
}
@@ -280,40 +277,10 @@ pub struct OutlineItem {
/// Fuzzy-match highlight: indices (in chars of the displayed string) that matched.
pub type MatchIndices = Vec<u32>;
#[cfg(test)]
#[cfg(all(test, unix))]
mod workspace_edit_tests {
use super::*;
#[test]
fn repeated_edits_within_one_entry_survive_but_repeats_across_entries_do_not() {
let path = PathBuf::from("missing-dir/main.ts");
let at = |line, text: &str| TextEdit {
range: Range::new(Pos::new(line, 0), Pos::new(line, 0)),
new_text: text.into(),
};
let entry = |edits: Vec<TextEdit>| FileEdit::Edit { path: path.clone().into(), edits, version: None };
let (nl, brace, other) = (at(0, "\n"), at(1, "}"), at(2, "x"));
for (entries, expected) in [
(vec![vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![brace.clone(), nl.clone(), brace.clone()]], vec![brace.clone(), nl.clone(), brace.clone()]),
(vec![vec![nl.clone(), nl.clone()], vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![nl.clone()], vec![nl.clone(), other.clone()]], vec![nl.clone(), other.clone()]),
(
vec![vec![nl.clone()], vec![other.clone(), other.clone()]],
vec![nl.clone(), other.clone(), other.clone()],
),
] {
let mut workspace =
WorkspaceEdit { files: entries.clone().into_iter().map(entry).collect(), ..Default::default() };
workspace.deduplicate();
assert_eq!(workspace.files, [entry(expected.clone())], "{entries:?}");
let once = workspace.clone();
workspace.deduplicate();
assert_eq!(workspace, once, "{entries:?}");
}
}
#[cfg(unix)]
#[test]
fn aliases_are_one_edit_but_resource_operations_preserve_order() {
let dir = tempfile::tempdir().unwrap();
+2 -5
Переглянути файл
@@ -67,13 +67,10 @@ fn is_executable(path: &Path) -> bool {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
return metadata.permissions().mode() & 0o111 != 0;
metadata.permissions().mode() & 0o111 != 0
}
#[cfg(not(unix))]
{
let _ = &metadata;
true
}
true
}
/// [`search_path`]'s directories, in order and once each. Only absolute ones: a
-57
Переглянути файл
@@ -192,33 +192,6 @@ pub struct IssueLink {
pub link: String,
}
/// Per-level colours for console log lines: a line whose level is detected
/// (`ERROR`, `WARN`, …) takes the level's colour; ANSI colours the program
/// prints itself win over it. Values are CSS colours, usually scheme tokens.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct LogLevelColors {
pub enabled: bool,
pub error: String,
pub warn: String,
pub info: String,
pub debug: String,
pub trace: String,
}
impl Default for LogLevelColors {
fn default() -> Self {
Self {
enabled: false,
error: "var(--ansi-red)".into(),
warn: "var(--ansi-yellow)".into(),
info: "var(--ansi-green)".into(),
debug: "var(--ansi-cyan)".into(),
trace: "var(--ansi-magenta)".into(),
}
}
}
/// Build, Execution, Deployment › Console.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
@@ -230,8 +203,6 @@ pub struct ConsoleSettings {
pub encoding: String,
/// Consecutive lines containing any of these are folded.
pub fold_patterns: Vec<String>,
/// Log lines take the colour of their level (Run/Debug consoles).
pub log_colors: LogLevelColors,
}
impl Default for ConsoleSettings {
@@ -248,7 +219,6 @@ impl Default for ConsoleSettings {
"at node:internal/".into(),
"<frozen importlib._bootstrap".into(),
],
log_colors: LogLevelColors::default(),
}
}
}
@@ -442,30 +412,3 @@ impl Default for ExternalTool {
}
}
}
/// Tools › Agents: Command Execution Policy (whitelist and blacklist patterns).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentSecuritySettings {
/// Commands or patterns (`git status*`, `pwd`) automatically approved without prompting.
pub allowed_commands: Vec<String>,
/// Commands or patterns (`rm -rf *`, `mkfs*`) automatically rejected.
pub blocked_commands: Vec<String>,
}
impl Default for AgentSecuritySettings {
fn default() -> Self {
Self {
allowed_commands: Vec::new(),
blocked_commands: vec![
"rm -rf *".into(),
"rm -rf /".into(),
"mkfs*".into(),
"dd *".into(),
":(){ :|:& };:".into(),
"shutdown*".into(),
"reboot*".into(),
],
}
}
}
-5
Переглянути файл
@@ -11,17 +11,12 @@ tempfile.workspace = true
ide-core.workspace = true
ide-language.workspace = true
ide-jsonrpc.workspace = true
ide-jvm-attach.workspace = true
ide-jvm-inspect.workspace = true
ide-jdwp.workspace = true
ide-heap.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
tokio = { workspace = true, features = ["net"] }
parking_lot.workspace = true
memchr = "2"
percent-encoding = "2"
portable-pty.workspace = true
tracing.workspace = true
+3 -19
Переглянути файл
@@ -72,13 +72,6 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install Kotlin Debug Adapter in Settings › Languages & Frameworks › Language Servers",
),
// Inside the language server: reached through `debugServer`, never started.
"kotlin-lsp" => cfg(
"kotlin-lsp",
&[],
"tcp",
"Install JetBrains Kotlin LSP in Settings › Languages & Frameworks › Language Servers",
),
// Windows installs `python.exe`; `python3.exe` there is at most a Store alias.
"debugpy" => cfg(
if cfg!(windows) { "python" } else { "python3" },
@@ -105,12 +98,9 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install netcoredbg (.NET debugger) in Settings › Languages & Frameworks › Language Servers",
),
"lldb-dap" => cfg(
"lldb-dap",
&[],
"stdio",
"Install LLVM's lldb-dap (brew install llvm), or on macOS use the one found by xcrun -f lldb-dap",
),
"lldb-dap" => {
cfg("lldb-dap", &[], "stdio", "Install LLVM's lldb-dap (brew install llvm; on macOS `xcrun -f lldb-dap`)")
}
// GDB's own DAP interpreter, stdio transport. The DAP support it needs shipped in GDB 14
// (`prepare_gdb` refuses older ones with the required version in the message).
"gdb" => cfg(
@@ -127,12 +117,6 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"tcp",
"Install Microsoft java-debug (Settings › Languages & Frameworks › Language Servers)",
),
"bun-dap" => cfg(
"bun-dap",
&["${port}", "127.0.0.1"],
"tcp",
"To install bun-dap: git clone https://github.com/morf-bls/bun-dap, run bun install && bun run build, then place dist/bun-dap in your PATH",
),
_ => return None,
})
}
-12
Переглянути файл
@@ -342,16 +342,4 @@ mod tests {
assert_eq!(error_message(&msg), "cannot find a.py");
assert_eq!(error_message(&json!({"message": "nope"})), "nope");
}
#[test]
fn a_failed_launch_response_keeps_delves_too_old_error_verbatim() {
// The UI's repair offer keys off this exact wording (`missingAdapter`), so the
// conversion to the IPC error must not fold it into a generic message.
let message = "Failed to launch /p/api: Version of Delve is too old for Go version go1.27.0 \
(maximum supported version 1.26, suppress this error with --check-go-version=false)";
let e = ide_core::IdeError::from(DapError::Failed { command: "launch".into(), message: message.into() });
assert_eq!(e.kind, ide_core::error::ErrorKind::InvalidInput);
assert_eq!(e.message, format!("launch failed: {message}"));
assert!(e.message.contains("Version of Delve is too old for Go version go1.27.0"));
}
}
+12 -99
Переглянути файл
@@ -12,35 +12,6 @@ use tokio::net::{TcpListener, TcpStream};
/// How long the relay waits for java-debug to attach once the JVM is connected.
const ATTACH_TIMEOUT: Duration = Duration::from_secs(60);
/// Loopback ports relaying a connected JVM, while their relay runs.
static RELAY_PORTS: std::sync::Mutex<Vec<u16>> = std::sync::Mutex::new(Vec::new());
fn relay_ports() -> std::sync::MutexGuard<'static, Vec<u16>> {
RELAY_PORTS.lock().unwrap_or_else(std::sync::PoisonError::into_inner)
}
/// Whether `port` is a relay of [`listen_for_jvm`]: a session attaching to it debugs a JVM that
/// connected to the IDE, so the JVM has no JDWP listener the memory view could find it by.
pub(crate) fn is_listen_relay(port: u16) -> bool {
relay_ports().contains(&port)
}
/// Registers a relay port for as long as it lives.
struct RelayPort(u16);
impl RelayPort {
fn new(port: u16) -> Self {
relay_ports().push(port);
Self(port)
}
}
impl Drop for RelayPort {
fn drop(&mut self) {
relay_ports().retain(|&p| p != self.0);
}
}
/// The address to listen on for `host`: the interface it names, or every interface when
/// it is not one of this machine's (a DNS name or a NAT address the JVM connects to).
async fn bind(host: &str, port: u16) -> IdeResult<TcpListener> {
@@ -69,11 +40,11 @@ pub async fn listen_for_jvm(host: &str, port: u16, cancel: CancellationToken) ->
let _ = jvm.set_nodelay(true);
let relay = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = relay.local_addr()?.port();
tokio::spawn(relay_one(relay, jvm, RelayPort::new(local)));
tokio::spawn(relay_one(relay, jvm));
Ok(local)
}
async fn relay_one(relay: TcpListener, mut jvm: TcpStream, _registered: RelayPort) {
async fn relay_one(relay: TcpListener, mut jvm: TcpStream) {
let debugger = match tokio::time::timeout(ATTACH_TIMEOUT, relay.accept()).await {
Ok(Ok((s, _))) => s,
_ => return,
@@ -87,64 +58,22 @@ async fn relay_one(relay: TcpListener, mut jvm: TcpStream, _registered: RelayPor
}
#[cfg(test)]
pub(crate) mod tests {
mod tests {
use super::*;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// A remote JVM (`server=n`) connected to [`listen_for_jvm`], and the relay port. A port
/// picked by binding to 0 can be taken by a parallel test before `listen_for_jvm` binds it,
/// so a failed listen is retried on a fresh one.
pub(crate) async fn connected_jvm() -> (TcpStream, u16) {
for _ in 0..20 {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
if let Some(connected) = jvm_on(port).await {
return connected;
}
}
panic!("listen_for_jvm never accepted a JVM");
}
/// One attempt of [`connected_jvm`]; `None` when the port turned out to be taken.
async fn jvm_on(port: u16) -> Option<(TcpStream, u16)> {
let mut waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
let mut jvm = None;
for _ in 0..100 {
// A listen that failed to bind leaves nothing to connect to.
if waiting.is_finished() {
break;
}
// macOS does not refuse a connect to a port a socket holds without listening: the
// SYN goes unanswered until the connect times out.
let connect = TcpStream::connect(("127.0.0.1", port));
if let Ok(Ok(s)) = tokio::time::timeout(Duration::from_secs(1), connect).await {
jvm = Some(s);
break;
}
tokio::time::sleep(Duration::from_millis(20)).await;
}
// A connection to another test's listener never reaches `listen_for_jvm`.
let relay = tokio::time::timeout(Duration::from_secs(2), &mut waiting).await;
if let (Some(jvm), Ok(Ok(Ok(relay)))) = (jvm, relay) {
return Some((jvm, relay));
}
waiting.abort();
None
}
#[tokio::test]
async fn a_port_another_socket_holds_is_given_up() {
// Bound but not listening, as a parallel test's outgoing connection holds its local port.
let held = tokio::net::TcpSocket::new_v4().unwrap();
held.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = held.local_addr().unwrap().port();
let attempt = tokio::time::timeout(Duration::from_secs(5), jvm_on(port)).await;
assert!(matches!(attempt, Ok(None)), "an attempt on a taken port gives up instead of waiting forever");
}
#[tokio::test]
async fn relays_the_jdwp_handshake_both_ways() {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
let waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
// The remote JVM (`server=n`) connects to the IDE.
let (mut jvm, local) = connected_jvm().await;
let mut jvm = loop {
if let Ok(s) = TcpStream::connect(("127.0.0.1", port)).await {
break s;
}
tokio::time::sleep(Duration::from_millis(20)).await;
};
let local = waiting.await.unwrap().unwrap();
// java-debug attaches to the relay and starts the handshake.
let mut debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
debugger.write_all(b"JDWP-Handshake").await.unwrap();
@@ -156,22 +85,6 @@ pub(crate) mod tests {
assert_eq!(&buf, b"JDWP-Handshake");
}
#[tokio::test]
async fn relay_port_is_known_while_the_relay_runs() {
let (jvm, local) = connected_jvm().await;
assert!(is_listen_relay(local));
let debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
drop((jvm, debugger));
for _ in 0..200 {
if !is_listen_relay(local) {
return;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("relay port {local} still registered after the relay ended");
}
#[tokio::test]
async fn waiting_is_cancellable() {
let cancel = CancellationToken::new();
-92
Переглянути файл
@@ -1,92 +0,0 @@
//! The class table counted by the debugger (JDWP `InstanceCounts`) where no histogram can be
//! taken: JDK 8-16 while suspended, Windows, a remote JVM. Only while suspended, and slow on a
//! large heap (seconds on JDK 17+).
use super::inspect::Call;
use super::snapshot::diff_rows;
use super::{ClassCounts, JvmMemoryResult, JvmMemorySnapshot, JvmMemorySource, JvmMemoryState, JvmMemoryUnavailable};
use crate::session::Session;
use ide_core::IdeResult;
use ide_jvm_attach::HistogramRow;
use ide_jvm_inspect::ClassCount;
/// Reasons the debugger may be able to work around; the others (the program exited, the JVM
/// did not answer) would fail for it too.
pub(super) fn falls_back(reason: &JvmMemoryUnavailable) -> bool {
use JvmMemoryUnavailable as U;
matches!(
reason,
U::UnsafeWhileSuspended { .. }
| U::UnsupportedPlatform
| U::AttachedJvmNotFound
| U::ListenerNotRunning
| U::ListenModeNotSupported
| U::NoJvmProcess
)
}
/// `histogram` unless it is unavailable for a reason `session`'s inspector can work around.
pub(super) async fn or_debugger_counts(
state: &mut JvmMemoryState,
histogram: Result<JvmMemorySnapshot, JvmMemoryUnavailable>,
session: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let reason = match histogram {
Ok(snapshot) => return Ok(JvmMemoryResult::Ready { snapshot }),
Err(reason) => reason,
};
let unavailable = |can_count_when_paused| Ok(JvmMemoryResult::Unavailable { reason, can_count_when_paused });
let Some(session) = session.filter(|_| falls_back(&reason)) else {
return unavailable(false);
};
let call = match Call::begin(session, None).await {
Ok(call) => call,
Err(super::JvmInspectUnavailable::NotSuspended) => return unavailable(true),
Err(_) => return unavailable(false),
};
let started = std::time::Instant::now();
let result = call.run(call.inspector.class_counts(&call.target)).await;
match call.finish(result) {
Ok(counts) => {
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let previous = state.baseline(JvmMemorySource::Debugger);
let (snapshot, counts) = debugger_snapshot(counts, previous.as_deref(), elapsed_ms);
state.remember(JvmMemorySource::Debugger, counts);
Ok(JvmMemoryResult::Ready { snapshot })
}
Err(e) => {
tracing::debug!("debugger class counts: {e}");
let running = !session.suspended.get();
unavailable(running && session.jvm_inspector.lock().is_some())
}
}
}
/// Classes without instances are left out; most instances first.
pub(super) fn debugger_snapshot(
counts: Vec<ClassCount>,
previous: Option<&ClassCounts>,
elapsed_ms: u32,
) -> (JvmMemorySnapshot, ClassCounts) {
let rows = counts
.into_iter()
.filter(|c| c.instances > 0)
.map(|c| HistogramRow { name: c.name, module: None, instances: c.instances, bytes: 0 })
.collect();
let (mut rows, counts) = diff_rows(rows, previous);
for row in &mut rows {
row.delta_bytes = None;
}
rows.sort_by(|a, b| b.instances.cmp(&a.instances).then_with(|| a.name.cmp(&b.name)));
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: 0,
rows,
java_version: String::new(),
taken_while_suspended: true,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Debugger,
};
(snapshot, counts)
}
-293
Переглянути файл
@@ -1,293 +0,0 @@
//! A heap dump of a debugged JVM, written over its attach listener into a snapshot directory.
use super::snapshot::{AttachProbe, JdkProbe, attach_failure, gone, pick_jvm_pid};
use super::{
JvmMemoryState, JvmMemoryUnavailable, JvmTarget, histogram_pid, keep_found, lookup, resolve_jvm,
safe_while_suspended, session_target,
};
use crate::session::Session;
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_jvm_attach::{AttachError, HeapDumpReply};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::sync::Arc;
const DUMP_FILE: &str = "dump.hprof";
/// For a refusal whose reply holds no reason: empty, or only the line announcing the dump.
const NO_REASON: &str = "the JVM wrote no heap dump and gave no reason";
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmHeapUnavailable {
/// The reason a class table could not be read applies to the dump too.
Memory {
reason: JvmMemoryUnavailable,
},
/// Bytes.
NotEnoughDiskSpace {
needed: u64,
available: u64,
},
DumpFailed {
message: String,
},
DumpTimedOut {
seconds: u32,
},
}
#[derive(Clone, Debug, PartialEq)]
pub struct JvmHeapDump {
pub path: PathBuf,
/// The class histogram taken just before the dump.
pub histogram: Vec<ide_heap::HistogramEntry>,
/// Raw `VM.flags`; empty when the JVM did not answer.
pub vm_flags: String,
/// 0 when unknown.
pub java_major: u32,
/// Empty when unknown.
pub java_version: String,
pub pid: u32,
pub taken_while_suspended: bool,
pub dump_ms: u32,
}
#[derive(Clone, Debug, PartialEq)]
pub enum JvmHeapOutcome {
Ready(JvmHeapDump),
/// The snapshot directory has been deleted.
Unavailable(JvmHeapUnavailable),
/// The snapshot directory has been deleted.
Cancelled,
}
pub(crate) async fn session_heap_dump(
session: Arc<Session>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let launched = histogram_pid(&session);
let capture = {
let (dir, cancel) = (dir.clone(), cancel.clone());
async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
let stopped = session.clone();
let suspended = move || capture_suspended(&stopped);
heap_dump_with(&session.jvm_memory, suspended, probe, target, dir, cancel).await
}
};
detached(capture, dir, cancel).await
}
/// The JVM can be suspended before the adapter reports the stop, so a session with a JDWP link
/// also counts the link's view.
pub(super) fn capture_suspended(session: &Session) -> bool {
session.suspended.get()
|| session
.jdwp_link
.lock()
.as_ref()
.is_some_and(|link| matches!(link.state(), ide_jdwp::LinkState::Suspended { .. }))
}
/// Runs `capture` in a task of its own, so a caller that stops waiting cannot release the
/// session's lock while the JVM is still writing the dump and let a class-table request run
/// next to it. On `cancel` returns at once and deletes `dir`.
pub(super) async fn detached(
capture: impl Future<Output = IdeResult<JvmHeapOutcome>> + Send + 'static,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let task = tokio::spawn(capture);
tokio::select! {
joined = task => joined.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?,
() = cancel.cancelled() => {
discard_later(dir).await;
Ok(JvmHeapOutcome::Cancelled)
}
}
}
/// Holds `state`'s lock throughout, as a class-table request does; `suspended` and then `probe`
/// are read once it is held, and `suspended` again by [`take_heap_dump`]. Anything but `Ready`
/// deletes `dir`.
pub(super) async fn heap_dump_with(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
// The capture's histogram never becomes the Memory view's baseline, so the state is unused.
let _turn = state.lock().await;
let outcome = capture(suspended, probe, target, dir.clone(), cancel).await;
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard_later(dir).await;
}
outcome
}
async fn capture(
suspended_now: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let suspended = suspended_now();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(memory(reason)),
};
let probe = probe(suspended);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_heap_dump(&*probe, &tree, suspended, &suspended_now, &dir, &cancel)),
Err(reason) => (None, Ok(memory(reason))),
})
.await
.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?;
keep_found(target, found);
outcome
}
/// `tree` starts with the session's reported pid, followed by its descendants. `suspended` is
/// the stop read when the capture began; `suspended_now` reads it again. Anything but `Ready`
/// deletes `dir`.
pub(crate) fn take_heap_dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let outcome = dump(probe, tree, suspended, suspended_now, dir, cancel);
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard(dir);
}
outcome
}
fn dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
use JvmHeapUnavailable as H;
use JvmMemoryUnavailable as U;
let Some(root) = tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(memory(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(tree, |p| probe.has_listener(p)) else {
return Ok(memory(gone(probe, root)));
};
let version = probe.version(jvm);
let java_major = version.as_ref().map_or(0, |v| v.major);
let safe = safe_while_suspended(version.as_ref());
let unsafe_stop = || Ok(memory(U::UnsafeWhileSuspended { java_major }));
if suspended && !safe {
return unsafe_stop();
}
// The program may stop during the histogram, and a stopped JDK below 17 hangs on the next
// request until it resumes.
let stopped_since = || !safe && suspended_now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(memory),
};
let histogram: Vec<ide_heap::HistogramEntry> = ide_jvm_attach::parse_class_histogram(&text)
.rows
.into_iter()
.map(|r| ide_heap::HistogramEntry { name: r.name, instances: r.instances, bytes: r.bytes })
.collect();
let (instances, bytes) =
histogram.iter().fold((0u64, 0u64), |(i, b), e| (i.saturating_add(e.instances), b.saturating_add(e.bytes)));
let needed = ide_heap::capture_need(bytes, instances);
if let Some(available) = probe.free_bytes(dir)
&& available < needed
{
return Ok(JvmHeapOutcome::Unavailable(H::NotEnoughDiskSpace { needed, available }));
}
if stopped_since() {
return unsafe_stop();
}
// Without flags the index assumes the default object layout.
let vm_flags = probe.vm_flags(jvm).unwrap_or_else(|e| {
tracing::debug!("VM.flags of JVM {jvm}: {e}");
String::new()
});
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
if stopped_since() {
return unsafe_stop();
}
let path = dir.join(DUMP_FILE);
let timeout = ide_jvm_attach::dump_timeout(bytes);
let started = std::time::Instant::now();
let reply = probe.heap_dump(jvm, &path, timeout);
let dump_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let failed = |message: String| Ok(JvmHeapOutcome::Unavailable(H::DumpFailed { message }));
match reply {
Ok(HeapDumpReply::Created { .. }) => {
if !std::fs::metadata(&path).is_ok_and(|m| m.is_file() && m.len() > 0) {
return failed(format!("the JVM reported a heap dump, but {} is missing or empty", path.display()));
}
Ok(JvmHeapOutcome::Ready(JvmHeapDump {
path,
histogram,
vm_flags,
java_major,
java_version: version.map(|v| v.text).unwrap_or_default(),
pid: jvm,
taken_while_suspended: suspended,
dump_ms,
}))
}
Ok(HeapDumpReply::Refused { message }) if message.is_empty() || message.starts_with("Dumping heap to") => {
failed(NO_REASON.into())
}
Ok(HeapDumpReply::Refused { message }) => failed(message),
Err(AttachError::Timeout) => {
let seconds = u32::try_from(timeout.as_secs()).unwrap_or(u32::MAX);
Ok(JvmHeapOutcome::Unavailable(H::DumpTimedOut { seconds }))
}
// A running JDK rejects an argument with a non-zero result code.
Err(AttachError::Command { message, .. }) => failed(message),
Err(e) => attach_failure(probe, jvm, e, "heap dump").map(memory),
}
}
fn memory(reason: JvmMemoryUnavailable) -> JvmHeapOutcome {
JvmHeapOutcome::Unavailable(JvmHeapUnavailable::Memory { reason })
}
fn discard(dir: &Path) {
if let Err(e) = std::fs::remove_dir_all(dir)
&& e.kind() != std::io::ErrorKind::NotFound
{
tracing::warn!("could not delete heap snapshot directory {}: {e}", dir.display());
}
}
/// A dump may be large, so it is deleted off the async threads.
async fn discard_later(dir: PathBuf) {
let _ = tokio::task::spawn_blocking(move || discard(&dir)).await;
}
-251
Переглянути файл
@@ -1,251 +0,0 @@
//! A suspended JVM's instances, fields and referrers, read through the session's
//! [`JvmInspector`]. An object is only valid within the stop it was read in.
use super::launch_pid;
use crate::session::Session;
use ide_core::event::{DebugEvent, JvmBackendState};
use ide_core::{IdeError, IdeResult};
use ide_jvm_inspect::{
DEFAULT_REFERRERS, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, MAX_INSTANCES,
MAX_REFERRERS, ObjectId, VmTarget,
};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::sync::Arc;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmObjectRef {
/// The `generation` of the [`JvmInstancesResult`] the object came from.
pub generation: u64,
pub id: ObjectId,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInspectUnavailable {
/// The session has no inspector: not a JVM session, not ready yet, or turned off.
NoBackend,
/// The debug adapter lacks what the inspector needs; turned off for the session.
Unsupported {
detail: String,
},
NotSuspended,
/// The object is from an earlier stop, or the program stopped or resumed meanwhile.
Stale,
Collected,
ClassNotLoaded,
VmNotFound,
/// Transient: ask again.
TimedOut,
BackendClosed,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInstancesResult {
Ready {
/// Goes into the [`JvmObjectRef`] of each instance.
generation: u64,
instances: Vec<JvmValue>,
/// The limit was reached; there may be more.
truncated: bool,
},
Unavailable {
reason: JvmInspectUnavailable,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmFieldsResult {
Ready { fields: Vec<JvmField> },
Unavailable { reason: JvmInspectUnavailable },
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmReferrersResult {
Ready { referrers: Vec<JvmReferrer>, truncated: bool },
Unavailable { reason: JvmInspectUnavailable },
}
/// The session's JVM for the inspector: java-debug's VM is found by the launched program's pid
/// or by the port of an attach (the thread lists of two JVMs can be identical).
pub(crate) fn vm_target(s: &Session) -> VmTarget {
let (pid, attach) = if s.request == "launch" {
(u32::try_from(launch_pid(s)).ok().filter(|&p| p > 0), None)
} else {
(None, s.attach_target.get())
};
VmTarget {
pid,
attach_host: attach.map(|(host, _)| host.clone()).filter(|h| !h.is_empty()),
attach_port: attach.map(|&(_, port)| port),
generation: s.suspended.ticket(),
}
}
pub(crate) fn set_inspector(s: &Session, inspector: Arc<dyn JvmInspector>) {
*s.jvm_inspector.lock() = Some(inspector);
s.emit(DebugEvent::Capabilities);
}
pub(crate) async fn instances(s: &Session, class_name: &str, limit: u32) -> IdeResult<JvmInstancesResult> {
let limit = limit.clamp(1, MAX_INSTANCES);
let call = match Call::begin(s, None).await {
Ok(call) => call,
Err(reason) => return Ok(JvmInstancesResult::Unavailable { reason }),
};
let generation = call.target.generation;
let result = call.run(call.inspector.instances(&call.target, class_name, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(instances) => {
JvmInstancesResult::Ready { generation, truncated: instances.len() >= limit as usize, instances }
}
Err(reason) => JvmInstancesResult::Unavailable { reason },
})
}
pub(crate) async fn fields(s: &Session, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmFieldsResult::Unavailable { reason }),
};
let result = call.run(call.inspector.fields(&call.target, object.id)).await;
Ok(match settle(call.finish(result))? {
Ok(fields) => JvmFieldsResult::Ready { fields },
Err(reason) => JvmFieldsResult::Unavailable { reason },
})
}
pub(crate) async fn referrers(s: &Session, object: JvmObjectRef, limit: u32) -> IdeResult<JvmReferrersResult> {
let limit = if limit == 0 { DEFAULT_REFERRERS } else { limit.min(MAX_REFERRERS) };
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmReferrersResult::Unavailable { reason }),
};
let result = call.run(call.inspector.referrers(&call.target, object.id, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(referrers) => JvmReferrersResult::Ready { truncated: referrers.len() >= limit as usize, referrers },
Err(reason) => JvmReferrersResult::Unavailable { reason },
})
}
/// One inspector request. Holds the session's inspect lock and a clone of its inspector, which
/// [`Call::run`] lets go of when the session ends.
pub(super) struct Call<'s> {
session: &'s Session,
pub(super) inspector: Arc<dyn JvmInspector>,
pub(super) target: VmTarget,
_turn: tokio::sync::MutexGuard<'s, ()>,
}
impl<'s> Call<'s> {
/// `object_generation`: the stop the request's object was read in.
pub(super) async fn begin(
s: &'s Session,
object_generation: Option<u64>,
) -> Result<Call<'s>, JvmInspectUnavailable> {
// Checked under the lock: a request queued behind another must see the state it runs in.
let turn = s.inspect_lock.lock().await;
let Some(inspector) = s.jvm_inspector.lock().clone() else {
return Err(JvmInspectUnavailable::NoBackend);
};
if !s.suspended.get() {
return Err(JvmInspectUnavailable::NotSuspended);
}
let target = vm_target(s);
if object_generation.is_some_and(|g| g != target.generation) {
return Err(JvmInspectUnavailable::Stale);
}
Ok(Call { session: s, inspector, target, _turn: turn })
}
pub(super) async fn run<T>(&self, request: InspectFuture<'_, T>) -> Result<T, InspectError> {
tokio::select! {
result = request => result,
() = self.session.ended.cancelled() => Err(InspectError::Closed),
}
}
/// The result unless the program stopped or resumed while it was read: a request of several
/// commands can straddle a stop. `Unsupported` turns the backend off.
pub(super) fn finish<T>(self, result: Result<T, InspectError>) -> Result<T, InspectError> {
let s = self.session;
match result {
Err(InspectError::Unsupported(detail)) => {
turn_off(s, &self.inspector, &detail);
Err(InspectError::Unsupported(detail))
}
_ if !s.suspended.get() || s.suspended.ticket() != self.target.generation => Err(InspectError::Stale),
result => result,
}
}
}
/// An adapter that lacks what the inspector needs will not gain it during the session.
fn turn_off(s: &Session, inspector: &Arc<dyn JvmInspector>, detail: &str) {
let removed = {
let mut slot = s.jvm_inspector.lock();
// Not one installed since this request began.
let current = slot.as_ref().is_some_and(|i| std::ptr::addr_eq(Arc::as_ptr(i), Arc::as_ptr(inspector)));
current && slot.take().is_some()
};
if removed {
s.emit(DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail: detail.to_owned() } });
s.emit(DebugEvent::Capabilities);
}
}
/// A failure the inspector could not classify is an error, not a reason shown to the user.
fn settle<T>(result: Result<T, InspectError>) -> IdeResult<Result<T, JvmInspectUnavailable>> {
use JvmInspectUnavailable as U;
let error = match result {
Ok(value) => return Ok(Ok(value)),
Err(error) => error,
};
Ok(Err(match error {
InspectError::NotSuspended => U::NotSuspended,
InspectError::Stale => U::Stale,
InspectError::Collected => U::Collected,
InspectError::ClassNotLoaded => U::ClassNotLoaded,
InspectError::VmNotFound => U::VmNotFound,
InspectError::Unsupported(detail) => U::Unsupported { detail },
// Also the Kotlin link's `Busy`: the backend stays.
InspectError::TimedOut => U::TimedOut,
InspectError::Closed => U::BackendClosed,
InspectError::Failed(message) => return Err(IdeError::internal(format!("JVM inspector: {message}"))),
}))
}
/// Reports a Kotlin link that stops inspecting or closes. Holds neither the link nor the
/// session: it ends once the session drops the link, and never drops it itself (after
/// `VirtualMachine.Dispose` the link is closed but still forwarding the reply).
pub(crate) fn watch_link(s: &Arc<Session>, link: &ide_jdwp::JdwpLink) {
let mut states = link.subscribe();
let session = Arc::downgrade(s);
tokio::spawn(async move {
let mut reported = None;
loop {
let state = match &*states.borrow_and_update() {
ide_jdwp::LinkState::Degraded(detail) => Some(JvmBackendState::Degraded { detail: detail.clone() }),
ide_jdwp::LinkState::Closed => Some(JvmBackendState::Closed),
_ => None,
};
if let Some(state) = state
&& reported.as_ref() != Some(&state)
{
let Some(s) = session.upgrade() else { return };
if s.ended.is_cancelled() || s.terminated.load(std::sync::atomic::Ordering::Acquire) {
return;
}
s.emit(DebugEvent::JvmBackend { state: state.clone() });
reported = Some(state);
}
if states.changed().await.is_err() {
return;
}
}
});
}
-338
Переглянути файл
@@ -1,338 +0,0 @@
//! Class-table snapshots of a debugged JVM, read over the HotSpot attach listener or, where
//! that is unavailable, counted by the debugger; and the JVM's objects through the debugger.
mod fallback;
mod heap;
mod inspect;
mod scan;
mod snapshot;
#[cfg(test)]
pub(crate) mod test_support;
#[cfg(test)]
mod tests;
use crate::session::Session;
use crate::session::Suspension;
use ide_core::IdeResult;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicI64, Ordering};
use std::time::Duration;
pub use heap::*;
pub use inspect::{JvmFieldsResult, JvmInspectUnavailable, JvmInstancesResult, JvmObjectRef, JvmReferrersResult};
pub(crate) use inspect::{fields, instances, referrers, set_inspector, watch_link};
#[cfg(all(test, unix))]
pub(crate) use scan::watch_for_attached_jvm;
pub(crate) use scan::{JvmFinder, attach_target, find_attached_jvm, find_local_jvm, local_jdwp_port};
use snapshot::{AttachProbe, JdkProbe, take_snapshot};
pub const JVM_MEMORY_ADAPTERS: &[&str] = &["java-debug", crate::kotlin::ADAPTER, crate::kotlin_lsp::ADAPTER];
pub const HISTOGRAM_TIMEOUT: Duration = Duration::from_secs(10);
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmClassRow {
pub name: String,
pub module: Option<String>,
pub instances: u64,
pub bytes: u64,
/// Change since the previous snapshot; `None` on the first one and for ambiguous rows.
pub delta_instances: Option<i64>,
pub delta_bytes: Option<i64>,
/// Several classes share this name (different class loaders), so it has no delta.
pub ambiguous: bool,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmMemorySnapshot {
/// Largest `bytes` first.
pub rows: Vec<JvmClassRow>,
pub total_instances: u64,
pub total_bytes: u64,
/// Empty when unknown.
pub java_version: String,
pub taken_while_suspended: bool,
/// A previous snapshot from the same source exists; deltas are against it.
pub has_baseline: bool,
/// Wall time of the histogram or count request.
pub elapsed_ms: u32,
pub source: JvmMemorySource,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum JvmMemorySource {
/// `GC.class_histogram -all`: sizes, and objects not yet collected.
Histogram,
/// JDWP `InstanceCounts`: reachable objects only, no sizes (`bytes` is 0).
Debugger,
}
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryUnavailable {
/// An attach session whose JVM is remote, or local but not found by its JDWP port.
AttachedJvmNotFound,
/// An attach through the IDE's listen-mode relay: the JVM connected to the IDE, so it has
/// no JDWP listener to be found by.
ListenModeNotSupported,
UnsupportedPlatform,
NoJvmProcess,
ListenerNotRunning,
/// `java_major` is 0 when the version is unknown.
UnsafeWhileSuspended {
java_major: u32,
},
ProgramExited,
TimedOut,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryResult {
Ready {
snapshot: JvmMemorySnapshot,
},
Unavailable {
reason: JvmMemoryUnavailable,
/// The debugger could count the classes if the program were paused.
can_count_when_paused: bool,
},
}
/// Instances and bytes per class name, for the next snapshot's deltas; `None` for a name
/// several classes shared, so it gets no delta next time either.
type ClassCounts = HashMap<String, Option<(u64, u64)>>;
/// Per-session baseline; its mutex also keeps one class-table request per session at a time.
#[derive(Default)]
pub struct JvmMemoryState {
/// Shared with the blocking task, so a task that panics leaves it in place.
previous: Option<(JvmMemorySource, Arc<ClassCounts>)>,
}
impl JvmMemoryState {
/// The previous snapshot's counts if it came from `source`: counts from the other source
/// are not comparable (only reachable objects versus all of them).
pub(crate) fn baseline(&self, source: JvmMemorySource) -> Option<Arc<ClassCounts>> {
self.previous.as_ref().filter(|(s, _)| *s == source).map(|(_, counts)| counts.clone())
}
pub(crate) fn remember(&mut self, source: JvmMemorySource, counts: ClassCounts) {
self.previous = Some((source, Arc::new(counts)));
}
}
pub(crate) async fn session_classes(session: Arc<Session>) -> IdeResult<JvmMemoryResult> {
until_located(&session).await;
let launched = histogram_pid(&session);
// Detached, so a caller that gives up cannot release the lock while the JVM is still
// answering and let a second histogram request run next to it.
tokio::spawn(async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
snapshot(&session.jvm_memory, &session.suspended, probe, target, Some(&session)).await
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?
}
/// Bounds the wait of [`until_located`]; `configure` has its own timeouts, this only guards them.
const LOCATE_WAIT: Duration = Duration::from_secs(30);
/// Until `configure` has recorded where `session`'s JVM is. The Memory view asks as soon as the
/// session appears; looked up earlier, an attached JVM is reported as missing.
async fn until_located(session: &Session) {
let mut located = session.jvm_located.subscribe();
tokio::select! {
_ = located.wait_for(|v| *v) => {}
() = session.ended.cancelled() => {}
() = tokio::time::sleep(LOCATE_WAIT) => {}
}
}
/// `launched`: [`histogram_pid`] of `session`.
fn session_target(session: &Session, launched: Result<Option<i32>, JvmMemoryUnavailable>) -> JvmTarget<'_> {
match launched {
Ok(Some(pid)) => JvmTarget::Launched(pid),
Ok(None) => JvmTarget::Attached { pid: &session.jvm_pid, jdwp_port: session.jdwp_port.get().copied() },
Err(reason) => JvmTarget::Unavailable(reason),
}
}
/// The pid of a launch session's program (`None` for an attach session), or why no histogram
/// can be taken.
fn histogram_pid(session: &Session) -> Result<Option<i32>, JvmMemoryUnavailable> {
if !JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str()) {
return Err(JvmMemoryUnavailable::NoJvmProcess);
}
if !cfg!(unix) {
return Err(JvmMemoryUnavailable::UnsupportedPlatform);
}
if session.request == "launch" {
return match i32::try_from(launch_pid(session)).ok().filter(|&p| p > 0) {
Some(pid) => Ok(Some(pid)),
None => Err(JvmMemoryUnavailable::NoJvmProcess),
};
}
if session.jvm_pid.load(Ordering::Acquire) <= 0 {
if session.terminated.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ProgramExited);
}
if session.listen_relay.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ListenModeNotSupported);
}
}
Ok(None)
}
/// The JVM a launch session started: the one the IDE started for an app-launched (Kotlin)
/// session, else the program the adapter reported; 0 when neither is known.
pub(crate) fn launch_pid(session: &Session) -> i64 {
match session.jvm_pid.load(Ordering::Acquire) {
p if p > 0 => p,
_ => session.debuggee_pid.load(Ordering::Acquire),
}
}
#[derive(Clone, Copy)]
pub(crate) enum JvmTarget<'a> {
/// The program a launch session started: its pid and everything it started.
Launched(i32),
/// The JVM of an attach session. While `pid` is 0 it is looked up by `jdwp_port`, which is
/// `None` for a remote JVM.
Attached { pid: &'a AtomicI64, jdwp_port: Option<u16> },
/// Known without asking any JVM: no histogram can be taken, for this reason.
Unavailable(JvmMemoryUnavailable),
}
enum Lookup {
Tree(i32),
Known(i32),
Scan(u16),
}
/// `probe` is built from the suspension read once the lock is held: a request queued behind
/// another must not decide on a state from before the program hit a breakpoint. Where no
/// histogram can be taken, `debugger` (the session) may count the classes instead.
pub(crate) async fn snapshot(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
debugger: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let mut state = state.lock().await;
let outcome = histogram(&mut state, suspended, probe, target).await?;
fallback::or_debugger_counts(&mut state, outcome, debugger).await
}
async fn histogram(
state: &mut JvmMemoryState,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
) -> IdeResult<Result<JvmMemorySnapshot, JvmMemoryUnavailable>> {
let suspended = suspended.get();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(Err(reason)),
};
let probe = probe(suspended);
let previous = state.baseline(JvmMemorySource::Histogram);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_snapshot(&*probe, &tree, suspended, previous.as_deref())),
Err(reason) => (None, Ok(Err(reason))),
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?;
keep_found(target, found);
Ok(outcome?.map(|(snapshot, counts)| {
state.remember(JvmMemorySource::Histogram, counts);
snapshot
}))
}
/// Where to look for `target`'s JVM, or why there is none to ask.
fn lookup(target: JvmTarget<'_>) -> Result<Lookup, JvmMemoryUnavailable> {
match target {
JvmTarget::Launched(pid) => Ok(Lookup::Tree(pid)),
JvmTarget::Attached { pid, jdwp_port } => match i32::try_from(pid.load(Ordering::Acquire)) {
Ok(p) if p > 0 => Ok(Lookup::Known(p)),
_ => jdwp_port.map(Lookup::Scan).ok_or(JvmMemoryUnavailable::AttachedJvmNotFound),
},
JvmTarget::Unavailable(reason) => Err(reason),
}
}
/// Blocking. The pids to pick the JVM from, and the pid a scan found (for [`keep_found`]).
fn resolve_jvm(
lookup: Lookup,
probe: &dyn AttachProbe,
suspended: bool,
) -> Result<(Vec<i32>, Option<i32>), JvmMemoryUnavailable> {
match lookup {
Lookup::Tree(pid) => Ok((pid_tree(pid), None)),
Lookup::Known(pid) => Ok((vec![pid], None)),
Lookup::Scan(port) => scan_for_jvm(probe, port, suspended).map(|pid| (vec![pid], Some(pid))),
}
}
fn keep_found(target: JvmTarget<'_>, found: Option<i32>) {
if let (Some(found), JvmTarget::Attached { pid, .. }) = (found, target) {
// The attach-time scan may have found it meanwhile; either answer is the same JVM.
let _ = pid.compare_exchange(0, i64::from(found), Ordering::AcqRel, Ordering::Acquire);
}
}
/// A suspended JDK below 17 hangs on an attach request, so while suspended only a JVM known to
/// be 17 or later is asked anything. Pass a version read without attaching (the `release` file).
fn safe_while_suspended(version: Option<&ide_jvm_attach::JavaVersion>) -> bool {
version.is_some_and(|v| ide_jvm_attach::histogram_safe_while_suspended(v.major))
}
/// The JVM listening on JDWP `port`. While suspended the scan reaches the session's own JVM
/// too, so JVMs unsafe to ask are skipped, and when one of them may be the session's the
/// answer is a refusal rather than "not found".
fn scan_for_jvm(probe: &dyn AttachProbe, port: u16, suspended: bool) -> Result<i32, JvmMemoryUnavailable> {
// Majors of the JVMs left out; 0 for an unknown version.
let skipped = std::cell::RefCell::new(std::collections::BTreeSet::new());
let may_query = |pid: u32| {
if !suspended {
return true;
}
let version = probe.version(pid);
let safe = safe_while_suspended(version.as_ref());
if !safe {
skipped.borrow_mut().insert(version.map_or(0, |v| v.major));
}
safe
};
match probe.find_by_jdwp_port(port, &may_query).and_then(|p| i32::try_from(p).ok()) {
Some(pid) if pid > 0 => Ok(pid),
_ => {
let skipped = skipped.into_inner();
Err(match skipped.first() {
None => JvmMemoryUnavailable::AttachedJvmNotFound,
Some(&major) => JvmMemoryUnavailable::UnsafeWhileSuspended {
java_major: if skipped.len() == 1 { major } else { 0 },
},
})
}
}
}
fn pid_tree(pid: i32) -> Vec<i32> {
#[cfg(unix)]
{
ide_core::process::process_tree(pid)
}
#[cfg(not(unix))]
{
vec![pid]
}
}
-146
Переглянути файл
@@ -1,146 +0,0 @@
//! Finding the JVM behind an attach session by its JDWP port.
use super::safe_while_suspended;
use crate::session::Session;
use serde_json::Value;
use std::net::IpAddr;
use std::sync::Arc;
use std::sync::atomic::Ordering;
use std::time::Duration;
/// One try of the scan at attach time, and the limit per socket within it: a silent socket's
/// query thread lives that long, so a longer per-socket limit piles threads up across tries.
const ATTACH_SCAN_TRY: Duration = Duration::from_millis(500);
const ATTACH_SCAN_INTERVAL: Duration = Duration::from_millis(150);
/// The JVM's attach socket can appear a little after the JDWP "Listening" banner, which is
/// when the IDE attaches.
const ATTACH_SCAN_WINDOW: Duration = Duration::from_secs(3);
/// The single scan a refresh makes while the attach-time one found nothing.
pub(super) const RESCAN_PER_SOCKET: Duration = Duration::from_secs(1);
pub(super) const RESCAN_BUDGET: Duration = Duration::from_secs(2);
/// JDWP port of an attach to this machine (`hostName` loopback or absent): only such a JVM has
/// an attach socket the IDE can find.
pub(crate) fn local_jdwp_port(arguments: &Value) -> Option<u16> {
let local = match arguments.get("hostName") {
None | Some(Value::Null) => true,
Some(Value::String(host)) => {
let host = host.trim();
let ip = host.strip_prefix('[').and_then(|h| h.strip_suffix(']')).unwrap_or(host);
host.eq_ignore_ascii_case("localhost") || ip.parse::<IpAddr>().is_ok_and(|ip| ip.is_loopback())
}
Some(_) => false,
};
jdwp_port(arguments).filter(|_| local)
}
/// `port` of an attach request: a number or a numeric string, never 0.
fn jdwp_port(arguments: &Value) -> Option<u16> {
let port = match arguments.get("port")? {
Value::Number(n) => n.as_u64().and_then(|p| u16::try_from(p).ok()),
Value::String(s) => s.trim().parse().ok(),
_ => None,
};
port.filter(|&p| p != 0)
}
/// `hostName` (empty when absent) and `port` of an attach request, local or not.
pub(crate) fn attach_target(arguments: &Value) -> Option<(String, u16)> {
let host = arguments.get("hostName").and_then(Value::as_str).unwrap_or_default();
Some((host.to_owned(), jdwp_port(arguments)?))
}
/// Pid of the local JVM listening on a JDWP port (first argument), given whether the session is
/// suspended (second). Replaceable so tests never send attach requests to real JVMs.
pub(crate) type JvmFinder = Arc<dyn Fn(u16, bool) -> Option<u32> + Send + Sync>;
/// The [`JvmFinder`] of the attach-time scan.
pub(crate) fn find_local_jvm(port: u16, suspended: bool) -> Option<u32> {
let may_query = |pid| !suspended || safe_while_suspended(ide_jvm_attach::java_version_of(pid).as_ref());
ide_jvm_attach::find_jvm_by_jdwp_port(port, ATTACH_SCAN_TRY, ATTACH_SCAN_TRY, may_query)
}
/// Looks for the JVM behind a local attach: one try now, the rest in the background, so a JVM
/// that never opens an attach socket holds the attach up by one try at most. Call it before the
/// debugger connects: the JVM clears its JDWP listener address then, and a scan can no longer
/// tell it by its port.
pub(crate) async fn find_attached_jvm(session: &Arc<Session>, port: u16, finder: JvmFinder) {
let _ = session.jdwp_port.set(port);
let find = move |suspended: bool| finder(port, suspended);
let first = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || first(false)).await {
let _ = session.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
return;
}
drop(watch_for_attached_jvm(session, port, find, ATTACH_SCAN_INTERVAL, ATTACH_SCAN_WINDOW));
}
/// Retries `find` for the session's `jvm_pid`, holding the session weakly so an ended session
/// is not kept alive. `find` is told whether the session is suspended.
pub(crate) fn watch_for_attached_jvm<F>(
session: &Arc<Session>,
port: u16,
find: F,
interval: Duration,
window: Duration,
) -> tokio::task::JoinHandle<()>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let weak = Arc::downgrade(session);
tokio::spawn(async move {
let gate = || match weak.upgrade() {
Some(s) if !s.terminated.load(Ordering::Acquire) && !s.conn.is_closed() => {
if s.jvm_pid.load(Ordering::Acquire) > 0 {
ScanGate::Ended
} else {
ScanGate::Scan { suspended: s.suspended.get() }
}
}
_ => ScanGate::Ended,
};
let Some(pid) = retry_find(find, gate, interval, window).await else {
tracing::debug!("no local JVM found for JDWP port {port} yet");
return;
};
if let Some(s) = weak.upgrade() {
let _ = s.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
}
})
}
#[derive(Clone, Copy, Debug, PartialEq)]
pub(crate) enum ScanGate {
Scan { suspended: bool },
Ended,
}
/// Calls `find` on a blocking thread, on each try `gate` allows, until it returns a pid, `gate`
/// reports the end or `window` has passed.
pub(crate) async fn retry_find<F>(
find: F,
gate: impl Fn() -> ScanGate,
interval: Duration,
window: Duration,
) -> Option<u32>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let deadline = tokio::time::Instant::now() + window;
loop {
match gate() {
ScanGate::Ended => break,
ScanGate::Scan { suspended } => {
let find = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || find(suspended)).await {
return Some(pid);
}
}
}
if tokio::time::Instant::now() + interval >= deadline {
break;
}
tokio::time::sleep(interval).await;
}
None
}
-214
Переглянути файл
@@ -1,214 +0,0 @@
//! One class histogram of a JVM and its deltas against the previous one.
use super::scan::{RESCAN_BUDGET, RESCAN_PER_SOCKET};
use super::{
ClassCounts, HISTOGRAM_TIMEOUT, JvmClassRow, JvmMemorySnapshot, JvmMemorySource, JvmMemoryUnavailable,
safe_while_suspended,
};
use ide_core::IdeResult;
use ide_jvm_attach::{AttachError, HeapDumpReply, HistogramRow, JavaVersion};
use std::collections::HashMap;
use std::path::Path;
use std::time::Duration;
/// Version probe for a JVM whose `release` file could not be read.
const VERSION_TIMEOUT: Duration = Duration::from_secs(2);
const FLAGS_TIMEOUT: Duration = Duration::from_secs(2);
/// The attach calls a snapshot or a heap dump makes, so tests can stand in for a JVM.
pub(crate) trait AttachProbe: Send + Sync {
fn has_listener(&self, pid: u32) -> bool;
/// Asks only the JVMs whose pid `may_query` accepts.
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32>;
fn version(&self, pid: u32) -> Option<JavaVersion>;
fn histogram(&self, pid: u32) -> Result<String, AttachError>;
fn alive(&self, pid: u32) -> bool;
fn vm_flags(&self, pid: u32) -> Result<String, AttachError>;
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError>;
/// `None` where the free space is not known.
fn free_bytes(&self, dir: &Path) -> Option<u64>;
}
pub(super) struct JdkProbe {
pub(super) suspended: bool,
}
impl AttachProbe for JdkProbe {
fn has_listener(&self, pid: u32) -> bool {
ide_jvm_attach::has_listener(pid)
}
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32> {
ide_jvm_attach::find_jvm_by_jdwp_port(port, RESCAN_PER_SOCKET, RESCAN_BUDGET, may_query)
}
fn version(&self, pid: u32) -> Option<JavaVersion> {
ide_jvm_attach::java_version_of(pid).or_else(|| {
// The version decides whether attaching to a suspended VM hangs it, so never
// attach to learn it while suspended.
if self.suspended {
return None;
}
let out = ide_jvm_attach::jcmd(pid, "VM.version", VERSION_TIMEOUT).ok()?;
ide_jvm_attach::parse_vm_version(&out)
})
}
fn histogram(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::jcmd(pid, "GC.class_histogram -all", HISTOGRAM_TIMEOUT)
}
fn alive(&self, pid: u32) -> bool {
alive(pid)
}
fn vm_flags(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::vm_flags(pid, FLAGS_TIMEOUT)
}
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError> {
ide_jvm_attach::heap_dump(pid, path, timeout)
}
fn free_bytes(&self, dir: &Path) -> Option<u64> {
ide_heap::free_bytes(dir)
}
}
fn alive(pid: u32) -> bool {
#[cfg(unix)]
{
// 0 and negative pids address process groups.
let Ok(pid) = i32::try_from(pid) else { return false };
// SAFETY: signal 0 only checks that the process exists and may be signalled.
pid > 0
&& (unsafe { libc::kill(pid, 0) } == 0
|| std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM))
}
#[cfg(not(unix))]
{
let _ = pid;
false
}
}
/// Why `pid`'s attach listener cannot answer: it stopped listening or the program ended.
pub(super) fn gone(probe: &dyn AttachProbe, pid: u32) -> JvmMemoryUnavailable {
if probe.alive(pid) { JvmMemoryUnavailable::ListenerNotRunning } else { JvmMemoryUnavailable::ProgramExited }
}
/// The reason for an attach request (`what`) to `jvm` that failed, or an error where no reason
/// fits.
pub(super) fn attach_failure(
probe: &dyn AttachProbe,
jvm: u32,
error: AttachError,
what: &str,
) -> IdeResult<JvmMemoryUnavailable> {
use JvmMemoryUnavailable as U;
match error {
AttachError::Timeout => Ok(U::TimedOut),
AttachError::Refused | AttachError::NoListener => Ok(gone(probe, jvm)),
AttachError::ForeignListener => {
tracing::warn!("attach socket of JVM {jvm} is no longer private to this user; not using it");
Ok(U::ListenerNotRunning)
}
AttachError::Unsupported => Ok(U::UnsupportedPlatform),
e => Err(ide_core::IdeError::internal(format!("{what} of JVM {jvm}: {e}"))),
}
}
pub(crate) fn pick_jvm_pid(tree: &[i32], has_listener: impl Fn(u32) -> bool) -> Option<u32> {
tree.iter().filter_map(|&p| u32::try_from(p).ok()).filter(|&p| p > 0).find(|&p| has_listener(p))
}
pub(crate) fn diff_rows(rows: Vec<HistogramRow>, previous: Option<&ClassCounts>) -> (Vec<JvmClassRow>, ClassCounts) {
let mut seen: HashMap<&str, u32> = HashMap::with_capacity(rows.len());
for r in &rows {
*seen.entry(r.name.as_str()).or_default() += 1;
}
let ambiguous: Vec<bool> = rows.iter().map(|r| seen[r.name.as_str()] > 1).collect();
drop(seen);
let mut counts = ClassCounts::with_capacity(rows.len());
let mut out: Vec<JvmClassRow> = rows
.into_iter()
.zip(ambiguous)
.map(|(r, ambiguous)| {
counts.insert(r.name.clone(), (!ambiguous).then_some((r.instances, r.bytes)));
let delta = match previous {
_ if ambiguous => None,
None => None,
Some(p) => match p.get(&r.name) {
Some(None) => None,
// Absent: new since the previous snapshot, so it grew from nothing.
before => {
let (instances, bytes) = before.copied().flatten().unwrap_or_default();
Some((delta(r.instances, instances), delta(r.bytes, bytes)))
}
},
};
JvmClassRow {
name: r.name,
module: r.module,
instances: r.instances,
bytes: r.bytes,
delta_instances: delta.map(|d| d.0),
delta_bytes: delta.map(|d| d.1),
ambiguous,
}
})
.collect();
out.sort_by(|a, b| b.bytes.cmp(&a.bytes).then_with(|| a.name.cmp(&b.name)));
(out, counts)
}
/// Exact while the difference fits in an i64 (two's complement).
fn delta(now: u64, before: u64) -> i64 {
now.wrapping_sub(before) as i64
}
/// `pid_tree` starts with the session's reported pid, followed by its descendants.
pub(crate) fn take_snapshot(
probe: &dyn AttachProbe,
pid_tree: &[i32],
suspended: bool,
previous: Option<&ClassCounts>,
) -> IdeResult<Result<(JvmMemorySnapshot, ClassCounts), JvmMemoryUnavailable>> {
use JvmMemoryUnavailable as U;
let Some(root) = pid_tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(Err(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(pid_tree, |p| probe.has_listener(p)) else {
return Ok(Err(gone(probe, root)));
};
let version = probe.version(jvm);
if suspended && !safe_while_suspended(version.as_ref()) {
return Ok(Err(U::UnsafeWhileSuspended { java_major: version.map_or(0, |v| v.major) }));
}
let started = std::time::Instant::now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(Err),
};
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let histogram = ide_jvm_attach::parse_class_histogram(&text);
if histogram.skipped_lines > 0 {
tracing::debug!("class histogram of JVM {jvm}: {} unreadable lines", histogram.skipped_lines);
}
let (rows, counts) = diff_rows(histogram.rows, previous);
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: rows.iter().map(|r| r.bytes).sum(),
rows,
java_version: version.map(|v| v.text).unwrap_or_default(),
taken_while_suspended: suspended,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Histogram,
};
Ok(Ok((snapshot, counts)))
}
-119
Переглянути файл
@@ -1,119 +0,0 @@
//! A scripted [`JvmInspector`] and a session without an adapter behind it.
use crate::SessionOwner;
use crate::adapter::{AdapterSpec, Transport};
use crate::connection::Connection;
use crate::session::Session;
use ide_core::{DebugSessionId, IdeEvent, SharedSink};
use ide_jvm_inspect::{
ClassCount, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, ObjectId, VmTarget,
};
use parking_lot::Mutex;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::sync::Notify;
#[derive(Default)]
pub(crate) struct FakeInspector {
pub targets: Mutex<Vec<VmTarget>>,
/// The `limit` of each `instances` / `referrers` call.
pub limits: Mutex<Vec<u32>>,
pub calls: AtomicUsize,
/// Returned by every call when set.
pub error: Mutex<Option<InspectError>>,
pub counts: Mutex<Vec<ClassCount>>,
pub instances: Mutex<Vec<JvmValue>>,
/// Every call signals `entered`, then waits for `release`.
pub hold: bool,
pub entered: Notify,
pub release: Notify,
}
impl FakeInspector {
pub fn failing(error: InspectError) -> Self {
Self { error: Mutex::new(Some(error)), ..Self::default() }
}
pub fn calls(&self) -> usize {
self.calls.load(Ordering::SeqCst)
}
async fn answer<T: Send>(&self, target: &VmTarget, limit: Option<u32>, ok: T) -> Result<T, InspectError> {
self.calls.fetch_add(1, Ordering::SeqCst);
self.targets.lock().push(target.clone());
self.limits.lock().extend(limit);
if self.hold {
self.entered.notify_one();
self.release.notified().await;
}
match self.error.lock().clone() {
Some(e) => Err(e),
None => Ok(ok),
}
}
}
impl JvmInspector for FakeInspector {
fn class_counts<'a>(&'a self, target: &'a VmTarget) -> InspectFuture<'a, Vec<ClassCount>> {
Box::pin(self.answer(target, None, self.counts.lock().clone()))
}
fn instances<'a>(&'a self, target: &'a VmTarget, _: &'a str, limit: u32) -> InspectFuture<'a, Vec<JvmValue>> {
Box::pin(self.answer(target, Some(limit), self.instances.lock().clone()))
}
fn fields<'a>(&'a self, target: &'a VmTarget, _: ObjectId) -> InspectFuture<'a, Vec<JvmField>> {
Box::pin(self.answer(target, None, Vec::new()))
}
fn referrers<'a>(&'a self, target: &'a VmTarget, _: ObjectId, limit: u32) -> InspectFuture<'a, Vec<JvmReferrer>> {
Box::pin(self.answer(target, Some(limit), Vec::new()))
}
}
pub(crate) struct BareSession {
pub session: Arc<Session>,
pub events: Arc<Mutex<Vec<IdeEvent>>>,
/// The adapter's end of the connection, kept open.
_adapter: tokio::io::DuplexStream,
}
/// A java-debug session that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session(request: &str) -> BareSession {
bare_session_of("java-debug", request)
}
/// A session of adapter `name` that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session_of(name: &str, request: &str) -> BareSession {
let events: Arc<Mutex<Vec<IdeEvent>>> = Arc::default();
let sink_events = events.clone();
let sink: SharedSink = Arc::new(move |e: IdeEvent| sink_events.lock().push(e));
let (client, adapter) = tokio::io::duplex(1 << 10);
let (reader, writer) = tokio::io::split(client);
let (conn, _) = Connection::start(reader, writer);
let spec = AdapterSpec {
adapter: name.into(),
command: String::new(),
args: Vec::new(),
transport: Transport::Stdio,
install_hint: None,
cwd: std::env::temp_dir(),
env: Vec::new(),
server_host: None,
connect: None,
pinned: None,
};
let session = Session::new(
DebugSessionId(1),
SessionOwner::new("test", sink),
"test".into(),
name.into(),
None,
request.into(),
conn,
spec,
None,
None,
);
BareSession { session: Arc::new(session), events, _adapter: adapter }
}
File diff suppressed because it is too large. Load diff
-11
Переглянути файл
@@ -9,17 +9,6 @@ use std::{
pub const ADAPTER: &str = "kotlin-debug-adapter";
/// fwcd's adapter or kotlin-lsp's: both debug a JVM the IDE starts, through the JDWP link.
pub fn is_kotlin(adapter: &str) -> bool {
adapter == ADAPTER || adapter == crate::kotlin_lsp::ADAPTER
}
/// fwcd's adapter resumes the VM while constructing its debuggee, before its breakpoint hooks
/// exist, so only its link holds resumes back until the attach is done.
pub(crate) fn link_options(adapter: &str) -> ide_jdwp::LinkOptions {
ide_jdwp::LinkOptions { hold_resumes: adapter == ADAPTER }
}
pub struct SourceMirror {
dir: tempfile::TempDir,
paths: HashMap<PathBuf, PathBuf>,
+166
Переглянути файл
@@ -0,0 +1,166 @@
//! Keep a suspended JVM stopped until Kotlin's adapter has installed its listeners.
//! The official 0.4.4 adapter resumes VMStart while constructing its debuggee, before
//! installing breakpoint hooks. DAP attach completes only after those hooks exist.
use ide_core::IdeResult;
use std::{io, time::Duration};
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt},
net::{TcpListener, TcpStream},
sync::oneshot,
task::JoinHandle,
};
pub(crate) struct StartupRelay {
release: Option<oneshot::Sender<()>>,
task: JoinHandle<()>,
}
impl StartupRelay {
pub async fn start(host: String, port: u16) -> IdeResult<(Self, u16)> {
let listener = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = listener.local_addr()?.port();
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move {
let connect = async {
let (adapter, _) = listener.accept().await?;
let jvm = TcpStream::connect((host.as_str(), port)).await?;
adapter.set_nodelay(true)?;
jvm.set_nodelay(true)?;
Ok::<_, io::Error>((adapter, jvm))
};
if let Ok(Ok((adapter, jvm))) = tokio::time::timeout(Duration::from_secs(60), connect).await {
let (mut ar, mut aw) = adapter.into_split();
let (mut jr, mut jw) = jvm.into_split();
let upstream = forward(&mut ar, &mut jw, ready);
let downstream = tokio::io::copy(&mut jr, &mut aw);
// Close both sockets as soon as either side disconnects.
tokio::select! {
r = upstream => if let Err(e) = r { tracing::debug!("Kotlin JDWP relay: {e}"); },
_ = downstream => {},
}
}
});
Ok((Self { release: Some(release), task }, local))
}
pub fn release(&mut self) {
if let Some(ready) = self.release.take() {
let _ = ready.send(());
}
}
}
impl Drop for StartupRelay {
fn drop(&mut self) {
self.task.abort();
}
}
async fn packet(reader: &mut (impl AsyncRead + Unpin)) -> io::Result<Vec<u8>> {
let mut header = [0; 11];
reader.read_exact(&mut header).await?;
let len = u32::from_be_bytes(header[..4].try_into().unwrap()) as usize;
if !(11..=16 * 1024 * 1024).contains(&len) {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP packet length"));
}
let mut packet = vec![0; len];
packet[..11].copy_from_slice(&header);
reader.read_exact(&mut packet[11..]).await?;
Ok(packet)
}
async fn forward(
reader: &mut (impl AsyncRead + Unpin),
writer: &mut (impl AsyncWrite + Unpin),
mut ready: oneshot::Receiver<()>,
) -> io::Result<()> {
let mut handshake = [0; 14];
reader.read_exact(&mut handshake).await?;
if &handshake != b"JDWP-Handshake" {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP handshake"));
}
writer.write_all(&handshake).await?;
let mut released = false;
let mut held: Vec<Vec<u8>> = Vec::new();
loop {
// Retain the in-flight read when attach completes, including partial packets.
let read = packet(reader);
tokio::pin!(read);
let packet = tokio::select! {
result = &mut read => result?,
result = &mut ready, if !released => {
result.map_err(|_| io::Error::new(io::ErrorKind::ConnectionAborted, "Kotlin attach cancelled"))?;
released = true;
for packet in held.drain(..) {
writer.write_all(&packet).await?;
}
read.await?
}
};
// JDWP VirtualMachine.Resume (1/9), ThreadReference.Resume (11/3).
// Other commands must proceed: attach itself queries threads and installs requests.
if !released && packet[8] == 0 && matches!((packet[9], packet[10]), (1, 9) | (11, 3)) {
if held.len() >= 1024 {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Too many pending JDWP resumes"));
}
held.push(packet);
} else {
writer.write_all(&packet).await?;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn command(id: u8, set: u8, cmd: u8) -> Vec<u8> {
vec![0, 0, 0, 11, 0, 0, 0, id, 0, set, cmd]
}
#[tokio::test]
async fn holds_resumes_but_allows_setup_and_preserves_partial_reads() {
let (mut adapter, mut reader) = tokio::io::duplex(4096);
let (mut writer, mut jvm) = tokio::io::duplex(4096);
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move { forward(&mut reader, &mut writer, ready).await });
adapter.write_all(b"JDWP-Handshake").await.unwrap();
let mut handshake = [0; 14];
jvm.read_exact(&mut handshake).await.unwrap();
assert_eq!(&handshake, b"JDWP-Handshake");
let resume = command(1, 1, 9);
let thread_resume = command(2, 11, 3);
let setup = command(3, 1, 4);
adapter.write_all(&resume).await.unwrap();
adapter.write_all(&thread_resume).await.unwrap();
adapter.write_all(&setup).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), setup);
let next = command(4, 1, 9);
adapter.write_all(&next[..6]).await.unwrap();
release.send(()).unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), resume);
assert_eq!(packet(&mut jvm).await.unwrap(), thread_resume);
adapter.write_all(&next[6..]).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), next);
drop(adapter);
assert!(task.await.unwrap().is_err());
}
#[tokio::test]
async fn dropping_session_closes_relay() {
let (relay, port) = StartupRelay::start("127.0.0.1".into(), 1).await.unwrap();
let mut adapter = TcpStream::connect(("127.0.0.1", port)).await.unwrap();
drop(relay);
let result = tokio::time::timeout(Duration::from_secs(1), adapter.read_u8()).await.unwrap();
assert!(result.is_err());
}
#[tokio::test]
async fn rejects_invalid_packet_lengths() {
for len in [0_u32, 10, 16 * 1024 * 1024 + 1] {
let mut bytes = command(1, 1, 9);
bytes[..4].copy_from_slice(&len.to_be_bytes());
assert_eq!(packet(&mut bytes.as_slice()).await.unwrap_err().kind(), io::ErrorKind::InvalidData);
}
}
}
-147
Переглянути файл
@@ -1,147 +0,0 @@
//! JetBrains kotlin-lsp's debugger: a DAP server inside the language server, reached through
//! the `debugServer` port it opens once the project is imported.
//!
//! Known limits, with no guard here: an evaluate that calls a function with a breakpoint, right
//! after a step over a suspension point, loses every later stop at that breakpoint; while
//! stepping, breakpoints hit by other coroutines are skipped.
use ide_core::{IdeError, IdeResult};
use parking_lot::Mutex;
use std::collections::HashSet;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
pub const ADAPTER: &str = "kotlin-lsp";
/// The `adapterID` its debug server expects in `initialize`.
pub const DAP_ADAPTER_ID: &str = "intellij_jvm";
/// The path of a `file:` URI, as kotlin-lsp reports some sources (percent-encoded); `None` for
/// anything else, including a URI naming another host.
pub(crate) fn source_path(path: &str) -> Option<PathBuf> {
let rest = path.strip_prefix("file:")?;
let path = match rest.strip_prefix("//") {
Some(authority) => {
let slash = authority.find('/')?;
if !matches!(&authority[..slash], "" | "localhost") {
return None;
}
&authority[slash..]
}
None => rest,
};
let decoded = percent_encoding::percent_decode_str(path).decode_utf8().ok()?;
// `/C:/p/App.kt` on Windows.
let decoded = match decoded.strip_prefix('/') {
Some(drive) if cfg!(windows) && drive.as_bytes().get(1) == Some(&b':') => drive,
_ => &decoded,
};
Some(PathBuf::from(decoded))
}
pub(crate) fn frame_name(name: &str) -> &str {
name.strip_prefix("at ").unwrap_or(name)
}
/// What kotlin-lsp's DAP server does unlike other adapters.
///
/// It never answers a request that carries a thread id from an earlier stop (K1 G4), so such a
/// request would wait out `REQUEST_TIMEOUT`: [`Self::check_thread`] refuses it first. Only thread
/// ids were measured; a stale `frameId` or `variablesReference` may wait the same way. A step
/// ends with `reason: "breakpoint"`; a real breakpoint hit during a step is then reported as a
/// step too, since the stop carries neither `hitBreakpointIds` nor a description.
#[derive(Default)]
pub(crate) struct Quirks {
step_pending: AtomicBool,
/// The stop (a `Suspension` ticket) and the thread ids seen in it.
threads: Mutex<(u64, HashSet<i64>)>,
}
impl Quirks {
/// Before `next`, `stepIn`, `stepOut` or a forced step.
pub fn step_sent(&self) {
self.step_pending.store(true, Ordering::Release);
}
/// Before `continue` or run to cursor, or after a step request failed.
pub fn resume_sent(&self) {
self.step_pending.store(false, Ordering::Release);
}
/// The reason to report for a `stopped` event.
pub fn stop_reason(&self, reason: String) -> String {
let stepped = self.step_pending.swap(false, Ordering::AcqRel);
if stepped && reason == "breakpoint" { "step".into() } else { reason }
}
/// Thread ids seen in stop `stop`: a `stopped` event's, or a `threads` reply to a request sent
/// during that stop. A reply from an older stop changes nothing.
pub fn saw_threads(&self, stop: u64, ids: impl IntoIterator<Item = i64>) {
let mut threads = self.threads.lock();
if stop > threads.0 {
*threads = (stop, ids.into_iter().collect());
} else if stop == threads.0 {
threads.1.extend(ids);
}
}
/// Refuses a thread id that is not one of current stop `stop`'s, once some are known.
pub fn check_thread(&self, stop: u64, suspended: bool, id: i64) -> IdeResult<()> {
let threads = self.threads.lock();
if suspended && threads.0 == stop && !threads.1.is_empty() && !threads.1.contains(&id) {
return Err(IdeError::invalid(format!("thread {id} belongs to an earlier stop")));
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
#[test]
fn file_uris_become_paths() {
assert_eq!(source_path("file:///p/a%20b/App.kt").as_deref(), Some(Path::new("/p/a b/App.kt")));
assert_eq!(source_path("file://localhost/p/x.kt").as_deref(), Some(Path::new("/p/x.kt")));
assert_eq!(source_path("/p/x.kt"), None);
}
#[test]
fn frame_names_lose_the_at_prefix() {
assert_eq!(frame_name("at com.x.AppKt.main(App.kt:13)"), "com.x.AppKt.main(App.kt:13)");
assert_eq!(frame_name("main"), "main");
}
#[test]
fn only_a_pending_step_turns_a_breakpoint_into_a_step() {
let quirks = Quirks::default();
quirks.step_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "step");
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
quirks.step_sent();
quirks.resume_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
}
#[test]
fn threads_are_checked_against_the_current_stop() {
let quirks = Quirks::default();
quirks.saw_threads(3, [18]);
assert!(quirks.check_thread(3, true, 18).is_ok());
assert!(quirks.check_thread(3, true, 6).is_err());
assert!(quirks.check_thread(4, true, 6).is_ok(), "nothing known about stop 4 yet");
assert!(quirks.check_thread(3, false, 6).is_ok());
}
#[test]
fn an_older_threads_reply_does_not_change_a_newer_stop() {
let quirks = Quirks::default();
quirks.saw_threads(4, [2]);
quirks.saw_threads(3, [1]);
assert!(quirks.check_thread(4, true, 1).is_err());
quirks.saw_threads(4, [7]);
assert!(quirks.check_thread(4, true, 2).is_ok() && quirks.check_thread(4, true, 7).is_ok());
quirks.saw_threads(5, [9]);
assert!(quirks.check_thread(5, true, 2).is_err());
}
}
+30 -183
Переглянути файл
@@ -13,9 +13,8 @@ pub mod attach;
mod cargo;
pub mod connection;
pub mod jdwp;
mod jvm_memory;
pub mod kotlin;
pub mod kotlin_lsp;
mod kotlin_jdwp;
mod session;
use adapter::{AdapterLauncher, AdapterSpec, ProcessLauncher};
@@ -23,7 +22,6 @@ use connection::{Connection, DapError};
use futures::future::join_all;
use ide_core::{DebugSessionId, IdeError, IdeResult, SharedSink};
use ide_language::LanguageRegistry;
pub use jvm_memory::*;
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
@@ -128,14 +126,6 @@ pub struct DebugCapabilities {
pub log_points: bool,
/// HotSwap ("Reload Changed Classes"): java-debug's `redefineClasses` request.
pub hot_code_replace: bool,
/// The session debugs a JVM, so it has a Memory view; [`DebugManager::jvm_memory_classes`]
/// says why when it has no data (platform, remote JVM, ...).
pub jvm_memory: bool,
/// The session has a JVM inspector: instances, fields and referrers while suspended. Can
/// change during the session ([`ide_core::event::DebugEvent::Capabilities`]).
pub jvm_instances: bool,
/// The session can be asked for a heap dump ([`DebugManager::jvm_heap_dump`]).
pub jvm_heap: bool,
}
/// Result of [`DebugManager::redefine_classes`].
@@ -312,7 +302,6 @@ pub struct DebugManager {
data_breakpoints: Mutex<Vec<DataBreakpoint>>,
/// In-flight stops, by session: a second Stop sets `stop_force`, the waiting one kills.
stopping: Mutex<HashMap<DebugSessionId, Arc<Session>>>,
jvm_finder: Mutex<jvm_memory::JvmFinder>,
}
/// rdbg (Ruby's debug gem) starts the debuggee itself and ignores `launch`'s program,
@@ -577,21 +566,9 @@ pub(crate) fn capabilities_from(caps: &Value) -> DebugCapabilities {
hit_conditional_breakpoints: has("supportsHitConditionalBreakpoints"),
log_points: has("supportsLogPoints"),
hot_code_replace: false,
jvm_memory: false,
jvm_instances: false,
jvm_heap: false,
}
}
pub(crate) fn session_capabilities(session: &Session) -> DebugCapabilities {
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_memory = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_instances = session.jvm_inspector.lock().is_some();
caps.jvm_heap = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps
}
/// States of a file's line breakpoints that no session has been sent.
fn unsent(breakpoints: &[Breakpoint]) -> Vec<BreakpointState> {
breakpoints
@@ -611,19 +588,9 @@ fn unverified(enabled: impl Iterator<Item = bool>) -> Vec<BreakpointState> {
.collect()
}
/// A step request's outcome; a failed one leaves no step pending, so the next stop keeps its reason.
fn step_settled(session: &Session, sent: Result<Value, DapError>) -> Result<Value, DapError> {
if sent.is_err()
&& let Some(quirks) = session.kotlin_lsp_quirks()
{
quirks.resume_sent();
}
sent
}
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (in DAP mode a
/// temp-file name, `__debug_bin<digits>`; on Windows with `.exe` before the digits) and
/// removes it on a graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (on Windows
/// `__debug_bin.exe` plus `__debug_bin.exe<digits>` in DAP mode) and removes it on a
/// graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// the leftovers once the session is down. Runs after `session_closed`, so Delve
/// has already exited or been killed and holds no file locks; a leftover of a
/// still-building run is removed on that run's own shutdown.
@@ -632,10 +599,14 @@ fn remove_delve_binaries(cwd: &Path) {
for entry in entries.flatten() {
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
// `__debug_bin`, then `.exe` on Windows, then DAP mode's temp-file digits.
// Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin").map(|s| s.strip_prefix(".exe").unwrap_or(s));
let leftover = suffix.is_some_and(|s| s.chars().all(|c| c.is_ascii_digit()));
// Windows writes `__debug_bin.exe` (and `.exe<digits>` in DAP mode); other
// platforms keep the bare name. Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin");
let leftover = suffix.is_some_and(|s| {
s.is_empty()
|| s == ".exe"
|| (s.starts_with(".exe") && s[".exe".len()..].chars().all(|c| c.is_ascii_digit()))
});
if !leftover || entry.file_type().is_ok_and(|t| !t.is_file()) {
continue;
}
@@ -686,15 +657,9 @@ impl DebugManager {
function_breakpoints: Mutex::default(),
data_breakpoints: Mutex::default(),
stopping: Mutex::default(),
jvm_finder: Mutex::new(Arc::new(jvm_memory::find_local_jvm)),
})
}
#[cfg(test)]
pub(crate) fn set_jvm_finder(&self, finder: jvm_memory::JvmFinder) {
*self.jvm_finder.lock() = finder;
}
fn session(&self, s: DebugSessionId) -> IdeResult<Arc<Session>> {
self.sessions.lock().get(&s).cloned().ok_or_else(|| no_session(s))
}
@@ -988,7 +953,6 @@ impl DebugManager {
arguments["projectRoot"] = json!(mirror.root());
*s.kotlin_mirror.lock() = Some(mirror);
}
let is_kotlin_lsp = s.adapter == kotlin_lsp::ADAPTER;
let caps = s
.conn
.request_timeout(
@@ -996,7 +960,7 @@ impl DebugManager {
json!({
"clientID": "openidea",
"clientName": "OpenIDEA",
"adapterID": if is_kotlin_lsp { kotlin_lsp::DAP_ADAPTER_ID } else { s.adapter.as_str() },
"adapterID": s.adapter,
"locale": "en",
"linesStartAt1": true,
"columnsStartAt1": true,
@@ -1006,8 +970,7 @@ impl DebugManager {
"supportsRunInTerminalRequest": true,
"supportsStartDebuggingRequest": true,
"supportsProgressReporting": true,
// kotlin-lsp sends `invalidated` either way.
"supportsInvalidatedEvent": is_kotlin_lsp,
"supportsInvalidatedEvent": false,
"supportsMemoryReferences": false,
"supportsArgsCanBeInterpretedByShell": false,
}),
@@ -1016,32 +979,14 @@ impl DebugManager {
.await?;
*s.caps.lock() = if caps.is_object() { caps } else { json!({}) };
// The Kotlin adapters ignore the resolved classpath and project runtime on launch.
let request = if kotlin::is_kotlin(&s.adapter) && s.request == "launch" {
// The Kotlin adapter ignores the resolved classpath and project runtime on launch.
let request = if s.adapter == kotlin::ADAPTER && s.request == "launch" {
session::start_kotlin_debuggee(s, &mut arguments).await?;
"attach"
} else {
&s.request
};
// Both blocks below run before the Kotlin link replaces the JVM's address with its own.
if request == "attach"
&& let Some(target) = jvm_memory::attach_target(&arguments)
{
let _ = s.attach_target.set(target);
}
if cfg!(unix)
&& s.request == "attach"
&& JVM_MEMORY_ADAPTERS.contains(&s.adapter.as_str())
&& let Some(port) = jvm_memory::local_jdwp_port(&arguments)
{
if jdwp::is_listen_relay(port) {
s.listen_relay.store(true, Ordering::Release);
} else {
let finder = self.jvm_finder.lock().clone();
jvm_memory::find_attached_jvm(s, port, finder).await;
}
}
if kotlin::is_kotlin(&s.adapter) {
if s.adapter == kotlin::ADAPTER {
let host = arguments
.get("hostName")
.and_then(Value::as_str)
@@ -1053,14 +998,11 @@ impl DebugManager {
.and_then(|p| u16::try_from(p).ok())
.filter(|p| *p != 0)
.ok_or_else(|| IdeError::invalid("Invalid Kotlin JDWP port"))?;
let (link, port) = ide_jdwp::JdwpLink::start(host, port, kotlin::link_options(&s.adapter)).await?;
*s.jvm_inspector.lock() = Some(Arc::new(ide_jdwp::LinkInspector::new(link.clone())));
jvm_memory::watch_link(s, &link);
*s.jdwp_link.lock() = Some(link);
let (relay, port) = kotlin_jdwp::StartupRelay::start(host, port).await?;
*s.kotlin_relay.lock() = Some(relay);
arguments["hostName"] = json!("127.0.0.1");
arguments["port"] = json!(port);
}
s.jvm_located.send_replace(true);
// Launch/attach may wait for a slow runtime or a remote target. Let the adapter report
// failure; Stop cancels configuration and a closed connection fails the pending reply.
// Many adapters answer only after `configurationDone`, so don't wait for the reply yet.
@@ -1155,8 +1097,8 @@ impl DebugManager {
reply.await.unwrap_or(Err(DapError::Closed))?;
}
}
if let Some(link) = s.jdwp_link.lock().as_ref() {
link.release();
if let Some(relay) = s.kotlin_relay.lock().as_mut() {
relay.release();
}
Ok(())
}
@@ -1389,11 +1331,8 @@ impl DebugManager {
}
pub async fn threads(&self, s: DebugSessionId) -> IdeResult<Vec<ThreadInfo>> {
let session = self.session(s)?;
// Before sending: a stop that overtakes the reply must not take the reply's ids as its own.
let stop = session.suspended.ticket();
let body = session.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
let threads: Vec<ThreadInfo> = body
let body = self.session(s)?.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
Ok(body
.get("threads")
.and_then(Value::as_array)
.map(|t| {
@@ -1404,16 +1343,11 @@ impl DebugManager {
})
.collect()
})
.unwrap_or_default();
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.saw_threads(stop, threads.iter().map(|t| t.id));
}
Ok(threads)
.unwrap_or_default())
}
pub async fn stack_trace(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<Vec<StackFrame>> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
let body = session
.conn
.request_timeout(
@@ -1422,15 +1356,11 @@ impl DebugManager {
REQUEST_TIMEOUT,
)
.await?;
let kotlin_lsp = session.kotlin_lsp_quirks().is_some();
let frame = |f: &Value| {
let mut frame = frame_from(f);
if let Some(client) = frame.path.as_deref().and_then(|p| session.client_path(&p.to_string_lossy())) {
frame.path = Some(client.into());
}
if kotlin_lsp {
frame.name = kotlin_lsp::frame_name(&frame.name).to_owned();
}
frame
};
Ok(body.get("stackFrames").and_then(Value::as_array).map(|f| f.iter().map(frame).collect()).unwrap_or_default())
@@ -1542,15 +1472,8 @@ impl DebugManager {
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.resume_sent();
}
let ticket = session.suspended.ticket();
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
// DAP: a response without the flag continued every thread.
let all = body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true);
session.suspended.resumed(ticket, thread_id, all);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: all });
Ok(())
}
@@ -1590,15 +1513,7 @@ impl DebugManager {
StepKind::Out => "stepOut",
};
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
let sent = session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await;
step_settled(&session, sent)?;
// A step runs its thread; other threads stopped by their own events stay stopped.
session.suspended.resumed(ticket, thread_id, false);
session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1615,8 +1530,6 @@ impl DebugManager {
force: bool,
) -> IdeResult<()> {
let session = self.session(s)?;
// Before touching breakpoints: a refused `continue` would leave them replaced.
session.check_thread(thread_id)?;
if force {
self.suspend_breakpoints(&session, Some(&path)).await?;
}
@@ -1691,7 +1604,6 @@ impl DebugManager {
/// Force Step Over: `next` with every breakpoint suspended until the step ends.
pub async fn force_step_over(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
self.suspend_breakpoints(&session, None).await?;
self.step(s, thread_id, StepKind::Over).await
}
@@ -1699,17 +1611,10 @@ impl DebugManager {
/// Step into a specific call of the current line (Smart Step Into).
pub async fn step_in_target(&self, s: DebugSessionId, thread_id: i64, target_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
let sent = session
session
.conn
.request_timeout("stepIn", json!({ "threadId": thread_id, "targetId": target_id }), REQUEST_TIMEOUT)
.await;
step_settled(&session, sent)?;
session.suspended.resumed(ticket, thread_id, false);
.await?;
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1743,68 +1648,10 @@ impl DebugManager {
}
pub fn capabilities(&self, s: DebugSessionId) -> IdeResult<DebugCapabilities> {
Ok(session_capabilities(&*self.session(s)?))
}
/// Class table of the session's JVM (`GC.class_histogram -all`, or counted by the debugger
/// while suspended where no histogram can be taken), with deltas against the session's
/// previous snapshot from the same source.
pub async fn jvm_memory_classes(&self, s: DebugSessionId) -> IdeResult<JvmMemoryResult> {
let session = self.session(s)?;
jvm_memory::session_classes(session).await
}
/// Writes a heap dump of the session's JVM to `dir/dump.hprof`, one class-table or dump
/// request per session at a time. Anything but `Ready` deletes `dir`, unless `s` is not a
/// session. On `cancel` the result is `Cancelled` at once, while a dump the JVM is writing
/// still holds the session's turn until the JVM answers.
pub async fn jvm_heap_dump(
&self,
s: DebugSessionId,
dir: PathBuf,
cancel: ide_core::CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let session = self.session(s)?;
jvm_memory::session_heap_dump(session, dir, cancel).await
}
/// Gives the session the backend for [`Self::jvm_memory_instances`] and the debugger's class
/// counts; the UI is told to reload the session's capabilities.
pub fn set_jvm_inspector(
&self,
s: DebugSessionId,
inspector: Arc<dyn ide_jvm_inspect::JvmInspector>,
) -> IdeResult<()> {
let session = self.session(s)?;
if session.ended.is_cancelled() {
return Err(no_session(s));
}
jvm_memory::set_inspector(&session, inspector);
Ok(())
}
/// Up to `limit` (clamped to `1..=MAX_INSTANCES`) instances of `class_name` while suspended.
pub async fn jvm_memory_instances(
&self,
s: DebugSessionId,
class_name: String,
limit: u32,
) -> IdeResult<JvmInstancesResult> {
jvm_memory::instances(&*self.session(s)?, &class_name, limit).await
}
pub async fn jvm_memory_fields(&self, s: DebugSessionId, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
jvm_memory::fields(&*self.session(s)?, object).await
}
/// Objects that point at `object`; `limit` 0 means `DEFAULT_REFERRERS`, at most `MAX_REFERRERS`.
pub async fn jvm_memory_referrers(
&self,
s: DebugSessionId,
object: JvmObjectRef,
limit: u32,
) -> IdeResult<JvmReferrersResult> {
jvm_memory::referrers(&*self.session(s)?, object, limit).await
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
Ok(caps)
}
/// HotSwap: reloads the classes compiled since the session started (or since the
+42 -348
Переглянути файл
@@ -3,11 +3,8 @@
use crate::adapter::AdapterSpec;
use crate::connection::{Connection, DapError, Incoming};
use crate::kotlin_lsp;
use crate::{Breakpoint, BreakpointState, DataBreakpoint, DebugManager, FunctionBreakpoint, SessionOwner};
use ide_core::event::DebugEvent;
#[cfg(unix)]
use ide_core::process::process_tree;
use ide_core::{DebugSessionId, IdeEvent, TaskId};
use parking_lot::Mutex;
use serde_json::{Value, json};
@@ -65,15 +62,6 @@ pub(crate) struct Session {
pub stop_force: AtomicBool,
/// `systemProcessId` from the adapter's `process` event: the program being debugged.
pub debuggee_pid: AtomicI64,
/// The JVM an app-launched (Kotlin) session started, or the local JVM an attach session
/// found by its JDWP port. Separate from `debuggee_pid`, which decides how Stop signals the
/// program and is 0 for Kotlin.
pub jvm_pid: AtomicI64,
/// JDWP port of an attach to a local JVM, to look the JVM up again; unset for a remote one.
pub jdwp_port: std::sync::OnceLock<u16>,
/// The attach goes through the listen-mode relay of [`crate::jdwp::listen_for_jvm`].
pub listen_relay: AtomicBool,
pub suspended: Suspension,
/// Pids signalled for the current stop: the reported pid and everything it started (an
/// adapter may report the launcher shell, whose child is the program).
#[cfg_attr(not(unix), allow(dead_code))]
@@ -87,107 +75,8 @@ pub(crate) struct Session {
/// Resolved path → the path as the IDE knows it, for adapters given resolved paths.
resolved_paths: Mutex<HashMap<PathBuf, PathBuf>>,
progress: Mutex<HashMap<String, TaskId>>,
/// Relay between a Kotlin adapter and its JVM. Dropping it closes both sockets.
pub jdwp_link: parking_lot::Mutex<Option<ide_jdwp::JdwpLink>>,
pub jvm_inspector: parking_lot::Mutex<Option<Arc<dyn ide_jvm_inspect::JvmInspector>>>,
/// One inspector request per session at a time.
pub inspect_lock: tokio::sync::Mutex<()>,
/// Cancelled when the session ends, so a request still waiting on the inspector lets go of
/// it (and of the JDWP link it holds).
pub ended: ide_core::CancellationToken,
/// `hostName` (empty when absent) and `port` of the session's `attach` request, as the IDE
/// was given them: Kotlin then replaces them with the link's. Unlike `jdwp_port`, also set
/// for a remote JVM.
pub attach_target: std::sync::OnceLock<(String, u16)>,
/// Set once `configure` has recorded where the program's JVM is (the fields above, the
/// Kotlin link), just before the `launch`/`attach` request goes out.
pub jvm_located: watch::Sender<bool>,
pub kotlin_relay: Mutex<Option<crate::kotlin_jdwp::StartupRelay>>,
pub kotlin_mirror: Mutex<Option<crate::kotlin::SourceMirror>>,
pub jvm_memory: tokio::sync::Mutex<crate::JvmMemoryState>,
/// Used only by a kotlin-lsp session: see [`Session::kotlin_lsp_quirks`].
pub(crate) quirks: kotlin_lsp::Quirks,
}
/// Which threads of the program are suspended: each from its `stopped` event until a
/// `continued` event or a successful resume/step request for it. Never cleared elsewhere, since
/// a stale "suspended" only makes a later check refuse. fwcd and java-debug stop a breakpoint's
/// thread alone, so a thread can run while another stays stopped.
///
/// A resume reply ends only the stops taken before the request was sent: a thread's next
/// `stopped` event can overtake the reply. The check and the removal happen under one lock, so
/// no stop can land between them and be lost.
#[derive(Default)]
pub(crate) struct Suspension(Mutex<Stops>);
#[derive(Default)]
struct Stops {
/// Counts stops, and moves back to an earlier stopped thread: an object read under one
/// ticket is stale under another.
ticket: u64,
/// Threads stopped alone by their event, oldest first, with the ticket of that stop.
threads: Vec<(i64, u64)>,
/// The ticket of a stop of every thread (`allThreadsStopped`, or no thread id): any resume
/// ends it.
all: Option<u64>,
}
impl Stops {
fn latest(&self) -> Option<u64> {
self.threads.last().map(|&(_, ticket)| ticket).max(self.all)
}
/// Ends the stops taken by ticket `upto` of `thread`, or of every thread.
fn release(&mut self, thread: Option<i64>, every: bool, upto: u64) {
let latest = self.latest();
if every || thread.is_none() || self.all.is_some_and(|t| t <= upto) {
self.all = self.all.filter(|&t| t > upto);
self.threads.retain(|&(_, t)| t > upto);
} else {
self.threads.retain(|&(id, t)| Some(id) != thread || t > upto);
}
// Back on an earlier thread's stop, as the JDWP link moves to it with a new generation.
let now = self.latest();
if now.is_some() && now != latest {
self.ticket += 1;
}
}
}
impl Suspension {
/// Whether some thread is stopped.
pub fn get(&self) -> bool {
let stops = self.0.lock();
stops.all.is_some() || !stops.threads.is_empty()
}
/// A `stopped` event; `all`: its `allThreadsStopped`.
pub fn stopped(&self, thread: Option<i64>, all: bool) {
let mut stops = self.0.lock();
stops.ticket += 1;
let ticket = stops.ticket;
match thread.filter(|_| !all) {
Some(thread) => {
stops.threads.retain(|&(id, _)| id != thread);
stops.threads.push((thread, ticket));
}
None => stops.all = Some(ticket),
}
}
/// A `continued` event; `all`: its `allThreadsContinued`.
pub fn continued(&self, thread: Option<i64>, all: bool) {
self.0.lock().release(thread, all, u64::MAX);
}
/// Take before sending a resume-like request and pass to [`Self::resumed`].
pub fn ticket(&self) -> u64 {
self.0.lock().ticket
}
/// After a successful resume-like request for `thread`; `all`: every thread ran.
pub fn resumed(&self, ticket: u64, thread: i64, all: bool) {
self.0.lock().release(Some(thread), all, ticket);
}
}
/// Adapters that only match files by their resolved path: Godot maps a script to
@@ -236,10 +125,6 @@ impl Session {
terminated: AtomicBool::new(false),
stop_force: AtomicBool::new(false),
debuggee_pid: AtomicI64::new(0),
jvm_pid: AtomicI64::new(0),
jdwp_port: std::sync::OnceLock::new(),
listen_relay: AtomicBool::new(false),
suspended: Suspension::default(),
stop_pids: parking_lot::Mutex::new(Vec::new()),
#[cfg(windows)]
stop_tree: Mutex::new(None),
@@ -247,27 +132,8 @@ impl Session {
breakpoint_ids: Mutex::new(HashMap::new()),
resolved_paths: Mutex::new(HashMap::new()),
progress: Mutex::new(HashMap::new()),
jdwp_link: parking_lot::Mutex::new(None),
jvm_inspector: parking_lot::Mutex::new(None),
inspect_lock: tokio::sync::Mutex::default(),
ended: ide_core::CancellationToken::new(),
attach_target: std::sync::OnceLock::new(),
jvm_located: watch::channel(false).0,
kotlin_relay: Mutex::new(None),
kotlin_mirror: Mutex::new(None),
jvm_memory: tokio::sync::Mutex::default(),
quirks: kotlin_lsp::Quirks::default(),
}
}
pub fn kotlin_lsp_quirks(&self) -> Option<&kotlin_lsp::Quirks> {
(self.adapter == kotlin_lsp::ADAPTER).then_some(&self.quirks)
}
/// Call before sending a request for thread `id`.
pub fn check_thread(&self, id: i64) -> ide_core::IdeResult<()> {
match self.kotlin_lsp_quirks() {
Some(quirks) => quirks.check_thread(self.suspended.ticket(), self.suspended.get(), id),
None => Ok(()),
}
}
@@ -299,11 +165,6 @@ impl Session {
/// A path from the adapter as the IDE knows it (undoes [`Session::adapter_path`]).
pub fn client_path(&self, path: &str) -> Option<PathBuf> {
if self.adapter == kotlin_lsp::ADAPTER
&& let Some(path) = kotlin_lsp::source_path(path)
{
return Some(path);
}
if let Some(real) = self.kotlin_mirror.lock().as_ref().and_then(|m| m.client_path(Path::new(path))) {
return Some(real);
}
@@ -433,10 +294,7 @@ impl Session {
/// Kills the adapter and every debuggee we started.
pub fn kill_processes(&self) {
self.ended.cancel();
// The inspector holds a clone of the link, so both go.
self.jvm_inspector.lock().take();
self.jdwp_link.lock().take();
self.kotlin_relay.lock().take();
#[cfg(unix)]
if self.request == "launch" {
for group in launch_groups(self) {
@@ -493,6 +351,32 @@ pub(crate) async fn remember_debuggee(s: &Arc<Session>) {
let _ = s;
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// Asks the program itself to exit (`force` = SIGKILL) without ending the adapter session:
/// its output keeps flowing over DAP while it shuts down. The first call remembers the pid
/// tree, because the reported pid may die before its program does.
@@ -716,31 +600,18 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
s.emit(DebugEvent::Initialized);
}
"stopped" => {
let all = body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false);
s.suspended.stopped(int("threadId"), all);
let mut reason = str_("reason").unwrap_or_else(|| "pause".into());
if let Some(quirks) = s.kotlin_lsp_quirks() {
quirks.saw_threads(s.suspended.ticket(), int("threadId"));
reason = quirks.stop_reason(reason);
}
mgr.restore_breakpoints(s).await;
s.emit(DebugEvent::Stopped {
thread_id: int("threadId").unwrap_or(0),
reason,
reason: str_("reason").unwrap_or_else(|| "pause".into()),
description: str_("description").or_else(|| str_("text")),
all_threads_stopped: all,
});
}
"continued" => {
let all = body.get("allThreadsContinued").and_then(Value::as_bool);
// DAP: an event without the flag continues its thread alone. The UI keeps treating
// it as all threads; the suspension, which holds only threads that stopped, need not.
s.suspended.continued(int("threadId"), all.unwrap_or(false));
s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: all.unwrap_or(true),
all_threads_stopped: body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false),
});
}
"continued" => s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true),
}),
"output" => {
let category = str_("category").unwrap_or_else(|| "console".into());
if category == "telemetry" {
@@ -824,14 +695,6 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
change_type: crate::hot_code_replace_type(body.get("changeType")),
message: str_("message").unwrap_or_default(),
}),
// Sent only by adapters told `supportsInvalidatedEvent` (kotlin-lsp).
"invalidated" => s.emit(DebugEvent::Invalidated {
areas: body
.get("areas")
.and_then(Value::as_array)
.map(|a| a.iter().filter_map(Value::as_str).map(str::to_owned).collect())
.unwrap_or_default(),
}),
_ => tracing::trace!("dap: ignored event {event}"),
}
}
@@ -1054,19 +917,6 @@ fn forward_output<R: AsyncRead + Unpin + Send + 'static>(s: &Session, mut stream
});
}
/// The attach listener must be on the command line before the user's arguments, which may
/// override it; the memory view needs it because the JVM is started by the IDE, not attached to.
/// Not on Windows (`windows`), which the memory view does not support.
fn kotlin_jvm_args(user: Vec<String>, windows: bool) -> Vec<String> {
if windows || !ide_jvm_attach::wants_listener_flag(&user.join(" ")) {
return user;
}
let mut args = Vec::with_capacity(user.len() + 1);
args.push(ide_jvm_attach::START_LISTENER_FLAG.to_owned());
args.extend(user);
args
}
/// Launch with the IDE's resolved runtime and classpath; the adapter only attaches.
pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) -> ide_core::IdeResult<()> {
use tokio::io::{AsyncBufReadExt, BufReader};
@@ -1081,7 +931,7 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
std::env::join_paths(strings("classPaths")).map_err(|e| ide_core::IdeError::invalid(e.to_string()))?;
let mut cmd = tokio::process::Command::new(java);
ide_core::process::prepare(cmd.as_std_mut());
cmd.args(kotlin_jvm_args(strings("vmArguments"), cfg!(windows)))
cmd.args(strings("vmArguments"))
// Let the JVM pick its own port (`:0`) and report it below: pre-picking a free
// port here races with other processes that may take it before the JVM binds.
.arg("-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=127.0.0.1:0")
@@ -1115,12 +965,11 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
// forced stop), the session's job on Windows, so a stop reaches what it forks.
ide_core::process::own_group(cmd.as_std_mut());
let mut child = cmd.spawn()?;
if let Some(pid) = child.id() {
s.jvm_pid.store(i64::from(pid), Ordering::Release);
#[cfg(windows)]
if let Some(job) = s.job.lock().as_ref() {
job.assign_pid(pid);
}
#[cfg(windows)]
if let Some(pid) = child.id()
&& let Some(job) = s.job.lock().as_ref()
{
job.assign_pid(pid);
}
if let Some(err) = child.stderr.take() {
forward_output(s, err, "stderr");
@@ -1161,162 +1010,7 @@ fn parse_jdwp_port(line: &str) -> Option<u16> {
#[cfg(test)]
mod tests {
use super::{Suspension, kotlin_jvm_args, parse_jdwp_port};
#[test]
fn a_resume_reply_clears_the_stop_it_was_sent_for() {
let s = Suspension::default();
assert!(!s.get());
s.stopped(Some(1), true);
let ticket = s.ticket();
s.resumed(ticket, 1, true);
assert!(!s.get());
}
#[test]
fn a_stop_after_the_resume_request_keeps_the_session_suspended() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
// The next breakpoint's `stopped` event overtakes the reply to `continue`.
s.continued(Some(1), true);
s.stopped(Some(1), true);
s.resumed(ticket, 1, true);
assert!(s.get());
// A reply for the new stop still clears it.
s.resumed(s.ticket(), 1, true);
assert!(!s.get());
}
#[test]
fn a_reply_after_a_continued_event_changes_nothing() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
s.continued(Some(1), true);
s.resumed(ticket, 1, true);
assert!(!s.get());
s.stopped(Some(1), true);
assert!(s.get());
}
#[test]
fn concurrent_stops_are_never_lost() {
let s = std::sync::Arc::new(Suspension::default());
s.stopped(Some(1), true);
let ticket = s.ticket();
let stopping = {
let s = s.clone();
std::thread::spawn(move || (0..10_000).for_each(|_| s.stopped(Some(1), true)))
};
for _ in 0..10_000 {
s.resumed(ticket, 1, true);
}
stopping.join().unwrap();
// However they interleave, a resume for the old ticket never clears a later stop.
assert!(s.get());
assert_eq!(s.ticket(), ticket + 10_000);
}
/// fwcd and java-debug stop a breakpoint's thread alone (`SUSPEND_EVENT_THREAD`): 7, then 8.
fn two_threads_stopped() -> Suspension {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), false);
s
}
#[test]
fn continuing_the_latest_stopped_thread_keeps_the_other_as_a_new_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(8), false);
assert!(s.get());
// Objects read on thread 8's stop are stale.
assert_eq!(s.ticket(), ticket + 1);
s.continued(Some(7), false);
assert!(!s.get());
}
#[test]
fn continuing_an_earlier_stopped_thread_keeps_the_latest_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(7), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn continuing_all_threads_ends_every_stop() {
let s = two_threads_stopped();
s.continued(Some(7), true);
assert!(!s.get());
let s = two_threads_stopped();
s.continued(None, false);
assert!(!s.get());
}
#[test]
fn continuing_a_thread_that_did_not_stop_changes_nothing() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(5), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn a_step_reply_resumes_its_thread_alone() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.resumed(ticket, 8, false);
assert!(s.get());
assert_eq!(s.ticket(), ticket + 1);
s.resumed(s.ticket(), 7, true);
assert!(!s.get());
}
#[test]
fn a_thread_stopping_again_before_its_resume_reply_stays_stopped() {
let s = two_threads_stopped();
let ticket = s.ticket();
// Thread 7's step ends, and its `stopped` event overtakes the reply to `next`.
s.stopped(Some(7), false);
s.resumed(ticket, 7, false);
s.continued(Some(8), false);
assert!(s.get());
}
#[test]
fn a_stop_of_every_thread_ends_with_any_resume() {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), true);
s.continued(Some(8), false);
assert!(!s.get());
// A stop without a thread.
s.stopped(None, false);
s.resumed(s.ticket(), 5, false);
assert!(!s.get());
}
#[test]
fn kotlin_launch_starts_attach_listener_first() {
assert_eq!(
kotlin_jvm_args(vec!["-Xmx1g".into()], false),
vec!["-XX:+StartAttachListener".to_string(), "-Xmx1g".into()]
);
assert_eq!(
kotlin_jvm_args(vec!["-XX:+DisableAttachMechanism".into()], false),
vec!["-XX:+DisableAttachMechanism".to_string()]
);
}
#[test]
fn kotlin_launch_on_windows_gets_no_attach_listener() {
assert_eq!(kotlin_jvm_args(vec!["-Xmx1g".into()], true), vec!["-Xmx1g".to_string()]);
}
use super::parse_jdwp_port;
#[test]
fn reads_the_port_the_jvm_chose() {
+23 -762
Переглянути файл
@@ -1,7 +1,5 @@
//! Protocol flow against an in-memory fake adapter.
mod jvm_inspect;
use super::*;
use crate::adapter::{AdapterIo, LaunchFuture};
use ide_core::IdeEvent;
@@ -122,28 +120,6 @@ struct Script {
supports_terminate: bool,
/// Answer `setBreakpoints` only once this many are waiting (a client that waits for each never gets one).
hold_set_breakpoints: usize,
/// Send a `continued` event right after the first `stopped` one.
continued_after_stop: bool,
/// Close the connection when a `pause` request arrives (the adapter dies).
hang_up_on_pause: bool,
/// Close the connection at the first request after the first `stopped` event.
hang_up_after_stop: bool,
/// Answer `continue` with this body and keep the program running, instead of letting it exit.
continue_reply: Option<Value>,
/// Answer `continue` and stop at a breakpoint again, instead of letting the program exit.
stop_after_continue: bool,
/// The `reason` of the stop a `next` ends with (`step` when unset; kotlin-lsp says `breakpoint`).
step_reason: Option<&'static str>,
/// Refuse `next`, then stop at a breakpoint (one the program hit meanwhile).
refuse_next: bool,
/// Send `invalidated { areas: ["variables"] }` right after the first `stopped` event.
invalidated_after_stop: bool,
/// On `threads`: stop again on thread 2 first, then answer with thread 1 of the earlier stop.
stop_during_threads: bool,
/// Name of the first stack frame (`main` when unset).
frame_name: Option<&'static str>,
/// Answer `initialize` only after this long (a slow adapter keeps the session connecting).
delay_initialize: Option<Duration>,
initialized_delay: Duration,
launch_delay: Duration,
/// Leave startup pending before `initialized` (false) or after configuration (true).
@@ -168,17 +144,6 @@ impl Default for Script {
output_after_terminate: false,
supports_terminate: true,
hold_set_breakpoints: 0,
continued_after_stop: false,
hang_up_on_pause: false,
hang_up_after_stop: false,
continue_reply: None,
stop_after_continue: false,
step_reason: None,
refuse_next: false,
invalidated_after_stop: false,
stop_during_threads: false,
frame_name: None,
delay_initialize: None,
initialized_delay: Duration::ZERO,
launch_delay: Duration::ZERO,
pending_startup: None,
@@ -223,7 +188,6 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
let mut seq = 1000;
let mut pending_launch: Option<i64> = None;
let mut held: Vec<Value> = Vec::new();
let mut stopped = false;
let mut running = false;
macro_rules! send {
($v:expr) => {{
@@ -244,14 +208,8 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
if msg["type"] == "response" {
continue;
}
if script.hang_up_after_stop && stopped {
break;
}
match msg["command"].as_str().unwrap_or_default() {
"initialize" => {
if let Some(delay) = script.delay_initialize {
tokio::time::sleep(delay).await;
}
ok!(
msg,
json!({
@@ -345,18 +303,9 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
ok!(msg, Value::Null);
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "restart", "threadId": 1}}));
}
"next" if script.refuse_next => {
send!(
json!({"type": "response", "request_seq": msg["seq"], "command": "next", "success": false, "message": "refused"})
);
// Lets the client handle the refusal before the stop.
tokio::time::sleep(Duration::from_millis(50)).await;
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1}}));
}
"next" => {
ok!(msg, Value::Null);
let reason = script.step_reason.unwrap_or("step");
send!(json!({"type": "event", "event": "stopped", "body": {"reason": reason, "threadId": 1}}));
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "step", "threadId": 1}}));
}
"setVariable" => {
ok!(msg, json!({"value": format!("{}!", msg["arguments"]["value"].as_str().unwrap()), "type": "int"}))
@@ -392,32 +341,15 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
stopped = true;
if script.invalidated_after_stop {
send!(
json!({"type": "event", "event": "invalidated", "body": {"areas": ["variables"], "threadId": 1}})
);
}
if script.continued_after_stop {
send!(json!({"type": "event", "event": "continued", "body": {"threadId": 1}}));
}
send!(
json!({"type": "event", "event": "breakpoint", "body": {"reason": "changed", "breakpoint": {"id": 1, "verified": false, "message": "moved", "line": 9, "source": {"path": script.source_path}}}})
);
}
"threads" if script.stop_during_threads => {
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 2, "allThreadsStopped": true}})
);
// Lets the client handle the stop before it reads the reply.
tokio::time::sleep(Duration::from_millis(50)).await;
ok!(msg, json!({"threads": [{"id": 1, "name": "\"main\"@1"}]}));
}
"threads" => ok!(msg, json!({"threads": [{"id": 1, "name": "MainThread"}]})),
"stackTrace" => ok!(
msg,
json!({"stackFrames": [
{"id": 10, "name": script.frame_name.unwrap_or("main"), "line": 5, "column": 3, "source": {"path": script.source_path.as_deref().unwrap_or("/p/a.py")}},
{"id": 10, "name": "main", "line": 5, "column": 3, "source": {"path": script.source_path.as_deref().unwrap_or("/p/a.py")}},
{"id": 11, "name": "lib", "line": 1, "column": 1, "source": {"path": "/usr/lib/x.py", "presentationHint": "deemphasize"}},
]})
),
@@ -440,19 +372,11 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
{"label": ""},
]})
),
"continue" if script.continue_reply.is_some() => ok!(msg, script.continue_reply.clone().unwrap()),
"continue" if script.stop_after_continue => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
}
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
send!(json!({"type": "event", "event": "terminated"}));
}
"pause" if script.hang_up_on_pause => break,
"stepIn" | "stepOut" | "pause" => ok!(msg, Value::Null),
"redefineClasses" => {
send!(
@@ -506,8 +430,6 @@ fn harness(script: Script) -> Harness {
Arc::new(LanguageRegistry::from_configs(Vec::new())),
Arc::new(FakeLauncher { log: log.clone(), script }),
);
// A local attach must not send requests to the developer's own JVMs.
manager.set_jvm_finder(Arc::new(|_, _| None));
Harness { manager, log, events, owner }
}
@@ -1430,36 +1352,6 @@ async fn hot_swap_only_for_java_debug() {
assert_eq!(failed.error_message.as_deref(), Some("boom"));
}
/// A loopback port nothing listens on until `listen` is called on the socket, and that no parallel
/// test can bind meanwhile. A port picked by binding to 0 and closing the listener can be taken
/// by another test before the code under test connects to it.
fn reserved_port() -> (tokio::net::TcpSocket, u16) {
// Without SO_REUSEADDR, which every `TcpListener::bind` sets: no other socket may bind it.
let socket = tokio::net::TcpSocket::new_v4().unwrap();
socket.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = socket.local_addr().unwrap().port();
(socket, port)
}
/// An editor that starts listening on the reserved port after `delay` and serves one session.
#[cfg(unix)]
fn fake_editor(socket: tokio::net::TcpSocket, delay: Duration, log: Log) {
tokio::spawn(async move {
tokio::time::sleep(delay).await;
let listener = socket.listen(1).unwrap();
let (stream, _) = listener.accept().await.unwrap();
// Closed once its session is accepted, so a later start finds no editor open.
drop(listener);
fake_adapter(stream, log, Script::default()).await;
});
}
#[tokio::test]
async fn a_reserved_port_cannot_be_taken_by_a_parallel_test() {
let (_socket, port) = reserved_port();
assert!(tokio::net::TcpListener::bind(("127.0.0.1", port)).await.is_err(), "port {port} is still free to bind");
}
fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegistry> {
let lang = ide_language::LanguageConfig {
id: ide_core::LanguageId::new("gdscript"),
@@ -1472,7 +1364,6 @@ fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegist
indent: None,
ui_mode: None,
framework_ui_modes: Vec::new(),
framework_files: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
@@ -1528,7 +1419,7 @@ async fn godot_connects_to_the_editor_adapter() {
#[tokio::test]
async fn godot_without_editor_or_executable_says_what_to_do() {
let (_no_editor, port) = reserved_port();
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(godot_registry(port, "openidea-no-such-godot", &[]));
let err = manager.start(godot_launch(&std::env::temp_dir()), owner).await.unwrap_err();
@@ -1541,9 +1432,15 @@ async fn godot_without_editor_or_executable_says_what_to_do() {
#[cfg(unix)]
#[tokio::test]
async fn godot_starts_a_headless_editor_when_none_is_open() {
let (editor, port) = reserved_port();
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let log: Log = Arc::default();
fake_editor(editor, Duration::from_millis(300), log.clone());
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
let (owner, _events) = recording_owner("main");
// `sleep` stands in for `godot --headless --editor --path ${root} …`.
let manager = DebugManager::new(godot_registry(port, "sleep", &["30"]));
@@ -1746,7 +1643,7 @@ async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
std::fs::write(&program, "#!/bin/sh\nexec sleep 30\n").unwrap();
std::fs::set_permissions(&program, std::fs::Permissions::from_mode(0o755)).unwrap();
let (editor, port) = reserved_port();
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let mut gdscript = parse_definition(include_str!("../../../languages/gdscript.toml")).unwrap();
let debugger = gdscript.debugger.as_mut().unwrap();
debugger.args.clear();
@@ -1763,7 +1660,13 @@ async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
let _language = store.add("acme.godot".into(), gdscript);
let log: Log = Arc::default();
fake_editor(editor, Duration::from_millis(300), log.clone());
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(registry.clone());
let id = manager.start(godot_launch(&dir), owner).await.unwrap();
@@ -2096,667 +1999,25 @@ async fn stop_kills_the_program_started_by_the_reported_launcher() {
fn delve_leftovers_are_removed_without_touching_other_files() {
let dir = std::env::temp_dir().join(format!("openidea-delve-bin-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
for name in ["__debug_bin", "__debug_bin833767175", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
for name in ["__debug_bin", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
std::fs::write(dir.join(name), b"bin").unwrap();
}
// Lookalikes that are not Delve's own binaries stay: sources, a future
// in-progress build, and a directory that happens to share the name.
for name in ["__debug_bin.go", "__debug_bin.go.exe", "__debug_bin12.go", "main.go"] {
for name in ["__debug_bin.go", "__debug_bin.go.exe", "main.go"] {
std::fs::write(dir.join(name), b"keep").unwrap();
}
std::fs::create_dir(dir.join("__debug_bin.dir")).unwrap();
remove_delve_binaries(&dir);
for name in ["__debug_bin", "__debug_bin833767175", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
for name in ["__debug_bin", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
assert!(!dir.join(name).exists(), "{name} should be gone");
}
for name in ["__debug_bin.go", "__debug_bin.go.exe", "__debug_bin12.go", "main.go", "__debug_bin.dir"] {
for name in ["__debug_bin.go", "__debug_bin.go.exe", "main.go", "__debug_bin.dir"] {
assert!(dir.join(name).exists(), "{name} should stay");
}
std::fs::remove_dir_all(&dir).unwrap();
}
fn is_suspended(h: &Harness, id: DebugSessionId) -> bool {
h.manager.session(id).unwrap().suspended.get()
}
#[tokio::test]
async fn stopped_event_marks_session_suspended() {
let h = harness(Script::default());
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert!(is_suspended(&h, id));
}
#[tokio::test]
async fn resume_clears_suspended_without_continued_event() {
let h = harness(Script::default());
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
// The fake adapter ends the session after `continue`, so keep the session alive here.
let session = h.manager.session(id).unwrap();
assert!(session.suspended.get());
h.manager.resume(id, 1).await.unwrap();
assert!(!session.suspended.get());
}
/// After the fake adapter's stop of every thread ends, threads 7 then 8 stop alone, as a
/// breakpoint does on fwcd and java-debug (`SUSPEND_EVENT_THREAD`).
async fn threads_stopped(script: Script) -> (Harness, Arc<Session>, DebugSessionId) {
let h = harness(script);
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
session.suspended.continued(Some(1), true);
session.suspended.stopped(Some(7), false);
session.suspended.stopped(Some(8), false);
(h, session, id)
}
#[tokio::test]
async fn resume_of_one_thread_keeps_the_others_suspended() {
let only_it = Script { continue_reply: Some(json!({"allThreadsContinued": false})), ..Script::default() };
let (h, session, id) = threads_stopped(only_it).await;
h.manager.resume(id, 7).await.unwrap();
assert!(session.suspended.get());
h.manager.resume(id, 8).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn resume_reply_without_the_flag_continues_every_thread() {
let (h, session, id) = threads_stopped(Script { continue_reply: Some(json!({})), ..Script::default() }).await;
h.manager.resume(id, 7).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn step_of_one_thread_keeps_the_others_suspended() {
let (h, session, id) = threads_stopped(Script::default()).await;
let ticket = session.suspended.ticket();
h.manager.step(id, 8, StepKind::Into).await.unwrap();
assert!(session.suspended.get());
// Thread 8 was the latest stop: what was read on it is stale.
assert_eq!(session.suspended.ticket(), ticket + 1);
h.manager.step_in_target(id, 7, 1).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn continued_event_clears_suspended() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
assert!(!is_suspended(&h, id));
}
fn java_config(request: &str) -> LaunchConfig {
LaunchConfig { adapter: "java-debug".into(), request: request.into(), extra_json: None, ..config() }
}
fn remote_attach(adapter: &str, project: &std::path::Path) -> LaunchConfig {
LaunchConfig {
adapter: adapter.into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(json!({"projectRoot": project, "hostName": "10.0.0.5", "port": 5005}).to_string()),
..java_config("attach")
}
}
#[tokio::test]
async fn attach_session_reports_capability() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
for adapter in ["java-debug", kotlin::ADAPTER] {
let id = h.start(remote_attach(adapter, project.path())).await.unwrap();
let caps = h.manager.capabilities(id).unwrap();
assert!(caps.jvm_memory, "{adapter}");
assert_eq!(caps.jvm_heap, caps.jvm_memory, "{adapter}");
}
let id = h.start(config()).await.unwrap();
let caps = h.manager.capabilities(id).unwrap();
assert!(!caps.jvm_memory);
assert_eq!(caps.jvm_heap, caps.jvm_memory);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::NoJvmProcess, can_count_when_paused: false }
);
}
#[cfg(unix)]
#[tokio::test]
async fn ended_attach_session_without_pid_reports_program_exited() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.manager.session(id).unwrap().terminated.store(true, Ordering::Release);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::ProgramExited, can_count_when_paused: false }
);
}
#[cfg(unix)]
#[tokio::test]
async fn listen_mode_attach_is_not_scanned() {
let h = harness(Script::default());
let scans = Arc::new(Mutex::new(Vec::new()));
let counted = scans.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
counted.lock().push(port);
None
}));
let (_jvm, relay) = jdwp::tests::connected_jvm().await;
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "127.0.0.1", "port": relay}).to_string());
let id = h.start(cfg).await.unwrap();
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable {
reason: JvmMemoryUnavailable::ListenModeNotSupported,
can_count_when_paused: false
}
);
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(scans.lock().is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn remote_host_is_not_scanned() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jdwp_port.get(), None);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable {
reason: JvmMemoryUnavailable::AttachedJvmNotFound,
can_count_when_paused: false
}
);
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 0);
}
/// The Memory view asks as soon as the session appears, while its JVM is still being looked up:
/// answering then would claim there is no JVM to ask.
#[cfg(unix)]
#[tokio::test]
async fn memory_asked_while_an_attach_connects_waits_for_its_jvm() {
let h = harness(Script { delay_initialize: Some(Duration::from_millis(300)), ..Script::default() });
let project = tempfile::tempdir().unwrap();
let (manager, owner) = (h.manager.clone(), h.owner.clone());
let config = remote_attach("java-debug", project.path());
let start = tokio::spawn(async move { manager.start(config, owner).await });
h.wait_event(|e| matches!(e, DebugEvent::Started { .. })).await;
let id = DebugSessionId(*event_sessions(&h.events).first().unwrap());
let session = h.manager.session(id).unwrap();
assert!(session.attach_target.get().is_none(), "asked too late to cover the connecting session");
h.manager.jvm_memory_classes(id).await.unwrap();
assert!(session.attach_target.get().is_some(), "answered before the attach was looked at");
start.await.unwrap().unwrap();
}
/// A JVM clears its JDWP listener address once a debugger connects, so only a scan made before
/// the `attach` request can tell it by its port.
#[cfg(unix)]
#[tokio::test]
async fn local_attach_finds_its_jvm_before_the_debugger_connects() {
let h = harness(Script::default());
let log = h.log.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
std::thread::sleep(Duration::from_millis(50));
let attached = log.lock().iter().any(|m| m["command"] == "attach");
(port == 5005 && !attached).then_some(4242)
}));
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "127.0.0.1", "port": 5005}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn local_attach_remembers_its_jdwp_port() {
let h = harness(Script::default());
let ports = Arc::new(Mutex::new(Vec::new()));
let asked = ports.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
asked.lock().push(port);
Some(4242)
}));
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "localhost", "port": "5005"}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jdwp_port.get(), Some(&5005));
assert_eq!(h.request("attach")["arguments"]["port"], "5005");
for _ in 0..200 {
if session.jvm_pid.load(Ordering::Acquire) != 0 {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
assert_eq!(ports.lock().first(), Some(&5005));
}
/// A fake `find_jvm_by_jdwp_port` for the JVM 4242, recording whether each scan ran while
/// suspended. A JVM not `safe_while_suspended` (JDK below 17) is skipped by a suspended scan.
#[cfg(unix)]
fn counting_find(
safe_while_suspended: bool,
) -> (Arc<Mutex<Vec<bool>>>, impl Fn(bool) -> Option<u32> + Clone + Send + 'static) {
let scans = Arc::new(Mutex::new(Vec::new()));
let counted = scans.clone();
(scans, move |suspended| {
counted.lock().push(suspended);
(!suspended || safe_while_suspended).then_some(4242)
})
}
#[cfg(unix)]
async fn suspended_remote_attach(h: &Harness) -> Arc<Session> {
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
assert!(session.suspended.get());
session
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_skips_an_old_jdk_while_suspended() {
let h = harness(Script::default());
let session = suspended_remote_attach(&h).await;
let (scans, find) = counting_find(false);
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(!scans.lock().is_empty() && scans.lock().iter().all(|&suspended| suspended));
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 0);
session.suspended.continued(None, true);
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(scans.lock().last(), Some(&false));
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_finds_a_jdk_17_or_later_while_suspended() {
let h = harness(Script::default());
let session = suspended_remote_attach(&h).await;
let (scans, find) = counting_find(true);
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(*scans.lock(), [true]);
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_keeps_a_known_pid_and_stops_with_the_session() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
session.suspended.continued(None, true);
session.jvm_pid.store(7, Ordering::Release);
let watch = |find| {
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5))
};
let (scans, find) = counting_find(true);
tokio::time::timeout(Duration::from_secs(2), watch(find)).await.unwrap().unwrap();
assert!(scans.lock().is_empty(), "a known pid needs no scan");
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 7);
// A refresh stores the pid while this scan is still running.
session.jvm_pid.store(0, Ordering::Release);
let racing = session.clone();
let find = move |_| {
racing.jvm_pid.store(7, Ordering::Release);
Some(4242)
};
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 7, "a pid stored first is kept");
h.manager.stop(id).await.unwrap();
let (scans, find) = counting_find(true);
tokio::time::timeout(Duration::from_secs(2), watch(find)).await.unwrap().unwrap();
assert!(scans.lock().is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn unknown_pid_reports_no_jvm_process() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_memory);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::NoJvmProcess, can_count_when_paused: false }
);
}
fn kotlin_attach(project: &std::path::Path, host: &str, port: u16) -> LaunchConfig {
LaunchConfig {
adapter: kotlin::ADAPTER.into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(json!({"projectRoot": project, "hostName": host, "port": port}).to_string()),
..java_config("attach")
}
}
#[test]
fn link_options_hold_only_for_fwcd() {
assert!(kotlin::link_options(kotlin::ADAPTER).hold_resumes);
assert!(!kotlin::link_options(kotlin_lsp::ADAPTER).hold_resumes);
assert!(!kotlin::link_options("x").hold_resumes);
}
#[tokio::test]
async fn kotlin_session_installs_link_inspector() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some());
assert!(session.jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn attach_target_recorded_before_kotlin_rewrite() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "10.0.0.5", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.attach_target.get(), Some(&("10.0.0.5".to_owned(), 5005)));
let sent = &h.request("attach")["arguments"];
assert_eq!(sent["hostName"], "127.0.0.1");
let link_port = sent["port"].as_u64().unwrap();
assert!(link_port != 0 && link_port != 5005);
}
#[tokio::test]
async fn java_attach_records_target() {
let h = harness(Script::default());
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "localhost", "port": 5005}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.attach_target.get(), Some(&("localhost".to_owned(), 5005)));
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
#[tokio::test]
async fn attach_target_recorded_for_any_adapter() {
let h = harness(Script::default());
let mut cfg = java_config("attach");
// "x" is not a registered adapter; debugpy has no JVM link or memory support.
cfg.adapter = "debugpy".into();
cfg.extra_json = Some(json!({"hostName": "h", "port": 7}).to_string());
let id = h.start(cfg).await.unwrap();
assert_eq!(h.manager.session(id).unwrap().attach_target.get(), Some(&("h".to_owned(), 7)));
}
#[tokio::test]
async fn launch_sessions_have_no_attach_target() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert_eq!(h.manager.session(id).unwrap().attach_target.get(), None);
}
#[tokio::test]
async fn closing_the_session_drops_the_link() {
let h = harness(Script { hang_up_on_pause: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some() && session.jvm_inspector.lock().is_some());
let _ = session.conn.request_timeout("pause", json!({"threadId": 1}), Duration::from_secs(2)).await;
for _ in 0..200 {
if h.manager.session(id).is_err() {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
/// kotlin-lsp's debugger, reached through the language server's `debugServer` port (the fake ignores it).
fn kotlin_lsp_attach(project: &std::path::Path) -> LaunchConfig {
LaunchConfig {
adapter: kotlin_lsp::ADAPTER.into(),
request: "attach".into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(
json!({"debugServer": 4711, "hostName": "127.0.0.1", "port": 5005, "projectRoot": project}).to_string(),
),
..config()
}
}
#[tokio::test]
async fn kotlin_lsp_initializes_as_intellij_jvm() {
let project = tempfile::tempdir().unwrap();
for (config, adapter_id, invalidated) in [
(kotlin_lsp_attach(project.path()), "intellij_jvm", true),
(kotlin_attach(project.path(), "127.0.0.1", 5005), "kotlin-debug-adapter", false),
] {
let h = harness(Script::default());
let id = h.start(config).await.unwrap();
let init = &h.request("initialize")["arguments"];
assert_eq!(init["adapterID"], adapter_id);
assert_eq!(init["supportsInvalidatedEvent"], invalidated, "{adapter_id}");
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn kotlin_lsp_gets_real_paths_and_its_link_holds_nothing() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let path = project.path().join("Main.kt");
std::fs::write(&path, "package org.example\nfun main() {}\n").unwrap();
h.manager.set_breakpoints(path.clone(), vec![bp(1)]).await.unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
assert_eq!(h.request("setBreakpoints")["arguments"]["source"]["path"], json!(path));
let sent = &h.request("attach")["arguments"];
assert_eq!(sent["projectRoot"], json!(project.path()));
assert!(sent.get("debugServer").is_none());
assert_eq!(sent["hostName"], "127.0.0.1");
let link_port = sent["port"].as_u64().unwrap();
assert!(link_port != 0 && link_port != 5005);
let session = h.manager.session(id).unwrap();
assert!(session.kotlin_mirror.lock().is_none());
assert!(session.jdwp_link.lock().is_some() && session.jvm_inspector.lock().is_some());
assert_eq!(session.attach_target.get(), Some(&("127.0.0.1".to_owned(), 5005)));
assert!(!kotlin::link_options(&session.adapter).hold_resumes);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn kotlin_lsp_waits_for_the_configuration_done_reply() {
let project = tempfile::tempdir().unwrap();
// fwcd never answers it (`kotlin_configures_without_configuration_done_response`).
let h = harness(Script { ignore_configuration_done: true, ..Default::default() });
let fwcd = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005));
let id = tokio::time::timeout(Duration::from_secs(2), fwcd).await.unwrap().unwrap();
h.manager.stop(id).await.unwrap();
tokio::time::pause();
let h = harness(Script { ignore_configuration_done: true, ..Default::default() });
let started = tokio::time::Instant::now();
let err = h.start(kotlin_lsp_attach(project.path())).await.unwrap_err();
assert!(started.elapsed() >= REQUEST_TIMEOUT, "{:?}", started.elapsed());
assert!(err.message.contains("configurationDone"), "{err:?}");
assert!(h.manager.sessions(None).is_empty());
}
#[tokio::test]
async fn kotlin_lsp_sessions_have_the_memory_tab() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_memory);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn losing_the_kotlin_lsp_connection_ends_the_session() {
let h = harness(Script { hang_up_after_stop: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some());
let _ = session.conn.request_timeout("threads", Value::Null, Duration::from_secs(2)).await;
h.wait_event(|e| matches!(e, DebugEvent::Terminated)).await;
assert!(h.manager.sessions(None).is_empty());
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
/// The reasons of the first `count` stops, once that many arrived.
async fn stop_reasons(h: &Harness, count: usize) -> Vec<String> {
for _ in 0..200 {
let reasons: Vec<String> = h
.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event: DebugEvent::Stopped { reason, .. }, .. } => Some(reason.clone()),
_ => None,
})
.collect();
if reasons.len() >= count {
return reasons;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("{count} stops not received; got {:?}", h.events.lock());
}
#[tokio::test]
async fn a_step_stop_is_reported_as_a_step() {
let project = tempfile::tempdir().unwrap();
for (config, reason) in [(kotlin_lsp_attach(project.path()), "step"), (config(), "breakpoint")] {
let h = harness(Script { step_reason: Some("breakpoint"), ..Script::default() });
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.step(id, 1, StepKind::Over).await.unwrap();
assert_eq!(stop_reasons(&h, 2).await, ["breakpoint", reason]);
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn a_breakpoint_after_continue_stays_a_breakpoint() {
let h = harness(Script { step_reason: Some("breakpoint"), stop_after_continue: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.step(id, 1, StepKind::Over).await.unwrap();
stop_reasons(&h, 2).await;
// The fake ends `stepIn` with no stop: the `continue` after it decides the next one.
h.manager.step(id, 1, StepKind::Into).await.unwrap();
h.manager.resume(id, 1).await.unwrap();
assert_eq!(stop_reasons(&h, 3).await, ["breakpoint", "step", "breakpoint"]);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn a_breakpoint_after_a_refused_step_stays_a_breakpoint() {
let h = harness(Script { refuse_next: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert!(h.manager.step(id, 1, StepKind::Over).await.is_err());
assert_eq!(stop_reasons(&h, 2).await, ["breakpoint", "breakpoint"]);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn a_thread_from_an_earlier_stop_is_refused() {
let project = tempfile::tempdir().unwrap();
for (config, kotlin_lsp) in [(kotlin_lsp_attach(project.path()), true), (config(), false)] {
let h = harness(Script::default());
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(h.manager.threads(id).await.unwrap().len(), 1);
let started = tokio::time::Instant::now();
let stale = h.manager.stack_trace(id, 99).await;
let asked =
|command: &str| h.log.lock().iter().any(|m| m["command"] == command && m["arguments"]["threadId"] == 99);
if kotlin_lsp {
assert!(stale.is_err() && started.elapsed() < Duration::from_millis(100), "{stale:?}");
assert!(h.manager.step(id, 99, StepKind::Over).await.is_err());
assert!(h.manager.resume(id, 99).await.is_err());
assert!(!asked("stackTrace") && !asked("next") && !asked("continue"));
} else {
assert!(stale.is_ok() && asked("stackTrace"), "{stale:?}");
}
assert!(h.manager.stack_trace(id, 1).await.is_ok());
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn a_threads_reply_from_before_a_stop_does_not_vouch_for_its_ids() {
let h = harness(Script { stop_during_threads: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let threads = h.manager.threads(id).await.unwrap();
// M3 maps kotlin-lsp's threads by these names.
assert_eq!((threads[0].id, threads[0].name.as_str()), (1, "\"main\"@1"));
stop_reasons(&h, 2).await;
assert!(h.manager.stack_trace(id, 1).await.is_err());
assert!(h.manager.stack_trace(id, 2).await.is_ok());
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn kotlin_lsp_frames_get_real_paths() {
let project = tempfile::tempdir().unwrap();
let at = "at com.x.AppKt.main(App.kt:13)";
for (config, path, name) in [
(kotlin_lsp_attach(project.path()), "/p/App.kt", "com.x.AppKt.main(App.kt:13)"),
(java_config("launch"), "file:///p/App.kt", at),
] {
let script = Script { source_path: Some("file:///p/App.kt".into()), frame_name: Some(at), ..Script::default() };
let h = harness(script);
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let frames = h.manager.stack_trace(id, 1).await.unwrap();
assert_eq!(frames[0].path.as_ref().map(|p| p.0.clone()), Some(PathBuf::from(path)));
assert_eq!(frames[0].name, name);
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn invalidated_events_reach_the_client() {
let h = harness(Script { invalidated_after_stop: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
match h.wait_event(|e| matches!(e, DebugEvent::Invalidated { .. })).await {
DebugEvent::Invalidated { areas } => assert_eq!(areas, ["variables"]),
other => panic!("{other:?}"),
}
h.manager.stop(id).await.unwrap();
}
#[cfg(unix)]
#[tokio::test]
async fn rdbg_drains_test_output_after_dap_closes_before_killing_the_process() {
-401
Переглянути файл
@@ -1,401 +0,0 @@
//! Instances, fields and referrers through a session's JVM inspector.
use super::*;
use crate::jvm_memory::test_support::FakeInspector;
use ide_core::event::JvmBackendState;
use ide_jvm_inspect::{InspectError, JvmValue, VmTarget};
use tokio::io::AsyncReadExt;
fn debug_events(h: &Harness) -> Vec<DebugEvent> {
h.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event, .. } => Some(event.clone()),
_ => None,
})
.collect()
}
fn backend_events(h: &Harness) -> Vec<JvmBackendState> {
debug_events(h)
.into_iter()
.filter_map(|e| match e {
DebugEvent::JvmBackend { state } => Some(state),
_ => None,
})
.collect()
}
/// A java-debug session stopped at its breakpoint, inspected by `fake`.
async fn stopped_with(h: &Harness, config: LaunchConfig, fake: &Arc<FakeInspector>) -> DebugSessionId {
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
id
}
fn object(id: u64) -> JvmValue {
JvmValue::Object { id, type_name: "A".into() }
}
const NOT_SUSPENDED: JvmInstancesResult =
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NotSuspended };
#[tokio::test]
async fn instances_need_a_suspended_session() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
let fake = Arc::new(FakeInspector::default());
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
assert_eq!(h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(), NOT_SUSPENDED);
assert_eq!(fake.calls(), 0);
}
#[tokio::test]
async fn instances_without_an_inspector_have_no_backend() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
}
#[tokio::test]
async fn object_from_previous_stop_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let JvmInstancesResult::Ready { generation, instances, truncated } =
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap()
else {
panic!("instances unavailable");
};
assert_eq!((generation, instances, truncated), (1, vec![object(7)], false));
h.manager.session(id).unwrap().suspended.stopped(None, true);
let stale = JvmObjectRef { generation, id: 7 };
assert_eq!(
h.manager.jvm_memory_fields(id, stale).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(
h.manager.jvm_memory_referrers(id, stale, 10).await.unwrap(),
JvmReferrersResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(fake.calls(), 1, "a stale object never reaches the inspector");
let current = JvmObjectRef { generation: 2, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, current).await.unwrap(), JvmFieldsResult::Ready { fields: Vec::new() });
}
#[tokio::test]
async fn instances_stay_readable_while_another_thread_is_stopped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let session = h.manager.session(id).unwrap();
// Threads 20, 21, then 22 stop alone (SUSPEND_EVENT_THREAD).
session.suspended.continued(Some(1), true);
for thread in [20, 21, 22] {
session.suspended.stopped(Some(thread), false);
}
let instances = || h.manager.jvm_memory_instances(id, "A".into(), 10);
let result = instances().await.unwrap();
let JvmInstancesResult::Ready { generation, .. } = result else { panic!("{result:?}") };
let read_on_22 = JvmObjectRef { generation, id: 7 };
// An earlier thread resumes: the stop on 22 goes on, and so does what was read on it.
session.suspended.continued(Some(20), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Ready { fields: Vec::new() }
);
// The latest one resumes: 21 is still stopped, but what was read on 22 is stale.
session.suspended.continued(Some(22), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
let result = instances().await.unwrap();
assert!(matches!(result, JvmInstancesResult::Ready { .. }), "{result:?}");
}
/// Starts `instances` on a held fake inspector; returns once the inspector was entered.
async fn held_request(
h: &Harness,
id: DebugSessionId,
fake: &Arc<FakeInspector>,
) -> tokio::task::JoinHandle<IdeResult<JvmInstancesResult>> {
let manager = h.manager.clone();
let request = tokio::spawn(async move { manager.jvm_memory_instances(id, "A".into(), 10).await });
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
request
}
#[tokio::test]
async fn stop_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.stopped(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn resume_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.continued(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn one_request_at_a_time_per_session() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let first = held_request(&h, id, &fake).await;
let manager = h.manager.clone();
let second = tokio::spawn(async move { manager.jvm_memory_instances(id, "B".into(), 10).await });
tokio::time::sleep(Duration::from_millis(50)).await;
assert_eq!(fake.calls(), 1, "the second request waits for the first");
fake.release.notify_one();
first.await.unwrap().unwrap();
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
fake.release.notify_one();
second.await.unwrap().unwrap();
assert_eq!(fake.calls(), 2);
}
#[tokio::test]
async fn session_end_lets_go_of_a_waiting_inspector() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().kill_processes();
assert_eq!(
tokio::time::timeout(Duration::from_secs(2), request).await.unwrap().unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::BackendClosed }
);
assert_eq!(Arc::strong_count(&fake), 1, "no clone of the inspector outlives the session");
}
#[tokio::test]
async fn target_carries_pid_for_launch() {
let h = harness(Script { debuggee_pid: Some(4242), java_process_id: true, ..Script::default() });
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: Some(4242), attach_host: None, attach_port: None, generation: 1 }]
);
}
#[tokio::test]
async fn target_carries_attach_port_and_host() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: None, attach_host: Some("10.0.0.5".into()), attach_port: Some(5005), generation: 1 }]
);
}
#[tokio::test]
async fn unsupported_backend_turns_instances_off() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Unsupported("x".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let before = debug_events(&h).len();
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Unsupported { detail: "x".into() } }
);
let after: Vec<_> = debug_events(&h).split_off(before);
assert!(
matches!(
after.as_slice(),
[DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail } }, DebugEvent::Capabilities]
if detail == "x"
),
"{after:?}"
);
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
assert_eq!(fake.calls(), 1);
}
/// `error` is reported but leaves the backend on.
async fn keeps_the_backend(error: InspectError, reason: JvmInspectUnavailable) {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(error));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let object = JvmObjectRef { generation: 1, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, object).await.unwrap(), JvmFieldsResult::Unavailable { reason });
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(backend_events(&h).is_empty());
assert!(h.manager.session(id).unwrap().jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn collected_keeps_the_backend() {
keeps_the_backend(InspectError::Collected, JvmInspectUnavailable::Collected).await;
}
#[tokio::test]
async fn timed_out_keeps_the_backend() {
keeps_the_backend(InspectError::TimedOut, JvmInspectUnavailable::TimedOut).await;
}
#[tokio::test]
async fn failed_inspection_is_an_error() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Failed("boom".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let err = h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap_err();
assert!(err.to_string().contains("boom"), "{err}");
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
}
#[tokio::test]
async fn limits_are_clamped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
for limit in [5000, 0] {
h.manager.jvm_memory_instances(id, "A".into(), limit).await.unwrap();
}
let object = JvmObjectRef { generation: 1, id: 7 };
for limit in [0, 900, 20] {
h.manager.jvm_memory_referrers(id, object, limit).await.unwrap();
}
assert_eq!(*fake.limits.lock(), [1000, 1, 50, 200, 20]);
}
#[tokio::test]
async fn a_full_page_is_truncated() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().extend([object(1), object(2)]);
let id = stopped_with(&h, java_config("launch"), &fake).await;
let truncated = |r: JvmInstancesResult| match r {
JvmInstancesResult::Ready { truncated, .. } => truncated,
other => panic!("{other:?}"),
};
assert!(truncated(h.manager.jvm_memory_instances(id, "A".into(), 2).await.unwrap()));
assert!(!truncated(h.manager.jvm_memory_instances(id, "A".into(), 3).await.unwrap()));
}
#[tokio::test]
async fn capability_follows_the_inspector() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
let before = debug_events(&h).len();
h.manager.set_jvm_inspector(id, Arc::new(FakeInspector::default())).unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(matches!(debug_events(&h).split_off(before).as_slice(), [DebugEvent::Capabilities]));
}
#[tokio::test]
async fn remote_attach_counts_classes_through_the_debugger() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
fake.counts.lock().push(ide_jvm_inspect::ClassCount { name: "A".into(), instances: 3 });
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
let result = h.manager.jvm_memory_classes(id).await.unwrap();
assert!(
matches!(&result, JvmMemoryResult::Ready { snapshot }
if snapshot.source == JvmMemorySource::Debugger && snapshot.rows.len() == 1),
"{result:?}"
);
assert_eq!(fake.targets.lock()[0].attach_port, Some(5005));
}
const HANDSHAKE: &[u8; 14] = b"JDWP-Handshake";
/// A Kotlin attach whose link reaches a fake JVM: (adapter end, JVM end) after both handshakes.
async fn linked_kotlin(h: &Harness) -> (DebugSessionId, tokio::net::TcpStream, tokio::net::TcpStream) {
let jvm = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", jvm.local_addr().unwrap().port())).await.unwrap();
let link_port = h.request("attach")["arguments"]["port"].as_u64().unwrap() as u16;
let mut adapter = tokio::net::TcpStream::connect(("127.0.0.1", link_port)).await.unwrap();
adapter.write_all(HANDSHAKE).await.unwrap();
let (mut jvm, _) = tokio::time::timeout(Duration::from_secs(2), jvm.accept()).await.unwrap().unwrap();
let mut buf = [0; 14];
jvm.read_exact(&mut buf).await.unwrap();
jvm.write_all(HANDSHAKE).await.unwrap();
adapter.read_exact(&mut buf).await.unwrap();
(id, adapter, jvm)
}
/// A command the adapter sends with an id from the link's own range: the link then only forwards.
async fn degrade(adapter: &mut tokio::net::TcpStream) {
let mut packet = 11u32.to_be_bytes().to_vec();
packet.extend(ide_jdwp::OWN_IDS.start.to_be_bytes());
packet.extend([0, 1, 1]);
adapter.write_all(&packet).await.unwrap();
}
async fn wait_backend(h: &Harness, want: impl Fn(&JvmBackendState) -> bool) {
h.wait_event(|e| matches!(e, DebugEvent::JvmBackend { state } if want(state))).await;
}
#[tokio::test]
async fn kotlin_link_reports_degraded_then_closed() {
let h = harness(Script::default());
let (id, mut adapter, jvm) = linked_kotlin(&h).await;
degrade(&mut adapter).await;
wait_backend(&h, |s| matches!(s, JvmBackendState::Degraded { .. })).await;
drop(jvm);
wait_backend(&h, |s| *s == JvmBackendState::Closed).await;
assert_eq!(backend_events(&h).len(), 2, "{:?}", backend_events(&h));
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some(), "a closed link is kept until the session ends");
}
#[tokio::test]
async fn kotlin_link_is_quiet_once_the_program_ended() {
let h = harness(Script::default());
let (id, _adapter, jvm) = linked_kotlin(&h).await;
let session = h.manager.session(id).unwrap();
// The adapter reported the end; the session keeps its link until it shuts down.
session.terminated.store(true, Ordering::Release);
drop(jvm);
for _ in 0..100 {
if session.jdwp_link.lock().as_ref().is_some_and(|l| l.state() == ide_jdwp::LinkState::Closed) {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(backend_events(&h).is_empty(), "{:?}", backend_events(&h));
}
-6
Переглянути файл
@@ -17,7 +17,6 @@ blake3.workspace = true
tokio.workspace = true
tracing.workspace = true
parking_lot.workspace = true
notify.workspace = true
futures.workspace = true
sqlx.workspace = true
tiberius.workspace = true
@@ -30,11 +29,6 @@ ide-http.workspace = true
reqwest = { workspace = true, features = ["query"] }
rustls.workspace = true
uuid.workspace = true
base64.workspace = true
flate2 = "1"
rsa = { version = "0.10.0-rc.18", features = ["sha2", "pkcs5"] }
zeroize = "1"
open = "5"
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
-188
Переглянути файл
@@ -1,188 +0,0 @@
//! The introspected catalog of a data source, kept on disk: the schemas, tables and views of
//! its databases, so completion knows them from the start of the next run without reading the
//! server again. Names only — no data, no credentials. One file per project and data source in
//! the app data directory.
use crate::TableInfo;
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::{Path, PathBuf};
/// A schema of a database and its tables and views.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbCatalogSchema")]
pub struct CatalogSchema {
pub name: String,
pub tables: Vec<TableInfo>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbCatalogDatabase")]
pub struct CatalogDatabase {
pub name: String,
/// When this database was read, in milliseconds since the Unix epoch.
pub read_at: f64,
pub schemas: Vec<CatalogSchema>,
}
/// What introspection has read of a data source.
#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", default)]
#[serde(rename = "DbSourceCatalog")]
pub struct SourceCatalog {
pub databases: Vec<CatalogDatabase>,
}
pub(crate) struct CatalogStore {
dir: PathBuf,
project: String,
/// One writer at a time: an update reads the file, changes a database and writes it back.
writing: tokio::sync::Mutex<()>,
}
fn now_ms() -> f64 {
std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0.0, |d| d.as_millis() as f64)
}
impl CatalogStore {
pub(crate) fn new(dir: PathBuf, root: &Path) -> Self {
Self { dir, project: root.to_string_lossy().into_owned(), writing: tokio::sync::Mutex::new(()) }
}
fn file(&self, id: &str) -> PathBuf {
let name = blake3::hash(format!("{}\0{id}", self.project).as_bytes()).to_hex();
self.dir.join(format!("{}.json", &name.as_str()[..32]))
}
fn read(file: &Path) -> SourceCatalog {
std::fs::read(file).ok().and_then(|bytes| serde_json::from_slice(&bytes).ok()).unwrap_or_default()
}
fn write(file: &Path, catalog: &SourceCatalog) -> IdeResult<()> {
if let Some(dir) = file.parent() {
std::fs::create_dir_all(dir)?;
}
// Written beside the file and moved over it, so a crash leaves the old catalog whole.
let partial = file.with_extension("json.part");
let bytes = serde_json::to_vec(catalog).map_err(|e| IdeError::internal(e.to_string()))?;
std::fs::write(&partial, bytes)?;
std::fs::rename(&partial, file)?;
Ok(())
}
async fn blocking<T: Send + 'static>(task: impl FnOnce() -> T + Send + 'static) -> IdeResult<T> {
tokio::task::spawn_blocking(task).await.map_err(|e| IdeError::internal(e.to_string()))
}
/// The saved catalog of data source `id` (`None`: never introspected).
pub(crate) async fn load(&self, id: &str) -> IdeResult<Option<SourceCatalog>> {
let file = self.file(id);
Self::blocking(move || file.exists().then(|| Self::read(&file))).await
}
/// Saves what was just read of `database`, replacing what was known of it.
pub(crate) async fn put(&self, id: &str, database: &str, schemas: Vec<CatalogSchema>) -> IdeResult<()> {
let _writing = self.writing.lock().await;
let (file, name) = (self.file(id), database.to_owned());
Self::blocking(move || {
let mut catalog = Self::read(&file);
catalog.databases.retain(|d| d.name != name);
catalog.databases.push(CatalogDatabase { name, read_at: now_ms(), schemas });
catalog.databases.sort_by(|a, b| a.name.cmp(&b.name));
Self::write(&file, &catalog)
})
.await?
}
/// Forgets the databases not in `keep` (dropped on the server, or no longer shown).
pub(crate) async fn retain(&self, id: &str, keep: Vec<String>) -> IdeResult<()> {
let _writing = self.writing.lock().await;
let file = self.file(id);
Self::blocking(move || {
if !file.exists() {
return Ok(());
}
let mut catalog = Self::read(&file);
let before = catalog.databases.len();
catalog.databases.retain(|d| keep.contains(&d.name));
if catalog.databases.len() == before { Ok(()) } else { Self::write(&file, &catalog) }
})
.await?
}
/// Deletes the saved catalog of data source `id` (the data source was removed).
pub(crate) fn forget(&self, id: &str) {
let _ = std::fs::remove_file(self.file(id));
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::DbTableKind;
fn schema(name: &str, tables: &[&str]) -> CatalogSchema {
CatalogSchema {
name: name.into(),
tables: tables.iter().map(|t| TableInfo { name: (*t).into(), kind: DbTableKind::Table }).collect(),
}
}
#[tokio::test]
async fn a_catalog_survives_a_restart_per_project_and_data_source() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/shop"));
assert_eq!(store.load("sf").await.unwrap(), None);
store.put("sf", "RAW", vec![schema("PUBLIC", &["EVENTS"])]).await.unwrap();
store
.put("sf", "ANALYTICS", vec![schema("PUBLIC", &["ORDERS", "USERS"]), schema("STAGING", &[])])
.await
.unwrap();
// A new store over the same directory: what the next run of the app sees.
let reopened = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/shop"));
let saved = reopened.load("sf").await.unwrap().unwrap();
assert_eq!(saved.databases.iter().map(|d| d.name.as_str()).collect::<Vec<_>>(), ["ANALYTICS", "RAW"]);
assert_eq!(saved.databases[0].schemas, vec![schema("PUBLIC", &["ORDERS", "USERS"]), schema("STAGING", &[])]);
assert!(saved.databases.iter().all(|d| d.read_at > 0.0));
// Another data source and another project keep their own.
assert_eq!(reopened.load("other").await.unwrap(), None);
let elsewhere = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/blog"));
assert_eq!(elsewhere.load("sf").await.unwrap(), None);
}
#[tokio::test]
async fn rereading_replaces_a_database_and_dropped_ones_are_forgotten() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().to_path_buf(), Path::new("/p"));
store.put("sf", "RAW", vec![schema("PUBLIC", &["OLD"])]).await.unwrap();
store.put("sf", "GONE", vec![schema("PUBLIC", &["T"])]).await.unwrap();
store.put("sf", "RAW", vec![schema("PUBLIC", &["NEW"])]).await.unwrap();
store.retain("sf", vec!["RAW".into()]).await.unwrap();
let saved = store.load("sf").await.unwrap().unwrap();
assert_eq!(saved.databases.len(), 1);
assert_eq!(saved.databases[0].schemas, vec![schema("PUBLIC", &["NEW"])]);
// Nothing half-written is left beside the file.
let files: Vec<_> = std::fs::read_dir(dir.path()).unwrap().map(|f| f.unwrap().file_name()).collect();
assert_eq!(files.len(), 1);
store.forget("sf");
assert_eq!(store.load("sf").await.unwrap(), None);
store.retain("sf", vec![]).await.unwrap();
assert_eq!(store.load("sf").await.unwrap(), None);
}
#[tokio::test]
async fn a_damaged_file_reads_as_an_empty_catalog() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().to_path_buf(), Path::new("/p"));
std::fs::write(store.file("sf"), b"{ not json").unwrap();
assert_eq!(store.load("sf").await.unwrap(), Some(SourceCatalog::default()));
store.put("sf", "RAW", vec![schema("PUBLIC", &["T"])]).await.unwrap();
assert_eq!(store.load("sf").await.unwrap().unwrap().databases.len(), 1);
}
}
+9 -85
Переглянути файл
@@ -238,7 +238,6 @@ pub enum Pool {
SqlServer(Bb8Pool<MssqlConnectionManager>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(crate::oracle::OraclePool),
Snowflake(crate::snowflake::SnowflakePool),
}
pub(crate) fn db_err(e: sqlx::Error) -> IdeError {
@@ -284,16 +283,6 @@ fn mssql_pool_err(e: bb8::RunError<tiberius::error::Error>) -> IdeError {
}
}
/// Tiberius requires TLS when a TLS backend is enabled. `trust_cert` only skips
/// certificate checks, so Disable has to turn encryption off.
fn apply_sqlserver_ssl(config: &mut tiberius::Config, ssl: DbSslMode) {
match ssl {
DbSslMode::Disable => config.encryption(tiberius::EncryptionLevel::NotSupported),
DbSslMode::Prefer => config.trust_cert(),
DbSslMode::Require | DbSslMode::VerifyCa | DbSslMode::VerifyFull => {}
}
}
const DAYS_1900_01_01: i64 = 693_595;
/// Formats a proleptic Gregorian date given days since `0001-01-01`.
@@ -704,9 +693,6 @@ impl Setup {
if source.driver == DbDriver::Clickhouse && !o.time_zone.trim().is_empty() {
s.params.push(("session_timezone".into(), o.time_zone.trim().into()));
}
if source.driver == DbDriver::Snowflake && !o.time_zone.trim().is_empty() {
s.params.push(("TIMEZONE".into(), o.time_zone.trim().into()));
}
let script = o.startup_script.trim();
if !script.is_empty() {
s.statements.extend(crate::split::split(script, source.driver).into_iter().map(|st| st.text));
@@ -724,7 +710,6 @@ impl Pool {
Pool::SqlServer(_) => DbDriver::SqlServer,
Pool::Clickhouse(_) => DbDriver::Clickhouse,
Pool::Oracle(_) => DbDriver::Oracle,
Pool::Snowflake(_) => DbDriver::Snowflake,
}
}
@@ -746,9 +731,6 @@ impl Pool {
if source.driver == DbDriver::Oracle {
return crate::oracle::OraclePool::connect(source, password, &setup).await.map(Pool::Oracle);
}
if source.driver == DbDriver::Snowflake {
return crate::snowflake::SnowflakePool::connect(source, password, setup).await.map(Pool::Snowflake);
}
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let init = Arc::new(setup.statements);
// Runs the session statements (startup script…) on every new connection.
@@ -894,7 +876,9 @@ impl Pool {
if !source.database.is_empty() {
config.database(&source.database);
}
apply_sqlserver_ssl(&mut config, source.ssl);
if matches!(source.ssl, DbSslMode::Disable | DbSslMode::Prefer) {
config.trust_cert();
}
let manager = MssqlConnectionManager(config);
let pool = bb8::Pool::builder()
.max_size(MAX)
@@ -907,7 +891,7 @@ impl Pool {
Ok(Pool::SqlServer(pool))
}
DbDriver::Clickhouse => unreachable!(),
DbDriver::Oracle | DbDriver::Snowflake => unreachable!(),
DbDriver::Oracle => unreachable!(),
}
}
@@ -915,7 +899,6 @@ impl Pool {
match self {
Pool::SqlServer(_) | Pool::Oracle(_) => {}
Pool::Clickhouse(p) => p.close(),
Pool::Snowflake(p) => p.close().await,
Pool::Sqlite(p) => p.close().await,
Pool::Postgres(p) => p.closed.store(true, Ordering::Relaxed),
Pool::Mysql(p) => p.closed.store(true, Ordering::Relaxed),
@@ -927,7 +910,6 @@ impl Pool {
Pool::SqlServer(_) => false,
Pool::Oracle(p) => p.is_closed(),
Pool::Clickhouse(p) => p.is_closed(),
Pool::Snowflake(p) => p.is_closed(),
Pool::Sqlite(p) => p.is_closed(),
Pool::Postgres(p) => p.closed.load(Ordering::Relaxed),
Pool::Mysql(p) => p.closed.load(Ordering::Relaxed),
@@ -939,9 +921,6 @@ impl Pool {
if let Pool::Clickhouse(p) = self {
return Ok(p.execute(sql, max, &CancellationToken::new()).await?.rows);
}
if let Pool::Snowflake(p) = self {
return Ok(p.metadata().execute(sql, max, &CancellationToken::new()).await?.rows);
}
let mut conn = Held::acquire(self).await?;
conn.apply_context(None).await?;
let fetched = conn.fetch_rows(sql, max).await;
@@ -966,9 +945,6 @@ impl Pool {
if let Pool::Clickhouse(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
if let Pool::Snowflake(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
let mut conn = Held::acquire(self).await?;
let r = match conn.apply_context(None).await {
Ok(()) => conn.execute_from(self, sql, skip, max_rows, cancel).await,
@@ -986,7 +962,6 @@ impl Pool {
pub async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
match self {
Pool::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Pool::Snowflake(p) => p.transaction(statements).await,
Pool::SqlServer(_) => {
let mut held = Held::acquire(self).await?;
held.apply_context(None).await?;
@@ -1068,8 +1043,6 @@ pub enum Held {
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(Box<crate::oracle::OracleConnection>),
/// The shared session, and the schema this holder's statements run in.
Snowflake(crate::snowflake::SnowflakePool, Option<String>),
}
impl Held {
@@ -1086,12 +1059,6 @@ impl Held {
Held::Clickhouse(p.clone())
}
Pool::Oracle(p) => Held::Oracle(Box::new(p.get().await?)),
Pool::Snowflake(p) => {
if p.is_closed() {
return Err(IdeError::new(ErrorKind::Io, "Snowflake connection is closed"));
}
Held::Snowflake(p.clone(), None)
}
})
}
@@ -1136,11 +1103,6 @@ impl Held {
}
pub async fn apply_context(&mut self, schema: Option<&str>) -> IdeResult<()> {
if let Held::Snowflake(_, wanted) = self {
// Applied with each statement, while it holds the shared session.
*wanted = schema.filter(|s| !s.is_empty()).map(str::to_owned);
return Ok(());
}
let driver = match self {
Held::Postgres(_) => DbDriver::Postgres,
Held::Mysql(_) => DbDriver::Mysql,
@@ -1196,7 +1158,7 @@ impl Held {
Held::Mysql(c) => fetch!(&mut ****c, sql, max, Cells::Text),
Held::SqlServer(c) => mssql_fetch(c, sql, 0, max).await,
Held::Oracle(c) => crate::oracle::run(c, sql, 0, max, Cells::Text, &CancellationToken::new()).await,
Held::Clickhouse(_) | Held::Snowflake(..) => unreachable!(),
Held::Clickhouse(_) => unreachable!(),
};
if result.is_ok() {
self.after_statement(sql);
@@ -1218,12 +1180,6 @@ impl Held {
match self {
Held::SqlServer(c) => mssql_command(c, sql).await,
Held::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Held::Snowflake(c, schema) => {
if sql.trim().is_empty() {
return Ok(());
}
c.execute_in(schema.as_deref(), sql, 0, 0, &CancellationToken::new()).await.map(drop)
}
Held::Oracle(c) => {
let sql = sql.trim();
if sql.is_empty() {
@@ -1237,9 +1193,7 @@ impl Held {
Held::Sqlite(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Postgres(c) => sqlx::raw_sql(sql).execute(&mut ****c).await.map(drop),
Held::Mysql(c) => sqlx::raw_sql(sql).execute(&mut ****c).await.map(drop),
Held::SqlServer(_) | Held::Clickhouse(_) | Held::Oracle(_) | Held::Snowflake(..) => {
unreachable!()
}
Held::SqlServer(_) | Held::Clickhouse(_) | Held::Oracle(_) => unreachable!(),
}
.map_err(db_err)
}
@@ -1264,7 +1218,7 @@ impl Held {
Held::SqlServer(mut c) => {
c.0.take();
}
Held::Clickhouse(_) | Held::Snowflake(..) => {}
Held::Clickhouse(_) => {}
Held::Oracle(mut c) => c.discard_inner(),
}
}
@@ -1294,9 +1248,6 @@ impl Held {
if let Held::Clickhouse(c) = self {
return c.execute_from(sql, skip, max_rows, cancel).await;
}
if let Held::Snowflake(c, schema) = self {
return c.execute_in(schema.as_deref(), sql, skip, max_rows, cancel).await;
}
self.before_statement(sql);
let started = Instant::now();
// `$id`: the query naming the server process, `$kill`: how another connection stops it.
@@ -1341,7 +1292,7 @@ impl Held {
}
}
Held::Oracle(c) => crate::oracle::run(c, sql, skip, max_rows, Cells::Display, cancel).await?,
Held::Clickhouse(_) | Held::Snowflake(..) => unreachable!(),
Held::Clickhouse(_) => unreachable!(),
};
self.after_statement(sql);
let (columns, rows, affected, has_more, first) = fetched;
@@ -1364,7 +1315,7 @@ pub fn begin_statement(driver: DbDriver) -> &'static str {
DbDriver::Mysql => "START TRANSACTION",
DbDriver::SqlServer => "BEGIN TRAN",
DbDriver::Oracle => "", // opens implicitly with autocommit off
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse | DbDriver::Snowflake => "BEGIN",
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse => "BEGIN",
}
}
@@ -1380,33 +1331,6 @@ mod tests {
use super::*;
use crate::source::{DbOptions, DbProperty};
fn sqlserver_debug(ssl: DbSslMode) -> String {
let mut config = tiberius::Config::new();
apply_sqlserver_ssl(&mut config, ssl);
format!("{config:?}")
}
#[test]
fn sqlserver_disable_does_not_require_tls() {
let debug = sqlserver_debug(DbSslMode::Disable);
assert!(debug.contains("encryption: NotSupported"), "{debug}");
assert!(debug.contains("bypass: false"), "{debug}");
}
#[test]
fn sqlserver_prefer_trusts_the_server_certificate() {
let debug = sqlserver_debug(DbSslMode::Prefer);
assert!(debug.contains("encryption: Required"), "{debug}");
assert!(debug.contains("bypass: true"), "{debug}");
}
#[test]
fn sqlserver_require_keeps_validated_tls() {
let debug = sqlserver_debug(DbSslMode::Require);
assert!(debug.contains("encryption: Required"), "{debug}");
assert!(debug.contains("bypass: false"), "{debug}");
}
#[test]
fn discarded_sqlserver_connections_are_broken() {
let manager = MssqlConnectionManager(tiberius::Config::new());
+14 -249
Переглянути файл
@@ -2,7 +2,6 @@
//! execution ([`conn`]), introspection, table data paging / editing ([`sql`])
//! and statement splitting ([`split`]), for SQLite, PostgreSQL and MySQL/MariaDB.
pub mod catalog;
mod clickhouse;
pub mod conn;
pub mod history;
@@ -10,20 +9,15 @@ mod oracle;
pub mod plan;
pub mod plugin;
pub mod secrets;
mod snowflake;
pub mod source;
pub mod split;
pub mod sql;
mod watch;
pub use watch::SourcesWatch;
pub use catalog::{CatalogDatabase, CatalogSchema, SourceCatalog};
pub use conn::{Held, Pool, QueryResult, ResultColumn};
pub use history::HistoryEntry;
pub use plan::{PlanNode, PlanProperty};
pub use source::{
DataSource, DbColor, DbDriver, DbIntrospection, DbOptions, DbProperty, DbSnowflake, DbSnowflakeAuth, DbSsh,
DbSshAuth, DbSslMode, DbTxMode,
DataSource, DbColor, DbDriver, DbIntrospection, DbOptions, DbProperty, DbSsh, DbSshAuth, DbSslMode, DbTxMode,
};
pub use sql::{CellValue, DbRowEdit, SortKey};
@@ -115,21 +109,6 @@ pub struct SchemaObjects {
pub routines: Vec<RoutineInfo>,
pub packages: Vec<PackageInfo>,
pub sequences: Vec<SequenceInfo>,
/// Objects of kinds only some databases have (Snowflake stages, streams, tasks…).
#[serde(default)]
pub others: Vec<SchemaObject>,
}
/// A schema object the tree lists by name under a folder of its kind.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSchemaObject")]
pub struct SchemaObject {
/// `stage`, `stream`, `task`, `pipe`, `fileFormat`.
pub kind: String,
pub name: String,
/// What the list says about it (a stage's type, a stream's table, a task's schedule).
pub detail: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
@@ -225,18 +204,6 @@ pub struct TablePage {
pub offset: f64,
}
/// What a Snowflake session computes with and acts as, and what it may switch to.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSessionContext")]
pub struct SessionContext {
/// Empty: no warehouse is in use.
pub warehouse: String,
pub role: String,
pub warehouses: Vec<String>,
pub roles: Vec<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbDatabases")]
@@ -261,8 +228,6 @@ pub struct DbManager {
/// Directory of the global data sources and their saved passwords ([`DbManager::with_global_sources`]).
global: Option<(PathBuf, SecretStore)>,
history: Arc<HistoryStore>,
/// What introspection read of each data source, kept between runs.
catalogs: catalog::CatalogStore,
/// Records executed statements in the history off the async threads, in order.
recorder: std::sync::mpsc::Sender<(String, String, f64)>,
/// Passwords entered this session (not saved, or not yet), by credential key.
@@ -270,9 +235,6 @@ pub struct DbManager {
/// Manual-commit data sources, each with the connection of its open
/// transaction (taken by the first statement after a commit / rollback).
sessions: Mutex<HashMap<String, Session>>,
/// The data sources as last read from disk, for [`DbManager::reconcile_sources`]
/// (`None` until [`DbManager::watch_sources`]).
on_disk: Mutex<Option<Vec<DataSource>>>,
}
type Session = Arc<tokio::sync::Mutex<Option<Held>>>;
@@ -374,9 +336,6 @@ impl DbManager {
/// `passwords` / `history` are the app-wide files of saved passwords
/// ([`SecretStore`]) and query history ([`HistoryStore`]).
pub fn new(root: PathBuf, passwords: PathBuf, history: PathBuf, ssh: Arc<SshManager>) -> Self {
// Beside the query history, in the app data directory.
let catalogs = history.parent().unwrap_or(&root).join("db-catalogs");
let catalogs = catalog::CatalogStore::new(catalogs, &root);
let history = Arc::new(HistoryStore::new(history, &root));
let (recorder, statements) = std::sync::mpsc::channel::<(String, String, f64)>();
{
@@ -396,7 +355,6 @@ impl DbManager {
saved_passwords: SecretStore::new(passwords, &root),
global: None,
history,
catalogs,
recorder,
root,
pools: Mutex::default(),
@@ -404,7 +362,6 @@ impl DbManager {
tunnel_open: tokio::sync::Mutex::new(()),
ssh,
session_passwords: Mutex::default(),
on_disk: Mutex::default(),
sessions: Mutex::default(),
}
}
@@ -460,10 +417,6 @@ impl DbManager {
.ok_or_else(|| IdeError::not_found(format!("no data source {id}")))?
.resolved()?;
if let Some(db) = database {
// The configured schema belongs to the configured database.
if source.driver == DbDriver::Snowflake && !source.database.eq_ignore_ascii_case(db) {
source.options.default_schema.clear();
}
source.database = db.to_owned();
}
Ok(source)
@@ -498,10 +451,9 @@ impl DbManager {
match sources.iter().find(|s| s.id == o.id) {
Some(n) if *n == o => {}
gone_or_changed => {
let keys = [o.id.clone(), secrets::ssh_key(&o.id), Self::sso_credential(&o.id)];
let keys = [o.id.clone(), secrets::ssh_key(&o.id)];
match gone_or_changed {
None => {
self.catalogs.forget(&o.id);
for k in &keys {
self.session_passwords.lock().remove(k);
self.store_of(&o).delete(k);
@@ -522,45 +474,9 @@ impl DbManager {
}
}
}
// Our own write is not reported as an edit on disk.
if let Some(known) = self.on_disk.lock().as_mut() {
*known = self.sources().unwrap_or(sources);
}
Ok(())
}
/// Watches the data source files (the project's and the global one) for edits made
/// outside the IDE; `on_change` runs on a background thread once they settle, and
/// should call [`DbManager::reconcile_sources`]. Stops when the result is dropped.
pub fn watch_sources(&self, on_change: Box<dyn Fn() + Send>) -> IdeResult<SourcesWatch> {
*self.on_disk.lock() = Some(self.sources().unwrap_or_default());
SourcesWatch::start(&self.root, self.global.as_ref().map(|(dir, _)| dir.as_path()), on_change)
}
/// Re-reads the data sources after their files changed on disk: connections of
/// removed or changed sources close, as on [`DbManager::save_sources`] (saved
/// passwords stay, the source may come back). Returns whether the list the tree
/// shows changed, or the files no longer parse (the tree reports the error).
pub async fn reconcile_sources(&self) -> bool {
let now = match self.sources() {
Ok(now) => now,
Err(e) => {
tracing::warn!("data sources: {}", e.message);
return true;
}
};
let before = self.on_disk.lock().replace(now.clone()).unwrap_or_default();
if before == now {
return false;
}
for o in &before {
if now.iter().find(|s| s.id == o.id) != Some(o) {
self.disconnect_source(&o.id).await;
}
}
true
}
/// Remembers `password` for the session, and in the saved passwords when
/// the data source saves passwords (`None` forgets it).
pub async fn set_password(&self, id: &str, password: Option<String>, save: bool) -> IdeResult<()> {
@@ -655,9 +571,7 @@ impl DbManager {
ssh_secret: Option<String>,
shared: bool,
) -> IdeResult<(DataSource, Option<Arc<Tunnel>>)> {
// Snowflake is HTTPS to a named host: a tunnel's local end cannot pass its certificate check.
let tunneled = !matches!(source.driver, DbDriver::Sqlite | DbDriver::Snowflake);
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && tunneled) else {
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && source.driver != DbDriver::Sqlite) else {
return Ok((source.clone(), None));
};
let local = |t: &Tunnel| DataSource { host: "127.0.0.1".into(), port: Some(t.port()), ..source.clone() };
@@ -711,10 +625,7 @@ impl DbManager {
None => self.password(source).await,
};
let (target, tunnel) = self.endpoint(source, ssh_secret, false).await?;
self.restore_sso(source).await;
let connected = Pool::connect(&target, password.as_deref(), Some(&self.root)).await;
self.keep_sso(source).await;
let pool = connected.map_err(|e| {
let pool = Pool::connect(&target, password.as_deref(), Some(&self.root)).await.map_err(|e| {
through_tunnel(
e,
source,
@@ -727,16 +638,10 @@ impl DbManager {
DbDriver::Mysql => "SELECT CONCAT(@@version_comment, ' ', VERSION())",
DbDriver::SqlServer => "SELECT @@VERSION",
DbDriver::Oracle => "SELECT banner FROM v$version WHERE banner LIKE 'Oracle%' AND ROWNUM = 1",
DbDriver::Snowflake => "SELECT 'Snowflake ' || CURRENT_VERSION()",
};
let result = match &pool {
// On the session the login opened: metadata has its own, which would sign in again.
Pool::Snowflake(p) => p.execute(version, 1, &CancellationToken::new()).await.map(|r| r.rows),
_ => pool.rows(version, 1).await,
};
let result = pool.rows(version, 1).await;
pool.close().await;
let strict = matches!(source.driver, DbDriver::Clickhouse | DbDriver::Snowflake);
let rows = if strict { result? } else { result.unwrap_or_default() };
let rows = if source.driver == DbDriver::Clickhouse { result? } else { result.unwrap_or_default() };
let v = rows.into_iter().next().and_then(|r| r.into_iter().next().flatten());
Ok(v.unwrap_or_else(|| "Connected".into()))
}
@@ -763,10 +668,7 @@ impl DbManager {
let source = self.source(id)?;
let password = self.password(&source).await;
let (target, tunnel) = self.endpoint(&source, None, true).await?;
self.restore_sso(&source).await;
let connected = Pool::connect(&target, password.as_deref(), Some(&self.root)).await;
self.keep_sso(&source).await;
let pool = match connected {
let pool = match Pool::connect(&target, password.as_deref(), Some(&self.root)).await {
Ok(p) => Arc::new(p),
Err(e) => {
self.drop_unused_tunnel(source_id);
@@ -779,10 +681,9 @@ impl DbManager {
};
// Another caller may have connected meanwhile; keep theirs.
let kept = self.pools.lock().entry(id.to_owned()).or_insert_with(|| pool.clone()).clone();
if !Arc::ptr_eq(&kept, &pool) {
// Ours lost the race: close it rather than leave its session open on the server.
pool.close().await;
} else if let Some(secs) = source.options.keep_alive_secs.filter(|s| *s > 0) {
if Arc::ptr_eq(&kept, &pool)
&& let Some(secs) = source.options.keep_alive_secs.filter(|s| *s > 0)
{
keep_alive(&pool, Duration::from_secs(secs.into()));
}
Ok(kept)
@@ -832,23 +733,6 @@ impl DbManager {
let names = first(pool.rows(sql::sqlserver_databases(), MAX_META_ROWS).await?).collect();
return Ok(Databases { current, names });
}
if pool.driver() == DbDriver::Snowflake {
let current = pool
.rows("SELECT CURRENT_DATABASE()", 1)
.await?
.into_iter()
.next()
.and_then(|r| r.into_iter().next().flatten())
.unwrap_or_default();
// `SHOW` needs no running warehouse and lists what the role may see. Columns: created_on, name…
let names = pool
.rows("SHOW TERSE DATABASES", MAX_META_ROWS)
.await?
.into_iter()
.filter_map(|r| r.into_iter().nth(1).flatten())
.collect();
return Ok(Databases { current, names });
}
if pool.driver() != DbDriver::Postgres {
return Err(IdeError::invalid("this data source does not list databases"));
}
@@ -863,95 +747,17 @@ impl DbManager {
Ok(Databases { current, names })
}
/// The schemas of `database` of data source `id` with their tables and views (Snowflake):
/// what completion needs to follow `database.schema.table` without the tree being opened.
/// The result is also saved ([`DbManager::saved_catalog`]) for the next run.
pub async fn database_catalog(&self, id: &str, database: &str) -> IdeResult<Vec<CatalogSchema>> {
let id = split_target(id).0;
let schemas = match &*self.pool(id).await? {
Pool::Snowflake(p) => snowflake::meta::catalog(p, database).await?,
_ => return Err(IdeError::invalid("this data source does not list other databases")),
};
if let Err(e) = self.catalogs.put(id, database, schemas.clone()).await {
// Completion still has it for this run.
tracing::warn!("could not save the catalog of {database}: {}", e.message);
}
Ok(schemas)
}
/// What earlier introspection of data source `id` saved (`None`: never introspected).
/// Read from disk: the data source is not connected to.
pub async fn saved_catalog(&self, id: &str) -> IdeResult<Option<SourceCatalog>> {
self.catalogs.load(split_target(id).0).await
}
/// Drops the saved databases of data source `id` that are not in `databases` any more.
pub async fn retain_catalog(&self, id: &str, databases: Vec<String>) -> IdeResult<()> {
self.catalogs.retain(split_target(id).0, databases).await
}
/// Credential key of the single sign-on ID token of data source `id`.
fn sso_credential(id: &str) -> String {
format!("{id}#sso")
}
/// Hands the saved single sign-on token of `source` to the driver, so it signs in without the browser.
async fn restore_sso(&self, source: &DataSource) {
let Some(key) = snowflake::sso_key(source) else { return };
if snowflake::sso_token(&key).is_none()
&& let Some(saved) = self.store_of(source).get(&Self::sso_credential(&source.id))
{
snowflake::sso_seed(&key, &saved);
}
}
/// Keeps the token the last sign-in of `source` left in the credential store (or forgets a refused one).
async fn keep_sso(&self, source: &DataSource) {
let Some(key) = snowflake::sso_key(source) else { return };
if source.id.is_empty() {
return;
}
let credential = Self::sso_credential(&source.id);
let store = self.store_of(source);
let (now, saved) = (snowflake::sso_token(&key), store.get(&credential));
if now == saved {
return;
}
match now {
Some(token) => {
if let Err(e) = store.set(&credential, &token) {
tracing::warn!("could not save the single sign-on token: {}", e.message);
}
}
None => store.delete(&credential),
}
}
/// The warehouse and role of the session of data source `id` (Snowflake), and its choices.
pub async fn session_context(&self, id: &str) -> IdeResult<SessionContext> {
match &*self.pool(id).await? {
Pool::Snowflake(p) => snowflake::meta::session(p).await,
_ => Err(IdeError::invalid("this data source has no warehouse or role")),
}
}
/// Every schema (the Schemas tab picks the ones the tree shows); system schemas
/// only with [`DbIntrospection::show_system`].
pub async fn schemas(&self, id: &str) -> IdeResult<Vec<String>> {
let system = self.source(id)?.introspection.show_system;
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::schemas(p, system).await;
}
let rows = pool.rows(sql::schemas(pool.driver(), system), MAX_META_ROWS).await?;
Ok(rows.into_iter().filter_map(|r| r.into_iter().next().flatten()).collect())
}
pub async fn tables(&self, id: &str, schema: &str) -> IdeResult<Vec<TableInfo>> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::tables(p, schema).await;
}
let rows = pool.rows(&sql::tables(pool.driver(), schema), MAX_META_ROWS).await?;
Ok(rows
.into_iter()
@@ -965,9 +771,6 @@ impl DbManager {
pub async fn table_details(&self, id: &str, schema: &str, table: &str) -> IdeResult<TableDetails> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::table_details(p, schema, table).await;
}
let driver = pool.driver();
let columns = pool
.rows(&sql::columns(driver, schema, table), MAX_META_ROWS)
@@ -995,7 +798,7 @@ impl DbManager {
_ => indexes.push(IndexInfo { name, unique: flag(&unique), columns: vec![col] }),
}
}
let fk_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default()
@@ -1012,7 +815,7 @@ impl DbManager {
.collect();
}
foreign_keys.retain(|k| k.columns.len() == k.ref_columns.len());
let trigger_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
let trigger_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS).await.unwrap_or_else(|e| {
@@ -1039,9 +842,6 @@ impl DbManager {
/// The routines and sequences of `schema`.
pub async fn schema_objects(&self, id: &str, schema: &str) -> IdeResult<SchemaObjects> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::schema_objects(p, schema).await;
}
let driver = pool.driver();
let mut out = SchemaObjects::default();
if let Some(q) = sql::routines(driver, schema) {
@@ -1091,14 +891,6 @@ impl DbManager {
/// The definition of a routine (every overload), package (spec and body), trigger or sequence.
pub async fn object_ddl(&self, id: &str, schema: &str, kind: DbObjectKind, name: &str) -> IdeResult<String> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
let defs = snowflake::meta::object_ddl(p, schema, kind, name).await?;
if defs.is_empty() {
return Err(IdeError::not_found(format!("{schema}.{name}")));
}
let defs: Vec<String> = defs.iter().map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).collect();
return Ok(defs.join("\n"));
}
if kind == DbObjectKind::Package {
let mut defs: Vec<String> = Vec::new();
for body in [false, true] {
@@ -1199,26 +991,6 @@ impl DbManager {
.next();
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::Snowflake => {
let path = match &*pool {
Pool::Snowflake(p) => p.database_ident().map(|db| format!("{db}.")).unwrap_or_default(),
_ => String::new(),
};
let target = sql::quote_literal(driver, &format!("{path}{}", sql::qualified(driver, schema, name)));
let mut ddl = None;
// Views answer to their own object type on some accounts.
for kind in ["TABLE", "VIEW"] {
match pool.rows(&format!("SELECT GET_DDL('{kind}', {target})"), 1).await {
Ok(rows) => {
ddl = cells(rows, 0).into_iter().next();
break;
}
Err(_) if kind == "TABLE" => {}
Err(e) => return Err(e),
}
}
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::Postgres => {
let rel = pool.rows(&sql::postgres_relation(schema, name), 1).await?.into_iter().next();
let (oid, kind) = match rel.as_deref() {
@@ -1267,14 +1039,7 @@ impl DbManager {
let pool = self.pool(id).await?;
let driver = pool.driver();
let mut tables: Vec<CompletionTable> = Vec::new();
let rows = match &*pool {
Pool::Snowflake(p) => {
tables = snowflake::meta::completion(p).await?;
Vec::new()
}
_ => pool.rows(sql::completion_columns(driver), MAX_META_ROWS).await?,
};
for r in rows {
for r in pool.rows(sql::completion_columns(driver), MAX_META_ROWS).await? {
let mut it = r.into_iter().map(Option::unwrap_or_default);
let mut next = || it.next().unwrap_or_default();
let (schema, name, kind) = (next(), next(), next());
@@ -1299,7 +1064,7 @@ impl DbManager {
}
}
// Foreign keys are a nicety (JOIN completion): a server that refuses the catalog query still completes.
let fk_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
let fk_rows = if driver == DbDriver::Clickhouse {
Vec::new()
} else {
pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
-98
Переглянути файл
@@ -39,7 +39,6 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
DbDriver::Mysql => format!("EXPLAIN {s}"),
DbDriver::SqlServer => format!("SET SHOWPLAN_TEXT ON; {s}"),
DbDriver::Oracle => format!("EXPLAIN PLAN FOR {s}"),
DbDriver::Snowflake => format!("EXPLAIN USING TABULAR {s}"),
}
}
@@ -78,10 +77,6 @@ pub fn parse(driver: DbDriver, result: &QueryResult) -> IdeResult<Vec<PlanNode>>
Ok(mysql(&names, &result.rows))
}
DbDriver::SqlServer => Ok(Vec::new()),
DbDriver::Snowflake => {
let names: Vec<&str> = result.columns.iter().map(|c| c.name.as_str()).collect();
Ok(snowflake(&names, &result.rows))
}
}
}
@@ -202,42 +197,6 @@ fn oracle(rows: &[Vec<Option<String>>]) -> Vec<PlanNode> {
out
}
/// Snowflake's `EXPLAIN USING TABULAR`: one row per operation with its parents' ids; the
/// first row totals the partitions and bytes the statement would scan.
fn snowflake(names: &[&str], rows: &[Vec<Option<String>>]) -> Vec<PlanNode> {
let column = |name: &str| names.iter().position(|n| n.eq_ignore_ascii_case(name));
let cell = |r: &[Option<String>], name: &str| column(name).and_then(|i| r.get(i).cloned().flatten());
let mut ids: Vec<String> = Vec::new();
let mut out = Vec::new();
for r in rows {
let operation = cell(r, "operation").unwrap_or_default();
let id = cell(r, "id").unwrap_or_default();
// `[0]`, `[1, 2]`: the first is the operation this one feeds.
let parent = cell(r, "parentOperators")
.and_then(|p| p.trim_matches(['[', ']']).split(',').next().map(|p| p.trim().to_owned()))
.and_then(|p| ids.iter().position(|x| *x == p))
.map(|i| i as u32);
let mut params = cell(r, "objects").unwrap_or_default();
if let Some(alias) = cell(r, "alias").filter(|a| !a.is_empty()) {
params = format!("{params} {alias}").trim().to_owned();
}
let mut properties = Vec::new();
for (label, name) in [
("Expressions", "expressions"),
("Partitions scanned", "partitionsAssigned"),
("Partitions total", "partitionsTotal"),
("Bytes scanned", "bytesAssigned"),
] {
if let Some(value) = cell(r, name).filter(|v| !v.is_empty()) {
properties.push(prop(label, value));
}
}
out.push(PlanNode { parent, operation, params, properties });
ids.push(id);
}
out
}
/// MySQL's tabular `EXPLAIN`: one row per table access, flat.
fn mysql(names: &[&str], rows: &[Vec<Option<String>>]) -> Vec<PlanNode> {
let col = |r: &[Option<String>], n: &str| {
@@ -418,61 +377,4 @@ mod tests {
assert_eq!((plan[1].operation.as_str(), plan[1].params.as_str()), ("Index Lookup", "table: c; key: PRIMARY;"));
assert!(plan[0].properties.iter().any(|p| p.name == "rows" && p.value == "20"));
}
#[test]
fn reads_snowflake_tabular_plans() {
let names = [
"step",
"id",
"parentOperators",
"operation",
"objects",
"alias",
"expressions",
"partitionsTotal",
"partitionsAssigned",
"bytesAssigned",
];
let row = |cells: [Option<&'static str>; 10]| cells.to_vec();
let r = result(
&names,
vec![
row([None, None, None, Some("GlobalStats"), None, None, None, Some("12"), Some("3"), Some("4096")]),
row([Some("1"), Some("0"), None, Some("Result"), None, None, Some("O.ID"), None, None, None]),
row([
Some("1"),
Some("1"),
Some("[0]"),
Some("Filter"),
None,
None,
Some("O.ID > 3"),
None,
None,
None,
]),
row([
Some("1"),
Some("2"),
Some("[1]"),
Some("TableScan"),
Some("DB.PUBLIC.ORDERS"),
Some("O"),
Some("ID"),
Some("12"),
Some("3"),
Some("4096"),
]),
],
);
let plan = parse(DbDriver::Snowflake, &r).unwrap();
assert_eq!(
plan.iter().map(|n| n.operation.as_str()).collect::<Vec<_>>(),
["GlobalStats", "Result", "Filter", "TableScan"]
);
assert_eq!(plan.iter().map(|n| n.parent).collect::<Vec<_>>(), [None, None, Some(1), Some(2)]);
assert_eq!(plan[3].params, "DB.PUBLIC.ORDERS O");
assert!(plan[3].properties.contains(&PlanProperty { name: "Partitions scanned".into(), value: "3".into() }));
assert!(plan[0].properties.contains(&PlanProperty { name: "Bytes scanned".into(), value: "4096".into() }));
}
}
+1 -1
Переглянути файл
@@ -171,7 +171,7 @@ impl DbManager {
}
/// Completes cleanup after removal from the file.
pub async fn forget_plugin_source(&self, id: &str) {
for key in [id.to_owned(), crate::secrets::ssh_key(id), format!("{id}#sso")] {
for key in [id.to_owned(), crate::secrets::ssh_key(id)] {
self.session_passwords.lock().remove(&key);
self.saved_passwords.delete(&key);
}
-121
Переглянути файл
@@ -1,121 +0,0 @@
//! Where a Snowflake account is reached, and the account name its login and JWT carry.
use crate::source::{DataSource, DbSslMode};
use ide_core::{IdeError, IdeResult};
use reqwest::Url;
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct Endpoint {
pub base: Url,
/// Upper-case account name: the first label of the account identifier.
pub account: String,
}
/// Reads the data source's host: an account identifier (`myorg-myaccount`,
/// `xy12345.eu-central-1`), a host name (`myorg-myaccount.snowflakecomputing.com`) or the
/// account URL.
pub(crate) fn endpoint(source: &DataSource) -> IdeResult<Endpoint> {
let raw = source.host.trim();
let raw = raw.strip_prefix("https://").or_else(|| raw.strip_prefix("http://")).unwrap_or(raw);
let raw = raw.split(['/', '?', '#']).next().unwrap_or_default().trim_end_matches('.');
let (host, port) = match raw.rsplit_once(':') {
Some((host, port)) if port.chars().all(|c| c.is_ascii_digit()) && !port.is_empty() => {
(host, port.parse::<u16>().ok())
}
_ => (raw, None),
};
let valid = |c: char| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.');
if host.is_empty() || host == "localhost" && source.ssl != DbSslMode::Disable || !host.chars().all(valid) {
return Err(IdeError::invalid(
"enter the Snowflake account identifier, for example myorg-myaccount or xy12345.eu-central-1",
));
}
let account = host.split('.').next().unwrap_or_default().to_ascii_uppercase();
let literal = host.parse::<std::net::IpAddr>().is_ok() || host == "localhost";
let full = if literal || host.to_ascii_lowercase().contains(".snowflakecomputing.") {
host.to_owned()
} else {
// Host names cannot hold underscores; Snowflake serves such accounts under the dashed name.
format!("{}.snowflakecomputing.com", host.replace('_', "-"))
};
let scheme = if source.ssl == DbSslMode::Disable { "http" } else { "https" };
let port = port.or(source.port).filter(|p| *p != 443 || scheme == "http");
let authority = match port {
Some(port) => format!("{full}:{port}"),
None => full,
};
let base = Url::parse(&format!("{scheme}://{authority}/"))
.map_err(|_| IdeError::invalid("invalid Snowflake account identifier"))?;
Ok(Endpoint { base, account })
}
#[cfg(test)]
mod tests {
use super::*;
use crate::source::DbDriver;
fn of(host: &str) -> Endpoint {
endpoint(&DataSource { driver: DbDriver::Snowflake, host: host.into(), ..Default::default() }).unwrap()
}
#[test]
fn account_identifiers_become_hosts() {
let e = of("myorg-myaccount");
assert_eq!(e.base.as_str(), "https://myorg-myaccount.snowflakecomputing.com/");
assert_eq!(e.account, "MYORG-MYACCOUNT");
let e = of("xy12345.eu-central-1");
assert_eq!(e.base.as_str(), "https://xy12345.eu-central-1.snowflakecomputing.com/");
assert_eq!(e.account, "XY12345");
let e = of("xy12345.us-east-2.aws");
assert_eq!(e.base.as_str(), "https://xy12345.us-east-2.aws.snowflakecomputing.com/");
assert_eq!(e.account, "XY12345");
}
#[test]
fn underscores_stay_in_the_account_name_but_not_in_the_host() {
let e = of("my_org-my_account");
assert_eq!(e.base.as_str(), "https://my-org-my-account.snowflakecomputing.com/");
assert_eq!(e.account, "MY_ORG-MY_ACCOUNT");
}
#[test]
fn hosts_and_urls_are_accepted() {
for host in [
"myorg-myaccount.snowflakecomputing.com",
"https://myorg-myaccount.snowflakecomputing.com",
"https://myorg-myaccount.snowflakecomputing.com/console/login?x=1",
"MyOrg-MyAccount.snowflakecomputing.com:443",
] {
let e = of(host);
assert_eq!(e.base.host_str().unwrap().to_ascii_lowercase(), "myorg-myaccount.snowflakecomputing.com");
assert_eq!(e.base.port(), None);
assert_eq!(e.account, "MYORG-MYACCOUNT");
}
let e = of("xy12345.eu-west-1.privatelink.snowflakecomputing.com");
assert_eq!(e.base.as_str(), "https://xy12345.eu-west-1.privatelink.snowflakecomputing.com/");
assert_eq!(e.account, "XY12345");
assert_eq!(of("acct.snowflakecomputing.cn").base.as_str(), "https://acct.snowflakecomputing.cn/");
}
#[test]
fn rejects_missing_and_malformed_identifiers() {
for host in ["", " ", "localhost", "my account", "user@host", "/path"] {
let source = DataSource { driver: DbDriver::Snowflake, host: host.into(), ..Default::default() };
let error = endpoint(&source).unwrap_err();
assert!(error.message.contains("account identifier"), "{host}: {}", error.message);
}
}
#[test]
fn plain_http_is_only_for_an_explicit_ssl_off() {
let source = DataSource {
driver: DbDriver::Snowflake,
host: "127.0.0.1".into(),
port: Some(8080),
ssl: DbSslMode::Disable,
..Default::default()
};
let e = endpoint(&source).unwrap();
assert_eq!(e.base.as_str(), "http://127.0.0.1:8080/");
}
}
-229
Переглянути файл
@@ -1,229 +0,0 @@
//! Single sign-on in the web browser (Snowflake's `externalbrowser` authenticator): Snowflake
//! names the identity provider's page, the browser signs in there and is sent back to a
//! listener on this computer with a one-time token, which the login request then presents
//! together with the proof key Snowflake issued for this attempt.
use super::client::{Api, Request};
use super::{APP, error, uuid};
use ide_core::{IdeError, IdeResult, error::ErrorKind};
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::collections::HashMap;
use std::sync::LazyLock;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
const AUTHENTICATOR: &str = "session/authenticator-request";
/// How long the user has to finish signing in.
const WAIT: Duration = Duration::from_secs(120);
const DONE_PAGE: &str = "<!doctype html><html><head><meta charset=\"utf-8\"><title>OpenIDEA</title></head>\
<body style=\"font-family: sans-serif; margin: 3em\"><h3>Signed in to Snowflake</h3>\
<p>You can close this tab and return to OpenIDEA.</p></body></html>";
/// One browser sign-in at a time: connections opened together share its outcome through [`cached`].
pub(crate) static FLOW: LazyLock<tokio::sync::Mutex<()>> = LazyLock::new(|| tokio::sync::Mutex::new(()));
/// ID tokens Snowflake issued after a browser sign-in, by account address and user. They let
/// further sessions of this run open without the browser. Kept in memory only.
static ID_TOKENS: LazyLock<Mutex<HashMap<String, String>>> = LazyLock::new(|| Mutex::new(HashMap::new()));
pub(crate) fn cached(key: &str) -> Option<String> {
ID_TOKENS.lock().get(key).cloned()
}
pub(crate) fn remember(key: &str, token: &str) {
ID_TOKENS.lock().insert(key.to_owned(), token.to_owned());
}
pub(crate) fn forget(key: &str) {
ID_TOKENS.lock().remove(key);
}
/// What opens the sign-in page; tests put a stand-in for the browser here.
#[cfg(test)]
type Opener = dyn Fn(&str, u16) -> IdeResult<()> + Send + Sync;
#[cfg(test)]
pub(crate) static TEST_OPENER: LazyLock<Mutex<Option<Box<Opener>>>> = LazyLock::new(|| Mutex::new(None));
fn open(url: &str, port: u16) -> IdeResult<()> {
#[cfg(test)]
if let Some(opener) = TEST_OPENER.lock().as_ref() {
return opener(url, port);
}
let _ = port;
open::that_detached(url)
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: cannot open the web browser: {e}")))
}
/// The one-time token and proof key of a completed browser sign-in.
pub(crate) async fn authenticate(
api: &Api,
account: &str,
user: &str,
timeout: Duration,
) -> IdeResult<(String, String)> {
let listener = TcpListener::bind("127.0.0.1:0")
.await
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: cannot listen for the browser sign-in: {e}")))?;
let port = listener.local_addr().map_err(|e| IdeError::new(ErrorKind::Io, e.to_string()))?.port();
let body = json!({ "data": {
"ACCOUNT_NAME": account,
"LOGIN_NAME": user,
"AUTHENTICATOR": "EXTERNALBROWSER",
"BROWSER_MODE_REDIRECT_PORT": port.to_string(),
"CLIENT_APP_ID": APP,
"CLIENT_APP_VERSION": env!("CARGO_PKG_VERSION"),
} });
let envelope = api.send(Request::post(AUTHENTICATOR, body).query("requestId", uuid()).timeout(timeout)).await?;
if !envelope.success {
return Err(api.scrub.clean(error::server(envelope.code.as_deref(), envelope.message.as_deref())));
}
let text = |name: &str| envelope.data.get(name).and_then(Value::as_str).map(str::to_owned);
let (Some(url), Some(proof_key)) = (text("ssoUrl"), text("proofKey")) else {
return Err(IdeError::new(
ErrorKind::Protocol,
"Snowflake did not name a sign-in page; check that the account allows browser sign-in",
));
};
if !(url.starts_with("https://") || url.starts_with("http://")) {
return Err(IdeError::new(ErrorKind::Protocol, "Snowflake named a sign-in page that is not a web address"));
}
api.scrub.add(&proof_key);
open(&url, port)?;
let token = tokio::time::timeout(WAIT, callback(&listener)).await.map_err(|_| {
IdeError::new(ErrorKind::TimedOut, "Snowflake: the browser sign-in was not finished within 2 minutes")
})??;
api.scrub.add(&token);
Ok((token, proof_key))
}
/// Waits for the browser to come back with the token (other requests, like the tab's icon, are refused).
async fn callback(listener: &TcpListener) -> IdeResult<String> {
loop {
let (mut stream, _) = listener
.accept()
.await
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: browser sign-in failed: {e}")))?;
let Some(request) = read(&mut stream).await else { continue };
let line = request.head.lines().next().unwrap_or_default();
let mut parts = line.split_whitespace();
let (method, target) = (parts.next().unwrap_or_default(), parts.next().unwrap_or_default());
let origin = request.header("origin").unwrap_or("*").to_owned();
let cors = format!(
"Access-Control-Allow-Origin: {origin}\r\nVary: Origin\r\nAccess-Control-Allow-Methods: POST, GET\r\n\
Access-Control-Allow-Headers: {}\r\n",
request.header("access-control-request-headers").unwrap_or("content-type")
);
if method == "OPTIONS" {
respond(&mut stream, "200 OK", &cors, "").await;
continue;
}
let token = match method {
"GET" => target.split_once('?').and_then(|(_, query)| parameter(query, "token")),
"POST" => serde_json::from_str::<Value>(&request.body)
.ok()
.and_then(|v| v.get("token").and_then(Value::as_str).map(str::to_owned))
.or_else(|| parameter(&request.body, "token")),
_ => None,
};
match token.filter(|t| !t.is_empty()) {
Some(token) => {
respond(&mut stream, "200 OK", &format!("{cors}Content-Type: text/html; charset=utf-8\r\n"), DONE_PAGE)
.await;
return Ok(token);
}
None => respond(&mut stream, "404 Not Found", "", "").await,
}
}
}
struct Received {
head: String,
body: String,
}
impl Received {
fn header(&self, name: &str) -> Option<&str> {
self.head.lines().skip(1).find_map(|line| {
let (key, value) = line.split_once(':')?;
key.eq_ignore_ascii_case(name).then(|| value.trim())
})
}
}
async fn read(stream: &mut TcpStream) -> Option<Received> {
const LIMIT: usize = 256 * 1024;
let mut bytes = Vec::new();
let mut buffer = [0; 8192];
loop {
let count = tokio::time::timeout(Duration::from_secs(10), stream.read(&mut buffer)).await.ok()?.ok()?;
if count == 0 || bytes.len() > LIMIT {
return None;
}
bytes.extend_from_slice(&buffer[..count]);
if let Some(end) = bytes.windows(4).position(|w| w == b"\r\n\r\n") {
let head = String::from_utf8_lossy(&bytes[..end]).into_owned();
let length: usize = head
.lines()
.find_map(|line| {
line.to_ascii_lowercase().strip_prefix("content-length:").and_then(|v| v.trim().parse().ok())
})
.unwrap_or(0);
if length > LIMIT {
return None;
}
if bytes.len() >= end + 4 + length {
return Some(Received { head, body: String::from_utf8_lossy(&bytes[end + 4..]).into_owned() });
}
}
}
}
async fn respond(stream: &mut TcpStream, status: &str, headers: &str, body: &str) {
let response =
format!("HTTP/1.1 {status}\r\n{headers}Content-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = stream.write_all(response.as_bytes()).await;
let _ = stream.shutdown().await;
}
/// The value of `name` in a URL query or form body.
fn parameter(query: &str, name: &str) -> Option<String> {
query.split('&').find_map(|pair| {
let (key, value) = pair.split_once('=')?;
(key == name).then(|| decode(value))
})
}
fn decode(text: &str) -> String {
let bytes = text.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let hex = |i: usize| bytes.get(i).and_then(|&c| (c as char).to_digit(16));
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'%'
&& let (Some(high), Some(low)) = (hex(i + 1), hex(i + 2))
{
out.push((high * 16 + low) as u8);
i += 3;
continue;
}
out.push(if bytes[i] == b'+' { b' ' } else { bytes[i] });
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_the_token_from_a_query_or_form() {
assert_eq!(parameter("token=abc%2Bdef%3D%3D&x=1", "token").as_deref(), Some("abc+def=="));
assert_eq!(parameter("x=1&token=t", "token").as_deref(), Some("t"));
assert_eq!(parameter("tokens=1", "token"), None);
assert_eq!(parameter("", "token"), None);
}
}
-225
Переглянути файл
@@ -1,225 +0,0 @@
//! The HTTP side of the Snowflake session protocol: one JSON envelope per response.
//! Every error leaves through [`Scrubber::clean`], so no credential reaches a message or a log.
use ide_core::{IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use reqwest::{Client, Method, StatusCode, Url};
use serde::Deserialize;
use serde_json::Value;
use std::sync::Arc;
use std::time::Duration;
/// Replaces known secrets (passphrase, password, tokens) in text that leaves the driver.
#[derive(Clone, Default)]
pub(crate) struct Scrubber(Arc<RwLock<Vec<String>>>);
impl Scrubber {
pub(crate) fn add(&self, secret: &str) {
// Short values would blank out ordinary words.
if secret.len() >= 4 {
let mut secrets = self.0.write();
if !secrets.iter().any(|s| s == secret) {
secrets.push(secret.to_owned());
}
}
}
pub(crate) fn clean(&self, mut error: IdeError) -> IdeError {
for secret in self.0.read().iter() {
if error.message.contains(secret.as_str()) {
error.message = error.message.replace(secret.as_str(), "[redacted]");
}
}
error
}
}
/// A response body: `data` on success, `code` and `message` on failure.
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
pub(crate) struct Envelope {
pub success: bool,
#[serde(deserialize_with = "code")]
pub code: Option<String>,
pub message: Option<String>,
pub data: Value,
}
/// Codes arrive as strings, numbers or null.
fn code<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result<Option<String>, D::Error> {
Ok(match Value::deserialize(deserializer)? {
Value::String(s) if !s.is_empty() => Some(s),
Value::Number(n) => Some(n.to_string()),
_ => None,
})
}
impl Envelope {
pub(crate) fn has_code(&self, codes: &[&str]) -> bool {
self.code.as_deref().is_some_and(|code| codes.contains(&code))
}
}
#[derive(Clone)]
pub(crate) struct Api {
http: Client,
base: Url,
pub scrub: Scrubber,
}
pub(crate) struct Request<'a> {
pub method: Method,
pub path: &'a str,
pub query: Vec<(&'a str, String)>,
/// The session or master token.
pub token: Option<&'a str>,
pub body: Option<Value>,
pub timeout: Option<Duration>,
}
impl<'a> Request<'a> {
pub(crate) fn post(path: &'a str, body: Value) -> Self {
Self { method: Method::POST, path, query: Vec::new(), token: None, body: Some(body), timeout: None }
}
pub(crate) fn get(path: &'a str) -> Self {
Self { method: Method::GET, path, query: Vec::new(), token: None, body: None, timeout: None }
}
pub(crate) fn token(mut self, token: &'a str) -> Self {
self.token = Some(token);
self
}
pub(crate) fn query(mut self, name: &'a str, value: impl Into<String>) -> Self {
self.query.push((name, value.into()));
self
}
pub(crate) fn timeout(mut self, timeout: Duration) -> Self {
self.timeout = Some(timeout);
self
}
}
impl Api {
pub(crate) fn new(http: Client, base: Url) -> Self {
Self { http, base, scrub: Scrubber::default() }
}
/// The account's address, for keys that tell accounts apart.
pub(crate) fn address(&self) -> String {
format!("{}:{}", self.host(), self.base.port_or_known_default().unwrap_or_default())
}
fn host(&self) -> &str {
self.base.host_str().unwrap_or_default()
}
/// Sends `request` and reads the envelope, whatever it reports.
pub(crate) async fn send(&self, request: Request<'_>) -> IdeResult<Envelope> {
let url = self.base.join(request.path).map_err(|_| IdeError::internal("invalid Snowflake request path"))?;
let mut builder = self
.http
.request(request.method, url)
.query(&request.query)
.header(reqwest::header::ACCEPT, "application/snowflake");
if let Some(token) = request.token {
builder = builder.header(reqwest::header::AUTHORIZATION, format!("Snowflake Token=\"{token}\""));
}
if let Some(body) = &request.body {
builder = builder.header(reqwest::header::CONTENT_TYPE, "application/json").body(body.to_string());
}
if let Some(timeout) = request.timeout {
builder = builder.timeout(timeout);
}
let response = builder.send().await.map_err(|e| self.transport(e))?;
let status = response.status();
let body = response.bytes().await.map_err(|e| self.transport(e))?;
match serde_json::from_slice::<Envelope>(&body) {
Ok(envelope) if status.is_success() || envelope.code.is_some() || envelope.message.is_some() => {
Ok(envelope)
}
_ => Err(self.scrub.clean(self.status_error(status))),
}
}
/// Downloads a result chunk from cloud storage (no Snowflake credentials: the URL is presigned).
pub(crate) async fn chunk(&self, url: &str, headers: &[(String, String)]) -> IdeResult<Vec<u8>> {
let mut builder = self.http.get(url);
for (name, value) in headers {
builder = builder.header(name.as_str(), value.as_str());
}
let response = builder.send().await.map_err(|e| self.transport(e))?;
let status = response.status();
if !status.is_success() {
return Err(IdeError::new(
ErrorKind::Io,
format!("Snowflake: cannot download a result chunk (HTTP {})", status.as_u16()),
));
}
Ok(response.bytes().await.map_err(|e| self.transport(e))?.to_vec())
}
fn status_error(&self, status: StatusCode) -> IdeError {
let host = self.host();
match status {
StatusCode::FORBIDDEN | StatusCode::NOT_FOUND => IdeError::invalid(format!(
"Snowflake has no account at {host} (HTTP {}): check the account identifier",
status.as_u16()
)),
s if s.is_server_error() || s == StatusCode::TOO_MANY_REQUESTS => IdeError::new(
ErrorKind::Io,
format!("Snowflake is temporarily unavailable (HTTP {}); try again", s.as_u16()),
),
s => IdeError::new(ErrorKind::Protocol, format!("Snowflake: unexpected response (HTTP {})", s.as_u16())),
}
}
fn transport(&self, error: reqwest::Error) -> IdeError {
use std::error::Error;
let kind = if error.is_timeout() { ErrorKind::TimedOut } else { ErrorKind::Io };
let connect = error.is_connect();
let error = error.without_url();
let mut message = error.to_string();
let mut source = error.source();
while let Some(error) = source {
let detail = error.to_string();
if !message.contains(&detail) {
message.push_str(": ");
message.push_str(&detail);
}
source = error.source();
}
let hint = if connect { "; check the account identifier and the network connection" } else { "" };
self.scrub.clean(IdeError::new(kind, format!("Snowflake ({}): {message}{hint}", self.host())))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn envelopes_read_any_code_shape() {
let e: Envelope =
serde_json::from_str(r#"{"success":false,"code":"390100","message":"no","data":null}"#).unwrap();
assert_eq!(e.code.as_deref(), Some("390100"));
assert!(e.has_code(&["390100"]));
let e: Envelope = serde_json::from_str(r#"{"success":false,"code":390100}"#).unwrap();
assert_eq!(e.code.as_deref(), Some("390100"));
let e: Envelope = serde_json::from_str(r#"{"success":true,"code":null,"data":{"a":1}}"#).unwrap();
assert!(e.success && e.code.is_none());
assert_eq!(e.data["a"], 1);
}
#[test]
fn scrubbing_removes_every_known_secret() {
let scrub = Scrubber::default();
scrub.add("s3cret-passphrase");
scrub.add("eyJ.token.value");
scrub.add("ab");
let e = scrub.clean(IdeError::invalid("failed with s3cret-passphrase and eyJ.token.value, ab stays"));
assert_eq!(e.message, "failed with [redacted] and [redacted], ab stays");
}
}
-161
Переглянути файл
@@ -1,161 +0,0 @@
//! Every SQL command of Snowflake's reference
//! (<https://docs.snowflake.com/en/sql-reference/sql-all>) through the parts of the client a
//! statement passes: the script splitter, the read-only guard and the wire. The samples are in
//! `tests/fixtures/snowflake/commands.json`, one per documented command; the SQL assistance of
//! the UI reads the same file (`src/features/database/sql/snowflakeCommands.test.ts`).
//!
//! The samples exercise the client, not Snowflake's grammar: a server is needed to prove a
//! statement valid, which the ignored live test does for the commands that only read.
use super::tests::{CLOSED_OK, LOGIN_OK, bind, ok, result, serve, source};
use super::{first_word, is_write};
use crate::conn::Pool;
use crate::source::DbDriver;
use crate::split::split;
use ide_core::CancellationToken;
use serde::Deserialize;
use serde_json::json;
#[derive(Deserialize)]
pub(crate) struct Command {
pub command: String,
pub sql: String,
/// Runs on a read-only data source: it reads, or changes the session only.
pub reads: bool,
}
#[derive(Deserialize)]
struct Fixture {
commands: Vec<Command>,
}
pub(crate) fn commands() -> Vec<Command> {
let fixture: Fixture =
serde_json::from_str(include_str!("../../tests/fixtures/snowflake/commands.json")).expect("the fixture parses");
fixture.commands
}
/// `PUT` and `GET` move files on the client's side.
fn transfers(c: &Command) -> bool {
matches!(first_word(&c.sql).as_str(), "PUT" | "GET")
}
#[test]
fn the_fixture_covers_the_reference() {
let commands = commands();
assert!(commands.len() >= 786, "{} commands", commands.len());
let mut names: Vec<&str> = commands.iter().map(|c| c.command.as_str()).collect();
names.sort_unstable();
names.dedup();
assert_eq!(names.len(), commands.len(), "a command is listed twice");
for family in ["ALTER", "CREATE", "DESCRIBE", "DROP", "SHOW", "UNDROP", "USE", "GRANT", "REVOKE", "EXECUTE"] {
assert!(names.iter().any(|n| n.starts_with(family)), "no {family} command");
}
}
#[test]
fn every_command_is_one_statement() {
let commands = commands();
for c in &commands {
let alone = split(&c.sql, DbDriver::Snowflake);
assert_eq!(alone.len(), 1, "{}: split into {} statements", c.command, alone.len());
assert_eq!(alone[0].text, c.sql.trim(), "{}", c.command);
// Terminated, commented and followed by another statement.
let script = format!("-- {}; a comment\n{};\n// next; one\nselect 1;", c.command, c.sql);
let parts: Vec<String> = split(&script, DbDriver::Snowflake).into_iter().map(|s| s.text).collect();
assert_eq!(parts.len(), 2, "{}: {parts:?}", c.command);
assert!(parts[0].ends_with(c.sql.trim()), "{}: {:?}", c.command, parts[0]);
assert_eq!(parts[1].trim_start_matches("// next; one\n"), "select 1", "{}", c.command);
}
// The whole reference as one script.
let script = commands.iter().map(|c| format!("{};\n", c.sql)).collect::<String>();
let parts = split(&script, DbDriver::Snowflake);
assert_eq!(parts.len(), commands.len());
for (part, c) in parts.iter().zip(&commands) {
assert_eq!(part.text, c.sql.trim(), "{}", c.command);
}
}
#[test]
fn every_command_is_known_to_read_or_to_write() {
for c in commands() {
assert_eq!(!is_write(&c.sql), c.reads, "{}: {}", c.command, c.sql);
assert!(!first_word(&c.sql).is_empty(), "{}", c.command);
// Comments before a statement do not change what it is.
let commented = format!("/* note; */ -- one\n// two\n{}", c.sql);
assert_eq!(!is_write(&commented), c.reads, "{} after comments", c.command);
}
}
#[tokio::test]
async fn every_command_reaches_the_server_as_written_and_its_answer_is_read() {
let commands = commands();
let (listener, port) = bind().await;
let status = json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string();
let rows = result(
json!([{ "name": "name", "type": "text" }, { "name": "n", "type": "fixed", "scale": 0 }]),
json!([["a", "1"], ["b", null]]),
);
let changed =
json!({ "success": true, "data": { "rowtype": [{ "name": "number of rows inserted", "type": "fixed" }],
"rowset": [["3"]], "queryResultFormat": "json", "statementTypeId": 0x3100, "total": 1 } })
.to_string();
// Login (which names the schema), then one answer per command.
let mut replies = vec![ok(LOGIN_OK)];
let mut entered = true;
for c in commands.iter().filter(|c| !transfers(c)) {
if !entered && first_word(&c.sql) != "USE" {
replies.push(ok(status.clone()));
}
// These leave the session wherever they went: its schema is entered again afterwards.
entered = !matches!(first_word(&c.sql).as_str(), "USE" | "CALL" | "EXECUTE");
let dml = matches!(first_word(&c.sql).as_str(), "INSERT" | "UPDATE" | "DELETE" | "MERGE");
replies.push(ok(if c.reads && first_word(&c.sql) != "USE" {
rows.clone()
} else if dml {
changed.clone()
} else {
status.clone()
}));
}
replies.push(ok(CLOSED_OK));
let server = serve(listener, replies);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
for c in &commands {
if transfers(c) {
// Refused before anything is sent: the files would not move.
let refused = pool.execute(&c.sql, 100, &cancel).await.unwrap_err();
assert!(refused.message.contains("Snowflake CLI"), "{}: {}", c.command, refused.message);
continue;
}
let answer = pool.execute(&c.sql, 100, &cancel).await;
let answer = answer.unwrap_or_else(|e| panic!("{}: {}", c.command, e.message));
let word = first_word(&c.sql);
if c.reads && word != "USE" {
assert_eq!(
answer.rows,
[[Some("a".to_owned()), Some("1".to_owned())], [Some("b".to_owned()), None]],
"{}",
c.command
);
assert_eq!(answer.affected, None, "{}", c.command);
} else if matches!(word.as_str(), "INSERT" | "UPDATE" | "DELETE" | "MERGE") {
assert_eq!(answer.affected, Some(3.0), "{}", c.command);
} else {
assert!(answer.columns.is_empty() && answer.rows.is_empty(), "{}: a command shows no grid", c.command);
}
}
pool.close().await;
let seen = server.await.unwrap();
let sent = seen[1..seen.len() - 1]
.iter()
.map(|r| r.json()["sqlText"].as_str().unwrap_or_default().to_owned())
.collect::<Vec<_>>();
let sent: Vec<&str> = sent.iter().map(String::as_str).collect();
let context = "USE SCHEMA \"ANALYTICS\".\"PUBLIC\"";
let written: Vec<&str> = sent.iter().copied().filter(|sql| *sql != context).collect();
assert_eq!(written, commands.iter().filter(|c| !transfers(c)).map(|c| c.sql.as_str()).collect::<Vec<_>>());
}
-98
Переглянути файл
@@ -1,98 +0,0 @@
//! Snowflake error codes as errors the UI can act on: `PermissionDenied` asks for the
//! data source's secret again, everything else shows its message.
use ide_core::{IdeError, error::ErrorKind};
/// The session token expired; the master token renews it.
pub(crate) const SESSION_EXPIRED: &str = "390112";
/// The session is gone (master token expired, session closed by the server): log in again.
pub(crate) const SESSION_GONE: &[&str] = &["390111", "390114", "390104"];
/// The statement still runs; its result is polled.
pub(crate) const IN_PROGRESS: &[&str] = &["333333", "333334"];
const KEY_HINT: &str = "Check that the public key of this private key is assigned to the user \
(ALTER USER … SET RSA_PUBLIC_KEY), and that the account identifier and user name are right.";
/// What to do about a failed login or statement, by Snowflake error code.
fn hint(code: &str, message: &str) -> Option<&'static str> {
Some(match code {
"390144" => KEY_HINT,
"394300" => "Snowflake found no such user: check the user name and the account identifier.",
"394302" => "The login token arrived too late: check this computer's clock.",
"394303" => "The login token had expired: check this computer's clock.",
"394304" => {
"The public key assigned to the user does not match this private key: compare the fingerprint below \
with RSA_PUBLIC_KEY_FP of DESC USER."
}
"394307" => "The account in the login token does not match: check the account identifier.",
"390189" => "The role is not granted to this user: change the role of the data source.",
"390201" => "Check the database, schema and warehouse of the data source, and that the role may use them.",
"000606" => "Set a warehouse in the data source, or run USE WAREHOUSE <name>.",
"390422" | "390420" => "A network policy of the account blocks this address.",
_ if message.contains("is not allowed to access Snowflake") => {
"A network policy of the account blocks this address."
}
_ => return None,
})
}
/// Whether the login was refused for reasons a different private key or account setting
/// fixes, so the key's fingerprint helps.
pub(crate) fn key_rejected(code: &str) -> bool {
code == "390144" || code.starts_with("3943")
}
fn kind(code: &str) -> ErrorKind {
match code {
// Wrong password or token; expired or revoked tokens.
"390100" | "390303" | "390318" | "394400" | "394401" | "394402" => ErrorKind::PermissionDenied,
"000604" | "000605" => ErrorKind::Cancelled,
"000630" => ErrorKind::TimedOut,
_ => ErrorKind::InvalidInput,
}
}
/// The error of a response that reports failure.
pub(crate) fn server(code: Option<&str>, message: Option<&str>) -> IdeError {
let code = code.unwrap_or_default();
let message = message.map(str::trim).filter(|m| !m.is_empty()).unwrap_or("the request failed");
let mut text =
if code.is_empty() { format!("Snowflake: {message}") } else { format!("Snowflake [{code}]: {message}") };
if let Some(hint) = hint(code, message) {
text.push('\n');
text.push_str(hint);
}
IdeError::new(kind(code), text)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn wrong_credentials_ask_again_and_other_failures_explain() {
let e = server(Some("390100"), Some("Incorrect username or password was specified."));
assert_eq!(e.kind, ErrorKind::PermissionDenied);
assert_eq!(e.message, "Snowflake [390100]: Incorrect username or password was specified.");
let e = server(Some("390144"), Some("JWT token is invalid. [abc]"));
assert_eq!(e.kind, ErrorKind::InvalidInput);
assert!(e.message.starts_with("Snowflake [390144]: JWT token is invalid. [abc]\nCheck that the public key"));
assert!(key_rejected("390144") && key_rejected("394304") && !key_rejected("390100"));
let e = server(Some("000606"), Some("No active warehouse selected in the current session."));
assert!(e.message.ends_with("run USE WAREHOUSE <name>."), "{}", e.message);
let e = server(Some("002003"), Some("SQL compilation error:\nObject 'T' does not exist or not authorized."));
assert_eq!(e.kind, ErrorKind::InvalidInput);
assert_eq!(
e.message,
"Snowflake [002003]: SQL compilation error:\nObject 'T' does not exist or not authorized."
);
}
#[test]
fn network_policy_and_bare_failures() {
let e = server(Some("390432"), Some("IP 203.0.113.9 is not allowed to access Snowflake."));
assert!(e.message.ends_with("A network policy of the account blocks this address."));
assert_eq!(server(None, None).message, "Snowflake: the request failed");
assert_eq!(server(Some("000604"), Some("SQL execution canceled")).kind, ErrorKind::Cancelled);
}
}
-265
Переглянути файл
@@ -1,265 +0,0 @@
//! Key-pair authentication: reads the user's RSA private key file and signs the login JWT.
//! Parsing, decryption and signing are RustCrypto's (`rsa`, `pkcs8`, `pkcs5`).
use base64::Engine;
use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD};
use ide_core::{IdeError, IdeResult, error::ErrorKind};
use rsa::RsaPrivateKey;
use rsa::pkcs1::DecodeRsaPrivateKey;
use rsa::pkcs1v15::SigningKey;
use rsa::pkcs8::{EncodePublicKey, EncryptedPrivateKeyInfoRef, PrivateKeyInfoRef, SecretDocument};
use rsa::sha2::{Digest, Sha256};
use rsa::signature::{SignatureEncoding, Signer};
use rsa::traits::PublicKeyParts;
use zeroize::Zeroizing;
/// Snowflake rejects a JWT that lives longer than an hour.
const LIFETIME_SECS: u64 = 59 * 60;
const MIN_BITS: usize = 2048;
const CONVERT: &str = "openssl pkcs8 -topk8 -v2 aes-256-cbc -in <key file> -out rsa_key.p8";
pub(crate) struct KeyPair {
key: RsaPrivateKey,
}
fn invalid(message: impl Into<String>) -> IdeError {
IdeError::invalid(message)
}
/// The UI answers `PermissionDenied` by asking for the secret, here the key's passphrase.
fn passphrase_error(message: &str) -> IdeError {
IdeError::new(ErrorKind::PermissionDenied, message)
}
/// Reads the private key file at `path` (`~/` is the home directory).
pub(crate) fn load(path: &str, passphrase: Option<&str>) -> IdeResult<KeyPair> {
let path = path.trim();
if path.is_empty() {
return Err(invalid("select the private key file of the Snowflake user"));
}
let file = ide_ssh::expand(path);
let bytes = Zeroizing::new(
std::fs::read(&file)
.map_err(|e| invalid(format!("cannot read the private key file {}: {e}", file.display())))?,
);
let Ok(text) = std::str::from_utf8(&bytes) else {
return Err(invalid(format!(
"{} is not a PEM file; convert a DER key with: openssl pkcs8 -inform DER -outform PEM -in <key file> -out rsa_key.p8",
file.display()
)));
};
parse(text, passphrase)
}
pub(crate) fn parse(pem: &str, passphrase: Option<&str>) -> IdeResult<KeyPair> {
let pem = pem.trim();
if pem.contains("Proc-Type: 4,ENCRYPTED") {
return Err(invalid(format!(
"the private key uses the legacy encrypted PEM format; convert it to PKCS#8 with: {CONVERT}"
)));
}
if pem.contains("BEGIN OPENSSH PRIVATE KEY") {
return Err(invalid(
"the private key is in OpenSSH format; convert it to PKCS#8 with: ssh-keygen -p -m PKCS8 -f <key file>",
));
}
let (label, document) = SecretDocument::from_pem(pem).map_err(|_| {
if pem.contains("PRIVATE KEY-----") {
invalid("the private key file is damaged")
} else {
invalid("the private key file holds no PEM private key (expected -----BEGIN PRIVATE KEY-----)")
}
})?;
let key = match label {
"PRIVATE KEY" => from_pkcs8(document.as_bytes(), "the private key file is damaged")?,
"ENCRYPTED PRIVATE KEY" => {
let Some(passphrase) = passphrase.filter(|p| !p.is_empty()) else {
return Err(passphrase_error("the private key is encrypted; enter its passphrase"));
};
let encrypted = EncryptedPrivateKeyInfoRef::try_from(document.as_bytes())
.map_err(|_| invalid("the encrypted private key file is damaged"))?;
let decrypted = encrypted.decrypt(passphrase).map_err(|error| match error {
rsa::pkcs8::Error::EncryptedPrivateKey(rsa::pkcs8::pkcs5::Error::DecryptFailed) => {
passphrase_error("incorrect passphrase for the private key")
}
rsa::pkcs8::Error::EncryptedPrivateKey(_) => invalid(format!(
"the private key is encrypted with an unsupported algorithm; re-encrypt it with: {CONVERT}"
)),
_ => invalid("the encrypted private key file is damaged"),
})?;
// A wrong passphrase can leave valid padding around bytes that are no key.
from_pkcs8(decrypted.as_bytes(), "incorrect passphrase for the private key").map_err(|error| {
if error.message.starts_with("incorrect passphrase") { passphrase_error(&error.message) } else { error }
})?
}
"RSA PRIVATE KEY" => RsaPrivateKey::from_pkcs1_der(document.as_bytes())
.map_err(|_| invalid("the private key file is damaged"))?,
"PUBLIC KEY" | "RSA PUBLIC KEY" => {
return Err(invalid("this file holds the public key; select the private key file"));
}
"EC PRIVATE KEY" => return Err(not_rsa()),
other => return Err(invalid(format!("unsupported private key format: {other}"))),
};
let bits = key.size() * 8;
if bits < MIN_BITS {
return Err(invalid(format!("the RSA key has {bits} bits; Snowflake requires at least {MIN_BITS}")));
}
Ok(KeyPair { key })
}
fn not_rsa() -> IdeError {
invalid("the private key is not an RSA key; Snowflake key-pair authentication needs an RSA key")
}
fn from_pkcs8(der: &[u8], malformed: &str) -> IdeResult<RsaPrivateKey> {
let info = PrivateKeyInfoRef::try_from(der).map_err(|_| invalid(malformed))?;
if info.algorithm.oid != rsa::pkcs1::ALGORITHM_OID {
return Err(not_rsa());
}
RsaPrivateKey::try_from(info).map_err(|_| invalid(malformed))
}
impl KeyPair {
/// `SHA256:<base64>` of the public key, as `DESC USER` shows it in `RSA_PUBLIC_KEY_FP`.
pub(crate) fn fingerprint(&self) -> IdeResult<String> {
let der = self
.key
.to_public_key()
.to_public_key_der()
.map_err(|e| IdeError::internal(format!("cannot encode the public key: {e}")))?;
Ok(format!("SHA256:{}", STANDARD.encode(Sha256::digest(der.as_bytes()))))
}
/// The login token for `user` of `account` (both as Snowflake stores them: upper case).
pub(crate) fn jwt(&self, account: &str, user: &str, now_secs: u64) -> IdeResult<String> {
let subject = format!("{}.{}", account.to_uppercase(), user.to_uppercase());
let header = URL_SAFE_NO_PAD.encode(br#"{"alg":"RS256","typ":"JWT"}"#);
let claims = serde_json::json!({
"iss": format!("{subject}.{}", self.fingerprint()?),
"sub": subject,
"iat": now_secs,
"exp": now_secs + LIFETIME_SECS,
});
let message = format!("{header}.{}", URL_SAFE_NO_PAD.encode(claims.to_string()));
let signature = SigningKey::<Sha256>::new(self.key.clone())
.try_sign(message.as_bytes())
.map_err(|e| IdeError::internal(format!("cannot sign the login token: {e}")))?;
Ok(format!("{message}.{}", URL_SAFE_NO_PAD.encode(signature.to_bytes())))
}
}
#[cfg(test)]
mod tests {
use super::*;
use rsa::pkcs1v15::{Signature, VerifyingKey};
use rsa::signature::Verifier;
const PLAIN: &str = include_str!("../../tests/fixtures/snowflake/rsa_key.p8");
const ENCRYPTED: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_encrypted.p8");
const SCRYPT: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_scrypt.p8");
const PKCS1: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_pkcs1.pem");
const LEGACY: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_legacy_encrypted.pem");
const SHORT: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_1024.p8");
const EC: &str = include_str!("../../tests/fixtures/snowflake/ec_key.p8");
const PUBLIC: &str = include_str!("../../tests/fixtures/snowflake/rsa_key.pub");
/// Computed with OpenSSL (see the fixtures' README).
const FINGERPRINT: &str = "SHA256:ajjL6Bf6l1FIIyTcJli4i2AWPYDVJGQnSZr+9q9ClgI=";
const PASSPHRASE: &str = "test-passphrase";
fn error(pem: &str, passphrase: Option<&str>) -> IdeError {
match parse(pem, passphrase) {
Ok(_) => panic!("the key was accepted"),
Err(error) => error,
}
}
#[test]
fn every_supported_format_yields_the_same_key() {
for (pem, passphrase) in [
(PLAIN, None),
(PLAIN, Some("ignored")),
(ENCRYPTED, Some(PASSPHRASE)),
(SCRYPT, Some(PASSPHRASE)),
(PKCS1, None),
] {
assert_eq!(parse(pem, passphrase).unwrap().fingerprint().unwrap(), FINGERPRINT);
}
assert_eq!(parse(&format!("\n\n{PLAIN}\n\n"), None).unwrap().fingerprint().unwrap(), FINGERPRINT);
}
#[test]
fn an_encrypted_key_asks_for_its_passphrase() {
for passphrase in [None, Some("")] {
let e = error(ENCRYPTED, passphrase);
assert_eq!(e.kind, ErrorKind::PermissionDenied);
assert!(e.message.contains("enter its passphrase"), "{}", e.message);
}
}
#[test]
fn a_wrong_passphrase_is_named_and_asks_again() {
for pem in [ENCRYPTED, SCRYPT] {
for wrong in ["wrong", "test-passphrase ", "Test-Passphrase"] {
let e = error(pem, Some(wrong));
assert_eq!(e.kind, ErrorKind::PermissionDenied, "{}", e.message);
assert_eq!(e.message, "incorrect passphrase for the private key");
}
}
}
#[test]
fn unusable_files_say_what_is_wrong() {
let cases = [
(LEGACY, "legacy encrypted PEM"),
(SHORT, "1024 bits"),
(EC, "not an RSA key"),
(PUBLIC, "public key"),
("", "no PEM private key"),
("not a key", "no PEM private key"),
("-----BEGIN OPENSSH PRIVATE KEY-----\nAAAA\n-----END OPENSSH PRIVATE KEY-----", "OpenSSH format"),
("-----BEGIN PRIVATE KEY-----\nAAAA\n-----END PRIVATE KEY-----", "damaged"),
];
for (pem, expected) in cases {
let e = error(pem, Some(PASSPHRASE));
assert_eq!(e.kind, ErrorKind::InvalidInput, "{expected}: {}", e.message);
assert!(e.message.contains(expected), "{expected}: {}", e.message);
}
}
#[test]
fn missing_and_unreadable_files_are_reported_with_their_path() {
let e = load("", None).err().unwrap();
assert!(e.message.contains("select the private key file"));
let e = load("/nonexistent/openidea/rsa_key.p8", None).err().unwrap();
assert_eq!(e.kind, ErrorKind::InvalidInput);
assert!(e.message.contains("/nonexistent/openidea/rsa_key.p8"), "{}", e.message);
let dir = tempfile::tempdir().unwrap();
let der = dir.path().join("key.der");
std::fs::write(&der, [0x30, 0x82, 0xff, 0xfe, 0x00]).unwrap();
let e = load(der.to_str().unwrap(), None).err().unwrap();
assert!(e.message.contains("not a PEM file"), "{}", e.message);
let good = dir.path().join("rsa_key.p8");
std::fs::write(&good, ENCRYPTED).unwrap();
assert_eq!(load(good.to_str().unwrap(), Some(PASSPHRASE)).unwrap().fingerprint().unwrap(), FINGERPRINT);
}
#[test]
fn the_token_carries_snowflake_claims_and_a_valid_rs256_signature() {
let pair = parse(PLAIN, None).unwrap();
let token = pair.jwt("myorg-myaccount", "[email protected]", 1_700_000_000).unwrap();
let parts: Vec<&str> = token.split('.').collect();
assert_eq!(parts.len(), 3);
let decode = |part: &str| URL_SAFE_NO_PAD.decode(part).unwrap();
let header: serde_json::Value = serde_json::from_slice(&decode(parts[0])).unwrap();
assert_eq!(header, serde_json::json!({"alg": "RS256", "typ": "JWT"}));
let claims: serde_json::Value = serde_json::from_slice(&decode(parts[1])).unwrap();
assert_eq!(claims["sub"], "[email protected]");
assert_eq!(claims["iss"], format!("[email protected].{FINGERPRINT}"));
assert_eq!(claims["iat"], 1_700_000_000u64);
assert_eq!(claims["exp"], 1_700_000_000u64 + 59 * 60);
let verifier = VerifyingKey::<Sha256>::new(pair.key.to_public_key());
let signature = Signature::try_from(decode(parts[2]).as_slice()).unwrap();
verifier.verify(format!("{}.{}", parts[0], parts[1]).as_bytes(), &signature).unwrap();
assert!(verifier.verify(format!("{}.{}x", parts[0], parts[1]).as_bytes(), &signature).is_err());
}
}
-511
Переглянути файл
@@ -1,511 +0,0 @@
//! The Database tree's metadata, read with `SHOW` and `DESCRIBE`: they need no running
//! warehouse, list only what the session's role may see, and are asked one level at a time.
use super::SnowflakePool;
use crate::sql::{quote_ident, quote_literal};
use crate::{
ColumnInfo, CompletionColumn, CompletionTable, DbDriver, DbObjectKind, DbTableKind, ForeignKey, RoutineInfo,
SchemaObject, SchemaObjects, SequenceInfo, TableDetails, TableInfo,
};
use ide_core::{CancellationToken, IdeError, IdeResult};
use std::collections::HashMap;
/// `SHOW … LIMIT` takes at most this many rows; longer lists continue `FROM` the last name.
const PAGE: usize = 10_000;
const MAX_ROWS: usize = 100_000;
const DRIVER: DbDriver = DbDriver::Snowflake;
/// A `SHOW` / `DESCRIBE` result, read by column name (their column order varies by version).
struct Shown {
columns: Vec<String>,
rows: Vec<Vec<Option<String>>>,
}
impl Shown {
fn index(&self, name: &str) -> Option<usize> {
self.columns.iter().position(|c| c.eq_ignore_ascii_case(name))
}
fn values<'a>(&'a self, name: &str) -> impl Iterator<Item = Option<&'a str>> + 'a {
let index = self.index(name);
self.rows.iter().map(move |row| index.and_then(|i| row.get(i)).and_then(|v| v.as_deref()))
}
fn records(&self) -> impl Iterator<Item = Record<'_>> {
self.rows.iter().map(|row| Record { shown: self, row })
}
}
struct Record<'a> {
shown: &'a Shown,
row: &'a [Option<String>],
}
impl<'a> Record<'a> {
fn get(&self, name: &str) -> Option<&'a str> {
self.shown.index(name).and_then(|i| self.row.get(i)).and_then(|v| v.as_deref())
}
fn text(&self, name: &str) -> String {
self.get(name).unwrap_or_default().to_owned()
}
}
async fn show(pool: &SnowflakePool, sql: &str) -> IdeResult<Shown> {
let result = pool.metadata().execute(sql, MAX_ROWS, &CancellationToken::new()).await?;
Ok(Shown { columns: result.columns.into_iter().map(|c| c.name).collect(), rows: result.rows })
}
/// A `SHOW` that lists by name, page after page.
async fn show_all(pool: &SnowflakePool, sql: &str) -> IdeResult<Shown> {
let mut all = show(pool, &format!("{sql} LIMIT {PAGE}")).await?;
let mut last_page = all.rows.len();
while last_page >= PAGE && all.rows.len() < MAX_ROWS {
let Some(last) = all.values("name").last().flatten().map(str::to_owned) else { break };
let next = show(pool, &format!("{sql} LIMIT {PAGE} FROM {}", quote_literal(DRIVER, &last))).await?;
last_page = next.rows.len();
all.rows.extend(next.rows);
}
Ok(all)
}
/// ` IN DATABASE "DB"`, or nothing for the session's current database.
fn in_database(pool: &SnowflakePool) -> String {
pool.database_ident().map(|db| format!(" IN DATABASE {db}")).unwrap_or_default()
}
/// `"DB"."SCHEMA"` (the schema as the tree names it: exactly).
fn schema_path(pool: &SnowflakePool, schema: &str) -> String {
match pool.database_ident() {
Some(db) => format!("{db}.{}", quote_ident(DRIVER, schema)),
None => quote_ident(DRIVER, schema),
}
}
fn object_path(pool: &SnowflakePool, schema: &str, name: &str) -> String {
format!("{}.{}", schema_path(pool, schema), quote_ident(DRIVER, name))
}
pub(crate) async fn schemas(pool: &SnowflakePool, system: bool) -> IdeResult<Vec<String>> {
let shown = show_all(pool, &format!("SHOW TERSE SCHEMAS{}", in_database(pool))).await?;
Ok(shown
.values("name")
.flatten()
.filter(|name| system || *name != "INFORMATION_SCHEMA")
.map(str::to_owned)
.collect())
}
fn table_kind(kind: &str) -> DbTableKind {
let kind = kind.to_ascii_uppercase().replace('_', " ");
if kind == "MATERIALIZED VIEW" {
DbTableKind::MaterializedView
} else if kind.ends_with("VIEW") {
DbTableKind::View
} else {
DbTableKind::Table
}
}
/// Tables of every kind (dynamic, external, hybrid…) and views of `schema`.
pub(crate) async fn tables(pool: &SnowflakePool, schema: &str) -> IdeResult<Vec<TableInfo>> {
let shown = show_all(pool, &format!("SHOW TERSE OBJECTS IN SCHEMA {}", schema_path(pool, schema))).await?;
Ok(shown
.records()
.filter_map(|r| {
Some(TableInfo { name: r.get("name")?.to_owned(), kind: table_kind(r.get("kind").unwrap_or("TABLE")) })
})
.collect())
}
pub(crate) async fn table_details(pool: &SnowflakePool, schema: &str, table: &str) -> IdeResult<TableDetails> {
let path = object_path(pool, schema, table);
let described = show(pool, &format!("DESCRIBE TABLE {path}")).await?;
let yes = |v: Option<&str>| v.is_some_and(|v| v.eq_ignore_ascii_case("Y"));
let columns = described
.records()
.filter(|r| r.get("kind").is_none_or(|kind| kind.eq_ignore_ascii_case("COLUMN")))
.filter_map(|r| {
Some(ColumnInfo {
name: r.get("name")?.to_owned(),
data_type: r.text("type"),
nullable: yes(r.get("null?")),
default: r.get("default").filter(|d| !d.is_empty()).map(str::to_owned),
primary_key: yes(r.get("primary key")),
})
})
.collect();
// Views have no keys; a role without the privilege still sees the columns.
let foreign_keys = match show(pool, &format!("SHOW IMPORTED KEYS IN TABLE {path}")).await {
Ok(keys) => foreign_keys(&keys),
Err(_) => Vec::new(),
};
Ok(TableDetails { columns, indexes: Vec::new(), foreign_keys, triggers: Vec::new() })
}
fn foreign_keys(shown: &Shown) -> Vec<ForeignKey> {
let mut rows: Vec<Record<'_>> = shown.records().collect();
let sequence = |r: &Record<'_>| r.get("key_sequence").and_then(|n| n.parse::<u32>().ok()).unwrap_or(0);
rows.sort_by_key(|a| (a.text("fk_name"), sequence(a)));
let mut keys: Vec<(String, ForeignKey)> = Vec::new();
for r in rows {
let name = r.text("fk_name");
if !keys.last().is_some_and(|(last, _)| *last == name) {
keys.push((
name,
ForeignKey {
schema: r.text("fk_schema_name"),
table: r.text("fk_table_name"),
columns: Vec::new(),
ref_schema: r.text("pk_schema_name"),
ref_table: r.text("pk_table_name"),
ref_columns: Vec::new(),
},
));
}
let key = &mut keys.last_mut().expect("pushed above").1;
key.columns.push(r.text("fk_column_name"));
key.ref_columns.push(r.text("pk_column_name"));
}
keys.into_iter().map(|(_, key)| key).collect()
}
/// Rows of `SHOW TERSE OBJECTS` for database `db` (quoted; `None`: the session's), each with
/// its `schema_name`. One listing for the database when that answers in full; schema by schema
/// (as the tree lists them) when it is refused or is cut off at a page, since a page
/// continues by object name and names repeat across schemas.
async fn database_objects(pool: &SnowflakePool, db: Option<&str>, schemas: &[String]) -> IdeResult<Shown> {
let scope = db.map(|db| format!(" IN DATABASE {db}")).unwrap_or_default();
match show(pool, &format!("SHOW TERSE OBJECTS{scope} LIMIT {PAGE}")).await {
Ok(whole) if whole.rows.len() < PAGE && whole.index("schema_name").is_some() => return Ok(whole),
Ok(_) => {}
Err(e) => tracing::warn!("listing a whole database failed, listing its schemas instead: {}", e.message),
}
let mut all = Shown { columns: vec!["name".into(), "kind".into(), "schema_name".into()], rows: Vec::new() };
let (mut listed, mut refused) = (0, None);
for schema in schemas {
let path = match db {
Some(db) => format!("{db}.{}", quote_ident(DRIVER, schema)),
None => quote_ident(DRIVER, schema),
};
// A schema the role may not list: the others still complete.
let shown = match show_all(pool, &format!("SHOW TERSE OBJECTS IN SCHEMA {path}")).await {
Ok(shown) => shown,
Err(e) => {
refused.get_or_insert(e);
continue;
}
};
listed += 1;
let (Some(name), Some(kind)) = (shown.index("name"), shown.index("kind")) else { continue };
all.rows.extend(shown.rows.into_iter().map(|mut row| {
let kind = row.get_mut(kind).and_then(Option::take);
let name = row.get_mut(name).and_then(Option::take);
vec![name, kind, Some(schema.clone())]
}));
if all.rows.len() >= MAX_ROWS {
break;
}
}
// Nothing could be listed at all: that is a failure, not an empty database.
match refused {
Some(e) if listed == 0 => Err(e),
_ => Ok(all),
}
}
/// Every schema of `database` with its tables and views, for completion across databases.
pub(crate) async fn catalog(pool: &SnowflakePool, database: &str) -> IdeResult<Vec<crate::CatalogSchema>> {
let db = quote_ident(DRIVER, database);
let schemas = show_all(pool, &format!("SHOW TERSE SCHEMAS IN DATABASE {db}")).await?;
let names: Vec<String> =
schemas.values("name").flatten().filter(|name| *name != "INFORMATION_SCHEMA").map(str::to_owned).collect();
let mut out: Vec<crate::CatalogSchema> =
names.iter().map(|name| crate::CatalogSchema { name: name.clone(), tables: Vec::new() }).collect();
let at: HashMap<String, usize> = out.iter().enumerate().map(|(i, s)| (s.name.clone(), i)).collect();
let objects = database_objects(pool, Some(&db), &names).await?;
for r in objects.records() {
let (Some(&index), Some(name)) = (at.get(r.get("schema_name").unwrap_or_default()), r.get("name")) else {
continue;
};
out[index].tables.push(TableInfo { name: name.to_owned(), kind: table_kind(r.get("kind").unwrap_or("TABLE")) });
}
for schema in &mut out {
schema.tables.sort_by(|a, b| a.name.cmp(&b.name));
schema.tables.dedup_by(|a, b| a.name == b.name);
}
Ok(out)
}
/// The session's warehouse and role, and the ones it may switch to.
pub(crate) async fn session(pool: &SnowflakePool) -> IdeResult<crate::SessionContext> {
// The consoles' session: `USE WAREHOUSE` / `USE ROLE` there is what the pickers show.
let sql = "SELECT CURRENT_WAREHOUSE(), CURRENT_ROLE(), CURRENT_AVAILABLE_ROLES()";
let current = pool.execute(sql, 1, &CancellationToken::new()).await?.rows;
let cell = |i: usize| current.first().and_then(|r| r.get(i).cloned().flatten()).unwrap_or_default();
let mut roles: Vec<String> = serde_json::from_str(&cell(2)).unwrap_or_default();
roles.sort();
let warehouses = match show(pool, "SHOW WAREHOUSES").await {
Ok(shown) => shown.values("name").flatten().map(str::to_owned).collect(),
Err(_) => Vec::new(),
};
Ok(crate::SessionContext { warehouse: cell(0), role: cell(1), warehouses, roles })
}
/// `NAME(NUMBER, VARCHAR) RETURN NUMBER` as its argument types and result type.
fn signature(arguments: &str) -> (String, Option<String>) {
let inner = arguments.split_once('(').map_or(arguments, |(_, rest)| rest);
// The result type may hold parentheses of its own (`TABLE (…)`): split at the marker.
let (types, returns) = match inner.split_once(") RETURN ") {
Some((types, returns)) => (types, Some(returns.trim().to_owned())),
None => (inner.strip_suffix(')').unwrap_or(inner), None),
};
(types.trim().to_owned(), returns.filter(|r| !r.is_empty()))
}
async fn routines(
pool: &SnowflakePool,
schema: &str,
kind: DbObjectKind,
like: Option<&str>,
) -> IdeResult<Vec<RoutineInfo>> {
let what = if kind == DbObjectKind::Procedure { "PROCEDURES" } else { "USER FUNCTIONS" };
let like = like.map(|name| format!(" LIKE {}", quote_literal(DRIVER, &escape_like(name)))).unwrap_or_default();
let shown = show(pool, &format!("SHOW {what}{like} IN SCHEMA {}", schema_path(pool, schema))).await?;
Ok(shown
.records()
.filter(|r| !r.get("is_builtin").is_some_and(|b| b.eq_ignore_ascii_case("Y")))
.filter_map(|r| {
let (arguments, returns) = signature(r.get("arguments").unwrap_or_default());
Some(RoutineInfo {
name: r.get("name")?.to_owned(),
kind,
arguments,
returns: returns.filter(|_| kind == DbObjectKind::Function),
})
})
.collect())
}
/// `LIKE` patterns match `_` and `%` loosely; escape them to match the name itself.
fn escape_like(name: &str) -> String {
name.replace('\\', "\\\\").replace('_', "\\_").replace('%', "\\%")
}
/// Kinds of the tree's other folders, the `SHOW` that lists them, and the column that says most about one.
const OTHERS: &[(&str, &str, &str)] = &[
("stage", "STAGES", "type"),
("stream", "STREAMS", "table_name"),
("task", "TASKS", "schedule"),
("pipe", "PIPES", "definition"),
("fileFormat", "FILE FORMATS", "type"),
];
pub(crate) async fn schema_objects(pool: &SnowflakePool, schema: &str) -> IdeResult<SchemaObjects> {
let mut out = SchemaObjects::default();
// A role may see a schema's tables and none of these: each list stands alone.
for kind in [DbObjectKind::Function, DbObjectKind::Procedure] {
out.routines.extend(routines(pool, schema, kind, None).await.unwrap_or_default());
}
out.routines.sort_by(|a, b| (&a.name, &a.arguments).cmp(&(&b.name, &b.arguments)));
if let Ok(shown) = show(pool, &format!("SHOW SEQUENCES IN SCHEMA {}", schema_path(pool, schema))).await {
out.sequences = shown
.values("name")
.flatten()
.map(|name| SequenceInfo { name: name.to_owned(), data_type: "NUMBER".into() })
.collect();
}
for (kind, what, detail) in OTHERS {
let Ok(shown) = show(pool, &format!("SHOW {what} IN SCHEMA {}", schema_path(pool, schema))).await else {
continue;
};
out.others.extend(shown.records().filter_map(|r| {
let detail: String = r.text(detail).split_whitespace().collect::<Vec<_>>().join(" ");
Some(SchemaObject {
kind: (*kind).to_owned(),
name: r.get("name")?.to_owned(),
detail: detail.chars().take(120).collect(),
})
}));
}
Ok(out)
}
/// The definitions of a routine's overloads, or of a sequence.
pub(crate) async fn object_ddl(
pool: &SnowflakePool,
schema: &str,
kind: DbObjectKind,
name: &str,
) -> IdeResult<Vec<String>> {
let path = object_path(pool, schema, name);
let mut out = Vec::new();
match kind {
DbObjectKind::Sequence => out.extend(ddl(pool, "SEQUENCE", path).await?),
DbObjectKind::Function | DbObjectKind::Procedure => {
let object = if kind == DbObjectKind::Procedure { "PROCEDURE" } else { "FUNCTION" };
for routine in routines(pool, schema, kind, Some(name)).await?.into_iter().filter(|r| r.name == name) {
// Overloads are told apart by their argument types (names and defaults dropped).
let types: Vec<&str> = routine
.arguments
.split(',')
.map(|a| a.trim().trim_start_matches("DEFAULT ").trim())
.filter(|a| !a.is_empty())
.collect();
out.extend(ddl(pool, object, format!("{path}({})", types.join(", "))).await?);
}
}
DbObjectKind::Trigger | DbObjectKind::Package => {
return Err(IdeError::invalid("Snowflake has no triggers or packages"));
}
}
Ok(out)
}
async fn ddl(pool: &SnowflakePool, object: &str, target: String) -> IdeResult<Option<String>> {
let sql = format!("SELECT GET_DDL('{object}', {})", quote_literal(DRIVER, &target));
let rows = pool.metadata().execute(&sql, 1, &CancellationToken::new()).await?.rows;
Ok(rows.into_iter().next().and_then(|r| r.into_iter().next().flatten()))
}
/// A column type of `SHOW COLUMNS` (`{"type":"FIXED","precision":38,"scale":0,…}`) the way SQL spells it.
fn column_type(json: &str) -> String {
let Ok(value) = serde_json::from_str::<serde_json::Value>(json) else { return json.to_owned() };
let kind = value.get("type").and_then(|t| t.as_str()).unwrap_or_default().to_ascii_uppercase();
let number = |name: &str| value.get(name).and_then(|n| n.as_i64());
match kind.as_str() {
"FIXED" => match (number("precision"), number("scale")) {
(Some(p), Some(s)) => format!("NUMBER({p},{s})"),
_ => "NUMBER".into(),
},
"REAL" => "FLOAT".into(),
"TEXT" => match number("length") {
Some(n) if n > 0 && n < 16_777_216 => format!("VARCHAR({n})"),
_ => "VARCHAR".into(),
},
"TIME" | "TIMESTAMP_NTZ" | "TIMESTAMP_LTZ" | "TIMESTAMP_TZ" => match number("scale") {
Some(s) => format!("{kind}({s})"),
None => kind,
},
_ => kind,
}
}
/// Above this many tables, a database's columns are too many to read at once.
const SMALL_DATABASE: usize = 1_000;
/// Every table of the target's database for SQL completion, with the columns of all of them
/// when the database is small, else of the session's schema only. A table listed without
/// columns is one whose columns were not read.
pub(crate) async fn completion(pool: &SnowflakePool) -> IdeResult<Vec<CompletionTable>> {
let names = schemas(pool, false).await?;
let objects = database_objects(pool, pool.database_ident().as_deref(), &names).await?;
let mut tables: Vec<CompletionTable> = objects
.records()
.filter(|r| r.get("schema_name") != Some("INFORMATION_SCHEMA"))
.filter_map(|r| {
Some(CompletionTable {
schema: r.text("schema_name"),
name: r.get("name")?.to_owned(),
kind: table_kind(r.get("kind").unwrap_or("TABLE")),
columns: Vec::new(),
})
})
.collect();
tables.sort_by(|a, b| (&a.schema, &a.name).cmp(&(&b.schema, &b.name)));
tables.dedup_by(|a, b| a.schema == b.schema && a.name == b.name);
let scope = if tables.len() <= SMALL_DATABASE {
Some(in_database(pool))
} else {
let current = show(pool, "SELECT CURRENT_SCHEMA()").await?;
let schema = current.rows.first().and_then(|r| r.first().cloned().flatten()).filter(|s| !s.is_empty());
schema.map(|schema| format!(" IN SCHEMA {}", schema_path(pool, &schema)))
};
let Some(scope) = scope else { return Ok(tables) };
// Columns are a help, not a need: without them the tables still complete.
let Ok(shown) = show(pool, &format!("SHOW COLUMNS{scope}")).await else { return Ok(tables) };
let at: HashMap<(String, String), usize> =
tables.iter().enumerate().map(|(i, t)| ((t.schema.clone(), t.name.clone()), i)).collect();
for r in shown.records() {
let Some(&index) = at.get(&(r.text("schema_name"), r.text("table_name"))) else { continue };
tables[index].columns.push(CompletionColumn {
name: r.text("column_name"),
data_type: column_type(r.get("data_type").unwrap_or_default()),
nullable: !r
.get("null?")
.is_some_and(|v| v.eq_ignore_ascii_case("NOT_NULL") || v.eq_ignore_ascii_case("N")),
primary_key: false,
has_default: r.get("default").is_some_and(|d| !d.is_empty())
|| r.get("autoincrement").is_some_and(|a| !a.is_empty()),
});
}
Ok(tables)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn signatures_split_into_arguments_and_result() {
assert_eq!(signature("ADD_ONE(NUMBER) RETURN NUMBER"), ("NUMBER".into(), Some("NUMBER".into())));
assert_eq!(
signature("PARSE(VARCHAR, NUMBER(38,0)) RETURN TABLE (A NUMBER)"),
("VARCHAR, NUMBER(38,0)".into(), Some("TABLE (A NUMBER)".into()))
);
assert_eq!(signature("NOW_UTC() RETURN TIMESTAMP_NTZ"), (String::new(), Some("TIMESTAMP_NTZ".into())));
assert_eq!(signature(""), (String::new(), None));
}
#[test]
fn column_types_read_like_sql() {
assert_eq!(column_type(r#"{"type":"FIXED","precision":38,"scale":0,"nullable":true}"#), "NUMBER(38,0)");
assert_eq!(column_type(r#"{"type":"TEXT","length":100,"byteLength":400,"nullable":false}"#), "VARCHAR(100)");
assert_eq!(column_type(r#"{"type":"TEXT","length":16777216}"#), "VARCHAR");
assert_eq!(column_type(r#"{"type":"TIMESTAMP_NTZ","precision":0,"scale":9}"#), "TIMESTAMP_NTZ(9)");
assert_eq!(column_type(r#"{"type":"VARIANT","nullable":true}"#), "VARIANT");
assert_eq!(column_type(r#"{"type":"REAL"}"#), "FLOAT");
assert_eq!(column_type("not json"), "not json");
}
#[test]
fn object_kinds_become_tree_kinds() {
for kind in ["TABLE", "DYNAMIC TABLE", "EXTERNAL TABLE", "EVENT TABLE", "HYBRID TABLE"] {
assert_eq!(table_kind(kind), DbTableKind::Table, "{kind}");
}
assert_eq!(table_kind("VIEW"), DbTableKind::View);
assert_eq!(table_kind("MATERIALIZED_VIEW"), DbTableKind::MaterializedView);
assert_eq!(table_kind("MATERIALIZED VIEW"), DbTableKind::MaterializedView);
}
#[test]
fn imported_keys_group_by_constraint_in_column_order() {
let columns = [
"pk_schema_name",
"pk_table_name",
"pk_column_name",
"fk_schema_name",
"fk_table_name",
"fk_column_name",
"key_sequence",
"fk_name",
];
let row = |values: [&str; 8]| values.iter().map(|v| Some((*v).to_owned())).collect::<Vec<_>>();
let shown = Shown {
columns: columns.iter().map(|c| (*c).to_owned()).collect(),
rows: vec![
row(["PUBLIC", "REGIONS", "CODE", "PUBLIC", "ORDERS", "REGION_CODE", "2", "FK_REGION"]),
row(["PUBLIC", "CUSTOMERS", "ID", "PUBLIC", "ORDERS", "CUSTOMER_ID", "1", "FK_CUSTOMER"]),
row(["PUBLIC", "REGIONS", "COUNTRY", "PUBLIC", "ORDERS", "COUNTRY", "1", "FK_REGION"]),
],
};
let keys = foreign_keys(&shown);
assert_eq!(keys.len(), 2);
assert_eq!(
(keys[0].ref_table.as_str(), keys[0].columns.as_slice()),
("CUSTOMERS", &["CUSTOMER_ID".to_owned()][..])
);
assert_eq!(keys[1].columns, ["COUNTRY", "REGION_CODE"]);
assert_eq!(keys[1].ref_columns, ["COUNTRY", "CODE"]);
}
}
-929
Переглянути файл
@@ -1,929 +0,0 @@
//! Snowflake over HTTPS: the session protocol of Snowflake's own drivers (login, statements,
//! cancellation, token renewal) with results in JSON. One session per pool; statements run
//! one at a time on it, so session state (`USE`, `ALTER SESSION`, transactions) holds.
mod account;
mod browser;
mod client;
mod error;
mod keypair;
pub(crate) mod meta;
mod rows;
use crate::conn::{QueryResult, Setup};
use crate::source::{DataSource, DbSnowflakeAuth};
use client::{Api, Envelope, Request};
use ide_core::{CancellationToken, IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use rows::RowType;
use serde::Deserialize;
use serde_json::{Value, json};
use std::collections::BTreeMap;
use std::collections::HashMap;
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::{Arc, LazyLock, Weak};
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use tokio::sync::Mutex;
const APP: &str = "OpenIDEA";
const LOGIN: &str = "session/v1/login-request";
const QUERY: &str = "queries/v1/query-request";
const ABORT: &str = "queries/v1/abort-request";
const RENEW: &str = "session/token-request";
const SESSION: &str = "session";
/// DML statement types (`INSERT`, `UPDATE`, `DELETE`, `MERGE`…): their result is a row of counts.
const DML: std::ops::Range<i64> = 0x3000..0x4000;
const SELECT: i64 = 0x1000;
/// A connection target (a database of a data source) on the Snowflake session its data source's
/// targets share: one login serves every database, and each statement runs under its target's
/// database and schema.
#[derive(Clone)]
pub struct SnowflakePool {
shared: Arc<State>,
/// The target's database and the data source's schema (empty: the user's defaults).
database: String,
schema: String,
read_only: Arc<AtomicBool>,
closed: CancellationToken,
/// Which of the data source's sessions this handle runs on.
lane: usize,
}
static CONNECTING: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
/// The key the single sign-on ID token of `source` is cached under (`None`: another method).
pub(crate) fn sso_key(source: &DataSource) -> Option<String> {
if source.snowflake().auth != DbSnowflakeAuth::Browser {
return None;
}
let endpoint = account::endpoint(source).ok()?;
let address = format!(
"{}:{}",
endpoint.base.host_str().unwrap_or_default(),
endpoint.base.port_or_known_default().unwrap_or_default()
);
Some(format!("{address}|{}", source.user.trim().to_uppercase()))
}
/// The ID token a browser sign-in left, which signs in again without the browser while Snowflake accepts it.
pub(crate) fn sso_token(key: &str) -> Option<String> {
browser::cached(key)
}
/// Offers a saved ID token for the next sign-in (one already in use is kept).
pub(crate) fn sso_seed(key: &str, token: &str) {
if browser::cached(key).is_none() {
browser::remember(key, token);
}
}
/// Live sessions by their connection settings, so targets opened with the same settings share one.
static SESSIONS: LazyLock<parking_lot::Mutex<HashMap<String, Weak<State>>>> =
LazyLock::new(|| parking_lot::Mutex::new(HashMap::new()));
struct State {
api: Api,
login: Login,
/// The consoles' session and the tree's ([`CONSOLE`], [`METADATA`]): a long statement in a
/// console does not hold up the tree. The second one logs in when it is first used.
lanes: [Lane; 2],
/// Open targets on this session; the last one to close ends it.
users: AtomicUsize,
key: String,
closed: CancellationToken,
}
/// One Snowflake session.
#[derive(Default)]
struct Lane {
tokens: RwLock<Option<Tokens>>,
/// Statements share the session, so they take turns.
turn: Mutex<()>,
sequence: AtomicU64,
/// The `USE` statement the session is known to be under (`None`: a statement may have
/// moved it; empty: as the login left it, without a database or schema).
context: parking_lot::Mutex<Option<String>>,
}
const CONSOLE: usize = 0;
const METADATA: usize = 1;
struct Login {
account: String,
user: String,
credentials: Credentials,
database: String,
schema: String,
warehouse: String,
role: String,
parameters: Vec<(String, String)>,
timeout: Duration,
}
enum Credentials {
KeyPair { file: String, passphrase: Option<String> },
Password(String),
Token(String),
OAuth(String),
Browser,
}
#[derive(Clone)]
struct Tokens {
session: String,
master: String,
}
#[derive(Debug, Default, Deserialize)]
#[serde(default, rename_all = "camelCase")]
struct ResultData {
rowtype: Vec<RowType>,
rowset: Vec<Vec<Value>>,
rowset_base64: Option<String>,
chunks: Vec<Chunk>,
chunk_headers: BTreeMap<String, String>,
qrmk: Option<String>,
query_result_format: Option<String>,
statement_type_id: Option<i64>,
get_result_url: Option<String>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(default, rename_all = "camelCase")]
struct Chunk {
url: String,
row_count: usize,
}
fn uuid() -> String {
uuid::Uuid::new_v4().to_string()
}
fn now_secs() -> u64 {
SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_secs())
}
fn closed() -> IdeError {
IdeError::new(ErrorKind::Io, "Snowflake connection is closed")
}
/// A session parameter as JSON: Snowflake types booleans and numbers.
fn parameter(value: &str) -> Value {
if value.eq_ignore_ascii_case("true") {
Value::Bool(true)
} else if value.eq_ignore_ascii_case("false") {
Value::Bool(false)
} else if let Ok(n) = value.parse::<i64>() {
json!(n)
} else {
Value::String(value.to_owned())
}
}
/// Settings of the Advanced tab the driver sets itself.
fn reserved(name: &str) -> bool {
matches!(
name.to_ascii_uppercase().as_str(),
"ROWS_PER_RESULTSET" | "MULTI_STATEMENT_COUNT" | "CLIENT_STORE_TEMPORARY_CREDENTIAL"
)
}
/// A name typed in the dialog, as SQL: typed in one case it is an unquoted name, which folds
/// to upper case; anything else is taken as the exact name.
fn typed(name: &str) -> String {
let plain = name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '$')
&& !name.starts_with(|c: char| c.is_ascii_digit());
let one_case = !name.chars().any(|c| c.is_ascii_lowercase()) || !name.chars().any(|c| c.is_ascii_uppercase());
let name = if plain && one_case { name.to_ascii_uppercase() } else { name.to_owned() };
crate::sql::quote_ident(crate::DbDriver::Snowflake, &name)
}
fn first_word(sql: &str) -> String {
let mut sql = sql.trim_start();
loop {
if let Some(rest) = sql.strip_prefix("--").or_else(|| sql.strip_prefix("//")) {
sql = rest.split_once('\n').map_or("", |(_, rest)| rest).trim_start();
} else if let Some(rest) = sql.strip_prefix("/*") {
sql = rest.split_once("*/").map_or("", |(_, rest)| rest).trim_start();
} else {
break;
}
}
sql.split(|c: char| !c.is_ascii_alphanumeric() && c != '_').next().unwrap_or_default().to_ascii_uppercase()
}
/// Whether `sql` may change data or objects (a read-only data source refuses it).
/// Snowflake answers a `USE` it will not perform with one message for every cause; this adds
/// the causes.
fn explain_use(sql: &str, mut error: IdeError) -> IdeError {
if first_word(sql) == "USE" && error.message.contains("002043") {
error.message.push_str(
"\nThe role, warehouse, database or schema does not exist, or is not granted: a role must be \
granted to your user (SHOW GRANTS TO USER <name>), the others to the current role.",
);
}
error
}
/// Whether `sql` may change the account. Statements that read, and those that change only the
/// session (`USE`, `SET`, `UNSET`), run on a read-only data source.
fn is_write(sql: &str) -> bool {
match first_word(sql).as_str() {
"SELECT" | "SHOW" | "DESCRIBE" | "DESC" | "EXPLAIN" | "LIST" | "LS" | "USE" | "SET" | "UNSET" | "VALUES"
| "TABLE" | "" => false,
// `WITH name AS PROCEDURE (…) … CALL name(…)` runs a procedure; other `WITH`s select.
"WITH" => {
sql.to_ascii_uppercase().split_whitespace().collect::<Vec<_>>().windows(2).any(|w| w == ["AS", "PROCEDURE"])
}
_ => true,
}
}
fn gunzip(bytes: Vec<u8>) -> IdeResult<Vec<u8>> {
if !bytes.starts_with(&[0x1f, 0x8b]) {
return Ok(bytes);
}
let mut out = Vec::new();
std::io::Read::read_to_end(&mut flate2::read::GzDecoder::new(bytes.as_slice()), &mut out)
.map_err(|e| IdeError::new(ErrorKind::Protocol, format!("Snowflake: cannot read a result chunk: {e}")))?;
Ok(out)
}
impl SnowflakePool {
pub(crate) async fn connect(source: &DataSource, secret: Option<&str>, setup: Setup) -> IdeResult<Self> {
let settings = source.snowflake();
let endpoint = account::endpoint(source)?;
let user = source.user.trim();
if user.is_empty() {
return Err(IdeError::invalid("enter the Snowflake user name"));
}
for (name, _) in &setup.params {
if reserved(name) {
return Err(IdeError::invalid(format!("the Snowflake driver manages the {name} parameter")));
}
}
let secret = secret.filter(|s| !s.is_empty());
let ask = |what: &str| IdeError::new(ErrorKind::PermissionDenied, format!("enter the {what}"));
let credentials = match settings.auth {
DbSnowflakeAuth::KeyPair => {
Credentials::KeyPair { file: settings.private_key_file.clone(), passphrase: secret.map(str::to_owned) }
}
DbSnowflakeAuth::Password => Credentials::Password(secret.ok_or_else(|| ask("password"))?.to_owned()),
DbSnowflakeAuth::Token => {
Credentials::Token(secret.ok_or_else(|| ask("programmatic access token"))?.to_owned())
}
DbSnowflakeAuth::OAuth => Credentials::OAuth(secret.ok_or_else(|| ask("OAuth access token"))?.to_owned()),
DbSnowflakeAuth::Browser => Credentials::Browser,
};
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let _ = rustls::crypto::ring::default_provider().install_default();
let builder = reqwest::Client::builder()
.connect_timeout(timeout)
.pool_max_idle_per_host(4)
.redirect(reqwest::redirect::Policy::none())
.user_agent(concat!("OpenIDEA/", env!("CARGO_PKG_VERSION")));
let proxy = ide_core::proxy_settings::proxy();
let http = ide_http::client::with_proxy(builder, &proxy)
.build()
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: cannot start the HTTP client: {e}")))?;
let api = Api::new(http, endpoint.base);
if let Some(secret) = secret {
api.scrub.add(secret);
}
let database = source.database.trim().to_owned();
let schema = source.options.default_schema.trim().to_owned();
let target = |shared: Arc<State>| Self {
shared,
database: database.clone(),
schema: schema.clone(),
read_only: Arc::new(AtomicBool::new(source.read_only)),
closed: CancellationToken::new(),
lane: CONSOLE,
};
// Everything that makes a session what it is, the secret included (hashed: the key is only compared).
let key = blake3::hash(
format!(
"{}\0{user}\0{:?}\0{}\0{}\0{}\0{}\0{:?}\0{:?}",
api.address(),
settings.auth,
secret.unwrap_or_default(),
settings.private_key_file,
settings.warehouse.trim(),
settings.role.trim(),
setup.params,
setup.statements
)
.as_bytes(),
)
.to_hex()
.to_string();
// One connect at a time: targets opened together (the tree, consoles, completion) find the
// session the first of them signs in to, instead of each signing in (and each opening the
// browser with single sign-on).
let _connecting = CONNECTING.lock().await;
let live = SESSIONS.lock().get(&key).and_then(Weak::upgrade).filter(|s| !s.closed.is_cancelled());
if let Some(shared) = live {
shared.users.fetch_add(1, Ordering::SeqCst);
return Ok(target(shared));
}
let pool = target(Arc::new(State {
api,
login: Login {
account: endpoint.account,
user: user.to_owned(),
credentials,
database: database.clone(),
schema: schema.clone(),
warehouse: settings.warehouse.trim().to_owned(),
role: settings.role.trim().to_owned(),
parameters: setup.params,
timeout,
},
lanes: Default::default(),
users: AtomicUsize::new(1),
key: key.clone(),
closed: CancellationToken::new(),
}));
pool.login().await?;
// The login asked for this target's database and schema.
*pool.lane().context.lock() = Some(pool.context(None).unwrap_or_default());
let cancel = CancellationToken::new();
// The startup script may write whatever the read-only setting says.
pool.read_only.store(false, Ordering::Relaxed);
for sql in setup.statements {
if let Err(error) = pool.execute(&sql, 0, &cancel).await {
pool.close().await;
return Err(error);
}
}
pool.read_only.store(source.read_only, Ordering::Relaxed);
SESSIONS.lock().insert(key, Arc::downgrade(&pool.shared));
Ok(pool)
}
/// The target's database as SQL (`None`: the session's current database).
pub(crate) fn database_ident(&self) -> Option<String> {
(!self.database.is_empty()).then(|| typed(&self.database))
}
/// The `USE` statement for this target with `schema` active (`None`: the data source's schema).
fn context(&self, schema: Option<&str>) -> Option<String> {
let quote = |name: &str| crate::sql::quote_ident(crate::DbDriver::Snowflake, name);
let schema = match schema.filter(|s| !s.is_empty()) {
// Picked from the list of schemas: the exact name.
Some(picked) => Some(quote(picked)),
None if self.schema.is_empty() => None,
None => Some(typed(&self.schema)),
};
match (self.database.is_empty(), schema) {
(false, Some(schema)) => Some(format!("USE SCHEMA {}.{schema}", typed(&self.database))),
(false, None) => Some(format!("USE DATABASE {}", typed(&self.database))),
(true, Some(schema)) => Some(format!("USE SCHEMA {schema}")),
(true, None) => None,
}
}
/// Ends the session on the server (best effort) and refuses further statements.
pub async fn close(&self) {
if self.closed.is_cancelled() {
return;
}
self.closed.cancel();
if self.shared.users.fetch_sub(1, Ordering::SeqCst) != 1 {
return;
}
self.shared.closed.cancel();
{
let mut sessions = SESSIONS.lock();
if sessions.get(&self.shared.key).is_some_and(|s| std::ptr::eq(s.as_ptr(), Arc::as_ptr(&self.shared))) {
sessions.remove(&self.shared.key);
}
}
for lane in &self.shared.lanes {
let tokens = lane.tokens.write().take();
if let Some(tokens) = tokens {
let request = Request::post(SESSION, json!({}))
.query("delete", "true")
.query("requestId", uuid())
.token(&tokens.session)
.timeout(Duration::from_secs(5));
let _ = self.shared.api.send(request).await;
}
}
}
fn lane(&self) -> &Lane {
&self.shared.lanes[self.lane]
}
/// This target on the session the tree reads metadata with.
pub(crate) fn metadata(&self) -> Self {
// A second browser sign-in would interrupt the user: without an ID token to sign in
// with, metadata takes turns with the consoles.
let second = !matches!(self.shared.login.credentials, Credentials::Browser)
|| browser::cached(&self.browser_key()).is_some();
Self { lane: if second { METADATA } else { CONSOLE }, ..self.clone() }
}
fn browser_key(&self) -> String {
format!("{}|{}", self.shared.api.address(), self.shared.login.user.to_uppercase())
}
pub fn is_closed(&self) -> bool {
self.closed.is_cancelled() || self.shared.closed.is_cancelled()
}
/// The fields of the login request that prove who the user is.
async fn proof(&self) -> IdeResult<(Vec<(&'static str, Value)>, Option<String>)> {
let login = &self.shared.login;
Ok(match &login.credentials {
Credentials::KeyPair { file, passphrase } => {
let (file, passphrase) = (file.clone(), passphrase.clone());
let (account, user) = (login.account.clone(), login.user.clone());
// Decrypting the key derives its encryption key, which takes a while by design.
let (token, fingerprint) = tokio::task::spawn_blocking(move || {
let pair = keypair::load(&file, passphrase.as_deref())?;
Ok::<_, IdeError>((pair.jwt(&account, &user, now_secs())?, pair.fingerprint()?))
})
.await
.map_err(|e| IdeError::internal(e.to_string()))??;
self.shared.api.scrub.add(&token);
(vec![("AUTHENTICATOR", json!("SNOWFLAKE_JWT")), ("TOKEN", json!(token))], Some(fingerprint))
}
Credentials::Password(password) => (vec![("PASSWORD", json!(password))], None),
Credentials::Token(token) => {
(vec![("AUTHENTICATOR", json!("PROGRAMMATIC_ACCESS_TOKEN")), ("TOKEN", json!(token))], None)
}
Credentials::OAuth(token) => (vec![("AUTHENTICATOR", json!("OAUTH")), ("TOKEN", json!(token))], None),
Credentials::Browser => unreachable!("browser sign-in has its own login"),
})
}
async fn login(&self) -> IdeResult<Tokens> {
if matches!(self.shared.login.credentials, Credentials::Browser) {
return self.browser_login().await;
}
let (proof, fingerprint) = self.proof().await?;
self.sign_in(proof, fingerprint, None).await
}
/// Signs in through the web browser, or without it while an ID token of an earlier
/// sign-in of this run is still accepted.
async fn browser_login(&self) -> IdeResult<Tokens> {
let login = &self.shared.login;
let key = self.browser_key();
let _flow = browser::FLOW.lock().await;
if let Some(id_token) = browser::cached(&key) {
self.shared.api.scrub.add(&id_token);
let proof = vec![("AUTHENTICATOR", json!("ID_TOKEN")), ("TOKEN", json!(id_token))];
match self.sign_in(proof, None, Some(&key)).await {
Ok(tokens) => return Ok(tokens),
// The network is down: the browser would not get further either.
Err(error) if matches!(error.kind, ErrorKind::Io | ErrorKind::TimedOut) => return Err(error),
Err(_) => browser::forget(&key),
}
}
let (token, proof_key) =
browser::authenticate(&self.shared.api, &login.account, &login.user, login.timeout).await?;
let proof =
vec![("AUTHENTICATOR", json!("EXTERNALBROWSER")), ("TOKEN", json!(token)), ("PROOF_KEY", json!(proof_key))];
self.sign_in(proof, None, Some(&key)).await
}
/// The login request. `id_token_key`: ask for an ID token and keep it under this key.
async fn sign_in(
&self,
proof: Vec<(&'static str, Value)>,
fingerprint: Option<String>,
id_token_key: Option<&str>,
) -> IdeResult<Tokens> {
let login = &self.shared.login;
let mut parameters: serde_json::Map<String, Value> =
login.parameters.iter().map(|(name, value)| (name.to_ascii_uppercase(), parameter(value))).collect();
if id_token_key.is_some() {
parameters.insert("CLIENT_STORE_TEMPORARY_CREDENTIAL".into(), Value::Bool(true));
}
let mut data = json!({
"CLIENT_APP_ID": APP,
"CLIENT_APP_VERSION": env!("CARGO_PKG_VERSION"),
"ACCOUNT_NAME": login.account,
"LOGIN_NAME": login.user,
"CLIENT_ENVIRONMENT": { "APPLICATION": APP, "OS": std::env::consts::OS },
"SESSION_PARAMETERS": parameters,
});
for (name, value) in proof {
data[name] = value;
}
let mut request = Request::post(LOGIN, json!({ "data": data }))
.query("requestId", uuid())
.query("request_guid", uuid())
.timeout(login.timeout);
for (name, value) in [
("databaseName", &login.database),
("schemaName", &login.schema),
("warehouse", &login.warehouse),
("roleName", &login.role),
] {
if !value.is_empty() {
request = request.query(name, value.as_str());
}
}
let envelope = self.shared.api.send(request).await?;
if !envelope.success {
let code = envelope.code.as_deref().unwrap_or_default();
let mut failure = error::server(envelope.code.as_deref(), envelope.message.as_deref());
if let Some(fingerprint) = fingerprint.filter(|_| error::key_rejected(code)) {
failure.message.push_str(&format!("\nFingerprint of this private key: {fingerprint}"));
}
return Err(self.shared.api.scrub.clean(failure));
}
let text = |name: &str| envelope.data.get(name).and_then(Value::as_str).map(str::to_owned);
let (Some(session), Some(master)) = (text("token"), text("masterToken")) else {
return Err(IdeError::new(ErrorKind::Protocol, "Snowflake: the login response has no session token"));
};
self.shared.api.scrub.add(&session);
self.shared.api.scrub.add(&master);
if let (Some(key), Some(id_token)) = (id_token_key, text("idToken")) {
self.shared.api.scrub.add(&id_token);
browser::remember(key, &id_token);
}
let tokens = Tokens { session, master };
if self.is_closed() {
return Err(closed());
}
// A new session: nothing is known about where it is until a statement says.
*self.lane().context.lock() = None;
*self.lane().tokens.write() = Some(tokens.clone());
Ok(tokens)
}
/// A fresh session token from the master token; a new login when that is refused.
async fn renew(&self, stale: &Tokens) -> IdeResult<Tokens> {
let request = Request::post(RENEW, json!({ "oldSessionToken": stale.session, "requestType": "RENEW" }))
.query("requestId", uuid())
.token(&stale.master)
.timeout(self.shared.login.timeout);
if let Ok(envelope) = self.shared.api.send(request).await
&& envelope.success
{
let text = |name: &str| envelope.data.get(name).and_then(Value::as_str).map(str::to_owned);
if let Some(session) = text("sessionToken") {
let master = text("masterToken").unwrap_or_else(|| stale.master.clone());
self.shared.api.scrub.add(&session);
self.shared.api.scrub.add(&master);
let tokens = Tokens { session, master };
*self.lane().tokens.write() = Some(tokens.clone());
return Ok(tokens);
}
}
self.login().await
}
async fn tokens(&self) -> IdeResult<Tokens> {
let current = self.lane().tokens.read().clone();
match current {
Some(tokens) => Ok(tokens),
None if self.is_closed() => Err(closed()),
None => self.login().await,
}
}
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
self.execute_from(sql, 0, max_rows, cancel).await
}
/// [`execute`](Self::execute), leaving out the first `skip` rows (a later page): whole
/// result chunks before the page are not downloaded.
pub async fn execute_from(
&self,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
self.execute_in(None, sql, skip, max_rows, cancel).await
}
/// [`execute_from`](Self::execute_from) with unqualified names resolving in `schema`
/// (a console's active schema; `None`: the data source's). The switch and the statement
/// hold the session together, so consoles with different schemas do not cross.
pub async fn execute_in(
&self,
schema: Option<&str>,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if self.read_only.load(Ordering::Relaxed) && is_write(sql) {
return Err(IdeError::invalid(format!(
"read-only data source: {} statements are not allowed",
first_word(sql)
)));
}
if matches!(first_word(sql).as_str(), "PUT" | "GET") {
// The server only plans these; the client moves the files, which this driver does not do.
return Err(IdeError::invalid(format!(
"{} copies files between this computer and a stage, which OpenIDEA does not do yet: \
run it with the Snowflake CLI (snow sql) or SnowSQL",
first_word(sql)
)));
}
let _turn = tokio::select! {
biased;
() = cancel.cancelled() => return Err(IdeError::cancelled()),
() = self.closed.cancelled() => return Err(closed()),
turn = self.lane().turn.lock() => turn,
};
// A `USE` needs no schema to run in, and may be what makes the schema reachable.
if first_word(sql) != "USE" {
self.enter(schema, cancel).await?;
}
// These can move the session to another database or schema.
let moves = matches!(first_word(sql).as_str(), "USE" | "CALL" | "EXECUTE");
let result = self.statement(sql, skip, max_rows, cancel).await;
if moves {
*self.lane().context.lock() = None;
}
result.map_err(|error| explain_use(sql, error))
}
/// Puts the session under this target's database and `schema` when it is not there already.
async fn enter(&self, schema: Option<&str>, cancel: &CancellationToken) -> IdeResult<()> {
if let Some(wanted) = self.context(schema)
&& self.lane().context.lock().as_deref() != Some(wanted.as_str())
{
*self.lane().context.lock() = None;
match self.statement(&wanted, 0, 0, cancel).await {
Ok(_) => {}
// The session's role may not use this database or schema (after `USE ROLE`, or a
// schema that is gone). The statement still runs, in the session's own context:
// it may be the `USE` that leads back, or name its objects in full. Asked again
// once a statement moves the session.
Err(refused) if refused.kind == ErrorKind::InvalidInput => {
tracing::warn!("Snowflake: the session did not enter its database and schema");
}
Err(error) => return Err(error),
}
*self.lane().context.lock() = Some(wanted);
}
Ok(())
}
/// Runs `statements` in one transaction; each must change one row when flagged
/// (table editor submits). Nothing else runs on the session meanwhile.
pub(crate) async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
if self.read_only.load(Ordering::Relaxed) {
return Err(IdeError::invalid("read-only data source: changes are not allowed"));
}
let _turn = self.lane().turn.lock().await;
let cancel = CancellationToken::new();
self.enter(None, &cancel).await?;
self.statement("BEGIN", 0, 0, &cancel).await?;
let changes = async {
let mut total = 0;
for (sql, expect_one) in statements {
let changed = self.statement(sql, 0, 0, &cancel).await?.affected.unwrap_or(0.0) as u64;
if *expect_one && changed != 1 {
return Err(IdeError::invalid(format!(
"{sql}: expected to change 1 row, changed {changed}; nothing was saved"
)));
}
total += changed;
}
Ok(total)
}
.await;
match changes {
Ok(total) => {
self.statement("COMMIT", 0, 0, &cancel).await?;
Ok(total)
}
Err(error) => {
let _ = self.statement("ROLLBACK", 0, 0, &cancel).await;
Err(error)
}
}
}
/// One statement on the session; the caller holds the session's turn.
async fn statement(
&self,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
let started = Instant::now();
let mut tokens = self.tokens().await?;
let mut renewed = false;
loop {
let request_id = uuid();
let run = self.run(sql, skip, max_rows, &tokens, &request_id, started);
let outcome = tokio::select! {
biased;
() = cancel.cancelled() => None,
() = self.closed.cancelled() => None,
outcome = run => Some(outcome),
};
let Some(outcome) = outcome else {
if self.is_closed() {
return Err(closed());
}
return Err(match self.abort(sql, &request_id, &tokens).await {
Ok(()) => IdeError::cancelled(),
Err(error) => IdeError::new(
ErrorKind::Cancelled,
format!("cancelled locally; Snowflake did not confirm the cancellation: {}", error.message),
),
});
};
match outcome {
Ok(Outcome::Done(result)) => return Ok(*result),
Ok(Outcome::Expired { gone }) if !renewed => {
renewed = true;
tokens = if gone { self.login().await? } else { self.renew(&tokens).await? };
}
Ok(Outcome::Expired { .. }) => {
return Err(IdeError::new(ErrorKind::Io, "Snowflake: the session expired; reconnect"));
}
Err(error) => return Err(self.shared.api.scrub.clean(error)),
}
}
}
async fn run(
&self,
sql: &str,
skip: usize,
max_rows: usize,
tokens: &Tokens,
request_id: &str,
started: Instant,
) -> IdeResult<Outcome> {
let wanted = skip.saturating_add(max_rows).saturating_add(1);
let body = json!({
"sqlText": sql,
"asyncExec": false,
"sequenceId": self.lane().sequence.fetch_add(1, Ordering::Relaxed) + 1,
"querySubmissionTime": SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_millis() as u64),
// The grid's page, plus one row that tells whether more exist.
"parameters": { "ROWS_PER_RESULTSET": wanted },
});
let request = Request::post(QUERY, body)
.query("requestId", request_id)
.query("request_guid", uuid())
.token(&tokens.session);
let mut envelope = self.shared.api.send(request).await?;
while envelope.has_code(error::IN_PROGRESS) {
let Some(url) = envelope.data.get("getResultUrl").and_then(Value::as_str).map(str::to_owned) else {
return Err(IdeError::new(ErrorKind::Protocol, "Snowflake: a running statement has no result address"));
};
tokio::time::sleep(Duration::from_millis(200)).await;
envelope = self.shared.api.send(Request::get(url.trim_start_matches('/')).token(&tokens.session)).await?;
}
if let Some(expired) = expiry(&envelope) {
return Ok(expired);
}
if !envelope.success {
return Err(error::server(envelope.code.as_deref(), envelope.message.as_deref()));
}
let execution_ms = started.elapsed().as_secs_f64() * 1000.0;
let data: ResultData = serde_json::from_value(envelope.data)
.map_err(|e| IdeError::new(ErrorKind::Protocol, format!("Snowflake: unreadable result: {e}")))?;
let arrow = data.query_result_format.as_deref().is_some_and(|f| !f.eq_ignore_ascii_case("json"));
if arrow || (data.rowset_base64.as_deref().is_some_and(|r| !r.is_empty()) && data.rowset.is_empty()) {
return Err(IdeError::new(
ErrorKind::Protocol,
"Snowflake sent the result in a format other than JSON, which this driver does not read",
));
}
let (rows, has_more) = self.rows(&data, skip, max_rows).await?;
let kind = data.statement_type_id.unwrap_or(SELECT);
let status_only = kind != SELECT && data.rowtype.len() == 1 && data.rowtype[0].name == "status";
let (columns, rows, affected, has_more) = if DML.contains(&kind) {
let changed: f64 = data
.rowset
.first()
.map(|row| row.iter().filter_map(|v| v.as_str().and_then(|s| s.parse::<f64>().ok())).sum())
.unwrap_or(0.0);
(Vec::new(), Vec::new(), Some(changed), false)
} else if status_only {
(Vec::new(), Vec::new(), Some(0.0), false)
} else {
(data.rowtype.iter().map(RowType::column).collect(), rows, None, has_more)
};
Ok(Outcome::Done(Box::new(QueryResult {
statement: sql.to_owned(),
columns,
rows,
affected,
has_more,
elapsed_ms: started.elapsed().as_secs_f64() * 1000.0,
execution_ms,
})))
}
/// The page of rows `skip..skip + max_rows`, and whether rows follow it.
async fn rows(
&self,
data: &ResultData,
mut skip: usize,
max_rows: usize,
) -> IdeResult<(Vec<Vec<Option<String>>>, bool)> {
let mut rows = Vec::new();
let mut more = false;
let mut take = |batch: Vec<Vec<Value>>, skip: &mut usize| -> IdeResult<bool> {
for row in batch {
if *skip > 0 {
*skip -= 1;
continue;
}
if rows.len() >= max_rows {
return Ok(true);
}
if row.len() != data.rowtype.len() {
return Err(IdeError::new(
ErrorKind::Protocol,
"Snowflake: a result row does not match the result's columns",
));
}
rows.push(
row.into_iter().zip(&data.rowtype).map(|(value, column)| rows::cell(column, value)).collect(),
);
}
Ok(false)
};
if take(data.rowset.clone(), &mut skip)? {
more = true;
}
let headers: Vec<(String, String)> = if !data.chunk_headers.is_empty() {
data.chunk_headers.iter().map(|(name, value)| (name.clone(), value.clone())).collect()
} else if let Some(key) = &data.qrmk {
vec![
("x-amz-server-side-encryption-customer-algorithm".into(), "AES256".into()),
("x-amz-server-side-encryption-customer-key".into(), key.clone()),
]
} else {
Vec::new()
};
for chunk in &data.chunks {
if more {
break;
}
if chunk.row_count > 0 && skip >= chunk.row_count {
skip -= chunk.row_count;
continue;
}
let body = gunzip(self.shared.api.chunk(&chunk.url, &headers).await?)?;
// A chunk is its rows separated by commas, without the enclosing brackets.
let mut text = Vec::with_capacity(body.len() + 2);
text.push(b'[');
text.extend_from_slice(&body);
text.push(b']');
let batch: Vec<Vec<Value>> = serde_json::from_slice(&text)
.map_err(|e| IdeError::new(ErrorKind::Protocol, format!("Snowflake: unreadable result chunk: {e}")))?;
if take(batch, &mut skip)? {
more = true;
}
}
Ok((rows, more))
}
async fn abort(&self, sql: &str, request_id: &str, tokens: &Tokens) -> IdeResult<()> {
let request = Request::post(ABORT, json!({ "sqlText": sql, "requestId": request_id }))
.query("requestId", uuid())
.token(&tokens.session)
.timeout(Duration::from_secs(5));
let envelope = self.shared.api.send(request).await?;
if envelope.success {
Ok(())
} else {
Err(self.shared.api.scrub.clean(error::server(envelope.code.as_deref(), envelope.message.as_deref())))
}
}
}
enum Outcome {
Done(Box<QueryResult>),
/// The session token no longer works (`gone`: nor does the master token).
Expired {
gone: bool,
},
}
fn expiry(envelope: &Envelope) -> Option<Outcome> {
if envelope.has_code(&[error::SESSION_EXPIRED]) {
Some(Outcome::Expired { gone: false })
} else if envelope.has_code(error::SESSION_GONE) {
Some(Outcome::Expired { gone: true })
} else {
None
}
}
#[cfg(test)]
mod commands;
#[cfg(test)]
mod tests;
-210
Переглянути файл
@@ -1,210 +0,0 @@
//! Result decoding: Snowflake's JSON result format sends every cell as text in an internal
//! form (dates as day numbers, timestamps as epoch seconds); these become the grid's text.
use crate::conn::ResultColumn;
use serde::Deserialize;
use serde_json::Value;
/// A column of the result's `rowtype`.
#[derive(Debug, Clone, Default, Deserialize)]
#[serde(default)]
pub(crate) struct RowType {
pub name: String,
#[serde(rename = "type")]
pub kind: String,
pub precision: Option<i64>,
pub scale: Option<i64>,
pub length: Option<i64>,
}
/// `VARCHAR` without a length: Snowflake reports its maximum.
const MAX_TEXT: i64 = 16_777_216;
impl RowType {
pub(crate) fn column(&self) -> ResultColumn {
let kind = self.kind.to_ascii_lowercase();
let scaled = |name: &str| match self.scale {
Some(scale) => format!("{name}({scale})"),
None => name.to_owned(),
};
let type_name = match kind.as_str() {
"fixed" => match (self.precision, self.scale) {
(Some(p), Some(s)) => format!("NUMBER({p},{s})"),
_ => "NUMBER".into(),
},
"real" => "FLOAT".into(),
"text" => match self.length {
Some(n) if n > 0 && n < MAX_TEXT => format!("VARCHAR({n})"),
_ => "VARCHAR".into(),
},
"time" => scaled("TIME"),
"timestamp_ntz" | "timestamp_ltz" | "timestamp_tz" => scaled(&kind.to_ascii_uppercase()),
other => other.to_ascii_uppercase(),
};
ResultColumn { name: self.name.clone(), type_name }
}
}
/// The grid text of one cell (`None`: SQL NULL).
pub(crate) fn cell(column: &RowType, value: Value) -> Option<String> {
let text = match value {
Value::Null => return None,
Value::String(text) => text,
other => other.to_string(),
};
let scale = column.scale.unwrap_or(0).clamp(0, 9) as usize;
let shown = match column.kind.to_ascii_lowercase().as_str() {
"boolean" => Some(if matches!(text.as_str(), "1" | "true" | "TRUE") { "true" } else { "false" }.to_owned()),
"date" => text.parse::<i64>().ok().map(date),
"time" => nanos(&text).map(|n| time(n.rem_euclid(DAY), scale)),
"timestamp_ntz" => nanos(&text).map(|n| timestamp(n, scale)),
"timestamp_ltz" => nanos(&text).map(|n| format!("{} Z", timestamp(n, scale))),
"timestamp_tz" => timestamp_tz(&text, scale),
"binary" => Some(format!("0x{text}")),
_ => None,
};
Some(shown.unwrap_or(text))
}
const SECOND: i128 = 1_000_000_000;
const DAY: i128 = 86_400 * SECOND;
/// `seconds[.fraction]` as nanoseconds.
fn nanos(text: &str) -> Option<i128> {
let text = text.trim();
let (negative, digits) = match text.strip_prefix('-') {
Some(rest) => (true, rest),
None => (false, text),
};
let (whole, fraction) = digits.split_once('.').unwrap_or((digits, ""));
if whole.is_empty() || !whole.bytes().all(|b| b.is_ascii_digit()) || !fraction.bytes().all(|b| b.is_ascii_digit()) {
return None;
}
let mut fraction = fraction.to_owned();
fraction.truncate(9);
while fraction.len() < 9 {
fraction.push('0');
}
let value = whole.parse::<i128>().ok()? * SECOND + fraction.parse::<i128>().ok()?;
Some(if negative { -value } else { value })
}
/// Days since 1970-01-01 as `YYYY-MM-DD` (proleptic Gregorian calendar).
fn date(days: i64) -> String {
let z = days + 719_468;
let era = z.div_euclid(146_097);
let doe = z.rem_euclid(146_097);
let yoe = (doe - doe / 1_460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let day = doy - (153 * mp + 2) / 5 + 1;
let month = if mp < 10 { mp + 3 } else { mp - 9 };
let year = yoe + era * 400 + i64::from(month <= 2);
if year < 0 { format!("-{:04}-{month:02}-{day:02}", -year) } else { format!("{year:04}-{month:02}-{day:02}") }
}
fn time(nanos_of_day: i128, scale: usize) -> String {
let seconds = nanos_of_day / SECOND;
let mut text = format!("{:02}:{:02}:{:02}", seconds / 3600, seconds / 60 % 60, seconds % 60);
if scale > 0 {
let fraction = format!("{:09}", nanos_of_day % SECOND);
text.push('.');
text.push_str(&fraction[..scale]);
}
text
}
fn timestamp(nanos: i128, scale: usize) -> String {
format!("{} {}", date(nanos.div_euclid(DAY) as i64), time(nanos.rem_euclid(DAY), scale))
}
/// `seconds[.fraction] offset`, the offset in minutes plus 1440.
fn timestamp_tz(text: &str, scale: usize) -> Option<String> {
let (instant, offset) = text.trim().split_once(' ')?;
let offset = offset.trim().parse::<i128>().ok()? - 1440;
let local = nanos(instant)? + offset * 60 * SECOND;
let sign = if offset < 0 { '-' } else { '+' };
Some(format!("{} {sign}{:02}:{:02}", timestamp(local, scale), offset.abs() / 60, offset.abs() % 60))
}
#[cfg(test)]
mod tests {
use super::*;
fn column(kind: &str, scale: Option<i64>) -> RowType {
RowType { name: "C".into(), kind: kind.into(), scale, ..Default::default() }
}
fn shown(kind: &str, scale: Option<i64>, value: &str) -> String {
cell(&column(kind, scale), Value::String(value.into())).unwrap()
}
#[test]
fn names_types_the_way_snowflake_spells_them() {
let named = |kind: &str, precision, scale, length| {
RowType { name: "C".into(), kind: kind.into(), precision, scale, length }.column().type_name
};
assert_eq!(named("fixed", Some(38), Some(0), None), "NUMBER(38,0)");
assert_eq!(named("fixed", Some(10), Some(2), None), "NUMBER(10,2)");
assert_eq!(named("real", None, None, None), "FLOAT");
assert_eq!(named("text", None, None, Some(100)), "VARCHAR(100)");
assert_eq!(named("text", None, None, Some(16_777_216)), "VARCHAR");
assert_eq!(named("text", None, None, Some(134_217_728)), "VARCHAR");
assert_eq!(named("timestamp_ntz", None, Some(9), None), "TIMESTAMP_NTZ(9)");
assert_eq!(named("timestamp_tz", None, Some(3), None), "TIMESTAMP_TZ(3)");
assert_eq!(named("time", None, Some(0), None), "TIME(0)");
for kind in ["boolean", "date", "variant", "object", "array", "binary", "geography", "vector"] {
assert_eq!(named(kind, None, None, None), kind.to_ascii_uppercase());
}
}
#[test]
fn null_and_plain_values_pass_through() {
assert_eq!(cell(&column("fixed", Some(0)), Value::Null), None);
assert_eq!(shown("fixed", Some(2), "12345678901234567890.12"), "12345678901234567890.12");
assert_eq!(shown("real", None, "1.5e-7"), "1.5e-7");
assert_eq!(shown("text", None, "a\nb"), "a\nb");
assert_eq!(shown("variant", None, "{\n \"a\": 1\n}"), "{\n \"a\": 1\n}");
assert_eq!(cell(&column("fixed", Some(0)), serde_json::json!(7)), Some("7".into()));
}
#[test]
fn booleans_and_binary() {
assert_eq!(shown("boolean", None, "1"), "true");
assert_eq!(shown("boolean", None, "true"), "true");
assert_eq!(shown("boolean", None, "0"), "false");
assert_eq!(shown("binary", None, "DEADBEEF"), "0xDEADBEEF");
}
#[test]
fn dates_count_days_from_the_epoch() {
assert_eq!(shown("date", None, "0"), "1970-01-01");
assert_eq!(shown("date", None, "19675"), "2023-11-14");
assert_eq!(shown("date", None, "-1"), "1969-12-31");
assert_eq!(shown("date", None, "11016"), "2000-02-29");
assert_eq!(shown("date", None, "-719162"), "0001-01-01");
assert_eq!(shown("date", None, "2932896"), "9999-12-31");
assert_eq!(shown("date", None, "not a number"), "not a number");
}
#[test]
fn times_and_timestamps_keep_their_scale() {
assert_eq!(shown("time", Some(0), "3723"), "01:02:03");
assert_eq!(shown("time", Some(3), "3723.5"), "01:02:03.500");
assert_eq!(shown("time", Some(9), "86399.999999999"), "23:59:59.999999999");
assert_eq!(shown("timestamp_ntz", Some(0), "1700000000"), "2023-11-14 22:13:20");
assert_eq!(shown("timestamp_ntz", Some(3), "1700000000.123000000"), "2023-11-14 22:13:20.123");
assert_eq!(shown("timestamp_ntz", Some(9), "1700000000.123456789"), "2023-11-14 22:13:20.123456789");
assert_eq!(shown("timestamp_ntz", Some(3), "-0.500"), "1969-12-31 23:59:59.500");
assert_eq!(shown("timestamp_ntz", Some(0), "-86401"), "1969-12-30 23:59:59");
assert_eq!(shown("timestamp_ltz", Some(0), "1700000000"), "2023-11-14 22:13:20 Z");
}
#[test]
fn zoned_timestamps_show_local_time_and_offset() {
assert_eq!(shown("timestamp_tz", Some(3), "1700000000.123 1440"), "2023-11-14 22:13:20.123 +00:00");
assert_eq!(shown("timestamp_tz", Some(0), "1700000000 1560"), "2023-11-15 00:13:20 +02:00");
assert_eq!(shown("timestamp_tz", Some(0), "1700000000 1110"), "2023-11-14 16:43:20 -05:30");
assert_eq!(shown("timestamp_tz", Some(0), "garbage"), "garbage");
}
}
-921
Переглянути файл
@@ -1,921 +0,0 @@
//! The driver against an in-process HTTP server that plays Snowflake's part.
use super::*;
use crate::conn::Pool;
use crate::source::{DbDriver, DbOptions, DbProperty, DbSnowflake, DbSslMode};
use base64::Engine;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
const KEY: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/tests/fixtures/snowflake/rsa_key.p8");
const ENCRYPTED_KEY: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/tests/fixtures/snowflake/rsa_key_encrypted.p8");
const FINGERPRINT: &str = "SHA256:ajjL6Bf6l1FIIyTcJli4i2AWPYDVJGQnSZr+9q9ClgI=";
pub(super) const LOGIN_OK: &str =
r#"{"success":true,"code":null,"message":null,"data":{"token":"session-token-1","masterToken":"master-token-1"}}"#;
pub(super) const CLOSED_OK: &str = r#"{"success":true,"data":null}"#;
pub(super) struct Reply {
status: &'static str,
body: Vec<u8>,
/// Answer only after this long (a statement that is still running).
delay: Duration,
}
pub(super) fn ok(body: impl Into<String>) -> Reply {
Reply { status: "200 OK", body: body.into().into_bytes(), delay: Duration::ZERO }
}
/// A recorded request: its request line and headers, and its body.
pub(super) struct Seen {
head: String,
body: String,
}
impl Seen {
pub(super) fn json(&self) -> Value {
serde_json::from_str(&self.body).unwrap()
}
fn line(&self) -> &str {
self.head.lines().next().unwrap()
}
fn header(&self, name: &str) -> Option<&str> {
self.head.lines().skip(1).find_map(|line| {
let (key, value) = line.split_once(':')?;
key.eq_ignore_ascii_case(name).then(|| value.trim())
})
}
}
async fn read_request(stream: &mut TcpStream) -> Seen {
let mut bytes = Vec::new();
let mut buffer = [0; 8192];
loop {
let count = tokio::time::timeout(Duration::from_secs(5), stream.read(&mut buffer)).await.unwrap().unwrap();
assert!(count > 0, "request ended early");
bytes.extend_from_slice(&buffer[..count]);
if let Some(end) = bytes.windows(4).position(|s| s == b"\r\n\r\n") {
let head = String::from_utf8_lossy(&bytes[..end]).into_owned();
let length: usize = head
.lines()
.find_map(|line| {
line.to_ascii_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse().unwrap())
})
.unwrap_or(0);
if bytes.len() >= end + 4 + length {
return Seen { head, body: String::from_utf8_lossy(&bytes[end + 4..]).into_owned() };
}
}
}
}
pub(super) async fn bind() -> (TcpListener, u16) {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
(listener, port)
}
/// Answers one request per reply, in order, and returns what it received.
pub(super) fn serve(listener: TcpListener, replies: Vec<Reply>) -> tokio::task::JoinHandle<Vec<Seen>> {
tokio::spawn(async move {
let mut seen = Vec::new();
let mut held = Vec::new();
for reply in replies {
let (mut stream, _) =
tokio::time::timeout(Duration::from_secs(5), listener.accept()).await.unwrap().unwrap();
seen.push(read_request(&mut stream).await);
let respond = async move {
tokio::time::sleep(reply.delay).await;
let head = format!(
"HTTP/1.1 {}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n",
reply.status,
reply.body.len()
);
let _ = stream.write_all(head.as_bytes()).await;
let _ = stream.write_all(&reply.body).await;
};
if reply.delay.is_zero() {
respond.await;
} else {
held.push(tokio::spawn(respond));
}
}
seen
})
}
pub(super) fn source(port: u16) -> DataSource {
DataSource {
id: "sf".into(),
name: "sf".into(),
driver: DbDriver::Snowflake,
host: "127.0.0.1".into(),
port: Some(port),
user: "jane".into(),
database: "ANALYTICS".into(),
ssl: DbSslMode::Disable,
options: DbOptions { default_schema: "PUBLIC".into(), ..Default::default() },
snowflake: Some(DbSnowflake {
warehouse: "COMPUTE_WH".into(),
role: "ANALYST".into(),
private_key_file: KEY.into(),
..Default::default()
}),
..Default::default()
}
}
pub(super) fn result(rowtype: Value, rowset: Value) -> String {
json!({ "success": true, "code": null, "data": { "rowtype": rowtype, "rowset": rowset, "queryResultFormat": "json",
"statementTypeId": 4096, "total": 0, "chunks": [] } })
.to_string()
}
fn one() -> String {
result(json!([{ "name": "N", "type": "fixed", "precision": 1, "scale": 0 }]), json!([["1"]]))
}
async fn failure(source: &DataSource, secret: Option<&str>) -> IdeError {
match Pool::connect(source, secret, None).await {
Ok(_) => panic!("the connection was accepted"),
Err(error) => error,
}
}
#[tokio::test]
async fn key_pair_login_then_a_statement_on_the_session() {
let (listener, port) = bind().await;
let server = serve(listener, vec![ok(LOGIN_OK), ok(one()), ok(CLOSED_OK)]);
let mut source = source(port);
source.options.time_zone = "Europe/Kyiv".into();
source.properties = vec![
DbProperty { name: "query_tag".into(), value: "ide".into() },
DbProperty { name: "statement_timeout_in_seconds".into(), value: "60".into() },
DbProperty { name: "client_session_keep_alive".into(), value: "true".into() },
];
let pool = Pool::connect(&source, None, None).await.unwrap();
let cancel = CancellationToken::new();
let rows = pool.execute("select 1", 10, &cancel).await.unwrap();
assert_eq!(rows.rows, vec![vec![Some("1".to_owned())]]);
assert_eq!(rows.columns[0].name, "N");
assert_eq!(rows.columns[0].type_name, "NUMBER(1,0)");
assert_eq!(rows.affected, None);
assert!(!rows.has_more);
pool.close().await;
assert!(pool.is_closed());
let seen = server.await.unwrap();
let login = &seen[0];
assert!(login.line().starts_with("POST /session/v1/login-request?"), "{}", login.line());
for parameter in
["databaseName=ANALYTICS", "schemaName=PUBLIC", "warehouse=COMPUTE_WH", "roleName=ANALYST", "requestId="]
{
assert!(login.line().contains(parameter), "{parameter}: {}", login.line());
}
assert_eq!(login.header("accept"), Some("application/snowflake"));
assert_eq!(login.header("authorization"), None);
let data = &login.json()["data"];
assert_eq!(data["AUTHENTICATOR"], "SNOWFLAKE_JWT");
assert_eq!(data["LOGIN_NAME"], "jane");
assert_eq!(data["ACCOUNT_NAME"], "127");
assert_eq!(data["CLIENT_APP_ID"], "OpenIDEA");
assert!(data.get("PASSWORD").is_none());
assert_eq!(
data["SESSION_PARAMETERS"],
json!({ "QUERY_TAG": "ide", "STATEMENT_TIMEOUT_IN_SECONDS": 60, "CLIENT_SESSION_KEEP_ALIVE": true, "TIMEZONE": "Europe/Kyiv" })
);
let token = data["TOKEN"].as_str().unwrap();
let claims: Value = serde_json::from_slice(
&base64::engine::general_purpose::URL_SAFE_NO_PAD.decode(token.split('.').nth(1).unwrap()).unwrap(),
)
.unwrap();
assert_eq!(claims["sub"], "127.JANE");
assert_eq!(claims["iss"], format!("127.JANE.{FINGERPRINT}"));
let query = &seen[1];
assert!(query.line().starts_with("POST /queries/v1/query-request?requestId="), "{}", query.line());
assert_eq!(query.header("authorization"), Some("Snowflake Token=\"session-token-1\""));
let body = query.json();
assert_eq!(body["sqlText"], "select 1");
assert_eq!(body["asyncExec"], false);
assert_eq!(body["parameters"]["ROWS_PER_RESULTSET"], 11);
assert_eq!(body["sequenceId"], 1);
let close = &seen[2];
assert!(close.line().starts_with("POST /session?delete=true"), "{}", close.line());
assert_eq!(close.header("authorization"), Some("Snowflake Token=\"session-token-1\""));
}
#[tokio::test]
async fn a_rejected_key_explains_itself_and_shows_the_fingerprint() {
let (listener, port) = bind().await;
let refused = r#"{"success":false,"code":"390144","message":"JWT token is invalid. [0ce9eb56-821d-4ca9-a774-04ae89a0cf5a]","data":{"authnMethod":"SNOWFLAKE_JWT"}}"#;
let server = serve(listener, vec![ok(refused)]);
let error = failure(&source(port), None).await;
assert_eq!(error.kind, ErrorKind::InvalidInput);
assert!(error.message.starts_with("Snowflake [390144]: JWT token is invalid."), "{}", error.message);
assert!(error.message.contains("ALTER USER"), "{}", error.message);
assert!(error.message.ends_with(&format!("Fingerprint of this private key: {FINGERPRINT}")), "{}", error.message);
server.await.unwrap();
}
#[tokio::test]
async fn key_problems_are_found_before_any_request() {
// Nothing listens: a request would fail with a network error instead.
let (listener, port) = bind().await;
drop(listener);
let mut encrypted = source(port);
encrypted.snowflake.as_mut().unwrap().private_key_file = ENCRYPTED_KEY.into();
let error = failure(&encrypted, None).await;
assert_eq!(error.kind, ErrorKind::PermissionDenied);
assert_eq!(error.message, "the private key is encrypted; enter its passphrase");
let error = failure(&encrypted, Some("wrong")).await;
assert_eq!(error.kind, ErrorKind::PermissionDenied);
assert_eq!(error.message, "incorrect passphrase for the private key");
let mut missing = source(port);
missing.snowflake.as_mut().unwrap().private_key_file = String::new();
assert_eq!(failure(&missing, None).await.message, "select the private key file of the Snowflake user");
let mut no_user = source(port);
no_user.user = " ".into();
assert_eq!(failure(&no_user, None).await.message, "enter the Snowflake user name");
let mut reserved = source(port);
reserved.properties = vec![DbProperty { name: "rows_per_resultset".into(), value: "5".into() }];
assert!(failure(&reserved, None).await.message.contains("manages the rows_per_resultset parameter"));
let mut no_account = source(port);
no_account.host = String::new();
assert!(failure(&no_account, None).await.message.contains("account identifier"));
for auth in [DbSnowflakeAuth::Password, DbSnowflakeAuth::Token, DbSnowflakeAuth::OAuth] {
let mut source = source(port);
source.snowflake.as_mut().unwrap().auth = auth;
let error = failure(&source, None).await;
assert_eq!(error.kind, ErrorKind::PermissionDenied, "{auth:?}");
assert!(error.message.starts_with("enter the "), "{}", error.message);
}
}
#[tokio::test]
async fn an_encrypted_key_signs_in_with_its_passphrase_and_never_leaks_it() {
let (listener, port) = bind().await;
// A server that echoes what it was sent must not get the secrets into an error message.
let echo =
r#"{"success":false,"code":"002003","message":"bad statement test-passphrase session-token-1 master-token-1"}"#;
let server = serve(listener, vec![ok(LOGIN_OK), ok(echo), ok(CLOSED_OK)]);
let mut source = source(port);
source.snowflake.as_mut().unwrap().private_key_file = ENCRYPTED_KEY.into();
let pool = Pool::connect(&source, Some("test-passphrase"), None).await.unwrap();
let error = pool.execute("select nope", 1, &CancellationToken::new()).await.unwrap_err();
assert_eq!(error.kind, ErrorKind::InvalidInput);
assert_eq!(error.message, "Snowflake [002003]: bad statement [redacted] [redacted] [redacted]");
pool.close().await;
let seen = server.await.unwrap();
assert!(!seen[0].body.contains("test-passphrase"));
assert!(!seen[0].head.contains("test-passphrase"));
}
#[tokio::test]
async fn password_and_token_methods_send_their_own_proof() {
for (auth, secret, expected) in [
(DbSnowflakeAuth::Password, "pa55word!", json!({ "PASSWORD": "pa55word!" })),
(
DbSnowflakeAuth::Token,
"pat-value-1",
json!({ "AUTHENTICATOR": "PROGRAMMATIC_ACCESS_TOKEN", "TOKEN": "pat-value-1" }),
),
(DbSnowflakeAuth::OAuth, "oauth-value-1", json!({ "AUTHENTICATOR": "OAUTH", "TOKEN": "oauth-value-1" })),
] {
let (listener, port) = bind().await;
let server = serve(listener, vec![ok(LOGIN_OK), ok(CLOSED_OK)]);
let mut source = source(port);
source.snowflake.as_mut().unwrap().auth = auth;
let pool = Pool::connect(&source, Some(secret), None).await.unwrap();
pool.close().await;
let seen = server.await.unwrap();
let data = &seen[0].json()["data"];
for (name, value) in expected.as_object().unwrap() {
assert_eq!(&data[name], value, "{auth:?} {name}");
}
if auth == DbSnowflakeAuth::Password {
assert!(data.get("AUTHENTICATOR").is_none());
}
}
}
#[tokio::test]
async fn a_wrong_password_asks_again() {
let (listener, port) = bind().await;
let refused = r#"{"success":false,"code":"390100","message":"Incorrect username or password was specified."}"#;
let server = serve(listener, vec![ok(refused)]);
let mut source = source(port);
source.snowflake.as_mut().unwrap().auth = DbSnowflakeAuth::Password;
let error = failure(&source, Some("pa55word!")).await;
assert_eq!(error.kind, ErrorKind::PermissionDenied);
assert_eq!(error.message, "Snowflake [390100]: Incorrect username or password was specified.");
server.await.unwrap();
}
#[tokio::test]
async fn an_unknown_account_is_named() {
let (listener, port) = bind().await;
let server = serve(
listener,
vec![Reply { status: "403 Forbidden", body: b"<html>Forbidden</html>".to_vec(), delay: Duration::ZERO }],
);
let error = failure(&source(port), None).await;
assert_eq!(error.kind, ErrorKind::InvalidInput);
assert_eq!(error.message, "Snowflake has no account at 127.0.0.1 (HTTP 403): check the account identifier");
server.await.unwrap();
}
#[tokio::test]
async fn pages_skip_whole_chunks_and_stop_after_the_page() {
let (listener, port) = bind().await;
let rowtype =
json!([{ "name": "ID", "type": "fixed", "precision": 38, "scale": 0 }, { "name": "D", "type": "date" }]);
let chunk = |n: u16| format!("http://127.0.0.1:{port}/chunk{n}");
let body = json!({ "success": true, "data": {
"rowtype": rowtype, "rowset": [["1", "0"], ["2", "1"]], "queryResultFormat": "json", "statementTypeId": 4096,
"chunks": [
{ "url": chunk(1), "rowCount": 2 }, { "url": chunk(2), "rowCount": 3 }, { "url": chunk(3), "rowCount": 2 },
],
"chunkHeaders": { "x-amz-server-side-encryption-customer-key": "key-1" },
} })
.to_string();
let mut gz = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::default());
std::io::Write::write_all(&mut gz, br#"["5","4"],["6","5"],["7",null]"#).unwrap();
let second = Reply { status: "200 OK", body: gz.finish().unwrap(), delay: Duration::ZERO };
// Rows 5 and 6 (skip 4, keep 2): the first chunk is skipped, the third never fetched.
let server = serve(listener, vec![ok(LOGIN_OK), ok(body), second, ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let page = pool.execute_from("select id, d from t", 4, 2, &CancellationToken::new()).await.unwrap();
assert_eq!(
page.rows,
vec![vec![Some("5".into()), Some("1970-01-05".into())], vec![Some("6".into()), Some("1970-01-06".into())]]
);
assert!(page.has_more);
assert_eq!(page.columns[1].type_name, "DATE");
pool.close().await;
let seen = server.await.unwrap();
assert_eq!(seen[1].json()["parameters"]["ROWS_PER_RESULTSET"], 7);
assert_eq!(seen[2].line(), "GET /chunk2 HTTP/1.1");
assert_eq!(seen[2].header("x-amz-server-side-encryption-customer-key"), Some("key-1"));
assert_eq!(seen[2].header("authorization"), None);
}
#[tokio::test]
async fn a_long_statement_is_polled_until_it_finishes() {
let (listener, port) = bind().await;
let running = |code: &str| {
json!({ "success": true, "code": code, "message": "Query execution in progress.",
"data": { "getResultUrl": "/queries/01b2c3/result", "queryId": "01b2c3" } })
.to_string()
};
let server =
serve(listener, vec![ok(LOGIN_OK), ok(running("333333")), ok(running("333334")), ok(one()), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let done = pool.execute("call system$wait(60)", 5, &CancellationToken::new()).await.unwrap();
assert_eq!(done.rows, vec![vec![Some("1".to_owned())]]);
pool.close().await;
let seen = server.await.unwrap();
for poll in &seen[2..4] {
assert_eq!(poll.line(), "GET /queries/01b2c3/result HTTP/1.1");
assert_eq!(poll.header("authorization"), Some("Snowflake Token=\"session-token-1\""));
}
}
#[tokio::test]
async fn an_expired_session_token_is_renewed_and_the_statement_repeated() {
let (listener, port) = bind().await;
let expired = r#"{"success":false,"code":"390112","message":"Your session has expired. Please login again."}"#;
let renewed = r#"{"success":true,"data":{"sessionToken":"session-token-2","masterToken":"master-token-2"}}"#;
let server = serve(listener, vec![ok(LOGIN_OK), ok(expired), ok(renewed), ok(one()), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let done = pool.execute("select 1", 5, &CancellationToken::new()).await.unwrap();
assert_eq!(done.rows.len(), 1);
pool.close().await;
let seen = server.await.unwrap();
assert!(seen[2].line().starts_with("POST /session/token-request?requestId="), "{}", seen[2].line());
assert_eq!(seen[2].header("authorization"), Some("Snowflake Token=\"master-token-1\""));
assert_eq!(seen[2].json(), json!({ "oldSessionToken": "session-token-1", "requestType": "RENEW" }));
assert_eq!(seen[3].header("authorization"), Some("Snowflake Token=\"session-token-2\""));
assert_eq!(seen[4].header("authorization"), Some("Snowflake Token=\"session-token-2\""));
}
#[tokio::test]
async fn a_lost_session_signs_in_again() {
let (listener, port) = bind().await;
let gone = r#"{"success":false,"code":"390114","message":"Authentication token has expired. The user must authenticate again."}"#;
let again = r#"{"success":true,"data":{"token":"session-token-3","masterToken":"master-token-3"}}"#;
let server = serve(listener, vec![ok(LOGIN_OK), ok(gone), ok(again), ok(one()), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
pool.execute("select 1", 5, &CancellationToken::new()).await.unwrap();
pool.close().await;
let seen = server.await.unwrap();
assert!(seen[2].line().starts_with("POST /session/v1/login-request?"), "{}", seen[2].line());
assert_eq!(seen[3].header("authorization"), Some("Snowflake Token=\"session-token-3\""));
}
#[tokio::test]
async fn cancelling_aborts_the_statement_on_the_server() {
let (listener, port) = bind().await;
let slow = Reply { status: "200 OK", body: one().into_bytes(), delay: Duration::from_secs(30) };
let server = serve(listener, vec![ok(LOGIN_OK), slow, ok(r#"{"success":true,"data":null}"#), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let statement = pool.execute("call system$wait(600)", 5, &cancel);
let stop = async {
tokio::time::sleep(Duration::from_millis(300)).await;
cancel.cancel();
};
let (outcome, ()) = tokio::join!(statement, stop);
let error = outcome.unwrap_err();
assert_eq!(error.kind, ErrorKind::Cancelled);
assert_eq!(error.message, "cancelled");
pool.close().await;
let seen = server.await.unwrap();
let request_id =
|line: &str| line.split(['?', '&', ' ']).find_map(|p| p.strip_prefix("requestId=")).unwrap().to_owned();
assert!(seen[2].line().starts_with("POST /queries/v1/abort-request?"), "{}", seen[2].line());
let abort = seen[2].json();
assert_eq!(abort["sqlText"], "call system$wait(600)");
assert_eq!(abort["requestId"], request_id(seen[1].line()));
assert_ne!(request_id(seen[2].line()), request_id(seen[1].line()));
}
#[tokio::test]
async fn changes_report_counts_and_commands_report_nothing() {
let (listener, port) = bind().await;
let merged = json!({ "success": true, "data": {
"rowtype": [{ "name": "number of rows inserted", "type": "fixed" }, { "name": "number of rows updated", "type": "fixed" }],
"rowset": [["3", "4"]], "queryResultFormat": "json", "statementTypeId": 0x3400 } })
.to_string();
let created = json!({ "success": true, "data": {
"rowtype": [{ "name": "status", "type": "text" }], "rowset": [["Table T successfully created."]],
"queryResultFormat": "json", "statementTypeId": 0x6100 } })
.to_string();
let server = serve(listener, vec![ok(LOGIN_OK), ok(merged), ok(created), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let merge = pool.execute("merge into t using s on …", 10, &cancel).await.unwrap();
assert_eq!((merge.affected, merge.columns.len(), merge.rows.len()), (Some(7.0), 0, 0));
let create = pool.execute("create table t (id int)", 10, &cancel).await.unwrap();
assert_eq!((create.affected, create.columns.len(), create.rows.len()), (Some(0.0), 0, 0));
pool.close().await;
server.await.unwrap();
}
#[tokio::test]
async fn a_read_only_data_source_refuses_writes_without_sending_them() {
let (listener, port) = bind().await;
let server = serve(listener, vec![ok(LOGIN_OK), ok(one()), ok(one()), ok(CLOSED_OK)]);
let mut source = source(port);
source.read_only = true;
source.options.startup_script = "alter session set week_start = 1".into();
let pool = Pool::connect(&source, None, None).await.unwrap();
let cancel = CancellationToken::new();
for write in
["delete from t", " -- note\n /* x */ INSERT INTO t VALUES (1)", "create table x (i int)", "// c\ncall p()"]
{
let error = pool.execute(write, 1, &cancel).await.unwrap_err();
assert_eq!(error.kind, ErrorKind::InvalidInput);
assert!(error.message.starts_with("read-only data source: "), "{}", error.message);
}
assert!(pool.transaction(&[("update t set a = 1".into(), true)]).await.is_err());
pool.execute("-- reads pass\nwith x as (select 1) select * from x", 1, &cancel).await.unwrap();
pool.close().await;
let seen = server.await.unwrap();
assert_eq!(seen[1].json()["sqlText"], "alter session set week_start = 1");
}
#[tokio::test]
async fn table_edits_commit_together_or_roll_back() {
let changed = |n: &str| {
json!({ "success": true, "data": { "rowtype": [{ "name": "number of rows updated", "type": "fixed" }],
"rowset": [[n]], "queryResultFormat": "json", "statementTypeId": 0x3200 } })
.to_string()
};
let status = || {
json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string()
};
let edits =
[("update t set a = 1 where id = 1".to_owned(), true), ("update t set a = 2 where id = 2".to_owned(), true)];
let (listener, port) = bind().await;
let server = serve(
listener,
vec![ok(LOGIN_OK), ok(status()), ok(changed("1")), ok(changed("1")), ok(status()), ok(CLOSED_OK)],
);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
assert_eq!(pool.transaction(&edits).await.unwrap(), 2);
pool.close().await;
let sent: Vec<String> =
server.await.unwrap()[1..5].iter().map(|r| r.json()["sqlText"].as_str().unwrap().to_owned()).collect();
assert_eq!(sent, ["BEGIN", "update t set a = 1 where id = 1", "update t set a = 2 where id = 2", "COMMIT"]);
let (listener, port) = bind().await;
let server = serve(listener, vec![ok(LOGIN_OK), ok(status()), ok(changed("0")), ok(status()), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let error = pool.transaction(&edits).await.unwrap_err();
assert!(error.message.contains("expected to change 1 row, changed 0; nothing was saved"), "{}", error.message);
pool.close().await;
let sent: Vec<String> =
server.await.unwrap()[1..4].iter().map(|r| r.json()["sqlText"].as_str().unwrap().to_owned()).collect();
assert_eq!(sent, ["BEGIN", "update t set a = 1 where id = 1", "ROLLBACK"]);
}
#[tokio::test]
async fn an_arrow_result_is_refused_instead_of_shown_empty() {
let (listener, port) = bind().await;
let arrow = json!({ "success": true, "data": { "rowtype": [{ "name": "N", "type": "fixed" }],
"rowsetBase64": "QVJST1c=", "queryResultFormat": "arrow", "statementTypeId": 4096 } })
.to_string();
let server = serve(listener, vec![ok(LOGIN_OK), ok(arrow), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let error = pool.execute("select 1", 1, &CancellationToken::new()).await.unwrap_err();
assert_eq!(error.kind, ErrorKind::Protocol);
pool.close().await;
server.await.unwrap();
}
#[test]
fn read_only_knows_reads_from_writes() {
for read in [
"select 1",
"WITH a AS (select 1) select * from a",
"show tables",
"desc table t",
"explain select 1",
"ls @stage",
"use schema s",
"",
] {
assert!(!is_write(read), "{read}");
}
for write in [
"insert into t values (1)",
"COPY INTO t FROM @s",
"put file://x @s",
"truncate t",
"grant select on t to role r",
"begin",
] {
assert!(is_write(write), "{write}");
}
}
#[tokio::test]
async fn browser_sign_in_presents_the_token_and_proof_key_then_reuses_the_id_token() {
let (listener, port) = bind().await;
let page =
r#"{"success":true,"data":{"ssoUrl":"https://idp.example/sso?SAMLRequest=abc","proofKey":"proof-key-1"}}"#;
let signed_in =
r#"{"success":true,"data":{"token":"session-token-1","masterToken":"master-token-1","idToken":"id-token-1"}}"#;
let again = r#"{"success":true,"data":{"token":"session-token-2","masterToken":"master-token-2"}}"#;
let stale = r#"{"success":false,"code":"390195","message":"The provided ID Token is invalid."}"#;
let server = serve(
listener,
vec![
// First connection: the browser.
ok(page),
ok(signed_in),
ok(CLOSED_OK),
// Second connection: the ID token alone.
ok(again),
ok(CLOSED_OK),
// Third: the ID token is refused, so the browser again.
ok(stale),
ok(page),
ok(signed_in),
ok(CLOSED_OK),
],
);
let opened = Arc::new(parking_lot::Mutex::new(Vec::<String>::new()));
let pages = opened.clone();
// The stand-in browser: the identity provider sends it back to the listener with a token.
*browser::TEST_OPENER.lock() = Some(Box::new(move |url, redirect_port| {
pages.lock().push(url.to_owned());
tokio::spawn(async move {
let mut stream = TcpStream::connect(("127.0.0.1", redirect_port)).await.unwrap();
// A browser also asks for the tab's icon; that request must not end the sign-in.
stream.write_all(b"GET /favicon.ico HTTP/1.1\r\nHost: localhost\r\n\r\n").await.unwrap();
let mut reply = Vec::new();
stream.read_to_end(&mut reply).await.unwrap();
assert!(reply.starts_with(b"HTTP/1.1 404"));
let mut stream = TcpStream::connect(("127.0.0.1", redirect_port)).await.unwrap();
stream.write_all(b"GET /?token=sso%2Btoken-1 HTTP/1.1\r\nHost: localhost\r\n\r\n").await.unwrap();
let mut reply = Vec::new();
stream.read_to_end(&mut reply).await.unwrap();
let reply = String::from_utf8_lossy(&reply).into_owned();
assert!(reply.starts_with("HTTP/1.1 200 OK"), "{reply}");
assert!(reply.contains("return to OpenIDEA"), "{reply}");
});
Ok(())
}));
let mut source = source(port);
source.snowflake.as_mut().unwrap().auth = DbSnowflakeAuth::Browser;
source.snowflake.as_mut().unwrap().private_key_file = String::new();
for _ in 0..3 {
let pool = Pool::connect(&source, None, None).await.unwrap();
pool.close().await;
}
*browser::TEST_OPENER.lock() = None;
let seen = server.await.unwrap();
assert_eq!(*opened.lock(), ["https://idp.example/sso?SAMLRequest=abc"; 2]);
assert!(seen[0].line().starts_with("POST /session/authenticator-request?requestId="), "{}", seen[0].line());
let asked = &seen[0].json()["data"];
assert_eq!(asked["AUTHENTICATOR"], "EXTERNALBROWSER");
assert_eq!(asked["LOGIN_NAME"], "jane");
assert_eq!(asked["ACCOUNT_NAME"], "127");
assert!(asked["BROWSER_MODE_REDIRECT_PORT"].as_str().unwrap().parse::<u16>().unwrap() > 0);
let first = &seen[1].json()["data"];
assert_eq!(first["AUTHENTICATOR"], "EXTERNALBROWSER");
assert_eq!(first["TOKEN"], "sso+token-1");
assert_eq!(first["PROOF_KEY"], "proof-key-1");
assert_eq!(first["SESSION_PARAMETERS"]["CLIENT_STORE_TEMPORARY_CREDENTIAL"], true);
assert!(first.get("PASSWORD").is_none());
let second = &seen[3].json()["data"];
assert_eq!(second["AUTHENTICATOR"], "ID_TOKEN");
assert_eq!(second["TOKEN"], "id-token-1");
assert!(second.get("PROOF_KEY").is_none());
assert_eq!(seen[5].json()["data"]["AUTHENTICATOR"], "ID_TOKEN");
assert!(seen[6].line().starts_with("POST /session/authenticator-request?"), "{}", seen[6].line());
assert_eq!(seen[7].json()["data"]["AUTHENTICATOR"], "EXTERNALBROWSER");
}
#[tokio::test]
async fn an_account_without_browser_sign_in_says_so() {
let (listener, port) = bind().await;
let refused = r#"{"success":false,"code":"390190","message":"There was an error related to the SAML Identity Provider account parameter."}"#;
let server = serve(listener, vec![ok(refused)]);
let mut source = source(port);
source.user = "nobody".into();
source.snowflake.as_mut().unwrap().auth = DbSnowflakeAuth::Browser;
let error = failure(&source, None).await;
assert_eq!(error.kind, ErrorKind::InvalidInput);
assert!(
error.message.starts_with("Snowflake [390190]: There was an error related to the SAML"),
"{}",
error.message
);
server.await.unwrap();
}
#[tokio::test]
async fn a_console_schema_is_switched_with_its_statement_and_only_when_needed() {
let status = || {
json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string()
};
let (listener, port) = bind().await;
let mut replies = vec![ok(LOGIN_OK)];
// use Sales, 1, 2 | restore PUBLIC, 3 | 4 | use database (the user's), then PUBLIC again, 5
for body in [status(), one(), one(), status(), one(), one(), status(), status(), one()] {
replies.push(ok(body));
}
replies.push(ok(CLOSED_OK));
let server = serve(listener, replies);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let mut console = crate::conn::Held::acquire(&pool).await.unwrap();
console.apply_context(Some("Sales \"2024\"")).await.unwrap();
console.execute(&pool, "select 1", 1, &cancel).await.unwrap();
console.execute(&pool, "select 2", 1, &cancel).await.unwrap();
// The tree and other consoles run in the data source's schema.
pool.execute("select 3", 1, &cancel).await.unwrap();
pool.execute("select 4", 1, &cancel).await.unwrap();
// After the user's own USE the session may be anywhere: the schema is set again.
pool.execute("use database OTHER", 0, &cancel).await.unwrap();
pool.execute("select 5", 1, &cancel).await.unwrap();
pool.close().await;
let sent: Vec<String> =
server.await.unwrap()[1..10].iter().map(|r| r.json()["sqlText"].as_str().unwrap().to_owned()).collect();
assert_eq!(
sent,
[
"USE SCHEMA \"ANALYTICS\".\"Sales \"\"2024\"\"\"",
"select 1",
"select 2",
"USE SCHEMA \"ANALYTICS\".\"PUBLIC\"",
"select 3",
"select 4",
"use database OTHER",
"USE SCHEMA \"ANALYTICS\".\"PUBLIC\"",
"select 5",
]
);
}
#[tokio::test]
async fn a_role_that_may_not_use_the_schema_still_runs_statements() {
let status = || {
json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string()
};
let refused = r#"{"success":false,"code":"002043","message":"SQL compilation error:\nObject does not exist, or operation cannot be performed."}"#;
let (listener, port) = bind().await;
// use role | the schema is refused once, the statement runs | the next one does not ask again |
// a role that is not granted | use role back, the schema is entered again
let replies = vec![
ok(LOGIN_OK),
ok(status()),
ok(refused),
ok(one()),
ok(one()),
ok(refused),
ok(status()),
ok(status()),
ok(one()),
ok(CLOSED_OK),
];
let server = serve(listener, replies);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
pool.execute("use role auditor", 0, &cancel).await.unwrap();
pool.execute("select 1", 1, &cancel).await.unwrap();
pool.execute("select 2", 1, &cancel).await.unwrap();
let missing = pool.execute("use role nobody", 0, &cancel).await.unwrap_err();
assert!(missing.message.contains("002043") && missing.message.contains("granted"), "{}", missing.message);
pool.execute("use role analyst", 0, &cancel).await.unwrap();
pool.execute("select 3", 1, &cancel).await.unwrap();
pool.close().await;
let sent: Vec<String> =
server.await.unwrap()[1..9].iter().map(|r| r.json()["sqlText"].as_str().unwrap().to_owned()).collect();
let schema = "USE SCHEMA \"ANALYTICS\".\"PUBLIC\"";
assert_eq!(
sent,
["use role auditor", schema, "select 1", "select 2", "use role nobody", "use role analyst", schema, "select 3"]
);
}
#[tokio::test]
async fn databases_of_one_data_source_share_a_login() {
let (listener, port) = bind().await;
let status = json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string();
// One login; `select 1` in ANALYTICS; a switch and `select 2` in "Raw Data"; one session end.
let server = serve(listener, vec![ok(LOGIN_OK), ok(one()), ok(status), ok(one()), ok(CLOSED_OK)]);
let first = Pool::connect(&source(port), None, None).await.unwrap();
let mut other = source(port);
other.database = "Raw Data".into();
other.options.default_schema = String::new();
other.read_only = true;
let second = Pool::connect(&other, None, None).await.unwrap();
let cancel = CancellationToken::new();
first.execute("select 1", 1, &cancel).await.unwrap();
second.execute("select 2", 1, &cancel).await.unwrap();
// Read-only is the target's own setting.
assert!(second.execute("delete from t", 1, &cancel).await.is_err());
first.close().await;
assert!(first.is_closed() && !second.is_closed());
assert!(first.execute("select 3", 1, &cancel).await.is_err());
second.close().await;
let seen = server.await.unwrap();
let sent: Vec<String> = seen[1..4].iter().map(|r| r.json()["sqlText"].as_str().unwrap().to_owned()).collect();
assert_eq!(sent, ["select 1", "USE DATABASE \"Raw Data\"", "select 2"]);
assert!(seen[4].line().starts_with("POST /session?delete=true"), "{}", seen[4].line());
}
#[tokio::test]
async fn metadata_runs_on_its_own_session_beside_a_busy_console() {
let (listener, port) = bind().await;
let slow = Reply { status: "200 OK", body: one().into_bytes(), delay: Duration::from_millis(1500) };
let second = r#"{"success":true,"data":{"token":"session-token-2","masterToken":"master-token-2"}}"#;
let status = json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string();
// Console login and its slow statement; the metadata session logs in, enters the schema and answers meanwhile.
let server =
serve(listener, vec![ok(LOGIN_OK), slow, ok(second), ok(status), ok(one()), ok(CLOSED_OK), ok(CLOSED_OK)]);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
let started = Instant::now();
let console = pool.execute("call system$wait(60)", 1, &cancel);
let tree = async {
tokio::time::sleep(Duration::from_millis(200)).await;
let rows = pool.rows("show terse schemas", 10).await.unwrap();
(rows, started.elapsed())
};
let (console, (rows, tree_took)) = tokio::join!(console, tree);
console.unwrap();
assert_eq!(rows.len(), 1);
assert!(tree_took < Duration::from_millis(1200), "the tree waited for the console: {tree_took:?}");
pool.close().await;
let seen = server.await.unwrap();
assert!(seen[2].line().starts_with("POST /session/v1/login-request?"), "{}", seen[2].line());
assert_eq!(seen[3].json()["sqlText"], "USE SCHEMA \"ANALYTICS\".\"PUBLIC\"");
assert_eq!(seen[4].header("authorization"), Some("Snowflake Token=\"session-token-2\""));
// Both sessions end with the connection.
let ended: Vec<&str> = seen[5..7].iter().filter_map(|r| r.header("authorization")).collect();
assert_eq!(ended.len(), 2);
assert!(
ended.contains(&"Snowflake Token=\"session-token-1\"")
&& ended.contains(&"Snowflake Token=\"session-token-2\"")
);
}
#[tokio::test]
async fn a_database_is_introspected_schema_by_schema_when_its_whole_listing_is_refused() {
let (listener, port) = bind().await;
let second = r#"{"success":true,"data":{"token":"session-token-2","masterToken":"master-token-2"}}"#;
let text = |name: &str| json!({ "name": name, "type": "text" });
let terse = json!([text("created_on"), text("name"), text("kind"), text("database_name"), text("schema_name")]);
let schemas = result(
terse.clone(),
json!([
["", "INFORMATION_SCHEMA", null, "RAW", null],
["", "PUBLIC", null, "RAW", null],
["", "STAGE", null, "RAW", null]
]),
);
let refused = r#"{"success":false,"code":"001003","message":"SQL compilation error: unsupported listing"}"#;
let public = result(
terse.clone(),
json!([["", "ORDERS", "TABLE", "RAW", "PUBLIC"], ["", "DAILY", "VIEW", "RAW", "PUBLIC"]]),
);
let stage = result(terse, json!([["", "ORDERS", "TABLE", "RAW", "STAGE"]]));
// Metadata session: login, schema context, the schemas, the refused listing, then each schema.
let replies = vec![
ok(LOGIN_OK),
ok(second),
ok(one()),
ok(schemas),
ok(refused),
ok(public),
ok(stage),
ok(CLOSED_OK),
ok(CLOSED_OK),
];
let server = serve(listener, replies);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let Pool::Snowflake(snowflake) = &pool else { panic!("not a Snowflake pool") };
let catalog = super::meta::catalog(snowflake, "RAW").await.unwrap();
let names = |i: usize| catalog[i].tables.iter().map(|t| t.name.as_str()).collect::<Vec<_>>();
assert_eq!(catalog.iter().map(|s| s.name.as_str()).collect::<Vec<_>>(), ["PUBLIC", "STAGE"]);
assert_eq!(names(0), ["DAILY", "ORDERS"]);
assert_eq!(names(1), ["ORDERS"]);
pool.close().await;
let seen = server.await.unwrap();
let sql = |i: usize| seen[i].json()["sqlText"].as_str().unwrap_or_default().to_owned();
assert_eq!(sql(3), "SHOW TERSE SCHEMAS IN DATABASE \"RAW\" LIMIT 10000");
assert_eq!(sql(4), "SHOW TERSE OBJECTS IN DATABASE \"RAW\" LIMIT 10000");
assert_eq!(sql(5), "SHOW TERSE OBJECTS IN SCHEMA \"RAW\".\"PUBLIC\" LIMIT 10000");
assert_eq!(sql(6), "SHOW TERSE OBJECTS IN SCHEMA \"RAW\".\"STAGE\" LIMIT 10000");
}
#[tokio::test]
async fn targets_opened_at_the_same_time_sign_in_once() {
let (listener, port) = bind().await;
// Had each connect signed in, the server would be asked for more than this one login.
let slow_login = Reply { status: "200 OK", body: LOGIN_OK.as_bytes().to_vec(), delay: Duration::from_millis(300) };
let server = serve(listener, vec![slow_login, ok(CLOSED_OK)]);
let base = source(port);
let mut others = Vec::new();
for database in ["RAW", "STAGING", "MARTS"] {
let mut other = source(port);
other.database = database.into();
others.push(other);
}
let (a, b, c, d) = tokio::join!(
Pool::connect(&base, None, None),
Pool::connect(&others[0], None, None),
Pool::connect(&others[1], None, None),
Pool::connect(&others[2], None, None),
);
let pools = [a.unwrap(), b.unwrap(), c.unwrap(), d.unwrap()];
for pool in &pools {
pool.close().await;
}
let seen = server.await.unwrap();
assert_eq!(seen.len(), 2);
assert!(seen[0].line().starts_with("POST /session/v1/login-request?"), "{}", seen[0].line());
assert!(seen[1].line().starts_with("POST /session?delete=true"), "{}", seen[1].line());
}
#[test]
fn single_sign_on_tokens_are_keyed_by_account_and_user_and_only_for_browser_sign_in() {
let mut source = source(443);
source.host = "myorg-myaccount".into();
source.port = None;
source.ssl = DbSslMode::Prefer;
assert_eq!(sso_key(&source), None);
source.snowflake.as_mut().unwrap().auth = DbSnowflakeAuth::Browser;
let key = sso_key(&source).unwrap();
assert_eq!(key, "myorg-myaccount.snowflakecomputing.com:443|JANE");
assert_eq!(sso_token(&key), None);
sso_seed(&key, "saved-id-token");
assert_eq!(sso_token(&key).as_deref(), Some("saved-id-token"));
// A token already in use is not replaced by an older saved one.
sso_seed(&key, "stale");
assert_eq!(sso_token(&key).as_deref(), Some("saved-id-token"));
browser::forget(&key);
}
+1 -138
Переглянути файл
@@ -8,7 +8,7 @@ use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::Path;
pub(crate) const FILE: &str = "data-sources.toml";
const FILE: &str = "data-sources.toml";
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
@@ -22,7 +22,6 @@ pub enum DbDriver {
Clickhouse,
/// Oracle Database (pure-Rust thin driver, no client library).
Oracle,
Snowflake,
}
impl DbDriver {
@@ -34,7 +33,6 @@ impl DbDriver {
DbDriver::SqlServer => Some(1433),
DbDriver::Clickhouse => Some(8123),
DbDriver::Oracle => Some(1521),
DbDriver::Snowflake => Some(443),
}
}
}
@@ -133,37 +131,6 @@ pub struct DbProperty {
pub value: String,
}
/// How a Snowflake data source signs in. The data source's one saved secret is what the
/// method needs: the private key's passphrase, the password, or the token.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum DbSnowflakeAuth {
/// RSA key pair: a JWT signed with the private key file.
#[default]
KeyPair,
/// User name and password.
Password,
/// Programmatic access token.
Token,
/// OAuth access token.
OAuth,
/// Single sign-on in the web browser.
Browser,
}
/// Snowflake connection settings beyond the shared ones: the account identifier is
/// [`DataSource::host`], the schema [`DbOptions::default_schema`].
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", default)]
pub struct DbSnowflake {
pub warehouse: String,
pub role: String,
pub auth: DbSnowflakeAuth,
/// PEM private key of [`DbSnowflakeAuth::KeyPair`]; `~/` is the home directory. Only the
/// path is stored: the key is read from the file at each login.
pub private_key_file: String,
}
/// How the SSH tunnel authenticates (shared with Settings › SSH Configurations).
pub use ide_core::remote_settings::SshAuth as DbSshAuth;
@@ -218,8 +185,6 @@ pub struct DataSource {
pub options: DbOptions,
pub introspection: DbIntrospection,
pub properties: Vec<DbProperty>,
/// Snowflake settings (`None`: not a Snowflake data source, or all defaults).
pub snowflake: Option<DbSnowflake>,
/// Stored app-wide and listed in every project. Implied by the file it is read from,
/// so not written to it.
pub global: bool,
@@ -257,7 +222,6 @@ impl Default for DataSource {
options: DbOptions::default(),
introspection: DbIntrospection::default(),
properties: Vec::new(),
snowflake: None,
global: false,
plugin_owner: None,
plugin_source_id: None,
@@ -309,7 +273,6 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
"sqlite" | "sqlite3" => Some(DbDriver::Sqlite),
"clickhouse" | "http" | "https" => Some(DbDriver::Clickhouse),
"oracle" | "oracledb" => Some(DbDriver::Oracle),
"snowflake" => Some(DbDriver::Snowflake),
_ => None,
}
}
@@ -412,9 +375,6 @@ impl DataSource {
}
let mut properties = Vec::new();
for (k, v) in u.params {
if s.driver == DbDriver::Snowflake && s.snowflake_url_parameter(&k, &v) {
continue;
}
match k.to_ascii_lowercase().as_str() {
"sslmode" | "ssl-mode" => {
s.ssl = match v.to_ascii_lowercase().replace('-', "_").as_str() {
@@ -445,35 +405,6 @@ impl DataSource {
Ok(s)
}
/// Applies a parameter of a Snowflake JDBC URL that names a connection setting.
fn snowflake_url_parameter(&mut self, name: &str, value: &str) -> bool {
let settings = self.snowflake.get_or_insert_with(DbSnowflake::default);
match name.to_ascii_lowercase().as_str() {
"db" | "database" => self.database = value.to_owned(),
"schema" => self.options.default_schema = value.to_owned(),
"warehouse" => settings.warehouse = value.to_owned(),
"role" => settings.role = value.to_owned(),
"private_key_file" => settings.private_key_file = value.to_owned(),
"authenticator" => {
settings.auth = match value.to_ascii_lowercase().as_str() {
"snowflake_jwt" => DbSnowflakeAuth::KeyPair,
"externalbrowser" => DbSnowflakeAuth::Browser,
"oauth" => DbSnowflakeAuth::OAuth,
"programmatic_access_token" => DbSnowflakeAuth::Token,
"snowflake" => DbSnowflakeAuth::Password,
_ => return false,
}
}
_ => return false,
}
true
}
/// The Snowflake settings (defaults when none were saved).
pub fn snowflake(&self) -> DbSnowflake {
self.snowflake.clone().unwrap_or_default()
}
/// The password written in the URL of a [`DataSource::url_only`] data source.
pub fn url_password(&self) -> Option<String> {
if !self.url_only {
@@ -682,72 +613,4 @@ mod tests {
let error = parse_url("clickhouse://reader:[email protected]:invalid/default").unwrap_err();
assert!(!error.message.contains("secret"));
}
#[test]
fn snowflake_settings_round_trip_and_stay_out_of_other_sources() {
let tmp = tempfile::tempdir().unwrap();
let snowflake = DataSource {
id: "sf".into(),
name: "warehouse".into(),
driver: DbDriver::Snowflake,
host: "myorg-myaccount".into(),
user: "jane".into(),
database: "ANALYTICS".into(),
save_password: true,
snowflake: Some(DbSnowflake {
warehouse: "COMPUTE_WH".into(),
role: "ANALYST".into(),
auth: DbSnowflakeAuth::KeyPair,
private_key_file: "~/.ssh/rsa_key.p8".into(),
}),
..Default::default()
};
let other = DataSource { id: "pg".into(), name: "pg".into(), driver: DbDriver::Postgres, ..Default::default() };
save(tmp.path(), &[snowflake.clone(), other.clone()]).unwrap();
assert_eq!(load(tmp.path()).unwrap(), vec![snowflake, other]);
let text = std::fs::read_to_string(project_dir::path(tmp.path(), FILE)).unwrap();
assert_eq!(text.matches("snowflake]").count(), 1, "{text}");
assert!(text.contains("driver = \"snowflake\""), "{text}");
assert!(text.contains("auth = \"keyPair\""), "{text}");
assert!(text.contains("privateKeyFile = \"~/.ssh/rsa_key.p8\""), "{text}");
assert_eq!(DbDriver::Snowflake.default_port(), Some(443));
assert_eq!(DataSource::default().snowflake(), DbSnowflake::default());
}
#[test]
fn snowflake_jdbc_urls_name_the_session_settings() {
let s = DataSource {
driver: DbDriver::Snowflake,
url_only: true,
url: "jdbc:snowflake://myorg-myaccount.snowflakecomputing.com:443/?warehouse=COMPUTE_WH&db=ANALYTICS\
&schema=PUBLIC&role=ANALYST&user=jane&authenticator=snowflake_jwt\
&private_key_file=%7E%2Fkeys%2Frsa_key.p8&QUERY_TAG=ide"
.into(),
..Default::default()
};
let r = s.resolved().unwrap();
assert_eq!(r.host, "myorg-myaccount.snowflakecomputing.com");
assert_eq!(r.port, Some(443));
assert_eq!(r.user, "jane");
assert_eq!(r.database, "ANALYTICS");
assert_eq!(r.options.default_schema, "PUBLIC");
assert_eq!(
r.snowflake(),
DbSnowflake {
warehouse: "COMPUTE_WH".into(),
role: "ANALYST".into(),
auth: DbSnowflakeAuth::KeyPair,
private_key_file: "~/keys/rsa_key.p8".into(),
}
);
assert_eq!(r.properties, vec![DbProperty { name: "QUERY_TAG".into(), value: "ide".into() }]);
let browser = DataSource {
url: "snowflake://xy12345.eu-central-1.snowflakecomputing.com/DB?authenticator=externalbrowser".into(),
..s
};
let r = browser.resolved().unwrap();
assert_eq!(r.database, "DB");
assert_eq!(r.snowflake().auth, DbSnowflakeAuth::Browser);
assert_eq!(scheme_driver("snowflake"), Some(DbDriver::Snowflake));
}
}
+2 -137
Переглянути файл
@@ -32,7 +32,6 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let clickhouse = driver == DbDriver::Clickhouse;
// PL/SQL: `;` inside BEGIN…END / DECLARE…END blocks does not terminate a statement.
let plsql = driver == DbDriver::Oracle;
let snowflake = driver == DbDriver::Snowflake;
let chars: Vec<(usize, char)> = text.char_indices().collect();
// UTF-16 offset of each char index (plus the end).
let mut u16_at = Vec::with_capacity(chars.len() + 1);
@@ -58,10 +57,6 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let mut block = false;
// `CREATE PACKAGE` / `TYPE BODY`: only `/` on its own line (or the end) ends it.
let mut package = false;
// The segment is a Snowflake Scripting block (`DECLARE … BEGIN … END`, `BEGIN … END`).
let mut script = false;
// The segment may carry a Scripting block as its body (`CREATE TASK … AS BEGIN … END`).
let mut defines = false;
let push = |out: &mut Vec<SqlStatement>, s: usize, e: usize, has_code: bool| {
if !has_code {
return;
@@ -96,29 +91,11 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
// Re-read this character as code (it may start BEGIN / DECLARE).
continue;
}
_ if snowflake && !has_code && c.is_alphabetic() => {
if word_at(&chars, i, "DECLARE") {
(script, block, declarations) = (true, true, true);
} else if word_at(&chars, i, "BEGIN") {
// `BEGIN;`, `BEGIN TRANSACTION`… open a transaction; anything else a block.
let after = skip_ws(&chars, i + 5);
let transaction = chars.get(after).is_none_or(|&(_, c)| c == ';')
|| ["TRANSACTION", "WORK", "NAME"].into_iter().any(|w| word_at(&chars, after, w));
(script, block) = (!transaction, !transaction);
} else {
defines = ["CREATE", "WITH", "EXECUTE"].into_iter().any(|w| word_at(&chars, i, w));
}
has_code = true;
// Re-read this character as code (it starts BEGIN / DECLARE).
continue;
}
';' if depth == 0 && !declarations && !package => {
push(&mut out, start, if block { i + 1 } else { i }, has_code);
start = i + 1;
has_code = false;
block = false;
script = false;
defines = false;
}
';' => has_code = true,
'\'' => {
@@ -155,16 +132,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
i += 1;
}
'#' if mysql => state = State::LineComment,
// The `//` of `file://…` (PUT, GET) or another URL is not a comment.
'/' if snowflake && next == Some('/') && chars[..i].last().is_some_and(|&(_, c)| c == ':') => {
has_code = true;
i += 1;
}
'/' if snowflake && next == Some('/') => {
state = State::LineComment;
i += 1;
}
'$' if driver == DbDriver::Postgres || snowflake => {
'$' if driver == DbDriver::Postgres => {
// `$tag$ … $tag$` (tag may be empty); `$1` parameters are not quotes.
let rest = &text[byte_at(i)..];
let tag_len = rest[1..].find('$').filter(|&n| {
@@ -183,31 +151,6 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
continue;
}
}
// An unquoted body: `… AS BEGIN … END`, `… AS DECLARE … BEGIN … END`, an alert's
// `THEN BEGIN … END`. A column named `begin` is followed by `,`, `)`, `;` or FROM.
_ if defines && !script && depth == 0 && scripting_body(&chars, i) => {
script = true;
let body = skip_ws(&chars, i + if word_at(&chars, i, "AS") { 2 } else { 4 });
declarations = word_at(&chars, body, "DECLARE");
i = body - 1;
}
_ if script && word_at(&chars, i, "BEGIN") => {
depth += 1;
declarations = false;
}
_ if script && word_at(&chars, i, "CASE") => depth += 1,
_ if script && word_at(&chars, i, "END") => {
// `END IF` / `FOR` / `WHILE` / `LOOP` / `REPEAT` close statements that opened no
// block; `END` and `END CASE` close what BEGIN and CASE opened.
let after = skip_ws(&chars, i + 3);
let closing = ["IF", "FOR", "WHILE", "LOOP", "REPEAT", "CASE"]
.into_iter()
.find(|w| word_at(&chars, after, w));
if matches!(closing, None | Some("CASE")) && depth > 0 {
depth -= 1;
}
i = closing.map_or(i + 3, |w| after + w.len()) - 1;
}
_ if plsql && word_at(&chars, i, "BEGIN") => {
depth += 1;
declarations = false;
@@ -240,7 +183,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
_ => {}
},
State::Single => {
if (mysql || clickhouse || snowflake) && c == '\\' {
if (mysql || clickhouse) && c == '\\' {
i += 1;
} else if c == '\'' {
if next == Some('\'') {
@@ -338,20 +281,6 @@ fn word_at(chars: &[(usize, char)], i: usize, word: &str) -> bool {
&& chars.get(i + word.len()).is_none_or(|&(_, c)| !c.is_alphanumeric() && c != '_' && c != '$')
}
/// Whether `AS` or `THEN` at `i` is followed by a Snowflake Scripting block.
fn scripting_body(chars: &[(usize, char)], i: usize) -> bool {
let Some(word) = ["AS", "THEN"].into_iter().find(|w| word_at(chars, i, w)) else { return false };
let body = skip_ws(chars, i + word.len());
if word_at(chars, body, "DECLARE") {
return true;
}
if !word_at(chars, body, "BEGIN") {
return false;
}
let after = skip_ws(chars, body + 5);
chars.get(after).is_some_and(|&(_, c)| !matches!(c, ',' | ')' | ';')) && !word_at(chars, after, "FROM")
}
/// Whether only whitespace follows `i` until the end of its line.
fn ends_line(chars: &[(usize, char)], i: usize) -> bool {
chars[i + 1..].iter().take_while(|&&(_, c)| c != '\n').all(|&(_, c)| c.is_whitespace())
@@ -538,68 +467,4 @@ mod tests {
vec!["create view v as select 1 x from dual", "create type t as object (a number)", "select 4"]
);
}
#[test]
fn snowflake_bodies_comments_and_escapes_keep_statements_intact() {
let procedure = "create procedure p() returns string language javascript as $$ var a = 1; return 'x;y'; $$";
assert_eq!(
texts(
&format!("{procedure};\nselect 2; // done; really\nselect $1, 'it\\'s;here' from @stage;"),
DbDriver::Snowflake
),
vec![procedure, "select 2", "// done; really\nselect $1, 'it\\'s;here' from @stage"]
);
assert_eq!(texts("select 1 / 2; select 3", DbDriver::Snowflake), vec!["select 1 / 2", "select 3"]);
}
#[test]
fn snowflake_unquoted_bodies_and_urls_keep_statements_intact() {
let task = "create task t warehouse = w as\nbegin\n insert into a values (1);\n delete from b;\nend";
assert_eq!(texts(&format!("{task};\nselect 2;"), DbDriver::Snowflake), vec![task, "select 2"]);
let procedure = "create procedure p() returns number language sql as\ndeclare\n n number default 1;\nbegin\n return n;\nend";
assert_eq!(texts(&format!("{procedure};\nselect 2"), DbDriver::Snowflake), vec![procedure, "select 2"]);
let alert =
"create alert a schedule = '1 minute' if (exists (select 1)) then begin insert into h values (1); end";
assert_eq!(texts(&format!("{alert}; select 2"), DbDriver::Snowflake), vec![alert, "select 2"]);
// A column or alias named `begin` opens nothing.
for plain in [
"create view v as select 1 as begin, 2 as n",
"create view v as select a as begin from t",
"with x as (select 1 as begin) select * from x",
] {
assert_eq!(texts(&format!("{plain}; select 2"), DbDriver::Snowflake), vec![plain, "select 2"]);
}
assert_eq!(
texts(
"put file:///tmp/a.csv @s; get @s/a.csv file:///tmp/; select 1 // note; here\n; select 2",
DbDriver::Snowflake
),
vec!["put file:///tmp/a.csv @s", "get @s/a.csv file:///tmp/", "select 1 // note; here", "select 2"]
);
}
#[test]
fn snowflake_scripting_blocks_are_one_statement_and_transactions_are_not() {
let block = "DECLARE\n total NUMBER DEFAULT 0;\n c CURSOR FOR SELECT n FROM t;\nBEGIN\n FOR r IN c DO\n \
IF (r.n > 1) THEN total := total + r.n; END IF;\n END FOR;\n \
LET label := CASE WHEN total > 9 THEN 'big' ELSE 'small' END;\n \
BEGIN\n INSERT INTO log VALUES (:label);\n EXCEPTION WHEN OTHER THEN RETURN 'failed';\n END;\n \
RETURN total;\nEND;";
assert_eq!(texts(&format!("{block}\nselect 2;"), DbDriver::Snowflake), vec![block, "select 2"]);
let plain = "BEGIN\n LET x := 1;\n WHILE (x < 3) DO x := x + 1; END WHILE;\n RETURN x;\nEND;";
assert_eq!(
texts(&format!("select 1;\n{plain}\nselect 3"), DbDriver::Snowflake),
vec!["select 1", plain, "select 3"]
);
for begin in ["begin", "BEGIN TRANSACTION", "begin work", "BEGIN NAME t1"] {
assert_eq!(
texts(&format!("{begin};\ninsert into t values (1);\ncommit;"), DbDriver::Snowflake),
vec![begin, "insert into t values (1)", "commit"]
);
}
assert_eq!(
texts("select case when a then 1 end, b from t; select 2", DbDriver::Snowflake),
vec!["select case when a then 1 end, b from t", "select 2"]
);
}
}
+7 -93
Переглянути файл
@@ -16,7 +16,7 @@ pub fn quote_ident(driver: DbDriver, name: &str) -> String {
pub fn quote_literal(driver: DbDriver, value: &str) -> String {
let mut s = value.replace('\'', "''");
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse | DbDriver::Snowflake) {
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse) {
s = s.replace('\\', "\\\\");
}
format!("'{s}'")
@@ -75,19 +75,12 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
DbDriver::Clickhouse => {
"SELECT name FROM system.databases WHERE lower(name) NOT IN ('system', 'information_schema') ORDER BY name"
}
// `information_schema.schemata` lists only owned schemas before PostgreSQL 10 (and on servers
// built on such a core): a read-only account would see none. Any schema it may use is listed.
DbDriver::Postgres if system => {
"SELECT nspname FROM pg_catalog.pg_namespace \
WHERE nspname NOT LIKE 'pg\\_toast%' AND nspname NOT LIKE 'pg\\_temp%' \
AND has_schema_privilege(nspname, 'USAGE') ORDER BY 1"
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
}
DbDriver::Mysql if system => "SELECT schema_name FROM information_schema.schemata ORDER BY 1",
DbDriver::SqlServer if system => "SELECT schema_name FROM information_schema.schemata ORDER BY 1",
DbDriver::Snowflake if system => "SELECT schema_name FROM information_schema.schemata ORDER BY 1",
DbDriver::Snowflake => {
"SELECT schema_name FROM information_schema.schemata WHERE schema_name <> 'INFORMATION_SCHEMA' ORDER BY 1"
}
DbDriver::Oracle if system => "SELECT username FROM all_users ORDER BY username",
DbDriver::Oracle => {
"SELECT username FROM all_users WHERE username NOT IN \
@@ -95,10 +88,9 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
'OUTLN', 'SYS', 'SYSTEM', 'XDB', 'XS$NULL') ORDER BY username"
}
DbDriver::Postgres => {
"SELECT nspname FROM pg_catalog.pg_namespace \
WHERE nspname NOT IN ('pg_catalog', 'information_schema') \
AND nspname NOT LIKE 'pg\\_toast%' AND nspname NOT LIKE 'pg\\_temp%' \
AND has_schema_privilege(nspname, 'USAGE') ORDER BY 1"
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT IN ('pg_catalog', 'information_schema') \
AND schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
}
DbDriver::Mysql => {
"SELECT schema_name FROM information_schema.schemata \
@@ -125,11 +117,6 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\' ORDER BY name",
quote_ident(driver, schema)
),
DbDriver::Snowflake => format!(
"SELECT table_name, CASE table_type WHEN 'VIEW' THEN 'view' WHEN 'MATERIALIZED VIEW' THEN 'materialized view' \
ELSE 'table' END FROM information_schema.tables WHERE table_schema = {} ORDER BY 1",
quote_literal(driver, schema)
),
DbDriver::Oracle => format!(
"SELECT table_name, 'table' FROM all_tables WHERE owner = {} \
UNION ALL SELECT view_name, 'view' FROM all_views WHERE owner = {} ORDER BY 1",
@@ -155,11 +142,6 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
}
}
/// A Snowflake column type with its size, from `information_schema.columns`.
const SNOWFLAKE_TYPE: &str = "CASE WHEN data_type = 'NUMBER' THEN 'NUMBER(' || numeric_precision || ',' || numeric_scale || ')' \
WHEN data_type = 'TEXT' THEN 'VARCHAR(' || character_maximum_length || ')' \
WHEN data_type LIKE 'TIME%' THEN data_type || '(' || datetime_precision || ')' ELSE data_type END";
/// Rows: `name, type, nullable (YES/NO or 0/1), default, is_pk (0/1)`.
pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
@@ -200,10 +182,6 @@ pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
FROM information_schema.columns c WHERE c.table_schema = {s} AND c.table_name = {t} \
ORDER BY c.ordinal_position"
),
DbDriver::Snowflake => format!(
"SELECT column_name, {SNOWFLAKE_TYPE}, is_nullable, column_default, 0 \
FROM information_schema.columns WHERE table_schema = {s} AND table_name = {t} ORDER BY ordinal_position"
),
DbDriver::Oracle => format!(
"SELECT c.column_name, c.data_type, DECODE(c.nullable, 'Y', 'YES', 'NO'), c.data_default, \
CASE WHEN k.column_name IS NOT NULL THEN 1 ELSE 0 END \
@@ -252,7 +230,6 @@ pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
WHERE OBJECT_SCHEMA_NAME(i.object_id) = {s} AND OBJECT_NAME(i.object_id) = {t} AND i.name IS NOT NULL \
ORDER BY i.name, ic.key_ordinal"
),
DbDriver::Snowflake => "SELECT '', 0, '' WHERE 1 = 0".to_owned(),
DbDriver::Oracle => format!(
"SELECT i.index_name, CASE WHEN i.uniqueness = 'UNIQUE' THEN 1 ELSE 0 END, ic.column_name \
FROM all_indexes i JOIN all_ind_columns ic \
@@ -319,15 +296,6 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
WHERE c.table_schema NOT IN ('sys', 'information_schema') \
ORDER BY c.table_schema, c.table_name, c.ordinal_position"
}
DbDriver::Snowflake => {
"SELECT c.table_schema, c.table_name, CASE t.table_type WHEN 'VIEW' THEN 'view' \
WHEN 'MATERIALIZED VIEW' THEN 'materialized view' ELSE 'table' END, c.column_name, c.data_type, \
c.is_nullable, 0, CASE WHEN c.column_default IS NOT NULL THEN 1 ELSE 0 END \
FROM information_schema.columns c JOIN information_schema.tables t \
ON t.table_schema = c.table_schema AND t.table_name = c.table_name \
WHERE c.table_schema <> 'INFORMATION_SCHEMA' \
ORDER BY c.table_schema, c.table_name, c.ordinal_position LIMIT 50000"
}
DbDriver::Oracle => {
"SELECT c.owner, c.table_name, CASE WHEN v.view_name IS NOT NULL THEN 'view' ELSE 'table' END, \
c.column_name, c.data_type, DECODE(c.nullable, 'Y', 'YES', 'NO'), \
@@ -352,7 +320,6 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
pub fn foreign_keys(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => "SELECT '', '', '', '', '', '', '' WHERE 0",
DbDriver::Snowflake => "SELECT '', '', '', '', '', '', '' WHERE 1 = 0",
DbDriver::Sqlite => {
"SELECT 'main', m.name, f.id, f.\"from\", 'main', f.\"table\", f.\"to\" \
FROM sqlite_master m JOIN pragma_foreign_key_list(m.name) f \
@@ -403,7 +370,7 @@ pub fn foreign_keys(driver: DbDriver) -> &'static str {
pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse | DbDriver::Snowflake => foreign_keys(driver).to_owned(),
DbDriver::Clickhouse => foreign_keys(driver).to_owned(),
DbDriver::Sqlite => format!(
"SELECT {s}, {t}, f.id, f.\"from\", {s}, f.\"table\", f.\"to\" FROM pragma_foreign_key_list({t}, {s}) f ORDER BY f.id, f.seq"
),
@@ -431,7 +398,6 @@ pub fn current_schema(driver: DbDriver) -> Option<&'static str> {
DbDriver::Mysql => Some("SELECT DATABASE()"),
DbDriver::SqlServer => Some("SELECT DB_NAME()"),
DbDriver::Oracle => Some("SELECT SYS_CONTEXT('USERENV', 'CURRENT_SCHEMA') FROM DUAL"),
DbDriver::Snowflake => Some("SELECT CURRENT_SCHEMA()"),
}
}
@@ -444,7 +410,6 @@ pub fn use_schema(driver: DbDriver, schema: &str) -> Option<String> {
DbDriver::Postgres => Some(format!("SET search_path TO {}", quote_ident(driver, schema))),
DbDriver::Mysql | DbDriver::SqlServer => Some(format!("USE {}", quote_ident(driver, schema))),
DbDriver::Oracle => Some(format!("ALTER SESSION SET CURRENT_SCHEMA = {}", quote_ident(driver, schema))),
DbDriver::Snowflake => Some(format!("USE SCHEMA {}", quote_ident(driver, schema))),
}
}
@@ -555,9 +520,6 @@ pub fn edit_statement(driver: DbDriver, schema: &str, table: &str, edit: &DbRowE
DbDriver::Mysql => format!("INSERT INTO {target} () VALUES ()"),
// Oracle has no DEFAULT VALUES; without columns there is nothing to insert.
DbDriver::Oracle => return Err("Oracle needs at least one column value to insert".into()),
DbDriver::Snowflake => {
return Err("Snowflake needs at least one column value to insert".into());
}
_ => format!("INSERT INTO {target} DEFAULT VALUES"),
});
}
@@ -626,7 +588,6 @@ pub fn triggers(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => "SELECT '', '', '' WHERE 0".to_owned(),
DbDriver::Snowflake => "SELECT '', '', '' WHERE 1 = 0".to_owned(),
DbDriver::Sqlite => format!(
"SELECT name, sql FROM {}.sqlite_master WHERE type = 'trigger' AND tbl_name = {t} ORDER BY name",
quote_ident(driver, schema)
@@ -695,12 +656,6 @@ pub fn routines(driver: DbDriver, schema: &str) -> Option<String> {
"SELECT r.routine_name, lower(r.routine_type), '', r.data_type \
FROM information_schema.routines r WHERE r.routine_schema = {s} ORDER BY 1"
)),
DbDriver::Snowflake => Some(format!(
"SELECT function_name, 'function', argument_signature, data_type FROM information_schema.functions \
WHERE function_schema = {s} UNION ALL \
SELECT procedure_name, 'procedure', argument_signature, data_type FROM information_schema.procedures \
WHERE procedure_schema = {s} ORDER BY 1, 3"
)),
DbDriver::Oracle => Some(format!(
"SELECT object_name, CASE object_type WHEN 'PROCEDURE' THEN 'procedure' ELSE 'function' END, '', NULL \
FROM all_objects WHERE owner = {s} AND object_type IN ('PROCEDURE', 'FUNCTION') \
@@ -742,10 +697,6 @@ pub fn sequences(driver: DbDriver, schema: &str) -> Option<String> {
"SELECT sequence_name, data_type FROM information_schema.sequences WHERE sequence_schema = {} ORDER BY 1",
quote_literal(driver, schema)
)),
DbDriver::Snowflake => Some(format!(
"SELECT sequence_name, data_type FROM information_schema.sequences WHERE sequence_schema = {} ORDER BY 1",
quote_literal(driver, schema)
)),
// Every sequence is numeric; ALL_SEQUENCES has no data_type column before 18c.
DbDriver::Oracle => Some(format!(
"SELECT sequence_name, 'NUMBER' FROM all_sequences WHERE sequence_owner = {} ORDER BY 1",
@@ -979,17 +930,6 @@ mod tests {
assert!(edit_statement(DbDriver::Postgres, "s", "t", &DbRowEdit::Delete { key: vec![] }).is_err());
}
#[test]
fn postgres_lists_the_schemas_the_user_may_use() {
for system in [false, true] {
let sql = schemas(DbDriver::Postgres, system);
assert!(sql.contains("FROM pg_catalog.pg_namespace"), "{sql}");
assert!(sql.contains("has_schema_privilege(nspname, 'USAGE')"), "{sql}");
assert!(sql.contains("NOT LIKE 'pg\\_toast%'"), "{sql}");
assert_eq!(sql.contains("NOT IN ('pg_catalog', 'information_schema')"), !system, "{sql}");
}
}
#[test]
fn sqlserver_quoting_and_schema() {
assert_eq!(quote_ident(DbDriver::SqlServer, "a]b"), "[a]]b]");
@@ -1146,30 +1086,4 @@ mod tests {
"create table \"public\".\"t\"\n(\n \"id\" integer not null,\n \"name\" character varying(40) default 'x'::character varying,\n constraint \"t_pkey\" PRIMARY KEY (id)\n);\n\nCREATE INDEX t_name ON public.t USING btree (name);\n"
);
}
#[test]
fn snowflake_quotes_names_and_reads_its_information_schema() {
let driver = DbDriver::Snowflake;
assert_eq!(quote_ident(driver, "My \"Table\""), "\"My \"\"Table\"\"\"");
assert_eq!(quote_literal(driver, "it's a \\ path"), "'it''s a \\\\ path'");
assert_eq!(qualified(driver, "PUBLIC", "ORDERS"), "\"PUBLIC\".\"ORDERS\"");
assert_eq!(current_schema(driver), Some("SELECT CURRENT_SCHEMA()"));
assert_eq!(use_schema(driver, "Sales"), Some("USE SCHEMA \"Sales\"".into()));
assert!(schemas(driver, true).contains("information_schema.schemata"));
assert!(schemas(driver, false).contains("<> 'INFORMATION_SCHEMA'"));
assert!(tables(driver, "PUBLIC").contains("table_schema = 'PUBLIC'"));
let columns = columns(driver, "PUBLIC", "ORDERS");
assert!(columns.contains("'NUMBER(' || numeric_precision"), "{columns}");
assert!(columns.contains("table_name = 'ORDERS' ORDER BY ordinal_position"), "{columns}");
assert!(completion_columns(driver).contains("LIMIT 50000"));
assert!(routines(driver, "PUBLIC").unwrap().contains("information_schema.procedures"));
assert!(sequences(driver, "PUBLIC").unwrap().contains("information_schema.sequences"));
assert_eq!(
table_page(driver, "PUBLIC", "ORDERS", None, &[], 50, 25),
"SELECT * FROM \"PUBLIC\".\"ORDERS\" LIMIT 26 OFFSET 50"
);
assert!(edit_statement(driver, "PUBLIC", "T", &DbRowEdit::Insert { values: vec![] }).is_err());
assert_eq!(crate::conn::begin_statement(driver), "BEGIN");
assert_eq!(crate::plan::explain_statement(driver, "select 1;"), "EXPLAIN USING TABULAR select 1");
}
}
-151
Переглянути файл
@@ -1,151 +0,0 @@
//! Data source files edited outside the Database tool window (by hand, by a coding
//! agent, by `git checkout`) are picked up while the project is open, as IDEA does
//! for `dataSources.xml`. The project file tree's watcher is no help here: it never
//! reports a modification inside a gitignored `.openidea/`.
use crate::source::FILE;
use ide_core::{IdeError, IdeResult, project_dir};
use notify::{RecommendedWatcher, RecursiveMode, Watcher};
use parking_lot::Mutex;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Weak, mpsc};
use std::time::Duration;
/// How long a burst of writes (an editor's save, an agent's rewrite) settles before
/// the files are read.
const SETTLE: Duration = Duration::from_millis(200);
/// Watches `<root>/.openidea/data-sources.toml` and `<global>/data-sources.toml`;
/// stops when dropped.
pub struct SourcesWatch {
_watcher: Arc<Mutex<RecommendedWatcher>>,
}
impl SourcesWatch {
/// Calls `on_change` once a burst of changes to either file settled, on a
/// background thread.
pub fn start(root: &Path, global: Option<&Path>, on_change: Box<dyn Fn() + Send>) -> IdeResult<Self> {
let config: PathBuf = root.join(project_dir::DIR);
let (tx, rx) = mpsc::channel::<()>();
let relevant = {
let config = config.clone();
move |p: &Path| p.file_name().is_some_and(|n| n == FILE) || p == config
};
let watcher = notify::recommended_watcher(move |res: notify::Result<notify::Event>| {
if let Ok(e) = res
&& e.paths.iter().any(|p| relevant(p))
{
let _ = tx.send(());
}
})
.map_err(|e| IdeError::internal(format!("data sources watcher: {e}")))?;
let watcher = Arc::new(Mutex::new(watcher));
{
let mut w = watcher.lock();
// The root (not recursively) sees `.openidea/` itself appear or go.
w.watch(root, RecursiveMode::NonRecursive)
.map_err(|e| IdeError::internal(format!("data sources watcher: {e}")))?;
if let Some(dir) = global.filter(|d| d.is_dir())
&& let Err(e) = w.watch(dir, RecursiveMode::NonRecursive)
{
tracing::warn!("global data sources watcher: {e}");
}
}
let mut watching = watch_config(&watcher, &config, false);
let weak: Weak<Mutex<RecommendedWatcher>> = Arc::downgrade(&watcher);
std::thread::Builder::new().name("openidea-db-sources-watch".into()).spawn(move || {
while rx.recv().is_ok() {
// Wait until the writes stop.
loop {
match rx.recv_timeout(SETTLE) {
Ok(()) => {}
Err(mpsc::RecvTimeoutError::Timeout) => break,
Err(mpsc::RecvTimeoutError::Disconnected) => return,
}
}
let Some(watcher) = weak.upgrade() else { return };
watching = watch_config(&watcher, &config, watching);
drop(watcher);
on_change();
}
})?;
Ok(Self { _watcher: watcher })
}
}
/// Watches `.openidea/` once it exists (again: a removed folder loses its watch);
/// returns whether it is watched.
fn watch_config(watcher: &Mutex<RecommendedWatcher>, config: &Path, watching: bool) -> bool {
if !config.is_dir() {
return false;
}
if watching {
return true;
}
match watcher.lock().watch(config, RecursiveMode::NonRecursive) {
Ok(()) => true,
Err(e) => {
tracing::warn!("data sources watcher {}: {e}", config.display());
false
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Instant;
fn wait_for(count: &AtomicUsize, at_least: usize) -> bool {
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline {
if count.load(Ordering::SeqCst) >= at_least {
return true;
}
std::thread::sleep(Duration::from_millis(20));
}
false
}
fn counting() -> (Arc<AtomicUsize>, Box<dyn Fn() + Send>) {
let count = Arc::new(AtomicUsize::new(0));
let c = count.clone();
(count, Box::new(move || _ = c.fetch_add(1, Ordering::SeqCst)))
}
#[test]
fn reports_project_and_global_files_and_a_config_folder_created_later() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let (project, global) = (root.join("p"), root.join("app"));
std::fs::create_dir_all(&project).unwrap();
std::fs::create_dir_all(&global).unwrap();
let (count, on_change) = counting();
let _watch = SourcesWatch::start(&project, Some(&global), on_change).unwrap();
// No `.openidea/` yet: it is created, then the file written into it.
std::fs::create_dir_all(project.join(".openidea")).unwrap();
assert!(wait_for(&count, 1));
let seen = count.load(Ordering::SeqCst);
std::fs::write(project.join(".openidea/data-sources.toml"), "").unwrap();
assert!(wait_for(&count, seen + 1), "project file");
let seen = count.load(Ordering::SeqCst);
std::fs::write(global.join("data-sources.toml"), "").unwrap();
assert!(wait_for(&count, seen + 1), "global file");
}
#[test]
fn ignores_other_files() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
std::fs::create_dir_all(root.join(".openidea")).unwrap();
let (count, on_change) = counting();
let _watch = SourcesWatch::start(&root, None, on_change).unwrap();
std::fs::write(root.join(".openidea/settings.toml"), "").unwrap();
std::fs::write(root.join("main.py"), "").unwrap();
std::thread::sleep(SETTLE * 3);
assert_eq!(count.load(Ordering::SeqCst), 0);
}
}
-24
Переглянути файл
@@ -1,24 +0,0 @@
Throwaway keys for the Snowflake key-pair tests. They were generated for this repository,
are registered with no account, and protect nothing.
| File | Format |
|---|---|
| `rsa_key.p8` | PKCS#8, unencrypted, RSA 2048 |
| `rsa_key_encrypted.p8` | PKCS#8, PBES2 (PBKDF2 + AES-256-CBC), passphrase `test-passphrase` |
| `rsa_key_scrypt.p8` | PKCS#8, PBES2 (scrypt + AES-256-CBC), passphrase `test-passphrase` |
| `rsa_key_pkcs1.pem` | PKCS#1 (`BEGIN RSA PRIVATE KEY`), unencrypted |
| `rsa_key_legacy_encrypted.pem` | PKCS#1 with `Proc-Type: 4,ENCRYPTED` (rejected) |
| `rsa_key_1024.p8` | RSA 1024 (rejected: too short) |
| `ec_key.p8` | EC P-256 (rejected: not RSA) |
| `rsa_key.pub` | Public key of `rsa_key.p8` |
Fingerprint of `rsa_key.pub` from OpenSSL
(`openssl rsa -pubin -in rsa_key.pub -outform DER | openssl dgst -sha256 -binary | openssl enc -base64`):
`ajjL6Bf6l1FIIyTcJli4i2AWPYDVJGQnSZr+9q9ClgI=`
`commands.json` holds one sample statement for every command of Snowflake's SQL reference
(<https://docs.snowflake.com/en/sql-reference/sql-all>, 786 names). The driver's tests
(`src/snowflake/commands.rs`) and the console's (`src/features/database/sql/snowflakeCommands.test.ts`)
run all of them; the ignored live test runs the listing ones against a real account. `reads`: allowed
on a read-only data source. `rerunnable`: safe to run again for another page of the result. The
samples exercise the client, not Snowflake's grammar, and name no real object.
File diff suppressed because it is too large. Load diff
-5
Переглянути файл
@@ -1,5 +0,0 @@
-----BEGIN PRIVATE KEY-----
MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgwYBQrdPGgoV9c58S
q6Fc5aso2M4wVyXE9WxrpPzcrHahRANCAAR0SjYZ1Pvc5jG2ll+pTSdBiVRccnSB
BirgCQL5f1g4CuEK4YNqTIvXMqJ2bBVx6f2MTbLeCmu8EebiJ99dI/ZM
-----END PRIVATE KEY-----
-28
Переглянути файл
@@ -1,28 +0,0 @@
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDPfjbU88uYwtpN
hthRdv7h1cJJ1p2nnb8pKLMi/SdPSfSZLm0eA2Tbjp6HtGqv91enrCPyy++hQOl0
KtfoPGbzDqxnzZPK4BtckmxHiBQifG7A0d3jpq8Y2JjIkkMItnF7kXQob89TOaFj
+pW8/XH6Y+w5cbMYdkf3RXKjZpYRLRBxnzAl4lMhP/boE7reoWuBIwPFVXnX8AVP
p0bjkpR2I1fgUKZxMw/mbRhWVy/9b7Vh79RwPG85rvCia2e7OdDLxNYUcEHocvfb
8phUHtI7ZvnCJcbXJX7ON9/JdU/DN9RHc+az1ZjB4Za1cHoLXlbnUisSf1e3tnaY
uZnGqGLPAgMBAAECggEAQ+8N4sEDSruZH8LjdiCgqK4E04oJygQyvzYNGIqL4UDU
bYauwSHZVq0mKEYgnAY+U65fgrHUfhYzmQ4h7G5CQpfuSHodRlIDg1aJTw41kWju
FgFgRZWgRGE9/IUun16kXxGrV0XsOlJnDcXQWrUcKR+oCtK4EW1AeKvrMaZ3wZ1J
i0xqMlU5bk77kvskBRL4k51V3gOfDoflus2nHpES8yZSAP1uiivbtTOXYB2dRHuD
omB4KlscUdIwT9vTMC7kxECEKDwd5fxJkdIJ/NsBi98tOZS0QrH7y7FrBA3K7gLY
VJSIckKP8rz3kUAbW9vYV2dRwxIMQj2kZJEKa0kA6QKBgQDsNsJPJdt5BcAPYjV0
1LX91ooLCDYux5yUNEno2rmQ/qv5g0d12L94aASklGIIfJV75dXmwa8q4erZU0hs
9/XhyUnjDlBLd6oEztDjDA49J57g2nPmzrOtjugucH7Qe8EAJzz+O0PbtkcQzRmV
8vnYptWHvEKmnjh8KOb5wOMVFQKBgQDg35SYkiys8V3tQApbp6U7Q3fq3vbVu+2A
avBYIE0qgG9fhFrQdihVX1MZ3L4TtXtiikMPO8MOGBjSgN/qbelChhkQV0l+CIf0
nOo1SeoEXp16q1NcZTEohnbaboMjWrkl6Br6SUNvEt7Y63UOi62E+ll50ye8wKgA
yBHrGXOZUwKBgQDoTaIivJRvHNyZRHhWAatJjGgOGXFP6Ddw+UuGmTTOB+eRZy4s
E5GaFJIcWWFf/0Xzp0EJw4lFnxCb0m/z2pgSCSeFCVqmHXUzbUG5SO3n7sMtVymM
39OM6HIm2bOdzVctk2lEjXxa45RYTZzP/zj/fHObHeZeflQGpyMglDW/UQKBgQDE
wRuFOzC3ySCDu6rUcyTRa8D60kdqz1mJKQhOM0URUPshue6zww6bVCKj2E6/c+UX
XR47r/CE3nSpinVg6vZTbH11nMoC6nf6gWkzYw7M7/QeJtVzXF7ctHo9uMApKPIi
xqkEISJNh1dQuQPczD3wwhOT0jtMnczhkiyLZeONXQKBgE4wo4IkUhH2SQRiu8Je
sg18oa1sABaL6bCQ7vqJ2+cZzj5Pz0WxMoyc7ZJlIiKt2joGQwqmRAqVohHpJylm
9RKvymaM0BGVKkAASfwyEf15YvZQx/9IK3uWpRoGrlu7jyu0UlzCgpHXfOu1FKCx
DsVT35WOOVJXuDrNhpYAdg5U
-----END PRIVATE KEY-----
-9
Переглянути файл
@@ -1,9 +0,0 @@
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAz3421PPLmMLaTYbYUXb+
4dXCSdadp52/KSizIv0nT0n0mS5tHgNk246eh7Rqr/dXp6wj8svvoUDpdCrX6Dxm
8w6sZ82TyuAbXJJsR4gUInxuwNHd46avGNiYyJJDCLZxe5F0KG/PUzmhY/qVvP1x
+mPsOXGzGHZH90Vyo2aWES0QcZ8wJeJTIT/26BO63qFrgSMDxVV51/AFT6dG45KU
diNX4FCmcTMP5m0YVlcv/W+1Ye/UcDxvOa7womtnuznQy8TWFHBB6HL32/KYVB7S
O2b5wiXG1yV+zjffyXVPwzfUR3Pms9WYweGWtXB6C15W51IrEn9Xt7Z2mLmZxqhi
zwIDAQAB
-----END PUBLIC KEY-----
-16
Переглянути файл
@@ -1,16 +0,0 @@
-----BEGIN PRIVATE KEY-----
MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBANqRvj6Xi55mKH0x
96UVh/4tMgBb+5TVT5MN39diDr6iP7vFycEnnHDVXaeQJBog+P7NiYC7XY+7yUL3
2J3VxMBzGirm5Sj12uU1C+U4f0Nxx4nVKqonQD8GHJGJxypSZvIculFV9CYSHfoc
JYHR95k1Ce7echs7VnXfjvaOBdaLAgMBAAECgYBYecC6E5xn7VCchTqK7/FR2bew
34syjzqb+3TN+hqf3ovQFdt6ve8c2bIEn1uN3CUO9+KeYroX8dshqbRdB+iRgx8i
EzHAmKLkQj/QF7j1bTLD3zMq9XmUAOaUQwtlzRKSJIOa6EoYlGTcsefZhLhctegE
dOfNbBM6En2el2O3MQJBAPX3BX2hkVU42nRYkmqoqz2DEFrcd7T3WpyxT8U42d4z
rn8iJJAYwpg5SPf23PNrT+0IGGcI2LFWji9ENXM7H5kCQQDjfJahG/PvElaDGGlh
JGjDn5FwR2bs8rII/gxhBXZJSyettkm0PIPnijlcQv3L6lRHi+7LXLFUbV8C6DWn
sg3DAkAGnd2M8qK5Oy1NQfC2Y9Evd24iM5oYySqr2m097fiWbKd+DgoTwkbB15Cp
8v8goTtmMfiOFQPXSW/Lv7b8YZPJAkEA0aC1jbEckGntntCYCxRqFhqtAzJI/oCG
3xJ8hctqvQ+seL9KG/TH+E2/ne19wrMlBulEG9JG9u3yZ9Nj3iyN0QJAM6ROygXj
2ReQpmst38WnAaQvT+0Nd8cWoliGz24Vx1cAmlHI4NrCQsODNuR+gJ4vCbfq8Sa/
qJQfaDmu34JhJw==
-----END PRIVATE KEY-----
-30
Переглянути файл
@@ -1,30 +0,0 @@
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFNTBfBgkqhkiG9w0BBQ0wUjAxBgkqhkiG9w0BBQwwJAQQrUgr104rqBLzsVP2
13KRWQICCAAwDAYIKoZIhvcNAgkFADAdBglghkgBZQMEASoEEKZLvVKRzfkt0lie
Hvjy+vYEggTQ05I/45BV/X9VcWZ0W4OC8JdtYC0nosK3ymyl4iQGo4sOSLkJGpzu
0LhexFRDICNLqHkcs2XQ4Jti8lZyArY3Wql+vl9Z/FVl9s4zJIg8VDXqrPSJ6Ha+
wvVhY59UreqlQeLoFdhCK8O9/QeJow/ZQnVshJCLLQ4jiwYUHpuTrRPInZp/Jt5X
KD7N4oLsXg7NrBDyQu/DxRiPbtOu07NqMFHel4mRXCz0oKhG0pQ0KuqP0arA9RGN
CpT2SuhA3xMl6DMqQhCsGyUVhJhD7FyH/2fxlWHH2WVGWbza3L6l293fpRP2d0Nh
VET2Ia93brNpYsXT1BVAvnXG+yddbCSgTjIh/YU+ciKcaaRsZXSpB3ON2yysBXo1
XRiXbqfzx22ah9p1sT3ed3QRZ5PPOAYLNqEvUz/eNDBBiwOAdSuS4JAYYS0VyJdG
v93pWEf6fuFXdzJsOcmXy86t6JMfgV45zuiUFBD7+coNh8Yn1O+SMSTe3kKjgcaw
j7xMVDlNy6okpDv5mIFTkzhLCPZPNhGTjYBs21edEmB7PyGf9l8d+Mv+ItT9Ualy
5MUJ4ZzOPIT+Umv97DUB8CIvRSuLq34K1puvjoPa1oz6etUkMVxOEG40+h+b3JNo
RiwONkO/0Mvew8uUywnwrdS08I8Y2NeFwQsYf90371xKqSqtwLKS9CLI225WSOXQ
CTeEItkmJRa+RQApLZIvhJZQ0Hreb89W0wjsz4VStsS0laLekfh1UCZnvVZRNxCn
FM0aTTLL5TJAFMiEYDzfrADgbgm56DCMW80ByBpHiqwmysXnAX20IhDcnP6A4iDJ
ukW0qlojrAEh79Y7nCHC8y4CCsEwwUxuxXfSRrFVQeS6K2crGoalnJfXFMhHr8SW
YBD9J4x6tVhKB++rmy3LkX7lHzVLlZbfd8lJouNxTlj9BWwzm3k96v9V6KO/vDLn
Iqh2qENSNAWbRrFoCn1d6XPYt4L9kFRUrKClGVbgMyVpKNX0izVb2TElViRGiA2+
Tq2gXNGuCNcz2Y5EURaJozjCb4udRazXTraZJ4YZgfwdrHzTz+neBSEsiXU7OGrY
R+Q5ijkahA7I0BUUabPEVmxup7iOkJmagjPu8ve75c+VTqs3QHMFjEkYLazxB7a3
N6LtM9fp9X5RtQQBM+ppGjMtQD6Jkp0/npUUCbfPNnH0ZXxiNh6GaYuTY58C9ob8
zBBgfiBH2Olzrd/bZv695Ohxtd97eEqMosixKJLcYsvSqeJlmx8XmcNoWesVmooT
gR4QB1x22UdcZa0ShQLSuDTDEGiU33uJF9VM1CAHWQNEHHfVMUpdCV/m5AkyPm/1
rp4IRG2oJkpId4PHu8aGD+/4T8mnALbSzooButfj9m1u9Mo1156hi+F9Z2jN63aA
Ehnzc0aRAJ5wbnq3RzaB7sfP0+/fh6e97gW+aNeLP5l6wYzDVqwLdIpt8o7s5rvq
FFFqChnzTRUXAkRMEn7WqMU1LeAF9S+rMTcawuolADVd2ntNWMmA6bqrpgq2C/0w
qk3npanYBmHg7KzJMg8jJNIPhRNV0PyMM9RYdwfDiuMW9Gw+pi3DgUOO027Vr7j1
VNlZgdaA9uYT2qETf4/N1tyqKxsrimdUaG9dQe3w2R9SEMXw28doihc=
-----END ENCRYPTED PRIVATE KEY-----
-30
Переглянути файл
@@ -1,30 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-256-CBC,A5A125F46CCED791DF3B43E3BFFC72EF
j/qWilDpPFE84GAHLgUM4m+yXGbgjePy/G9GB8jV3doCyDRJhjvqKfJmXmypj5IM
tz2Ab059Gz396FK2N0OEtj4Us9TgPtvW/fTPNoLQp9mcIrRB7+s3TFoxNn9sUxuv
fnEcYWmrrTElin7DbWn1RrilCkf3jiTINysFt2ypUwCqpDKebQ2WRXdUCjIevul4
8hmNoS8YCbr++HkcRqnDTYPrNU1sV4Fl8cXs4BnEBlWULxlA6Y6xBsDur58X/UO3
Up+WLZ8ptCWZMNYlOGNnS3TUzMn6sWWVJaShChvJBaqwTfNoFThzD8bBxf1PXN+7
PkdkDcX5oHt+p0q1EtteFZ55VwWWMDdQTfG7dYAqTm+K4YCVaRh2xm4QTke5Kbol
0B9C3/RaR9zLA4XEf2mq+5yc3PPBcKq+aWX7pGL+9kxMXz2HsiMWb9ir+FNxb2nc
4mG5P/bmersRXhlmUl3bGXx3m5+4nnfBzO88S1MgZo4Z2L4TzJEaF7KjQENicDlA
UVMaVvq9+xUPPtIT8o/VTc5x/drO0HayhMfkz8Yxka2DLsMnLhU9NiDO6nKREEjL
GQvsan0/gIDweXzRFUY5AhPQHAOpjf/BnmTa80/DvW+WxA/B2EMMABbjxGcKK1aP
FyJf0oVvNuC91RPNhCJ2jiGKlLxnWspcEG1hiFZJvQ4Z8u1TRItqIadE0Ar0z54A
qJIr56SoU+xPyfXEt2GWwTPRu9W6l7EKo1yVKvqSdz1UGzrW3WuHI3G/UYvkqAs9
fBQK47sYlrpLYS3qnFbYlrbR0RCHlNLVCs0V1Xb6U+qU3ExQRlo1YvkvH7dkvhqn
Bdqv9iYnlryVP5SJcAeKZ7VqmM/LXq+bF7g/hTcHkg1yISqC1qaYDNi5D7uOVwW6
ALiD5zFyLArnG7girpuT1hc8N+dkh1ZckOGf2nW3I27czY4Xsaz+9keXFvAy5F2K
OGSu5iu/8p7M1bPRdiutcEVrT07FFvVZufASo4brMXptxpozd9o14uMhudneIqtb
qgw+F4nl6LZkTmbdn2suXjG1jNxlv86bKsLuY/IoaOO70YTho2HS1zpQ7SS1Jj63
5v4y6205m9xMJVZywbCwo7FQhxfBT3cmS2KkD9mYSZiFtYkRDLPxneNHKyegnuRy
+rYelwbKCiFoUXg7QMOYfYDfC2UckCblcWlYgtSkL6TRnzl0aKBO8SQiph8LMvtB
+/Om76T4OGlkk4PJsbLjt6uSEgcAwJBjG2ox0FpE75tLIO6ZBWdgn6TnBLFiIniw
apomJZNc3bybpiuH+w9F1gOy/1c+nS6O82559Zn03ez6tX2SHqO7ROX06zLdcdVO
Yg0PJiBaAPNgfKeSb/K4XX4L/bJi9lH6VjUBcQqDqHKSAonmZDLGrWxHyuHbBE1u
wHXigKzIxR0nw8VUPIFDpdgnNlKe9eputx0+7rgrh6sW1rZL3EfuyX/62SLG8Y4L
YYrIx1AcrkKct1yPUXgwUKYn1TItPhK3JSXxmCYbkEuVdZTKKtf/irqoauEIPjm2
ufvhM0Y+PIGGOwAh7EvuDUUemeHd4Q2HnHwnROx5L1rSf2izC1GoxCtP8/UE2OTK
-----END RSA PRIVATE KEY-----
-27
Переглянути файл
@@ -1,27 +0,0 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAz3421PPLmMLaTYbYUXb+4dXCSdadp52/KSizIv0nT0n0mS5t
HgNk246eh7Rqr/dXp6wj8svvoUDpdCrX6Dxm8w6sZ82TyuAbXJJsR4gUInxuwNHd
46avGNiYyJJDCLZxe5F0KG/PUzmhY/qVvP1x+mPsOXGzGHZH90Vyo2aWES0QcZ8w
JeJTIT/26BO63qFrgSMDxVV51/AFT6dG45KUdiNX4FCmcTMP5m0YVlcv/W+1Ye/U
cDxvOa7womtnuznQy8TWFHBB6HL32/KYVB7SO2b5wiXG1yV+zjffyXVPwzfUR3Pm
s9WYweGWtXB6C15W51IrEn9Xt7Z2mLmZxqhizwIDAQABAoIBAEPvDeLBA0q7mR/C
43YgoKiuBNOKCcoEMr82DRiKi+FA1G2GrsEh2VatJihGIJwGPlOuX4Kx1H4WM5kO
IexuQkKX7kh6HUZSA4NWiU8ONZFo7hYBYEWVoERhPfyFLp9epF8Rq1dF7DpSZw3F
0Fq1HCkfqArSuBFtQHir6zGmd8GdSYtMajJVOW5O+5L7JAUS+JOdVd4Dnw6H5brN
px6REvMmUgD9boor27Uzl2AdnUR7g6JgeCpbHFHSME/b0zAu5MRAhCg8HeX8SZHS
CfzbAYvfLTmUtEKx+8uxawQNyu4C2FSUiHJCj/K895FAG1vb2FdnUcMSDEI9pGSR
CmtJAOkCgYEA7DbCTyXbeQXAD2I1dNS1/daKCwg2LseclDRJ6Nq5kP6r+YNHddi/
eGgEpJRiCHyVe+XV5sGvKuHq2VNIbPf14clJ4w5QS3eqBM7Q4wwOPSee4Npz5s6z
rY7oLnB+0HvBACc8/jtD27ZHEM0ZlfL52KbVh7xCpp44fCjm+cDjFRUCgYEA4N+U
mJIsrPFd7UAKW6elO0N36t721bvtgGrwWCBNKoBvX4Ra0HYoVV9TGdy+E7V7YopD
DzvDDhgY0oDf6m3pQoYZEFdJfgiH9JzqNUnqBF6deqtTXGUxKIZ22m6DI1q5Jega
+klDbxLe2Ot1DouthPpZedMnvMCoAMgR6xlzmVMCgYEA6E2iIryUbxzcmUR4VgGr
SYxoDhlxT+g3cPlLhpk0zgfnkWcuLBORmhSSHFlhX/9F86dBCcOJRZ8Qm9Jv89qY
EgknhQlaph11M21BuUjt5+7DLVcpjN/TjOhyJtmznc1XLZNpRI18WuOUWE2cz/84
/3xzmx3mXn5UBqcjIJQ1v1ECgYEAxMEbhTswt8kgg7uq1HMk0WvA+tJHas9ZiSkI
TjNFEVD7Ibnus8MOm1Qio9hOv3PlF10eO6/whN50qYp1YOr2U2x9dZzKAup3+oFp
M2MOzO/0HibVc1xe3LR6PbjAKSjyIsapBCEiTYdXULkD3Mw98MITk9I7TJ3M4ZIs
i2XjjV0CgYBOMKOCJFIR9kkEYrvCXrINfKGtbAAWi+mwkO76idvnGc4+T89FsTKM
nO2SZSIirdo6BkMKpkQKlaIR6ScpZvUSr8pmjNARlSpAAEn8MhH9eWL2UMf/SCt7
lqUaBq5bu48rtFJcwoKR13zrtRSgsQ7FU9+VjjlSV7g6zYaWAHYOVA==
-----END RSA PRIVATE KEY-----
-30
Переглянути файл
@@ -1,30 +0,0 @@
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkrBgEEAdpHBAswHAQQRIkBIQM391D42515
x8qX7AICQAACAQgCAQEwHQYJYIZIAWUDBAEqBBAV4ik0WbzhBWcA0q2nB8/oBIIE
0KHh9w0Hi98G2eAdpBRCpAC2FryLKvfPcD6puChTFFwSIryfPt9dSrLJHB83c6/+
uYfaQskgdoA1aCx97A+lMAXFemURUBnI59+2eqdkFdMPsWTkRQC0iL0XiRzuwUA7
SbhSopssc21ZtiVvlxpDWaFupOqwu8mb79Lp17Rriirc/dy/e2ODStFzQyCvBJmh
rSbWwFBsZc0zB4Ptp4Ekeah5DEbxKXG5Lw56DetDiLukXlVeVJg+Gw2ZXHlipj4s
/wqbrnbX3VQCWuBJot3F3wYqhGZO7e52Ge+jiojfU1zT4NwMgIAjIv7jIxcxHINj
c+Iuql1Vq3QSM+sU7MYDMxsfgsEEzYgZDtcBMqJTYy0Dq4MixkSgc8zYI+V2axl9
RKkBQAOQQcgCN+hrPgV8JKmeNP8OBsIX3VmXgTBhRt6xpTgW2YtV0Xc4HbTY9YoF
tm1NI6gtcOlBPewJWI6GD8JueqY88lmrG/Qf2g4x7cQz3TCTlwpB6DXHX/blRSVe
kGJVu0CWTRy9yM49xnnZGOrK8UjL61Nd6on8vb9HC+wFsDFh2kJ1/8RvCWLyqPJY
xM8PVmPOwKhigEF2QN/bt1eESm3D1OTeV05TnBDgRPcDwVm/ym80D7tS4Vwj7xgf
gWzVQNoI1wtXMaJalmxSQ5qDdm78T/Mj3EJIDLoBtnkGJGQFkDxpZ4y5jQSHAAc6
7kXnM7SAI4LUvME5LcwJZlHcUnZ7FHqCdJWh+FrMUpsL807Gcd7KMcsx++lQAAVR
B/OUVykJHD4JnZtt3W5yyTA8E8nWbiDHro9n1w6n/fdSTj1AnwWncEx4o0CrSulf
eZ/78eE2ba4cOyOnE1VuMmvqfH98xcXwk1mME59HWbxxuryUgqoJX40dVSay0tTt
Ccfn0oi0WN+P8Qoq/7AueyeBbzN2dTEyDTj/C7dFFeerJT/gV3bfke3DL+J3E4zq
+U5qrI0Z+FXw3cSF66pCY+uWruSllkwGXLjTYWSXJ7oXaANu56zAjhjGgdzZ3o9F
Ub2JuvIuAZ+cYCJnSDPU1r4hUvMoJcfBu7YqkFQXztLU0kvdWR3fmOCapKFHaKXh
y0KVUzi2tkTwj22+hYOTuHTLtleiFmtJERWeCkmn2eyMhSJsbxanlSRddVsMs5mJ
vMpnXh1AlDbKHRqQIxOUhf01Dw1Q4VpQfMuxPcN4GQxaj2N0Kn6OmvxF42ze8OuA
vO1orsV8VRDApFAVmhsfjet0hXo/EnWL+oJ5WadY5hfYX1cXe8BdjiDRyCCDv9ob
lZUHadw2sbr1FkJ3TsxnX0OkOQ+07pDq03EjyScHyNyCForw8F0a867F23TBKunq
q0XuwGG+EgkVEHvy/Ksqrfc60jbDyu3kAusmnLMQv07OHUiiqlBbtLQO31FQH7QE
KIm6BxiZ8A2DxdsbyWO1LZ9+SHttUA3OAAKjFeOO89A7iK+8asRTjIRSt6NtDI6V
rbbWFKrZY9ACfXWoFuCrfIjipPrc8q293rJ8mVxZf34rvEjt0WuV17VDEbozt8XZ
4F2mnYeVF8LX0sdy+OBVfpTGpmn8Ubsxond4OQyzxI3MbT4WOey2BdznmdROEYbJ
RmVzKCBEWVuHptyL6LrWbO3pu+Kdkh/1mK/E+UhFyrjk
-----END ENCRYPTED PRIVATE KEY-----
-365
Переглянути файл
@@ -1,365 +0,0 @@
//! Against a real Snowflake account, with key-pair authentication:
//!
//! ```sh
//! OPENIDEA_TEST_SNOWFLAKE_ACCOUNT=myorg-myaccount OPENIDEA_TEST_SNOWFLAKE_USER=jane \
//! OPENIDEA_TEST_SNOWFLAKE_KEY_FILE=~/.ssh/rsa_key.p8 \
//! cargo test -p ide-db --test snowflake -- --ignored --nocapture
//! ```
//!
//! Optional: `…_KEY_PASSPHRASE`, `…_WAREHOUSE`, `…_ROLE`, `…_DATABASE`, `…_SCHEMA`.
//! The output holds no credential.
use ide_core::{CancellationToken, error::ErrorKind};
use ide_db::{DataSource, DbDriver, DbOptions, DbSnowflake, DbSnowflakeAuth, Pool};
use std::time::{Duration, Instant};
fn setting(name: &str) -> Option<String> {
std::env::var(format!("OPENIDEA_TEST_SNOWFLAKE_{name}")).ok().filter(|v| !v.trim().is_empty())
}
fn source() -> DataSource {
DataSource {
id: "snowflake".into(),
name: "snowflake".into(),
driver: DbDriver::Snowflake,
host: setting("ACCOUNT").expect("OPENIDEA_TEST_SNOWFLAKE_ACCOUNT"),
user: setting("USER").expect("OPENIDEA_TEST_SNOWFLAKE_USER"),
database: setting("DATABASE").unwrap_or_default(),
options: DbOptions { default_schema: setting("SCHEMA").unwrap_or_default(), ..Default::default() },
snowflake: Some(DbSnowflake {
warehouse: setting("WAREHOUSE").unwrap_or_default(),
role: setting("ROLE").unwrap_or_default(),
auth: DbSnowflakeAuth::KeyPair,
private_key_file: setting("KEY_FILE").expect("OPENIDEA_TEST_SNOWFLAKE_KEY_FILE"),
}),
..Default::default()
}
}
#[tokio::test]
#[ignore = "requires OPENIDEA_TEST_SNOWFLAKE_ACCOUNT, _USER and _KEY_FILE"]
async fn key_pair_session_against_a_real_account() {
let passphrase = setting("KEY_PASSPHRASE");
let started = Instant::now();
let pool = match Pool::connect(&source(), passphrase.as_deref(), None).await {
Ok(pool) => pool,
Err(error) => panic!("login failed ({:?}): {}", error.kind, error.message),
};
println!("[1] logged in with the key pair in {:?}", started.elapsed());
let cancel = CancellationToken::new();
let cell = |rows: &[Vec<Option<String>>], i: usize| rows[0][i].clone().unwrap_or_else(|| "<null>".into());
let version = pool.execute("SELECT 'Snowflake ' || CURRENT_VERSION()", 1, &cancel).await.unwrap();
println!("[2] {}", cell(&version.rows, 0));
let context = pool
.execute("SELECT CURRENT_WAREHOUSE(), CURRENT_DATABASE(), CURRENT_SCHEMA(), CURRENT_ROLE()", 1, &cancel)
.await
.unwrap();
println!(
"[3] context: warehouse={} database={} schema={} role={}",
cell(&context.rows, 0),
cell(&context.rows, 1),
cell(&context.rows, 2),
cell(&context.rows, 3)
);
for (wanted, i) in
[(setting("WAREHOUSE"), 0), (setting("DATABASE"), 1), (setting("SCHEMA"), 2), (setting("ROLE"), 3)]
{
if let Some(wanted) = wanted {
assert!(cell(&context.rows, i).eq_ignore_ascii_case(&wanted), "context {i} is not {wanted}");
}
}
let types = pool
.execute(
"SELECT 12345.67::NUMBER(10,2) AS n, 1.5::FLOAT AS f, 'text' AS t, TRUE AS b, \
'2023-11-14'::DATE AS d, '01:02:03.5'::TIME(3) AS tm, \
'2023-11-14 22:13:20.123'::TIMESTAMP_NTZ(3) AS ntz, \
'2023-11-14 22:13:20 +02:00'::TIMESTAMP_TZ(0) AS tz, \
'2023-11-14 22:13:20'::TIMESTAMP_LTZ(0) AS ltz, \
PARSE_JSON('{\"a\":[1,2]}') AS v, TO_BINARY('DEADBEEF', 'HEX') AS bin, NULL AS nothing",
1,
&cancel,
)
.await
.unwrap();
for (column, value) in types.columns.iter().zip(&types.rows[0]) {
println!("[4] {:8} {:18} {}", column.name, column.type_name, value.as_deref().unwrap_or("<null>"));
}
let shown: Vec<Option<&str>> = types.rows[0].iter().map(|v| v.as_deref()).collect();
assert_eq!(shown[0], Some("12345.67"));
assert_eq!(shown[2], Some("text"));
assert_eq!(shown[3], Some("true"));
assert_eq!(shown[4], Some("2023-11-14"));
assert_eq!(shown[5], Some("01:02:03.500"));
assert_eq!(shown[6], Some("2023-11-14 22:13:20.123"));
assert_eq!(shown[7], Some("2023-11-14 22:13:20 +02:00"));
assert_eq!(shown[10], Some("0xDEADBEEF"));
assert_eq!(shown[11], None);
assert_eq!(types.columns[0].type_name, "NUMBER(10,2)");
pool.execute("SET openidea_probe = 41", 0, &cancel).await.unwrap();
let kept = pool.execute("SELECT $openidea_probe + 1", 1, &cancel).await.unwrap();
assert_eq!(cell(&kept.rows, 0), "42");
println!("[5] session state is kept between statements");
let databases = pool.execute("SHOW TERSE DATABASES", 1000, &cancel).await.unwrap();
println!(
"[6] SHOW TERSE DATABASES: {} databases, columns {:?}",
databases.rows.len(),
databases.columns.iter().map(|c| c.name.as_str()).collect::<Vec<_>>()
);
let missing = pool.execute("SELECT * FROM openidea_no_such_table_x", 1, &cancel).await.unwrap_err();
println!("[7] error for a missing table ({:?}): {}", missing.kind, missing.message.replace('\n', " | "));
assert_eq!(missing.kind, ErrorKind::InvalidInput);
let limited = pool.execute("SELECT SEQ4() FROM TABLE(GENERATOR(ROWCOUNT => 100))", 2, &cancel).await;
match limited {
Ok(limited) => {
assert_eq!(limited.rows.len(), 2);
assert!(limited.has_more);
println!("[8] a row limit keeps 2 of 100 rows and reports more");
let page = pool
.execute_from(
"SELECT SEQ4() AS n, REPEAT('x', 200) FROM TABLE(GENERATOR(ROWCOUNT => 300000)) ORDER BY n",
250_000,
3,
&cancel,
)
.await
.unwrap();
assert_eq!(
page.rows.iter().map(|r| r[0].clone().unwrap()).collect::<Vec<_>>(),
["250000", "250001", "250002"]
);
println!(
"[9] a page deep in a large result: {:?} total, {:?} to first answer",
page.elapsed_ms, page.execution_ms
);
let stop = CancellationToken::new();
let waited = Instant::now();
let statement = pool.execute("CALL SYSTEM$WAIT(120)", 1, &stop);
let cancelling = async {
tokio::time::sleep(Duration::from_secs(2)).await;
stop.cancel();
};
let (outcome, ()) = tokio::join!(statement, cancelling);
let error = outcome.unwrap_err();
assert_eq!(error.kind, ErrorKind::Cancelled, "{}", error.message);
assert!(waited.elapsed() < Duration::from_secs(30));
println!("[10] cancelled a 120 s statement after {:?}: {}", waited.elapsed(), error.message);
let after = pool.execute("SELECT 1", 1, &cancel).await.unwrap();
assert_eq!(cell(&after.rows, 0), "1");
println!("[11] the session works after the cancellation");
}
Err(error) => {
println!("[8] skipped warehouse checks ({:?}): {}", error.kind, error.message.replace('\n', " | "))
}
}
pool.close().await;
assert!(pool.is_closed());
println!("[12] session closed");
}
/// The Database tree's metadata calls. Prints counts and column shapes only, never object names.
#[tokio::test]
#[ignore = "requires OPENIDEA_TEST_SNOWFLAKE_ACCOUNT, _USER and _KEY_FILE"]
async fn explorer_metadata_against_a_real_account() {
let root = tempfile::tempdir().unwrap();
let dir = root.path().to_path_buf();
let ssh = std::sync::Arc::new(ide_ssh::SshManager::new(dir.join("ssh-passwords.json")));
let db = ide_db::DbManager::new(dir.clone(), dir.join("passwords.json"), dir.join("history.json"), ssh);
db.save_sources(vec![source()]).await.unwrap();
db.set_password("snowflake", setting("KEY_PASSPHRASE"), false).await.unwrap();
let databases = db.databases("snowflake").await.unwrap();
println!("[m1] databases: {} (current set: {})", databases.names.len(), !databases.current.is_empty());
assert!(!databases.names.is_empty());
let database = setting("DATABASE")
.map(|d| d.to_uppercase())
.filter(|d| databases.names.contains(d))
.unwrap_or_else(|| databases.names[0].clone());
let target = format!("snowflake:{database}");
let schemas = db.schemas(&target).await.unwrap();
println!("[m2] schemas in the chosen database: {}", schemas.len());
assert!(!schemas.iter().any(|s| s == "INFORMATION_SCHEMA"));
// The first schema that has tables, so the deeper calls have something to read.
let mut found = None;
for schema in schemas.iter().take(25) {
let tables = db.tables(&target, schema).await.unwrap();
if !tables.is_empty() {
found = Some((schema.clone(), tables));
break;
}
}
match found {
None => println!("[m3] no tables in the first 25 schemas: table checks skipped"),
Some((schema, tables)) => {
let views = tables.iter().filter(|t| t.kind != ide_db::DbTableKind::Table).count();
println!("[m3] tables and views in one schema: {} ({} views)", tables.len(), views);
let details = db.table_details(&target, &schema, &tables[0].name).await.unwrap();
let typed = details.columns.iter().filter(|c| !c.data_type.is_empty()).count();
println!(
"[m4] first table: {} columns ({} typed, {} primary key, {} nullable), {} foreign keys; first type: {}",
details.columns.len(),
typed,
details.columns.iter().filter(|c| c.primary_key).count(),
details.columns.iter().filter(|c| c.nullable).count(),
details.foreign_keys.len(),
details.columns.first().map_or("", |c| c.data_type.as_str())
);
assert!(!details.columns.is_empty());
assert_eq!(typed, details.columns.len());
let objects = db.schema_objects(&target, &schema).await.unwrap();
let mut kinds = std::collections::BTreeMap::new();
for other in &objects.others {
*kinds.entry(other.kind.as_str()).or_insert(0) += 1;
}
println!(
"[m5] schema objects: {} routines, {} sequences, others {:?}",
objects.routines.len(),
objects.sequences.len(),
kinds
);
let ddl = db.ddl(&target, &schema, &tables[0].name).await.unwrap();
println!(
"[m6] DDL of the first table: {} characters, starts with {:?}",
ddl.len(),
ddl.split_whitespace().next()
);
assert!(ddl.to_lowercase().starts_with("create"));
let page =
db.table_page(&target, &schema, &tables[0].name, None, &[], 0, 5, &CancellationToken::new()).await;
match page {
Ok(page) => println!(
"[m7] table data page: {} rows, {} columns",
page.result.rows.len(),
page.result.columns.len()
),
Err(error) => {
println!("[m7] table data page failed ({:?}): {}", error.kind, error.message.replace('\n', " | "))
}
}
}
}
let loading = Instant::now();
let completion = db.completion(&target).await.unwrap();
println!("[m8] completion model loaded in {:?}", loading.elapsed());
println!(
"[m8] completion model: {} tables ({} with columns), {} columns, default schema set: {}",
completion.tables.len(),
completion.tables.iter().filter(|t| !t.columns.is_empty()).count(),
completion.tables.iter().map(|t| t.columns.len()).sum::<usize>(),
!completion.default_schema.is_empty()
);
let session = db.session_context(&target).await.unwrap();
println!(
"[m9] session: warehouse set: {}, role set: {}, {} warehouses, {} roles to pick from",
!session.warehouse.is_empty(),
!session.role.is_empty(),
session.warehouses.len(),
session.roles.len()
);
assert!(!session.role.is_empty());
let plan = db.explain(&target, "SELECT 1 AS n", &CancellationToken::new(), None).await;
match plan {
Ok(plan) => println!(
"[m10] explain plan: {} operations, first {:?}",
plan.len(),
plan.first().map(|n| n.operation.clone())
),
Err(error) => println!("[m10] explain failed ({:?}): {}", error.kind, error.message.replace('\n', " | ")),
}
// Introspection as the app runs it: every database through the data source itself.
let listed = db.databases("snowflake").await.unwrap();
let (mut read, mut failed, mut schemas, mut tables) = (0, 0, 0, 0);
let mut first_failure = None;
for database in &listed.names {
match db.database_catalog("snowflake", database).await {
Ok(catalog) => {
read += 1;
schemas += catalog.len();
tables += catalog.iter().map(|s| s.tables.len()).sum::<usize>();
}
Err(error) => {
failed += 1;
// Names stay out of the output: quoted parts of the message are masked.
let mut masked = String::new();
let mut quote = None;
for c in error.message.replace('\n', " | ").chars() {
match quote {
Some(q) if c == q => quote = None,
Some(_) => continue,
None if c == '\'' || c == '"' => quote = Some(c),
None => {}
}
masked.push(c);
}
first_failure.get_or_insert(format!("{:?}: {masked}", error.kind));
}
}
}
println!(
"[m11] introspection: {} databases listed, {read} read, {failed} failed, {schemas} schemas, {tables} tables and views",
listed.names.len()
);
if let Some(failure) = first_failure {
println!("[m11] first failure: {failure}");
}
assert!(read > 0, "no database could be introspected");
// The reference's commands that only list things and name no object, against the real server:
// each is either answered or refused by Snowflake (an edition or privilege it needs), never
// misread by the client. `USE` goes to what the session already has.
let fixture: serde_json::Value =
serde_json::from_str(include_str!("fixtures/snowflake/commands.json")).expect("the fixture parses");
let mut statements: Vec<(String, String)> = fixture["commands"]
.as_array()
.expect("commands")
.iter()
.map(|c| {
(c["command"].as_str().unwrap_or_default().to_owned(), c["sql"].as_str().unwrap_or_default().to_owned())
})
.filter(|(_, sql)| sql.starts_with("SHOW ") && !sql.to_lowercase().contains("my_") && !sql.contains("analyst"))
.collect();
let quoted = |name: &str| format!("\"{}\"", name.replace('"', "\"\""));
statements.push(("USE ROLE".into(), format!("USE ROLE {}", quoted(&session.role))));
if !session.warehouse.is_empty() {
statements.push(("USE WAREHOUSE".into(), format!("USE WAREHOUSE {}", quoted(&session.warehouse))));
}
statements.push(("USE DATABASE".into(), format!("USE DATABASE {}", quoted(&database))));
statements.push(("USE SECONDARY ROLES".into(), "USE SECONDARY ROLES ALL".into()));
let (mut answered, mut refused, mut broken) = (0, Vec::new(), Vec::new());
for (command, sql) in &statements {
match db.execute(&target, sql, 1, &CancellationToken::new()).await {
Ok(_) => answered += 1,
Err(error) if error.kind == ErrorKind::InvalidInput => {
// Snowflake's own refusal: the command and the error code only.
let code = error.message.split(['[', ']']).nth(1).unwrap_or("?").to_owned();
refused.push(format!("{command} [{code}]"));
}
Err(error) => broken.push(format!("{command} ({:?})", error.kind)),
}
}
println!(
"[m12] commands: {} run, {answered} answered, {} refused by Snowflake, {} failed in the client",
statements.len(),
refused.len(),
broken.len()
);
if !refused.is_empty() {
println!("[m12] refused: {}", refused.join(", "));
}
assert!(broken.is_empty(), "the client failed on: {}", broken.join(", "));
assert!(answered > 0);
db.disconnect_source("snowflake").await;
println!("[m13] disconnected");
}
-28
Переглянути файл
@@ -342,31 +342,3 @@ async fn applies_driver_properties_and_the_startup_script_on_connect() -> IdeRes
assert_eq!(db.schemas("s1").await?, vec!["main"]);
Ok(())
}
#[tokio::test]
async fn data_sources_edited_on_disk_close_their_connections() -> IdeResult<()> {
let (tmp, db) = setup().await;
std::fs::File::create(tmp.path().join("other.db")).unwrap();
let (tx, rx) = std::sync::mpsc::channel();
let _watch = db.watch_sources(Box::new(move || _ = tx.send(())))?;
db.pool("s1").await?;
assert_eq!(db.connected(), vec!["s1"]);
// Our own save is no edit on disk.
let mut sources = db.sources()?;
sources[0].name = "renamed".into();
db.save_sources(sources.clone()).await?;
assert!(!db.reconcile_sources().await);
// Edited outside the IDE (by hand, by an agent): the watcher reports it, and the
// connection made with the old settings closes.
db.pool("s1").await?;
sources[0].file = Some("other.db".into());
ide_db::source::save(tmp.path(), &sources)?;
rx.recv_timeout(std::time::Duration::from_secs(10)).expect("the watcher reports the edit");
assert!(db.reconcile_sources().await);
assert!(db.connected().is_empty());
assert_eq!(db.sources()?[0].file.as_ref().map(|f| f.0.clone()), Some("other.db".into()));
assert!(!db.reconcile_sources().await);
Ok(())
}
Loaded 100 of 1447 files, more files were not shown because too many files have changed in this diff. Show more