дзеркало
https://github.com/l0ng-ai/openidea.git
синхронізовано 2026-11-18 15:46:14 +02:00
Порівняти коміти
100
Коміти
| Автор | SHA1 | Дата | |
|---|---|---|---|
|
|
1c00a1125d | ||
|
|
1dc1e3cca1 | ||
|
|
4a82c22fdc | ||
|
|
010e7ba6b4 | ||
|
|
9984e40b83 | ||
|
|
36a6e6359d | ||
|
|
4f64326f89 | ||
|
|
b5c2d0dc5e | ||
|
|
c9dca15972 | ||
|
|
41b581eafd | ||
|
|
13e18af777 | ||
|
|
488e819b46 | ||
|
|
ad8df8fe00 | ||
|
|
19d6eab21a | ||
|
|
5c0befeff0 | ||
|
|
07bd8ded5e | ||
|
|
4ab85d5f49 | ||
|
|
3c91adfae2 | ||
|
|
5cfa6614a1 | ||
|
|
e875d97f7c | ||
|
|
31016ee684 | ||
|
|
6a93f1cce6 | ||
|
|
2b4e3c79f8 | ||
|
|
4e90d9fd5c | ||
|
|
ffc2f75d3b | ||
|
|
19587af8b1 | ||
|
|
b2c22ccec6 | ||
|
|
f795aa3232 | ||
|
|
9239b99f3a | ||
|
|
c5be2998ec | ||
|
|
b83df2dbda | ||
|
|
9f7a35f170 | ||
|
|
52b3e14a8c | ||
|
|
934e4ca0e7 | ||
|
|
a2809bda80 | ||
|
|
0fd8389b33 | ||
|
|
19c0ae07f2 | ||
|
|
5a5b53d86b | ||
|
|
ac7a81db1c | ||
|
|
9681e1a118 | ||
|
|
293972baaa | ||
|
|
b173a53004 | ||
|
|
f6e54f581b | ||
|
|
276a30849c | ||
|
|
63abedaadd | ||
|
|
c22a7fa112 | ||
|
|
a46f187428 | ||
|
|
ff826279f2 | ||
|
|
759fb3278d | ||
|
|
f7d19e1711 | ||
|
|
97d202ade0 | ||
|
|
f0f15366e4 | ||
|
|
fecbf8ed14 | ||
|
|
2d00f09393 | ||
|
|
f154bf8c83 | ||
|
|
d3720c0e11 | ||
|
|
ec862b3668 | ||
|
|
735830d642 | ||
|
|
65c0413dbb | ||
|
|
14bb7bcc6f | ||
|
|
8113d101ef | ||
|
|
5f7e05adc4 | ||
|
|
f2b85eca4c | ||
|
|
e41cf32a94 | ||
|
|
0cd19a8c0f | ||
|
|
fb76a6dab7 | ||
|
|
f4502ffffe | ||
|
|
2430c892ba | ||
|
|
8876d04cb8 | ||
|
|
eeb20f467c | ||
|
|
4ccff88ccf | ||
|
|
fdf29989ec | ||
|
|
3c3b6a6fd1 | ||
|
|
49bb6b2d76 | ||
|
|
632e564d58 | ||
|
|
08333bcff8 | ||
|
|
ae542b3044 | ||
|
|
4cb98ff162 | ||
|
|
cc8ccea888 | ||
|
|
75344c21ef | ||
|
|
91ac7f417c | ||
|
|
2fd43aa29f | ||
|
|
5b7d258adb | ||
|
|
fd1cebdf98 | ||
|
|
c9362247d0 | ||
|
|
8ab9c688ac | ||
|
|
ba0b3d9c7d | ||
|
|
96f8ee27dd | ||
|
|
5529db667c | ||
|
|
9c55c28d5b | ||
|
|
009773c0d4 | ||
|
|
ba0f4329ea | ||
|
|
4d88d44616 | ||
|
|
5e5d804b89 | ||
|
|
83b9744a36 | ||
|
|
4ab8959bac | ||
|
|
d715643ff2 | ||
|
|
a5214cae38 | ||
|
|
54dd3a167f | ||
|
|
0938a507da |
No files matched your search
@@ -7,7 +7,7 @@ BASEDPYRIGHT=1.40.2
|
||||
RUFF=0.17.3
|
||||
GOPLS=v0.24.0
|
||||
JDTLS=1.63.0-202610191322
|
||||
KOTLIN_LSP=263.7084.0
|
||||
KOTLIN_LSP=271.2085.0
|
||||
GRADLE=8.14.3
|
||||
INTELEPHENSE=1.18.5
|
||||
# The JDK projects compile against (JAVA_HOME): library navigation opens its sources.
|
||||
|
||||
@@ -119,7 +119,7 @@ jobs:
|
||||
- run: cargo test --locked -p ide-host --test language_smoke --no-run
|
||||
- run: cargo test --locked -p ide-host --test language_smoke -- --ignored --exact ${{ matrix.language }}
|
||||
- if: matrix.language == 'java'
|
||||
run: cargo test --locked -p ide-host --test project_smoke -- --ignored --exact maven_sync jdk
|
||||
run: cargo test --locked -p ide-host --test project_smoke -- --ignored --exact maven_sync jdk maven_launch_classpath_uses_the_runner_jdk_classifier
|
||||
- if: matrix.language == 'kotlin'
|
||||
run: cargo test --locked -p ide-host --test project_smoke -- --ignored --exact gradle_sync
|
||||
- if: matrix.language == 'python'
|
||||
|
||||
Створено
+45
-44
@@ -4144,8 +4144,9 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
|
||||
|
||||
[[package]]
|
||||
name = "ide-agent"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"dunce",
|
||||
"encoding_rs",
|
||||
"ide-core",
|
||||
@@ -4168,7 +4169,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-analysis"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ast-grep-core",
|
||||
"ide-core",
|
||||
@@ -4182,7 +4183,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-android"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"serde",
|
||||
@@ -4193,7 +4194,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-build"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"chardetng",
|
||||
@@ -4211,7 +4212,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-cli"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dirs 6.0.0",
|
||||
"serde",
|
||||
@@ -4225,7 +4226,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-clipboard"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"gtk",
|
||||
"objc2 0.6.5",
|
||||
@@ -4235,7 +4236,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-core"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dunce",
|
||||
"encoding_rs",
|
||||
@@ -4252,7 +4253,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-coverage"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"ide-run",
|
||||
@@ -4266,7 +4267,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-dap"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"futures",
|
||||
"ide-core",
|
||||
@@ -4293,7 +4294,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-db"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"bb8",
|
||||
@@ -4329,7 +4330,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-docker"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"serde",
|
||||
@@ -4342,7 +4343,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-extension"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"blake3",
|
||||
@@ -4363,7 +4364,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-fonts"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"objc2-core-foundation",
|
||||
"objc2-core-text",
|
||||
@@ -4375,7 +4376,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-github"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"ide-vcs",
|
||||
@@ -4388,7 +4389,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-heap"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"ide-core",
|
||||
@@ -4406,7 +4407,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-host"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"blake3",
|
||||
@@ -4477,7 +4478,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-http"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"ide-vfs",
|
||||
@@ -4496,7 +4497,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-index"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"criterion",
|
||||
@@ -4518,7 +4519,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-jdwp"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-jvm-inspect",
|
||||
"proptest",
|
||||
@@ -4530,7 +4531,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-jsonrpc"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"encoding_rs",
|
||||
"futures",
|
||||
@@ -4547,7 +4548,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-jvm-attach"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"tempfile",
|
||||
@@ -4556,7 +4557,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-jvm-inspect"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"futures",
|
||||
"serde",
|
||||
@@ -4567,7 +4568,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-language"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"criterion",
|
||||
@@ -4624,7 +4625,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-lsp"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dunce",
|
||||
"encoding_rs",
|
||||
@@ -4652,7 +4653,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-mcp"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"dunce",
|
||||
@@ -4671,7 +4672,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-notebook"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"futures",
|
||||
"ide-core",
|
||||
@@ -4694,7 +4695,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-plugin"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"dunce",
|
||||
@@ -4717,7 +4718,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-profiler"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"ide-jvm-attach",
|
||||
@@ -4731,7 +4732,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-remote"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dunce",
|
||||
"futures",
|
||||
@@ -4751,7 +4752,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-run"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dunce",
|
||||
"encoding_rs",
|
||||
@@ -4775,7 +4776,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-scaffold"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"ide-tools",
|
||||
@@ -4791,7 +4792,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-sdk"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dunce",
|
||||
"ide-core",
|
||||
@@ -4812,7 +4813,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-search"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"encoding_rs",
|
||||
@@ -4833,7 +4834,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-spell"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"parking_lot",
|
||||
@@ -4846,7 +4847,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-ssh"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dirs 6.0.0",
|
||||
"flate2",
|
||||
@@ -4871,7 +4872,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-syntax"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"ide-language",
|
||||
@@ -4886,7 +4887,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-terminal"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"ide-core",
|
||||
"libc",
|
||||
@@ -4899,7 +4900,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-text"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"ide-core",
|
||||
@@ -4912,7 +4913,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-tools"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"flate2",
|
||||
"hex",
|
||||
@@ -4933,7 +4934,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-vcs"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"dunce",
|
||||
@@ -4954,7 +4955,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-vfs"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"chardetng",
|
||||
"dunce",
|
||||
@@ -4978,7 +4979,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-web"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"dirs 6.0.0",
|
||||
"ide-core",
|
||||
@@ -4994,7 +4995,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "ide-workspace"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"blake3",
|
||||
"dunce",
|
||||
@@ -6595,7 +6596,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "openidea"
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"blake3",
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
|
||||
exclude = ["sdk", "plugins"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.1.44"
|
||||
version = "0.1.45"
|
||||
edition = "2024"
|
||||
rust-version = "1.96"
|
||||
license = "Apache-2.0"
|
||||
|
||||
@@ -7,6 +7,7 @@ license.workspace = true
|
||||
|
||||
[dependencies]
|
||||
encoding_rs.workspace = true
|
||||
blake3.workspace = true
|
||||
ide-vfs.workspace = true
|
||||
ide-core.workspace = true
|
||||
ide-mcp.workspace = true
|
||||
@@ -24,3 +25,6 @@ dunce.workspace = true
|
||||
uuid.workspace = true
|
||||
tempfile.workspace = true
|
||||
which = "8"
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { workspace = true, features = ["test-util"] }
|
||||
@@ -3,6 +3,7 @@
|
||||
//! `initialize` alone).
|
||||
|
||||
use super::{Conn, Dialect};
|
||||
use crate::catalogs::Discovered;
|
||||
use crate::config;
|
||||
use crate::model::AgentConfigOption;
|
||||
use crate::provider::ProviderSpec;
|
||||
@@ -15,7 +16,7 @@ use std::time::Duration;
|
||||
const DISCOVER_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(3) } else { Duration::from_secs(120) };
|
||||
|
||||
/// Opens `session/new`, reads `configOptions`, and records per-model `thought_level` choices.
|
||||
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(), String> {
|
||||
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<Discovered, String> {
|
||||
let root = shared.paths.project_root.clone();
|
||||
let launch = spec.clone();
|
||||
let launch = tokio::task::spawn_blocking({
|
||||
@@ -25,12 +26,12 @@ pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(),
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let cwd = dunce::canonicalize(&root).unwrap_or(root);
|
||||
let stderr: Arc<Mutex<Vec<String>>> = Arc::default();
|
||||
let lines = stderr.clone();
|
||||
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(move |l| lines.lock().push(l));
|
||||
let client = Dialect::wrap(Arc::new(ide_jsonrpc::NoopHandler));
|
||||
let updates = Arc::new(DiscoveryUpdates { provider: spec.id.clone(), commands: Mutex::new(None) });
|
||||
let client = Dialect::wrap(updates.clone());
|
||||
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(|_| {});
|
||||
let peer = (shared.connector)(&launch, &cwd, client.clone(), on_stderr)
|
||||
.map_err(|e| format!("Couldn't start {} (`{}`): {e}", launch.name, launch.command))?;
|
||||
let _stop = crate::catalogs::KillPeer(peer.clone());
|
||||
let conn = Conn::new(peer.clone(), client);
|
||||
let work = async {
|
||||
conn.initialize(&client_info()).await.map_err(|e| e.to_string())?;
|
||||
@@ -51,34 +52,41 @@ pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(),
|
||||
(Some(model), Some(effort)) => model.current.clone().map(|m| vec![(m, effort)]).unwrap_or_default(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let option_ids: Vec<_> = options.iter().map(|o| o.id.as_str()).collect();
|
||||
tracing::info!(
|
||||
"agent {}: settings discovery got {} config option(s) {:?}, {} model effort list(s)",
|
||||
launch.id,
|
||||
options.len(),
|
||||
option_ids,
|
||||
levels.len()
|
||||
);
|
||||
shared.remember_discovered(&launch.id, &options, &[], &levels);
|
||||
Ok::<(), String>(())
|
||||
let commands = updates
|
||||
.commands
|
||||
.lock()
|
||||
.clone()
|
||||
.or_else(|| new.get("availableCommands").map(|_| config::commands_of(&new, &launch.id)));
|
||||
Ok::<_, String>(Discovered { config: options, commands, levels: levels.into_iter().collect() })
|
||||
};
|
||||
let result = tokio::time::timeout(DISCOVER_TIMEOUT, work).await;
|
||||
peer.kill();
|
||||
match result {
|
||||
Ok(Ok(())) => Ok(()),
|
||||
Ok(Err(e)) => Err(match stderr.lock().last() {
|
||||
Some(line) => format!("{e}: {line}"),
|
||||
None => e,
|
||||
}),
|
||||
Err(_) => {
|
||||
let tail = stderr.lock().join("\n");
|
||||
if tail.is_empty() {
|
||||
Err("the agent did not answer in time".into())
|
||||
} else {
|
||||
Err(format!("the agent did not answer in time:\n{tail}"))
|
||||
}
|
||||
Ok(result) => result,
|
||||
Err(_) => Err("the agent did not answer in time".into()),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct DiscoveryUpdates {
|
||||
provider: String,
|
||||
commands: Mutex<Option<Vec<crate::AgentCommand>>>,
|
||||
}
|
||||
|
||||
impl ide_jsonrpc::Handler for DiscoveryUpdates {
|
||||
fn on_notification(&self, method: &str, params: Value) {
|
||||
if method == "session/update"
|
||||
&& let Some(update) = params.get("update")
|
||||
&& update["sessionUpdate"] == "available_commands_update"
|
||||
{
|
||||
// Commands are protocol data, not stderr (which may contain credentials).
|
||||
*self.commands.lock() = Some(config::commands_of(update, &self.provider));
|
||||
}
|
||||
}
|
||||
|
||||
fn on_request(&self, method: &str, _: Value) -> Result<Value, (i64, String)> {
|
||||
Err((ide_jsonrpc::codes::METHOD_NOT_FOUND, format!("unsupported discovery request {method}")))
|
||||
}
|
||||
}
|
||||
|
||||
fn is_thought_level(o: &AgentConfigOption) -> bool {
|
||||
|
||||
@@ -108,6 +108,7 @@ fn manager(dir: &Path, agent: Arc<FakeV2Agent>) -> AgentManager {
|
||||
custom_agents_dir: None,
|
||||
home_dir: None,
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
connector,
|
||||
|
||||
@@ -155,6 +155,7 @@ async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent()
|
||||
custom_agents_dir: None,
|
||||
home_dir: None,
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
connector,
|
||||
|
||||
@@ -0,0 +1,427 @@
|
||||
//! Model availability for new tasks. Sessions own selections; discovery owns catalogs.
|
||||
|
||||
use crate::{
|
||||
AgentCommand, AgentConfigOption, AgentEvent, AgentModelCatalog, Known, Shared, acp, config, native, provider,
|
||||
};
|
||||
use ide_core::{IdeError, IdeResult};
|
||||
use parking_lot::Mutex;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::{BTreeMap, HashMap};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Weak};
|
||||
use std::time::Duration;
|
||||
use tokio::sync::watch;
|
||||
use tokio::time::Instant;
|
||||
|
||||
const FRESH_FOR: Duration = Duration::from_secs(15 * 60);
|
||||
const COOLDOWN: Duration = Duration::from_secs(30);
|
||||
|
||||
type Store = Arc<Mutex<HashMap<String, Known>>>;
|
||||
type Key = (PathBuf, PathBuf, String);
|
||||
|
||||
/// Backend-wide single-flight model lookups, scoped to project and effective launch configuration.
|
||||
#[derive(Default)]
|
||||
pub struct AgentCatalogs {
|
||||
state: Mutex<State>,
|
||||
#[cfg(test)]
|
||||
clock_ms: std::sync::atomic::AtomicU64,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct State {
|
||||
stores: HashMap<PathBuf, Store>,
|
||||
entries: HashMap<Key, Entry>,
|
||||
subscribers: Vec<Weak<Shared>>,
|
||||
generation: u64,
|
||||
}
|
||||
|
||||
struct Entry {
|
||||
// Launch credentials remain only in memory, never serialized or logged.
|
||||
context: LaunchContext,
|
||||
generation: u64,
|
||||
completed: Option<Instant>,
|
||||
successful: Option<Instant>,
|
||||
retry_at_ms: Option<f64>,
|
||||
error: Option<String>,
|
||||
flight: Option<watch::Receiver<bool>>,
|
||||
abort: Option<tokio::task::AbortHandle>,
|
||||
catalog: Option<CatalogData>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Default, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub(crate) struct CatalogData {
|
||||
pub config: Vec<AgentConfigOption>,
|
||||
pub thought_levels: BTreeMap<String, Vec<AgentConfigOption>>,
|
||||
pub refreshed_at_ms: Option<f64>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub(crate) struct SavedCatalog {
|
||||
/// An opaque cryptographic digest, never raw launch arguments or credentials.
|
||||
fingerprint: String,
|
||||
#[serde(flatten)]
|
||||
catalog: CatalogData,
|
||||
}
|
||||
|
||||
#[derive(PartialEq)]
|
||||
struct LaunchContext {
|
||||
spec: provider::ProviderSpec,
|
||||
executable: PathBuf,
|
||||
modified: Option<std::time::SystemTime>,
|
||||
len: Option<u64>,
|
||||
}
|
||||
|
||||
impl LaunchContext {
|
||||
fn new(spec: provider::ProviderSpec, root: &std::path::Path) -> Self {
|
||||
let resolved = spec.clone().resolve_for_launch(root);
|
||||
let executable = dunce::canonicalize(&resolved.command).unwrap_or_else(|_| PathBuf::from(&resolved.command));
|
||||
let metadata = std::fs::metadata(&executable).ok();
|
||||
Self {
|
||||
spec,
|
||||
executable,
|
||||
modified: metadata.as_ref().and_then(|m| m.modified().ok()),
|
||||
len: metadata.as_ref().map(std::fs::Metadata::len),
|
||||
}
|
||||
}
|
||||
|
||||
fn fingerprint(&self) -> String {
|
||||
let mut hash = blake3::Hasher::new();
|
||||
fn field(hash: &mut blake3::Hasher, value: &[u8]) {
|
||||
hash.update(&(value.len() as u64).to_le_bytes());
|
||||
hash.update(value);
|
||||
}
|
||||
field(&mut hash, b"openidea-model-catalog-v1");
|
||||
field(
|
||||
&mut hash,
|
||||
match self.spec.driver {
|
||||
provider::Driver::Acp => b"acp",
|
||||
provider::Driver::Codex => b"codex",
|
||||
provider::Driver::Claude => b"claude",
|
||||
},
|
||||
);
|
||||
field(&mut hash, self.spec.command.as_bytes());
|
||||
hash.update(&(self.spec.args.len() as u64).to_le_bytes());
|
||||
for arg in &self.spec.args {
|
||||
field(&mut hash, arg.as_bytes());
|
||||
}
|
||||
hash.update(&(self.spec.env.len() as u64).to_le_bytes());
|
||||
for (name, value) in &self.spec.env {
|
||||
field(&mut hash, name.as_bytes());
|
||||
field(&mut hash, value.as_bytes());
|
||||
}
|
||||
field(&mut hash, self.executable.to_string_lossy().as_bytes());
|
||||
if let Some(modified) = self.modified.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) {
|
||||
hash.update(&modified.as_nanos().to_le_bytes());
|
||||
}
|
||||
if let Some(len) = self.len {
|
||||
hash.update(&len.to_le_bytes());
|
||||
}
|
||||
hash.finalize().to_hex().to_string()
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct Discovered {
|
||||
pub config: Vec<AgentConfigOption>,
|
||||
pub commands: Option<Vec<AgentCommand>>,
|
||||
pub levels: BTreeMap<String, Vec<AgentConfigOption>>,
|
||||
}
|
||||
|
||||
pub(crate) struct KillPeer(pub ide_jsonrpc::Peer);
|
||||
|
||||
impl Drop for KillPeer {
|
||||
fn drop(&mut self) {
|
||||
self.0.kill();
|
||||
}
|
||||
}
|
||||
|
||||
impl AgentCatalogs {
|
||||
fn now(&self) -> Instant {
|
||||
let now = Instant::now();
|
||||
#[cfg(test)]
|
||||
let now = now + Duration::from_millis(self.clock_ms.load(std::sync::atomic::Ordering::Relaxed));
|
||||
now
|
||||
}
|
||||
|
||||
fn epoch_ms(&self) -> f64 {
|
||||
let now = crate::now_ms();
|
||||
#[cfg(test)]
|
||||
let now = now + self.clock_ms.load(std::sync::atomic::Ordering::Relaxed) as f64;
|
||||
now
|
||||
}
|
||||
|
||||
pub(crate) fn store(&self, path: &std::path::Path) -> Store {
|
||||
self.state
|
||||
.lock()
|
||||
.stores
|
||||
.entry(path.to_path_buf())
|
||||
.or_insert_with(|| {
|
||||
Arc::new(Mutex::new(
|
||||
std::fs::read(path).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default(),
|
||||
))
|
||||
})
|
||||
.clone()
|
||||
}
|
||||
|
||||
pub(crate) fn subscribe(&self, shared: &Arc<Shared>) {
|
||||
let mut state = self.state.lock();
|
||||
state.subscribers.retain(|s| s.strong_count() > 0);
|
||||
state.subscribers.push(Arc::downgrade(shared));
|
||||
}
|
||||
|
||||
fn key(shared: &Shared, provider: &str) -> Key {
|
||||
(shared.paths.models_file.clone(), shared.paths.project_root.clone(), provider.to_owned())
|
||||
}
|
||||
|
||||
fn fallback(shared: &Shared, provider: &str) -> CatalogData {
|
||||
let cache = shared.known.lock();
|
||||
let Some(known) = cache.get(provider) else { return CatalogData::default() };
|
||||
known.catalogs.get(&shared.root_key()).map(|saved| saved.catalog.clone()).unwrap_or_else(|| CatalogData {
|
||||
config: known.config.clone(),
|
||||
thought_levels: known.thought_levels.clone(),
|
||||
refreshed_at_ms: None,
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn cached(&self, shared: &Shared, provider: &str) -> CatalogData {
|
||||
self.state
|
||||
.lock()
|
||||
.entries
|
||||
.get(&Self::key(shared, provider))
|
||||
.and_then(|e| e.catalog.clone())
|
||||
.unwrap_or_else(|| Self::fallback(shared, provider))
|
||||
}
|
||||
|
||||
fn snapshot(&self, shared: &Shared, provider: &str, entry: &Entry) -> AgentModelCatalog {
|
||||
let catalog = entry.catalog.clone().unwrap_or_else(|| Self::fallback(shared, provider));
|
||||
AgentModelCatalog {
|
||||
provider: provider.to_owned(),
|
||||
config: shared.config_for_new_task(catalog.config, provider),
|
||||
thought_levels: catalog.thought_levels,
|
||||
refreshing: entry.flight.is_some(),
|
||||
refreshed_at_ms: catalog.refreshed_at_ms,
|
||||
retry_at_ms: entry
|
||||
.retry_at_ms
|
||||
.filter(|_| entry.completed.is_some_and(|at| self.now().duration_since(at) < COOLDOWN)),
|
||||
error: entry.error.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Start or join a lookup. No queued refresh can outlive the request that asked for it.
|
||||
pub(crate) fn request(
|
||||
self: &Arc<Self>,
|
||||
shared: &Arc<Shared>,
|
||||
spec: provider::ProviderSpec,
|
||||
force: bool,
|
||||
) -> (AgentModelCatalog, Option<watch::Receiver<bool>>) {
|
||||
let key = Self::key(shared, &spec.id);
|
||||
let context = LaunchContext::new(spec.clone(), &shared.paths.project_root);
|
||||
let now = self.now();
|
||||
let mut state = self.state.lock();
|
||||
let changed = state.entries.get(&key).is_none_or(|e| e.context != context);
|
||||
if changed {
|
||||
if let Some(abort) = state.entries.get_mut(&key).and_then(|entry| entry.abort.take()) {
|
||||
abort.abort();
|
||||
}
|
||||
state.generation += 1;
|
||||
let generation = state.generation;
|
||||
let saved =
|
||||
shared.known.lock().get(&spec.id).and_then(|known| known.catalogs.get(&shared.root_key())).cloned();
|
||||
let matching = saved.as_ref().is_some_and(|saved| saved.fingerprint == context.fingerprint());
|
||||
let age = saved
|
||||
.as_ref()
|
||||
.filter(|_| matching)
|
||||
.and_then(|saved| saved.catalog.refreshed_at_ms)
|
||||
.map(|at| self.epoch_ms() - at)
|
||||
.filter(|age| *age >= 0.0 && *age < FRESH_FOR.as_millis() as f64)
|
||||
.map(|age| Duration::from_secs_f64(age / 1000.0));
|
||||
let successful = age.and_then(|age| now.checked_sub(age));
|
||||
let completed = age.filter(|age| *age < COOLDOWN).and_then(|age| now.checked_sub(age));
|
||||
let retry_at_ms = completed
|
||||
.and_then(|_| saved.as_ref().and_then(|s| s.catalog.refreshed_at_ms))
|
||||
.map(|at| at + COOLDOWN.as_millis() as f64);
|
||||
let catalog = saved.map(|saved| {
|
||||
let mut catalog = saved.catalog;
|
||||
if !matching {
|
||||
catalog.refreshed_at_ms = None;
|
||||
}
|
||||
catalog
|
||||
});
|
||||
state.entries.insert(
|
||||
key.clone(),
|
||||
Entry {
|
||||
context,
|
||||
generation,
|
||||
completed,
|
||||
successful,
|
||||
retry_at_ms,
|
||||
catalog,
|
||||
error: None,
|
||||
flight: None,
|
||||
abort: None,
|
||||
},
|
||||
);
|
||||
}
|
||||
let entry = state.entries.get_mut(&key).expect("catalog entry");
|
||||
let cooling = entry.completed.is_some_and(|at| now.duration_since(at) < COOLDOWN);
|
||||
let fresh = entry.successful.is_some_and(|at| now.duration_since(at) < FRESH_FOR);
|
||||
let mut start = None;
|
||||
if entry.flight.is_none() && !cooling && (force || !fresh) {
|
||||
let (done, ready) = watch::channel(false);
|
||||
entry.flight = Some(ready);
|
||||
entry.error = None;
|
||||
entry.retry_at_ms = None;
|
||||
start = Some((entry.generation, done));
|
||||
}
|
||||
let snapshot = self.snapshot(shared, &spec.id, entry);
|
||||
let flight = entry.flight.clone();
|
||||
drop(state);
|
||||
if let Some((generation, done)) = start {
|
||||
self.broadcast(shared, &snapshot);
|
||||
let coordinator = self.clone();
|
||||
let shared = shared.clone();
|
||||
let flight_key = key.clone();
|
||||
let task = shared.runtime.clone().spawn(async move {
|
||||
let result = discover(&shared, spec).await;
|
||||
coordinator.complete(&shared, flight_key, generation, result);
|
||||
let _ = done.send(true);
|
||||
});
|
||||
// A subscriber can re-enter and change configuration during the start event.
|
||||
let mut state = self.state.lock();
|
||||
if let Some(entry) = state.entries.get_mut(&key).filter(|entry| entry.generation == generation) {
|
||||
if entry.flight.is_some() {
|
||||
entry.abort = Some(task.abort_handle());
|
||||
}
|
||||
} else {
|
||||
task.abort();
|
||||
}
|
||||
}
|
||||
(snapshot, flight)
|
||||
}
|
||||
|
||||
fn complete(&self, shared: &Shared, key: Key, generation: u64, result: Result<Discovered, String>) {
|
||||
let current = provider::load(shared.paths.custom_agents_dir.as_deref(), shared.env_secrets())
|
||||
.into_iter()
|
||||
.find(|spec| spec.id == key.2)
|
||||
.map(|spec| LaunchContext::new(spec, &shared.paths.project_root));
|
||||
let mut state = self.state.lock();
|
||||
let Some(entry) = state.entries.get_mut(&key).filter(|e| e.generation == generation) else { return };
|
||||
// Configuration may have changed without another refresh request arriving.
|
||||
if current.as_ref() != Some(&entry.context) {
|
||||
entry.flight = None;
|
||||
entry.abort = None;
|
||||
entry.completed = None;
|
||||
entry.retry_at_ms = None;
|
||||
entry.error = None;
|
||||
let snapshot = self.snapshot(shared, &key.2, entry);
|
||||
state.entries.remove(&key);
|
||||
drop(state);
|
||||
self.broadcast(shared, &snapshot);
|
||||
return;
|
||||
}
|
||||
let now = self.now();
|
||||
entry.completed = Some(now);
|
||||
entry.retry_at_ms = Some(self.epoch_ms() + COOLDOWN.as_millis() as f64);
|
||||
entry.flight = None;
|
||||
entry.abort = None;
|
||||
let success = match result {
|
||||
Ok(found) => {
|
||||
let mut cache = shared.known.lock();
|
||||
let known = cache.entry(key.2.clone()).or_default();
|
||||
let catalog = CatalogData {
|
||||
config: found.config,
|
||||
thought_levels: found.levels,
|
||||
refreshed_at_ms: Some(self.epoch_ms()),
|
||||
};
|
||||
known.config = catalog.config.clone();
|
||||
known.thought_levels = catalog.thought_levels.clone();
|
||||
if let Some(commands) = found.commands {
|
||||
known.commands.insert(shared.root_key(), commands);
|
||||
}
|
||||
known.authoritative = true;
|
||||
known.refreshed_at_ms = catalog.refreshed_at_ms;
|
||||
known.catalogs.insert(
|
||||
shared.root_key(),
|
||||
SavedCatalog { fingerprint: entry.context.fingerprint(), catalog: catalog.clone() },
|
||||
);
|
||||
entry.catalog = Some(catalog);
|
||||
shared.persist_known(&cache);
|
||||
entry.successful = Some(now);
|
||||
entry.error = None;
|
||||
true
|
||||
}
|
||||
Err(error) => {
|
||||
entry.error = Some(error);
|
||||
false
|
||||
}
|
||||
};
|
||||
let snapshot = self.snapshot(shared, &key.2, entry);
|
||||
drop(state);
|
||||
self.broadcast(shared, &snapshot);
|
||||
if success {
|
||||
for subscriber in self.subscribers(shared) {
|
||||
subscriber.emit(AgentEvent::Providers);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn subscribers(&self, shared: &Shared) -> Vec<Arc<Shared>> {
|
||||
self.state
|
||||
.lock()
|
||||
.subscribers
|
||||
.iter()
|
||||
.filter_map(Weak::upgrade)
|
||||
.filter(|s| {
|
||||
s.paths.models_file == shared.paths.models_file && s.paths.project_root == shared.paths.project_root
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn broadcast(&self, shared: &Shared, catalog: &AgentModelCatalog) {
|
||||
for subscriber in self.subscribers(shared) {
|
||||
subscriber.emit(AgentEvent::ModelCatalog { catalog: catalog.clone() });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl crate::AgentManager {
|
||||
/// Return cached choices immediately and schedule discovery when eligible.
|
||||
pub fn refresh_models(&self, provider: &str, force: bool) -> IdeResult<AgentModelCatalog> {
|
||||
// Settings and environment secrets can change while this window is open.
|
||||
*self.shared.providers.write() =
|
||||
provider::load(self.shared.paths.custom_agents_dir.as_deref(), self.shared.env_secrets());
|
||||
let spec = self.shared.spec(provider).ok_or_else(|| IdeError::invalid(format!("unknown agent {provider}")))?;
|
||||
Ok(self.shared.paths.catalogs.request(&self.shared, spec, force).0)
|
||||
}
|
||||
}
|
||||
|
||||
async fn discover(shared: &Arc<Shared>, spec: provider::ProviderSpec) -> Result<Discovered, String> {
|
||||
match spec.driver {
|
||||
provider::Driver::Acp => acp::discover::run(shared, &spec).await,
|
||||
provider::Driver::Codex => native::codex::discover(spec, shared.paths.project_root.clone()).await,
|
||||
provider::Driver::Claude => {
|
||||
let found = native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await?;
|
||||
Ok(Discovered {
|
||||
commands: Some(config::commands_of(
|
||||
&serde_json::json!({ "availableCommands": found.commands }),
|
||||
&spec.id,
|
||||
)),
|
||||
config: config::options_of(&serde_json::json!({ "configOptions": found.options })),
|
||||
levels: found
|
||||
.efforts
|
||||
.into_iter()
|
||||
.map(|(model, option)| {
|
||||
let levels = option
|
||||
.map(|o| config::options_of(&serde_json::json!({ "configOptions": [o] })))
|
||||
.unwrap_or_default();
|
||||
(model, levels)
|
||||
})
|
||||
.collect(),
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
@@ -0,0 +1,501 @@
|
||||
use super::*;
|
||||
use crate::{AgentManager, AgentPaths, Connector};
|
||||
use ide_jsonrpc::{Framing, Peer};
|
||||
use serde_json::{Value, json};
|
||||
use std::path::Path;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
|
||||
const PROVIDER: &str = "catalog-test";
|
||||
|
||||
#[derive(Debug)]
|
||||
struct Pending {
|
||||
params: Value,
|
||||
reply: oneshot::Sender<Result<Value, String>>,
|
||||
}
|
||||
|
||||
struct Harness {
|
||||
connector: Connector,
|
||||
pending: mpsc::UnboundedReceiver<Pending>,
|
||||
launches: Arc<AtomicUsize>,
|
||||
live: Arc<AtomicUsize>,
|
||||
}
|
||||
|
||||
struct Live(Arc<AtomicUsize>);
|
||||
impl Drop for Live {
|
||||
fn drop(&mut self) {
|
||||
self.0.fetch_sub(1, Ordering::SeqCst);
|
||||
}
|
||||
}
|
||||
|
||||
impl Harness {
|
||||
fn new() -> Self {
|
||||
let (pending_tx, pending) = mpsc::unbounded_channel();
|
||||
let launches = Arc::new(AtomicUsize::new(0));
|
||||
let live = Arc::new(AtomicUsize::new(0));
|
||||
let connector: Connector = Arc::new({
|
||||
let launches = launches.clone();
|
||||
let live = live.clone();
|
||||
move |_, _, client, _| {
|
||||
launches.fetch_add(1, Ordering::SeqCst);
|
||||
live.fetch_add(1, Ordering::SeqCst);
|
||||
let guard = Live(live.clone());
|
||||
let pending = pending_tx.clone();
|
||||
let (client_io, server_io) = tokio::io::duplex(1 << 16);
|
||||
let (read, mut write) = tokio::io::split(server_io);
|
||||
tokio::spawn(async move {
|
||||
let _guard = guard;
|
||||
let mut lines = BufReader::new(read).lines();
|
||||
while let Some(line) = lines.next_line().await.unwrap() {
|
||||
let request: Value = serde_json::from_str(&line).unwrap();
|
||||
let id = &request["id"];
|
||||
if id.is_null() {
|
||||
continue;
|
||||
}
|
||||
let response = match request["method"].as_str().unwrap() {
|
||||
"initialize" => json!({ "id": id, "jsonrpc": "2.0", "result": {
|
||||
"protocolVersion": 1, "agentCapabilities": {}, "authMethods": [] } }),
|
||||
"session/new" => {
|
||||
let (reply, result) = oneshot::channel();
|
||||
pending.send(Pending { params: request["params"].clone(), reply }).unwrap();
|
||||
let result = tokio::select! {
|
||||
result = result => result.unwrap(),
|
||||
_ = lines.next_line() => break,
|
||||
};
|
||||
match result {
|
||||
Ok(result) => json!({ "id": id, "jsonrpc": "2.0", "result": result }),
|
||||
Err(error) => {
|
||||
json!({ "id": id, "jsonrpc": "2.0", "error": { "code": -32000, "message": error } })
|
||||
}
|
||||
}
|
||||
}
|
||||
method => panic!("discovery must not call {method}"),
|
||||
};
|
||||
if write.write_all(format!("{response}\n").as_bytes()).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
let (read, write) = tokio::io::split(client_io);
|
||||
Ok(Peer::connect_framed(read, write, client, Framing::Lines))
|
||||
}
|
||||
});
|
||||
Self { connector, pending, launches, live }
|
||||
}
|
||||
|
||||
async fn next(&mut self) -> Pending {
|
||||
tokio::time::timeout(Duration::from_secs(5), self.pending.recv()).await.unwrap().unwrap()
|
||||
}
|
||||
}
|
||||
|
||||
fn configure(root: &Path, version: &str) {
|
||||
std::fs::create_dir_all(root.join("agents")).unwrap();
|
||||
let executable = root.join("catalog-test-bin");
|
||||
if !executable.exists() {
|
||||
std::fs::write(&executable, "first executable").unwrap();
|
||||
}
|
||||
std::fs::write(
|
||||
root.join("agents/catalog-test.toml"),
|
||||
format!(
|
||||
"name = \"Catalog Test\"\ncommand = {}\n[env]\nTOKEN = \"secret-{version}\"\n",
|
||||
serde_json::to_string(&executable.to_string_lossy()).unwrap()
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
fn make_manager(root: &Path, catalogs: Arc<AgentCatalogs>, harness: &Harness) -> AgentManager {
|
||||
make_manager_at(root, &root.join("models.json"), catalogs, harness)
|
||||
}
|
||||
|
||||
fn make_manager_at(root: &Path, models_file: &Path, catalogs: Arc<AgentCatalogs>, harness: &Harness) -> AgentManager {
|
||||
AgentManager::with_connector(
|
||||
AgentPaths {
|
||||
project_root: root.to_path_buf(),
|
||||
store_dir: root.join("store"),
|
||||
worktrees_dir: root.join("worktrees"),
|
||||
models_file: models_file.to_path_buf(),
|
||||
catalogs,
|
||||
custom_agents_dir: Some(root.join("agents")),
|
||||
home_dir: Some(root.to_path_buf()),
|
||||
env_secrets: None,
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
harness.connector.clone(),
|
||||
)
|
||||
}
|
||||
|
||||
fn events(manager: &AgentManager) -> mpsc::UnboundedReceiver<AgentEvent> {
|
||||
let (tx, rx) = mpsc::unbounded_channel();
|
||||
manager.subscribe(Arc::new(move |event| {
|
||||
let _ = tx.send(event);
|
||||
}));
|
||||
rx
|
||||
}
|
||||
|
||||
async fn completed(events: &mut mpsc::UnboundedReceiver<AgentEvent>) -> AgentModelCatalog {
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
loop {
|
||||
if let AgentEvent::ModelCatalog { catalog } = events.recv().await.unwrap()
|
||||
&& !catalog.refreshing
|
||||
{
|
||||
return catalog;
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn response(models: &[&str], effort: &[&str]) -> Value {
|
||||
json!({ "sessionId": "discovery-only", "configOptions": [
|
||||
{ "id": "model", "type": "select", "category": "model", "currentValue": models.first(),
|
||||
"options": models.iter().map(|m| json!({ "value": m, "name": m })).collect::<Vec<_>>() },
|
||||
{ "id": "effort", "type": "select", "category": "thought_level", "currentValue": effort.first(),
|
||||
"options": effort.iter().map(|e| json!({ "value": e })).collect::<Vec<_>>() }
|
||||
] })
|
||||
}
|
||||
|
||||
fn model_values(catalog: &AgentModelCatalog) -> Vec<&str> {
|
||||
catalog
|
||||
.config
|
||||
.iter()
|
||||
.filter(|o| o.category.as_deref() == Some("model"))
|
||||
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
async fn stopped(harness: &Harness) {
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
while harness.live.load(Ordering::SeqCst) != 0 {
|
||||
tokio::task::yield_now().await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
fn advance(catalogs: &AgentCatalogs, seconds: u64) {
|
||||
catalogs.clock_ms.fetch_add(seconds * 1000, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn successful_refresh_replaces_models_and_efforts_and_old_sessions_cannot_revert_it() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let mut events = events(&manager);
|
||||
let old = config::options_of(&response(&["old", "retained"], &["low", "high"]));
|
||||
manager.shared.remember(PROVIDER, Some(&old), None, Some("old"));
|
||||
let cached = manager.refresh_models(PROVIDER, false).unwrap();
|
||||
assert!(cached.refreshing);
|
||||
assert_eq!(model_values(&cached), ["old", "retained"]);
|
||||
assert!(cached.refreshed_at_ms.is_none(), "session reports are not fresh discovery");
|
||||
let pending = harness.next().await;
|
||||
assert_eq!(pending.params["mcpServers"], json!([]));
|
||||
pending.reply.send(Ok(response(&["retained", "new"], &["medium"]))).unwrap();
|
||||
let fresh = completed(&mut events).await;
|
||||
assert_eq!(model_values(&fresh), ["retained", "new"]);
|
||||
assert_eq!(fresh.thought_levels.keys().map(String::as_str).collect::<Vec<_>>(), ["new", "retained"]);
|
||||
assert_eq!(fresh.thought_levels["retained"][0].values[0].value, "medium");
|
||||
assert!(fresh.refreshed_at_ms.is_some() && fresh.retry_at_ms.is_some());
|
||||
manager.shared.remember(PROVIDER, Some(&old), None, Some("old"));
|
||||
assert_eq!(manager.refresh_models(PROVIDER, false).unwrap().config, fresh.config);
|
||||
let provider = manager.providers().into_iter().find(|p| p.id == PROVIDER).unwrap();
|
||||
assert_eq!(provider.config, fresh.config);
|
||||
let persisted = std::fs::read_to_string(tmp.path().join("models.json")).unwrap();
|
||||
assert!(!persisted.contains("secret-one") && !persisted.contains("TOKEN"));
|
||||
let restored = make_manager(tmp.path(), Arc::default(), &harness);
|
||||
let restored_snapshot = restored.refresh_models(PROVIDER, false).unwrap();
|
||||
assert!(!restored_snapshot.refreshing, "matching persisted launch context remains fresh");
|
||||
assert_eq!(restored_snapshot.refreshed_at_ms, fresh.refreshed_at_ms);
|
||||
let provider = restored.providers().into_iter().find(|p| p.id == PROVIDER).unwrap();
|
||||
assert_eq!(provider.config, fresh.config);
|
||||
restored.shared.remember(PROVIDER, Some(&old), None, Some("old"));
|
||||
assert_eq!(restored.shared.known_config(PROVIDER), fresh.config);
|
||||
assert_eq!(restored.shared.known.lock()[PROVIDER].refreshed_at_ms, fresh.refreshed_at_ms);
|
||||
assert!(!tmp.path().join("store").exists(), "refresh never creates tasks");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn legacy_cache_loads_without_claiming_authoritative_freshness() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let options = config::options_of(&response(&["legacy"], &["low"]));
|
||||
std::fs::write(
|
||||
tmp.path().join("models.json"),
|
||||
serde_json::to_vec(&json!({
|
||||
(PROVIDER): { "config": options, "thoughtLevels": {}, "projectCommands": {} }
|
||||
}))
|
||||
.unwrap(),
|
||||
)
|
||||
.unwrap();
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), Arc::default(), &harness);
|
||||
let mut events = events(&manager);
|
||||
let cached = manager.refresh_models(PROVIDER, false).unwrap();
|
||||
assert_eq!(model_values(&cached), ["legacy"]);
|
||||
assert!(cached.refreshing && cached.refreshed_at_ms.is_none());
|
||||
harness.next().await.reply.send(Err("Sign in required".into())).unwrap();
|
||||
let failed = completed(&mut events).await;
|
||||
assert_eq!(failed.config, cached.config);
|
||||
assert!(failed.error.as_deref().unwrap().contains("Sign in required"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn concurrent_windows_commands_and_manual_requests_share_one_flight_and_cooldown() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let first = Arc::new(make_manager(tmp.path(), catalogs.clone(), &harness));
|
||||
let second = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let mut first_events = events(&first);
|
||||
let mut second_events = events(&second);
|
||||
assert!(first.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
let pending = harness.next().await;
|
||||
for _ in 0..10 {
|
||||
assert!(second.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
assert!(first.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
}
|
||||
let commands = tokio::spawn({
|
||||
let first = first.clone();
|
||||
async move { first.commands(PROVIDER, None).await }
|
||||
});
|
||||
tokio::task::yield_now().await;
|
||||
assert_eq!(harness.launches.load(Ordering::SeqCst), 1);
|
||||
pending.reply.send(Ok(response(&["one"], &["low"]))).unwrap();
|
||||
let done = completed(&mut first_events).await;
|
||||
assert_eq!(completed(&mut second_events).await, done);
|
||||
commands.await.unwrap().unwrap();
|
||||
for force in [false, true, false, true] {
|
||||
assert!(!second.refresh_models(PROVIDER, force).unwrap().refreshing);
|
||||
}
|
||||
advance(&catalogs, 29);
|
||||
assert!(!first.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
advance(&catalogs, 1);
|
||||
assert!(!first.refresh_models(PROVIDER, false).unwrap().refreshing, "automatic refresh remains fresh");
|
||||
assert!(first.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
harness.next().await.reply.send(Ok(response(&["two"], &["high"]))).unwrap();
|
||||
let newer = completed(&mut first_events).await;
|
||||
assert_eq!(model_values(&newer), ["two"]);
|
||||
assert_eq!(completed(&mut second_events).await, newer);
|
||||
advance(&catalogs, 15 * 60 - 1);
|
||||
assert!(!second.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
advance(&catalogs, 1);
|
||||
assert!(second.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
harness.next().await.reply.send(Ok(response(&["three"], &["low"]))).unwrap();
|
||||
completed(&mut first_events).await;
|
||||
assert_eq!(harness.launches.load(Ordering::SeqCst), 3);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn failure_keeps_successful_choices_and_throttles_manual_and_auto_retries() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let mut events = events(&manager);
|
||||
manager.refresh_models(PROVIDER, true).unwrap();
|
||||
harness.next().await.reply.send(Ok(response(&["cached"], &["low"]))).unwrap();
|
||||
let success = completed(&mut events).await;
|
||||
let disk = std::fs::read(tmp.path().join("models.json")).unwrap();
|
||||
advance(&catalogs, 15 * 60);
|
||||
manager.refresh_models(PROVIDER, false).unwrap();
|
||||
harness.next().await.reply.send(Err("Authentication expired".into())).unwrap();
|
||||
let failed = completed(&mut events).await;
|
||||
assert_eq!(failed.config, success.config);
|
||||
assert_eq!(failed.thought_levels, success.thought_levels);
|
||||
assert_eq!(failed.refreshed_at_ms, success.refreshed_at_ms);
|
||||
assert!(failed.error.as_deref().unwrap().contains("Authentication expired"));
|
||||
assert_eq!(std::fs::read(tmp.path().join("models.json")).unwrap(), disk);
|
||||
assert!(!manager.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
assert!(!manager.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
advance(&catalogs, 29);
|
||||
assert!(!manager.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
advance(&catalogs, 1);
|
||||
assert!(manager.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
harness.next().await.reply.send(Ok(response(&["recovered"], &["high"]))).unwrap();
|
||||
let recovered = completed(&mut events).await;
|
||||
assert_eq!(model_values(&recovered), ["recovered"]);
|
||||
assert!(recovered.error.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn timeout_cleans_up_and_enters_the_same_failure_cooldown() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let mut events = events(&manager);
|
||||
manager.refresh_models(PROVIDER, false).unwrap();
|
||||
let pending = harness.next().await;
|
||||
tokio::time::pause();
|
||||
tokio::time::advance(Duration::from_secs(4)).await;
|
||||
let failed = completed(&mut events).await;
|
||||
assert!(failed.error.as_deref().unwrap().contains("time"));
|
||||
assert!(failed.retry_at_ms.is_some());
|
||||
assert!(!manager.refresh_models(PROVIDER, true).unwrap().refreshing);
|
||||
assert!(!manager.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
tokio::time::resume();
|
||||
stopped(&harness).await;
|
||||
assert_eq!(harness.live.load(Ordering::SeqCst), 0, "timed-out connection must close");
|
||||
assert!(pending.reply.is_closed());
|
||||
advance(&catalogs, 30);
|
||||
assert!(manager.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
harness.next().await.reply.send(Ok(response(&["retry"], &["low"]))).unwrap();
|
||||
completed(&mut events).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn changed_effective_config_supersedes_old_completion_without_queueing() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let first = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let second = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let mut events = events(&second);
|
||||
first.refresh_models(PROVIDER, false).unwrap();
|
||||
let obsolete = harness.next().await;
|
||||
configure(tmp.path(), "two");
|
||||
assert!(second.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
let latest = harness.next().await;
|
||||
latest.reply.send(Ok(response(&["latest"], &["high"]))).unwrap();
|
||||
let fresh = completed(&mut events).await;
|
||||
let _ = obsolete.reply.send(Ok(response(&["obsolete"], &["low"])));
|
||||
stopped(&harness).await;
|
||||
let shown = first.refresh_models(PROVIDER, false).unwrap();
|
||||
assert_eq!(shown.config, fresh.config);
|
||||
assert_eq!(shown.refreshed_at_ms, fresh.refreshed_at_ms);
|
||||
assert!(!shown.refreshing);
|
||||
assert_eq!(harness.launches.load(Ordering::SeqCst), 2);
|
||||
assert!(!std::fs::read_to_string(tmp.path().join("models.json")).unwrap().contains("obsolete"));
|
||||
// A changed configuration also invalidates successful freshness/cooldown.
|
||||
configure(tmp.path(), "three");
|
||||
assert!(first.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
harness.next().await.reply.send(Ok(response(&["new-config"], &["medium"]))).unwrap();
|
||||
completed(&mut events).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn invalid_provider_is_a_request_error_and_does_not_launch() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), Arc::default(), &harness);
|
||||
assert!(manager.refresh_models("missing-provider", true).is_err());
|
||||
assert_eq!(harness.launches.load(Ordering::SeqCst), 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn replacing_the_executable_at_the_same_path_invalidates_freshness_and_obsolete_responses() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "unchanged-environment");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), catalogs, &harness);
|
||||
let mut events = events(&manager);
|
||||
manager.refresh_models(PROVIDER, false).unwrap();
|
||||
let obsolete = harness.next().await;
|
||||
std::fs::write(tmp.path().join("catalog-test-bin"), "replacement executable with different size").unwrap();
|
||||
manager.refresh_models(PROVIDER, false).unwrap();
|
||||
let latest = harness.next().await;
|
||||
latest.reply.send(Ok(response(&["updated"], &["high"]))).unwrap();
|
||||
let fresh = completed(&mut events).await;
|
||||
let _ = obsolete.reply.send(Ok(response(&["obsolete"], &["low"])));
|
||||
stopped(&harness).await;
|
||||
assert_eq!(manager.refresh_models(PROVIDER, false).unwrap().config, fresh.config);
|
||||
std::fs::write(tmp.path().join("catalog-test-bin"), "another executable").unwrap();
|
||||
assert!(manager.refresh_models(PROVIDER, false).unwrap().refreshing);
|
||||
harness.next().await.reply.send(Ok(response(&["new-install"], &["medium"]))).unwrap();
|
||||
assert_eq!(model_values(&completed(&mut events).await), ["new-install"]);
|
||||
assert_eq!(harness.launches.load(Ordering::SeqCst), 3);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn project_catalogs_and_persisted_freshness_do_not_leak_between_contexts() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let a = tmp.path().join("project-a");
|
||||
let b = tmp.path().join("project-b");
|
||||
configure(&a, "account-a");
|
||||
configure(&b, "account-b");
|
||||
let models_file = tmp.path().join("shared-models.json");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let first = make_manager_at(&a, &models_file, catalogs.clone(), &harness);
|
||||
let second = make_manager_at(&b, &models_file, catalogs, &harness);
|
||||
let mut first_events = events(&first);
|
||||
let mut second_events = events(&second);
|
||||
first.refresh_models(PROVIDER, false).unwrap();
|
||||
harness.next().await.reply.send(Ok(response(&["account-a-model"], &["low"]))).unwrap();
|
||||
let a_catalog = completed(&mut first_events).await;
|
||||
second.refresh_models(PROVIDER, false).unwrap();
|
||||
harness.next().await.reply.send(Ok(response(&["account-b-model"], &["high"]))).unwrap();
|
||||
let b_catalog = completed(&mut second_events).await;
|
||||
assert_eq!(first.refresh_models(PROVIDER, false).unwrap().config, a_catalog.config);
|
||||
assert_eq!(second.refresh_models(PROVIDER, false).unwrap().config, b_catalog.config);
|
||||
assert_eq!(first.providers().into_iter().find(|p| p.id == PROVIDER).unwrap().config, a_catalog.config);
|
||||
for (root, expected) in [(&a, &a_catalog), (&b, &b_catalog)] {
|
||||
let restored = make_manager_at(root, &models_file, Arc::default(), &harness);
|
||||
let cached = restored.refresh_models(PROVIDER, false).unwrap();
|
||||
assert!(!cached.refreshing, "matching persisted context should retain freshness");
|
||||
assert_eq!(cached.config, expected.config);
|
||||
assert_eq!(cached.refreshed_at_ms, expected.refreshed_at_ms);
|
||||
}
|
||||
assert_eq!(harness.launches.load(Ordering::SeqCst), 2);
|
||||
configure(&a, "new-account");
|
||||
let changed = make_manager_at(&a, &models_file, Arc::default(), &harness);
|
||||
let mut events = events(&changed);
|
||||
let stale = changed.refresh_models(PROVIDER, false).unwrap();
|
||||
assert!(stale.refreshing, "a different launch environment cannot reuse persisted freshness");
|
||||
assert!(stale.refreshed_at_ms.is_none());
|
||||
assert_eq!(stale.config, a_catalog.config, "stale cached choices remain selectable");
|
||||
harness.next().await.reply.send(Ok(response(&["new-account-model"], &["medium"]))).unwrap();
|
||||
completed(&mut events).await;
|
||||
let persisted = std::fs::read_to_string(models_file).unwrap();
|
||||
assert!(!persisted.contains("secret-") && !persisted.contains("TOKEN"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn configuration_changed_without_another_request_also_discards_old_discovery() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "before");
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), Arc::default(), &harness);
|
||||
let mut events = events(&manager);
|
||||
manager.refresh_models(PROVIDER, false).unwrap();
|
||||
let obsolete = harness.next().await;
|
||||
configure(tmp.path(), "after");
|
||||
obsolete.reply.send(Ok(response(&["obsolete"], &["low"]))).unwrap();
|
||||
let discarded = completed(&mut events).await;
|
||||
assert!(model_values(&discarded).is_empty());
|
||||
assert!(discarded.refreshed_at_ms.is_none());
|
||||
assert!(!tmp.path().join("models.json").exists());
|
||||
manager.refresh_models(PROVIDER, false).unwrap();
|
||||
harness.next().await.reply.send(Ok(response(&["current"], &["high"]))).unwrap();
|
||||
assert_eq!(model_values(&completed(&mut events).await), ["current"]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn successful_empty_catalog_removes_all_previous_models_and_efforts() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
configure(tmp.path(), "one");
|
||||
let catalogs = Arc::new(AgentCatalogs::default());
|
||||
let mut harness = Harness::new();
|
||||
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
|
||||
let mut events = events(&manager);
|
||||
manager.refresh_models(PROVIDER, true).unwrap();
|
||||
harness.next().await.reply.send(Ok(response(&["removed"], &["low"]))).unwrap();
|
||||
completed(&mut events).await;
|
||||
advance(&catalogs, 30);
|
||||
manager.refresh_models(PROVIDER, true).unwrap();
|
||||
harness.next().await.reply.send(Ok(response(&[], &[]))).unwrap();
|
||||
let empty = completed(&mut events).await;
|
||||
assert!(model_values(&empty).is_empty() && empty.thought_levels.is_empty());
|
||||
assert!(empty.refreshed_at_ms.is_some() && empty.error.is_none());
|
||||
}
|
||||
@@ -8,6 +8,7 @@
|
||||
//! next prompt when the agent supports `session/load`.
|
||||
|
||||
mod acp;
|
||||
mod catalogs;
|
||||
mod config;
|
||||
mod conversation;
|
||||
mod external;
|
||||
@@ -21,6 +22,7 @@ mod skills;
|
||||
mod timeline;
|
||||
mod worktree;
|
||||
|
||||
pub use catalogs::AgentCatalogs;
|
||||
pub use external::ExternalSession;
|
||||
pub use model::*;
|
||||
|
||||
@@ -88,6 +90,8 @@ pub struct AgentPaths {
|
||||
pub worktrees_dir: PathBuf,
|
||||
/// Settings each agent offered last time (shared by all projects), and its commands per project.
|
||||
pub models_file: PathBuf,
|
||||
/// Shared by every project/window on this backend.
|
||||
pub catalogs: Arc<AgentCatalogs>,
|
||||
/// User-defined agents, one `<id>.toml` each (`name`, `command`, `args`, `env`).
|
||||
pub custom_agents_dir: Option<PathBuf>,
|
||||
/// The user's home: its `.claude` holds their own skills and commands.
|
||||
@@ -123,12 +127,10 @@ struct Shared {
|
||||
sink: RwLock<Option<AgentSink>>,
|
||||
security: RwLock<Option<SecurityHook>>,
|
||||
new_task_approvals: RwLock<Option<NewTaskApprovalsHook>>,
|
||||
known: Mutex<HashMap<String, Known>>,
|
||||
known: Arc<Mutex<HashMap<String, Known>>>,
|
||||
providers: RwLock<Vec<provider::ProviderSpec>>,
|
||||
/// Last probe of each provider.
|
||||
probes: Mutex<HashMap<String, AgentProbe>>,
|
||||
/// One pre-session lookup of commands and settings per provider, including failed attempts.
|
||||
command_fetches: Mutex<HashMap<String, tokio::sync::watch::Receiver<bool>>>,
|
||||
/// [`IDLE_RELEASE`], in milliseconds (shorter in tests).
|
||||
idle_release_ms: AtomicU64,
|
||||
mcp_servers_hook: RwLock<Option<Arc<McpServersFn>>>,
|
||||
@@ -145,6 +147,14 @@ struct Known {
|
||||
/// By project root: skills and custom commands differ per project.
|
||||
#[serde(default, rename = "projectCommands")]
|
||||
commands: HashMap<String, Vec<AgentCommand>>,
|
||||
/// Successful discovery, unlike a session report, owns availability.
|
||||
#[serde(default)]
|
||||
authoritative: bool,
|
||||
#[serde(default, rename = "refreshedAtMs")]
|
||||
refreshed_at_ms: Option<f64>,
|
||||
/// Authoritative availability and freshness are scoped to project and launch context.
|
||||
#[serde(default, rename = "projectCatalogs")]
|
||||
catalogs: HashMap<String, catalogs::SavedCatalog>,
|
||||
}
|
||||
|
||||
impl Shared {
|
||||
@@ -171,7 +181,11 @@ impl Shared {
|
||||
}
|
||||
|
||||
fn known_config(&self, provider: &str) -> Vec<AgentConfigOption> {
|
||||
let mut config = self.known.lock().get(provider).map(|k| k.config.clone()).unwrap_or_default();
|
||||
let config = self.paths.catalogs.cached(self, provider).config;
|
||||
self.config_for_new_task(config, provider)
|
||||
}
|
||||
|
||||
fn config_for_new_task(&self, mut config: Vec<AgentConfigOption>, provider: &str) -> Vec<AgentConfigOption> {
|
||||
// Model-specific options are offered separately, never as provider-wide defaults.
|
||||
config.retain(|o| o.category.as_deref() != Some("thought_level"));
|
||||
config::add_client_options(provider, &mut config);
|
||||
@@ -197,80 +211,6 @@ impl Shared {
|
||||
cache.get(provider).and_then(|k| k.commands.get(&self.root_key())).cloned().unwrap_or_default()
|
||||
}
|
||||
|
||||
async fn fetch_settings(self: &Arc<Self>, spec: provider::ProviderSpec) {
|
||||
let mut ready = {
|
||||
let mut fetches = self.command_fetches.lock();
|
||||
fetches
|
||||
.entry(spec.id.clone())
|
||||
.or_insert_with(|| {
|
||||
let (done, ready) = tokio::sync::watch::channel(false);
|
||||
let shared = self.clone();
|
||||
// Keep the lookup alive if a composer unmounts while it is waiting.
|
||||
self.runtime.spawn(async move {
|
||||
tracing::info!("agent {}: settings discovery started", spec.id);
|
||||
let discovered = if spec.driver == provider::Driver::Acp {
|
||||
acp::discover::run(&shared, &spec).await.map(|_| None)
|
||||
} else if spec.driver == provider::Driver::Claude {
|
||||
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await.map(Some)
|
||||
} else {
|
||||
return;
|
||||
};
|
||||
if let Ok(Some(found)) = discovered {
|
||||
let commands =
|
||||
config::commands_of(&json!({ "availableCommands": found.commands }), &spec.id);
|
||||
let options = config::options_of(&json!({ "configOptions": found.options }));
|
||||
let levels: Vec<_> = found
|
||||
.efforts
|
||||
.into_iter()
|
||||
.map(|(model, o)| {
|
||||
let options = o.map(|o| config::options_of(&json!({ "configOptions": [o] })));
|
||||
(model, options.unwrap_or_default())
|
||||
})
|
||||
.collect();
|
||||
let _ = tokio::task::spawn_blocking(move || {
|
||||
// Notifies subscribers itself when the discovery
|
||||
// changed the cached config; unchanged results
|
||||
// stay silent.
|
||||
shared.remember_discovered(&spec.id, &options, &commands, &levels);
|
||||
})
|
||||
.await;
|
||||
} else if let Err(e) = discovered {
|
||||
tracing::info!("agent {}: settings discovery failed: {e}", spec.id);
|
||||
}
|
||||
let _ = done.send(true);
|
||||
});
|
||||
ready
|
||||
})
|
||||
.clone()
|
||||
};
|
||||
while !*ready.borrow_and_update() {
|
||||
if ready.changed().await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn has_config(&self, provider: &str) -> bool {
|
||||
self.known.lock().get(provider).is_some_and(|k| !k.config.is_empty())
|
||||
}
|
||||
|
||||
/// Whether the thinking levels of every model the agent offers are known (#1686).
|
||||
fn knows_all_levels(&self, provider: &str) -> bool {
|
||||
let cache = self.known.lock();
|
||||
let Some(known) = cache.get(provider) else { return false };
|
||||
let models: Vec<&str> = known
|
||||
.config
|
||||
.iter()
|
||||
.filter(|o| o.category.as_deref() == Some("model"))
|
||||
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
|
||||
.collect();
|
||||
// No advertised models yet: nothing to attach effort levels to.
|
||||
if models.is_empty() {
|
||||
return false;
|
||||
}
|
||||
models.iter().all(|id| known.thought_levels.contains_key(*id))
|
||||
}
|
||||
|
||||
fn root_key(&self) -> String {
|
||||
self.paths.project_root.to_string_lossy().into_owned()
|
||||
}
|
||||
@@ -283,78 +223,48 @@ impl Shared {
|
||||
commands: Option<&[AgentCommand]>,
|
||||
model: Option<&str>,
|
||||
) {
|
||||
self.remember_inner(provider, config, commands, model, &[], false);
|
||||
}
|
||||
|
||||
/// `levels`: each model's thinking-level options, so a new task offers them for any model
|
||||
/// it picks (#1686). A session's report for a model wins over these.
|
||||
fn remember_discovered(
|
||||
&self,
|
||||
provider: &str,
|
||||
config: &[AgentConfigOption],
|
||||
commands: &[AgentCommand],
|
||||
levels: &[(String, Vec<AgentConfigOption>)],
|
||||
) {
|
||||
self.remember_inner(provider, Some(config), Some(commands), model_of(config).as_deref(), levels, true);
|
||||
}
|
||||
|
||||
fn remember_inner(
|
||||
&self,
|
||||
provider: &str,
|
||||
config: Option<&[AgentConfigOption]>,
|
||||
commands: Option<&[AgentCommand]>,
|
||||
model: Option<&str>,
|
||||
discovered_levels: &[(String, Vec<AgentConfigOption>)],
|
||||
discovered: bool,
|
||||
) {
|
||||
// Keep the update and its disk write ordered; a session report wins over discovery.
|
||||
let mut cache = self.known.lock();
|
||||
let known = cache.entry(provider.to_owned()).or_default();
|
||||
let mut changed = false;
|
||||
for (model, levels) in discovered_levels {
|
||||
if !known.thought_levels.contains_key(model) {
|
||||
known.thought_levels.insert(model.clone(), levels.clone());
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
if let Some(c) = config.filter(|_| !discovered || known.config.is_empty()) {
|
||||
// Running tasks retain their own reported settings. Once discovery has
|
||||
// succeeded, a session's older list/defaults cannot change New Task choices.
|
||||
if !known.authoritative
|
||||
&& let Some(config) = config.filter(|c| !c.is_empty())
|
||||
{
|
||||
if let Some(model) = model {
|
||||
let levels: Vec<_> =
|
||||
c.iter().filter(|o| o.category.as_deref() == Some("thought_level")).cloned().collect();
|
||||
config.iter().filter(|o| o.category.as_deref() == Some("thought_level")).cloned().collect();
|
||||
if known.thought_levels.get(model) != Some(&levels) {
|
||||
known.thought_levels.insert(model.to_owned(), levels);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
if !c.is_empty() && known.config != c {
|
||||
known.config = c.to_vec();
|
||||
if known.config != config {
|
||||
known.config = config.to_vec();
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
let root = self.root_key();
|
||||
if let Some(c) = commands.filter(|c| {
|
||||
!c.is_empty()
|
||||
&& (!discovered || known.commands.get(&root).is_none_or(Vec::is_empty))
|
||||
&& known.commands.get(&root).map(Vec::as_slice) != Some(c)
|
||||
}) {
|
||||
known.commands.insert(root, c.to_vec());
|
||||
if let Some(commands) = commands.filter(|c| !c.is_empty())
|
||||
&& known.commands.get(&self.root_key()).map(Vec::as_slice) != Some(commands)
|
||||
{
|
||||
known.commands.insert(self.root_key(), commands.to_vec());
|
||||
changed = true;
|
||||
}
|
||||
if !changed {
|
||||
return;
|
||||
if changed {
|
||||
self.persist_known(&cache);
|
||||
}
|
||||
if let Ok(bytes) = serde_json::to_vec_pretty(&*cache)
|
||||
drop(cache);
|
||||
if changed {
|
||||
self.emit(AgentEvent::Providers);
|
||||
}
|
||||
}
|
||||
|
||||
fn persist_known(&self, cache: &HashMap<String, Known>) {
|
||||
if let Ok(bytes) = serde_json::to_vec_pretty(cache)
|
||||
&& let Err(e) = write_file(&self.paths.models_file, &bytes)
|
||||
{
|
||||
tracing::warn!("saving agent settings: {e}");
|
||||
}
|
||||
// The cached provider config actually changed (a session report, discovery,
|
||||
// or a turn update): tell subscribers to re-list, so a changed model list
|
||||
// reaches the composer without waiting for the next task (#1490). Unchanged
|
||||
// reports stay silent, so routine session traffic emits nothing. The cache
|
||||
// lock is released first; the sink never runs under it.
|
||||
drop(cache);
|
||||
self.emit(AgentEvent::Providers);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -469,8 +379,7 @@ impl AgentManager {
|
||||
}
|
||||
|
||||
fn with_connector(paths: AgentPaths, runtime: tokio::runtime::Handle, connector: Connector) -> Self {
|
||||
let known =
|
||||
std::fs::read(&paths.models_file).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default();
|
||||
let known = paths.catalogs.store(&paths.models_file);
|
||||
let providers = RwLock::new(provider::load(paths.custom_agents_dir.as_deref(), paths.env_secrets.as_deref()));
|
||||
let shared = Arc::new(Shared {
|
||||
paths,
|
||||
@@ -479,13 +388,13 @@ impl AgentManager {
|
||||
sink: RwLock::new(None),
|
||||
security: RwLock::new(None),
|
||||
new_task_approvals: RwLock::new(None),
|
||||
known: Mutex::new(known),
|
||||
known,
|
||||
providers,
|
||||
probes: Mutex::new(HashMap::new()),
|
||||
command_fetches: Mutex::new(HashMap::new()),
|
||||
idle_release_ms: AtomicU64::new(IDLE_RELEASE.as_millis() as u64),
|
||||
mcp_servers_hook: RwLock::new(None),
|
||||
});
|
||||
shared.paths.catalogs.subscribe(&shared);
|
||||
let mut tasks = HashMap::new();
|
||||
for file in std::fs::read_dir(&shared.paths.store_dir).into_iter().flatten().flatten() {
|
||||
let path = file.path();
|
||||
@@ -542,9 +451,9 @@ impl AgentManager {
|
||||
.filter(|p| !provider::is_unlisted(p) || in_use.contains(&p.id))
|
||||
.map(|p| {
|
||||
let mut info = provider::describe(p);
|
||||
info.config = self.shared.known_config(&p.id);
|
||||
info.thought_levels =
|
||||
self.shared.known.lock().get(&p.id).map(|k| k.thought_levels.clone()).unwrap_or_default();
|
||||
let catalog = self.shared.paths.catalogs.cached(&self.shared, &p.id);
|
||||
info.config = self.shared.config_for_new_task(catalog.config, &p.id);
|
||||
info.thought_levels = catalog.thought_levels;
|
||||
info.commands = self.shared.known_commands(&p.id);
|
||||
info
|
||||
})
|
||||
@@ -632,17 +541,20 @@ impl AgentManager {
|
||||
// merged without duplicating what `availableCommands` or the disk offer.
|
||||
let mut offered = offered;
|
||||
if task.is_none()
|
||||
&& (offered.is_empty() || !self.shared.has_config(&provider) || !self.shared.knows_all_levels(&provider))
|
||||
&& let Some(spec) = self
|
||||
.shared
|
||||
.spec(&provider)
|
||||
.filter(|s| matches!(s.driver, provider::Driver::Claude | provider::Driver::Acp))
|
||||
&& let Some(spec) =
|
||||
provider::load(self.shared.paths.custom_agents_dir.as_deref(), self.shared.env_secrets())
|
||||
.into_iter()
|
||||
.find(|spec| spec.id == provider)
|
||||
{
|
||||
let claude = spec.driver == provider::Driver::Claude;
|
||||
self.shared.fetch_settings(spec).await;
|
||||
if claude {
|
||||
offered = self.shared.known_commands(&provider);
|
||||
let (_, flight) = self.shared.paths.catalogs.request(&self.shared, spec, false);
|
||||
if let Some(mut ready) = flight {
|
||||
while !*ready.borrow_and_update() {
|
||||
if ready.changed().await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
offered = self.shared.known_commands(&provider);
|
||||
}
|
||||
for &(name, description) in builtin_commands(&provider) {
|
||||
if !offered.iter().any(|c| c.name == name) {
|
||||
@@ -794,16 +706,18 @@ impl AgentManager {
|
||||
}
|
||||
};
|
||||
// Until the session reports its own, show what the agent offered last time with the choices applied.
|
||||
let mut config = self.shared.known_config(&spec.id);
|
||||
let catalog = self.shared.paths.catalogs.cached(&self.shared, &spec.id);
|
||||
let mut levels = catalog.thought_levels;
|
||||
let mut config = self.shared.config_for_new_task(catalog.config, &spec.id);
|
||||
for choice in &req.config {
|
||||
if let Some(o) = config.iter_mut().find(|o| o.id == choice.id) {
|
||||
o.current = Some(choice.value.clone());
|
||||
}
|
||||
}
|
||||
if let Some(model) = model_of(&config)
|
||||
&& let Some(levels) = self.shared.known.lock().get(&spec.id).and_then(|k| k.thought_levels.get(&model))
|
||||
&& let Some(levels) = levels.remove(&model)
|
||||
{
|
||||
config.extend(levels.iter().cloned().map(|mut o| {
|
||||
config.extend(levels.into_iter().map(|mut o| {
|
||||
if let Some(choice) =
|
||||
req.config.iter().find(|c| c.id == o.id && o.values.iter().any(|v| v.value == c.value))
|
||||
{
|
||||
|
||||
@@ -538,6 +538,19 @@ pub struct AgentChanges {
|
||||
pub commits: Vec<AgentCommit>,
|
||||
}
|
||||
|
||||
/// Authoritative choices for new tasks, independent of any running session's settings.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct AgentModelCatalog {
|
||||
pub provider: String,
|
||||
pub config: Vec<AgentConfigOption>,
|
||||
pub thought_levels: BTreeMap<String, Vec<AgentConfigOption>>,
|
||||
pub refreshing: bool,
|
||||
pub refreshed_at_ms: Option<f64>,
|
||||
pub retry_at_ms: Option<f64>,
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
/// Streamed to the window that subscribed (`AgentManager::subscribe`).
|
||||
#[derive(Clone, Debug, Serialize, Type)]
|
||||
#[serde(tag = "type", rename_all = "camelCase", rename_all_fields = "camelCase")]
|
||||
@@ -575,6 +588,10 @@ pub enum AgentEvent {
|
||||
provider: String,
|
||||
limits: Vec<AgentLimit>,
|
||||
},
|
||||
/// A model lookup started or completed, including unsuccessful lookups.
|
||||
ModelCatalog {
|
||||
catalog: AgentModelCatalog,
|
||||
},
|
||||
/// The installed or configured agent providers changed.
|
||||
Providers,
|
||||
}
|
||||
|
||||
@@ -85,6 +85,8 @@ pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<Discove
|
||||
let (spec, cwd) = spec.await.map_err(|e| e.to_string())?;
|
||||
let mut args = spec.args;
|
||||
args.extend(Start::New { id: uuid::Uuid::new_v4().to_string() }.args());
|
||||
// Metadata discovery must not launch project-configured MCP servers.
|
||||
args.extend(["--strict-mcp-config".into(), "--mcp-config".into(), "{\"mcpServers\":{}}".into()]);
|
||||
let launch = Launch { program: spec.command, args, cwd, env: spec.env };
|
||||
let mut io = cli::Process.launch(&launch, Arc::new(|_| {})).map_err(|e| e.to_string())?;
|
||||
let _stop = io.kill.clone().drop_guard();
|
||||
|
||||
@@ -44,6 +44,92 @@ pub(crate) fn connect(
|
||||
})
|
||||
}
|
||||
|
||||
/// A model lookup uses app-server directly: no thread, task, prompt or project MCP.
|
||||
pub(crate) async fn discover(
|
||||
spec: ProviderSpec,
|
||||
cwd: std::path::PathBuf,
|
||||
) -> Result<crate::catalogs::Discovered, String> {
|
||||
let spec = tokio::task::spawn_blocking({
|
||||
let cwd = cwd.clone();
|
||||
move || spec.resolve_for_launch(&cwd)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let peer = Peer::spawn_framed(
|
||||
&spec.command,
|
||||
&spec.args,
|
||||
&cwd,
|
||||
&spec.env,
|
||||
Arc::new(ide_jsonrpc::NoopHandler),
|
||||
Box::new(|_| {}),
|
||||
Framing::Lines,
|
||||
)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let _stop = crate::catalogs::KillPeer(peer.clone());
|
||||
tokio::time::timeout(Duration::from_secs(30), async {
|
||||
let _: Value = peer.request("initialize", initialize_params(), None).await.map_err(|e| e.to_string())?;
|
||||
peer.notify("initialized", Value::Null).map_err(|e| e.to_string())?;
|
||||
let mut models = list_models(&peer).await.map_err(|e| e.message)?;
|
||||
models.retain(|model| !model.hidden);
|
||||
let mut settings = Settings::of(&json!({}), &models);
|
||||
if settings.model.is_empty()
|
||||
&& let Some(model) = models.first()
|
||||
{
|
||||
settings.model = model.id.clone();
|
||||
settings.fit_effort(&models);
|
||||
}
|
||||
let options = translate::config_options(&models, &settings);
|
||||
let config = crate::config::options_of(&json!({ "configOptions": options }));
|
||||
let levels = models
|
||||
.iter()
|
||||
.map(|model| {
|
||||
let mut settings = settings.clone();
|
||||
settings.model = model.id.clone();
|
||||
settings.fit_effort(&models);
|
||||
let options = translate::config_options(&models, &settings);
|
||||
let levels = crate::config::options_of(&json!({ "configOptions": options }))
|
||||
.into_iter()
|
||||
.filter(|o| o.category.as_deref() == Some("thought_level"))
|
||||
.collect();
|
||||
(model.id.clone(), levels)
|
||||
})
|
||||
.collect();
|
||||
Ok(crate::catalogs::Discovered { config, commands: None, levels })
|
||||
})
|
||||
.await
|
||||
.map_err(|_| "Codex model discovery timed out".to_owned())?
|
||||
}
|
||||
|
||||
fn initialize_params() -> Value {
|
||||
json!({
|
||||
"clientInfo": { "name": "openidea", "title": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") },
|
||||
"capabilities": { "experimentalApi": true, "optOutNotificationMethods": OPT_OUT },
|
||||
})
|
||||
}
|
||||
|
||||
async fn list_models(peer: &Peer) -> Result<Vec<ModelInfo>, ResponseError> {
|
||||
let mut all = Vec::new();
|
||||
let mut cursor = Value::Null;
|
||||
let mut seen = HashSet::new();
|
||||
loop {
|
||||
let page: Value = peer.request("model/list", json!({ "cursor": cursor }), None).await.map_err(from_codex)?;
|
||||
let Some(data) = page.get("data").and_then(Value::as_array) else {
|
||||
return Err(error(codes::INTERNAL_ERROR, "Codex returned an invalid model list"));
|
||||
};
|
||||
let models = translate::models_of(&page);
|
||||
if models.len() != data.len() {
|
||||
return Err(error(codes::INTERNAL_ERROR, "Codex returned an invalid model list"));
|
||||
}
|
||||
all.extend(models);
|
||||
match page.get("nextCursor") {
|
||||
None | Some(Value::Null) => return Ok(all),
|
||||
Some(Value::String(next)) if next.is_empty() => return Ok(all),
|
||||
Some(Value::String(next)) if seen.insert(next.clone()) => cursor = json!(next),
|
||||
_ => return Err(error(codes::INTERNAL_ERROR, "Codex returned an invalid model-list cursor")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Connects `client` to a shim driving the codex connection `connect_codex` opens (it
|
||||
/// gets the handler for what codex sends). Each side's end closes the other.
|
||||
fn attach(
|
||||
@@ -250,15 +336,7 @@ impl CodexShim {
|
||||
}
|
||||
|
||||
async fn initialize(&self) -> Result<Value, ResponseError> {
|
||||
let info = self
|
||||
.call(
|
||||
"initialize",
|
||||
json!({
|
||||
"clientInfo": { "name": "openidea", "title": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") },
|
||||
"capabilities": { "experimentalApi": true, "optOutNotificationMethods": OPT_OUT },
|
||||
}),
|
||||
)
|
||||
.await?;
|
||||
let info = self.call("initialize", initialize_params()).await?;
|
||||
self.codex().notify("initialized", Value::Null).map_err(from_codex)?;
|
||||
self.state.lock().initialized = true;
|
||||
// `<client>/<codex version> (<platform>) …`
|
||||
@@ -282,32 +360,15 @@ impl CodexShim {
|
||||
}))
|
||||
}
|
||||
|
||||
/// The models codex offers; empty when it can't list them.
|
||||
async fn models(&self) -> Arc<Vec<ModelInfo>> {
|
||||
/// Cache only complete, successful listings; a failed page must never become availability.
|
||||
async fn models(&self) -> Result<Arc<Vec<ModelInfo>>, ResponseError> {
|
||||
let mut cached = self.models.lock().await;
|
||||
if let Some(m) = cached.as_ref() {
|
||||
return m.clone();
|
||||
if let Some(models) = cached.as_ref() {
|
||||
return Ok(models.clone());
|
||||
}
|
||||
let mut all = Vec::new();
|
||||
let mut cursor = Value::Null;
|
||||
loop {
|
||||
match self.call("model/list", json!({ "cursor": cursor })).await {
|
||||
Ok(page) => {
|
||||
all.extend(translate::models_of(&page));
|
||||
match page.get("nextCursor").filter(|c| c.is_string()) {
|
||||
Some(next) if *next != cursor => cursor = next.clone(),
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!("codex: listing models failed: {}", e.message);
|
||||
return Arc::new(all);
|
||||
}
|
||||
}
|
||||
}
|
||||
let models = Arc::new(all);
|
||||
let models = Arc::new(list_models(self.codex()).await?);
|
||||
*cached = Some(models.clone());
|
||||
models
|
||||
Ok(models)
|
||||
}
|
||||
|
||||
async fn publish_skills(&self, session: &str) {
|
||||
@@ -341,7 +402,7 @@ impl CodexShim {
|
||||
}
|
||||
|
||||
async fn new_session(&self, params: &Value) -> Result<Value, ResponseError> {
|
||||
let models = self.models().await;
|
||||
let models = self.models().await?;
|
||||
let mut request = json!({ "cwd": params.get("cwd") });
|
||||
if let Some(config) = translate::mcp_config(params) {
|
||||
request["config"] = config;
|
||||
@@ -358,7 +419,7 @@ impl CodexShim {
|
||||
/// Resumes the thread and replays its history before answering.
|
||||
async fn load_session(&self, params: &Value) -> Result<Value, ResponseError> {
|
||||
let id = str_param(params, "sessionId")?;
|
||||
let models = self.models().await;
|
||||
let models = self.models().await?;
|
||||
let mut request = json!({ "threadId": id, "cwd": params.get("cwd") });
|
||||
if let Some(config) = translate::mcp_config(params) {
|
||||
request["config"] = config;
|
||||
@@ -376,7 +437,7 @@ impl CodexShim {
|
||||
|
||||
async fn fork_session(&self, params: &Value) -> Result<Value, ResponseError> {
|
||||
let source = str_param(params, "sessionId")?;
|
||||
let models = self.models().await;
|
||||
let models = self.models().await?;
|
||||
let settings = self.state.lock().threads.get(source).and_then(|t| t.settings.clone());
|
||||
let mut request = json!({ "threadId": source, "cwd": params.get("cwd") });
|
||||
if let Some(config) = translate::mcp_config(params) {
|
||||
@@ -397,7 +458,7 @@ impl CodexShim {
|
||||
let session = str_param(params, "sessionId")?;
|
||||
let option = str_param(params, "configId")?;
|
||||
let value = str_param(params, "value")?;
|
||||
let models = self.models().await;
|
||||
let models = self.models().await?;
|
||||
let mut state = self.state.lock();
|
||||
let settings = state
|
||||
.threads
|
||||
@@ -414,7 +475,7 @@ impl CodexShim {
|
||||
if input.is_empty() {
|
||||
return Err(invalid("empty prompt"));
|
||||
}
|
||||
let models = self.models().await;
|
||||
let models = self.models().await?;
|
||||
let (done, ended) = oneshot::channel();
|
||||
let settings = {
|
||||
let mut state = self.state.lock();
|
||||
|
||||
@@ -11,6 +11,155 @@ use std::collections::VecDeque;
|
||||
use std::path::PathBuf;
|
||||
use tokio::io::{AsyncBufReadExt, AsyncRead, AsyncWrite, AsyncWriteExt, BufReader};
|
||||
|
||||
struct ModelPages {
|
||||
pages: Mutex<VecDeque<Result<Value, (i64, String)>>>,
|
||||
cursors: Mutex<Vec<Value>>,
|
||||
}
|
||||
|
||||
impl Handler for ModelPages {
|
||||
fn on_notification(&self, _: &str, _: Value) {}
|
||||
|
||||
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
|
||||
assert_eq!(method, "model/list", "listing models must never start a thread or turn");
|
||||
self.cursors.lock().push(params["cursor"].clone());
|
||||
self.pages.lock().pop_front().expect("another expected page")
|
||||
}
|
||||
}
|
||||
|
||||
fn model_peer(pages: Arc<ModelPages>) -> (Peer, Peer) {
|
||||
let (ours, theirs) = tokio::io::duplex(1 << 16);
|
||||
let (read, write) = tokio::io::split(theirs);
|
||||
let server = Peer::connect_framed(read, write, pages, Framing::Lines);
|
||||
let (read, write) = tokio::io::split(ours);
|
||||
(Peer::connect_framed(read, write, Arc::new(ide_jsonrpc::NoopHandler), Framing::Lines), server)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn model_pagination_failure_does_not_cache_an_incomplete_catalog() {
|
||||
let first = json!({ "data": [{ "id": "first" }], "nextCursor": "next" });
|
||||
let pages = Arc::new(ModelPages {
|
||||
pages: Mutex::new(VecDeque::from([
|
||||
Ok(first.clone()),
|
||||
Err((-32000, "authentication expired".into())),
|
||||
Ok(first),
|
||||
Ok(json!({ "data": [{ "id": "second" }], "nextCursor": null })),
|
||||
])),
|
||||
cursors: Mutex::default(),
|
||||
});
|
||||
let (peer, server) = model_peer(pages.clone());
|
||||
let shim = CodexShim {
|
||||
runtime: tokio::runtime::Handle::current(),
|
||||
codex: OnceLock::from(peer.clone()),
|
||||
client: OnceLock::new(),
|
||||
state: Mutex::default(),
|
||||
models: tokio::sync::Mutex::new(None),
|
||||
skills: tokio::sync::Mutex::new(()),
|
||||
};
|
||||
assert_eq!(shim.models().await.unwrap_err().message, "authentication expired");
|
||||
let models = shim.models().await.unwrap();
|
||||
assert_eq!(models.iter().map(|m| m.id.as_str()).collect::<Vec<_>>(), ["first", "second"]);
|
||||
assert_eq!(shim.models().await.unwrap().len(), 2);
|
||||
assert_eq!(*pages.cursors.lock(), [Value::Null, json!("next"), Value::Null, json!("next")]);
|
||||
peer.kill();
|
||||
server.kill();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn repeated_or_invalid_model_cursors_and_malformed_pages_are_failures() {
|
||||
for pages in [
|
||||
vec![json!({ "data": [], "nextCursor": "a" }), json!({ "data": [], "nextCursor": "a" })],
|
||||
vec![
|
||||
json!({ "data": [], "nextCursor": "a" }),
|
||||
json!({ "data": [], "nextCursor": "b" }),
|
||||
json!({ "data": [], "nextCursor": "a" }),
|
||||
],
|
||||
vec![json!({ "data": [], "nextCursor": 7 })],
|
||||
vec![json!({ "nextCursor": null })],
|
||||
vec![json!({ "data": [{}], "nextCursor": null })],
|
||||
] {
|
||||
let pages =
|
||||
Arc::new(ModelPages { pages: Mutex::new(pages.into_iter().map(Ok).collect()), cursors: Mutex::default() });
|
||||
let (peer, server) = model_peer(pages);
|
||||
assert!(list_models(&peer).await.is_err());
|
||||
peer.kill();
|
||||
server.kill();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn native_discovery_only_initializes_and_pages_models_then_stops_the_process() {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let script = tmp.path().join("fake-codex");
|
||||
std::fs::write(
|
||||
&script,
|
||||
r#"#!/usr/bin/env python3
|
||||
import json, os, pathlib, sys
|
||||
pathlib.Path('pid').write_text(str(os.getpid()))
|
||||
for line in sys.stdin:
|
||||
request = json.loads(line)
|
||||
with pathlib.Path('requests').open('a') as f:
|
||||
f.write(json.dumps(request) + '\n')
|
||||
method = request['method']
|
||||
if method == 'initialized':
|
||||
continue
|
||||
if method == 'initialize':
|
||||
result = {}
|
||||
elif method == 'model/list':
|
||||
cursor = request['params']['cursor']
|
||||
model = 'one' if cursor is None else 'two'
|
||||
result = {'data': [{'id': model, 'isDefault': cursor is None,
|
||||
'defaultReasoningEffort': 'low', 'supportedReasoningEfforts': [{'reasoningEffort': 'low'}]}],
|
||||
'nextCursor': 'second-page' if cursor is None else None}
|
||||
else:
|
||||
raise RuntimeError('must not create a thread or prompt: ' + method)
|
||||
print(json.dumps({'jsonrpc': '2.0', 'id': request['id'], 'result': result}), flush=True)
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap();
|
||||
let spec = ProviderSpec {
|
||||
id: "codex".into(),
|
||||
name: "Codex".into(),
|
||||
driver: crate::provider::Driver::Codex,
|
||||
command: script.to_string_lossy().into_owned(),
|
||||
args: Vec::new(),
|
||||
env: Vec::new(),
|
||||
login_hint: String::new(),
|
||||
source: crate::AgentSource::Builtin,
|
||||
};
|
||||
let found = discover(spec, tmp.path().to_path_buf()).await.unwrap();
|
||||
let models: Vec<_> = found
|
||||
.config
|
||||
.iter()
|
||||
.filter(|o| o.category.as_deref() == Some("model"))
|
||||
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
|
||||
.collect();
|
||||
assert_eq!(models, ["one", "two"]);
|
||||
assert_eq!(found.levels.keys().map(String::as_str).collect::<Vec<_>>(), ["one", "two"]);
|
||||
let requests = std::fs::read_to_string(tmp.path().join("requests")).unwrap();
|
||||
let requests: Vec<Value> = requests.lines().map(|line| serde_json::from_str(line).unwrap()).collect();
|
||||
assert_eq!(
|
||||
requests.iter().map(|r| r["method"].as_str().unwrap()).collect::<Vec<_>>(),
|
||||
["initialize", "initialized", "model/list", "model/list"]
|
||||
);
|
||||
assert_eq!(requests[2]["params"]["cursor"], Value::Null);
|
||||
assert_eq!(requests[3]["params"]["cursor"], "second-page");
|
||||
let pid = std::fs::read_to_string(tmp.path().join("pid")).unwrap();
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
loop {
|
||||
let alive = std::process::Command::new("kill").args(["-0", pid.trim()]).output().unwrap().status.success();
|
||||
if !alive {
|
||||
break;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.expect("discovery must kill and reap its child");
|
||||
}
|
||||
|
||||
// ---- Pure translations -------------------------------------------------------------
|
||||
|
||||
fn models() -> Vec<ModelInfo> {
|
||||
|
||||
@@ -25,6 +25,7 @@ fn paths(dir: &Path) -> AgentPaths {
|
||||
custom_agents_dir: None,
|
||||
home_dir: None,
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -405,6 +405,7 @@ fn manager_at(dir: &Path, models_file: &Path, agent: Arc<FakeAgent>) -> AgentMan
|
||||
// user's profile, when temp lives under it) leaks into the test.
|
||||
home_dir: Some(dir.to_path_buf()),
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
connector,
|
||||
@@ -442,7 +443,7 @@ async fn saved_conversation_reads_do_not_start_or_load_the_agent() {
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn claude_commands_before_session_are_fetched_once_and_saved() {
|
||||
async fn claude_commands_share_catalog_discovery_and_persist_results() {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
|
||||
for outcome in ["success", "session_report", "empty", "settings_timeout", "error", "exit", "timeout", "missing"] {
|
||||
@@ -502,14 +503,18 @@ for line in sys.stdin:
|
||||
std::fs::write(root.join(".claude/skills/triage/SKILL.md"), "---\nname: triage\ndescription: Triage\n---\n")
|
||||
.unwrap();
|
||||
let mgr = manager(root, Arc::new(FakeAgent::default()));
|
||||
{
|
||||
let mut providers = mgr.shared.providers.write();
|
||||
let spec = providers.iter_mut().find(|p| p.id == "claude").unwrap();
|
||||
if outcome != "missing" {
|
||||
spec.command = script.to_string_lossy().into_owned();
|
||||
}
|
||||
spec.env = vec![("OUTCOME".into(), outcome.into())];
|
||||
if outcome != "missing" {
|
||||
provider::set_builtin_command(&root.join("agents"), None, "claude", Some(&script.to_string_lossy()))
|
||||
.unwrap();
|
||||
}
|
||||
provider::set_builtin_env(
|
||||
&root.join("agents"),
|
||||
None,
|
||||
"claude",
|
||||
BTreeMap::from([("OUTCOME".into(), outcome.into())]),
|
||||
)
|
||||
.unwrap();
|
||||
mgr.reload_providers();
|
||||
let concurrent = async {
|
||||
if outcome == "session_report" {
|
||||
tokio::time::timeout(Duration::from_secs(10), async {
|
||||
@@ -553,7 +558,6 @@ for line in sys.stdin:
|
||||
effort.unwrap().current.as_deref(),
|
||||
Some(match outcome {
|
||||
"settings_timeout" => "default",
|
||||
"session_report" => "low",
|
||||
_ => "high",
|
||||
})
|
||||
);
|
||||
@@ -571,6 +575,8 @@ for line in sys.stdin:
|
||||
assert_eq!(launches.lines().count(), 1, "{outcome}");
|
||||
let args: Vec<String> = serde_json::from_str(launches.trim()).unwrap();
|
||||
assert!(args.windows(2).any(|a| a == ["--input-format", "stream-json"]));
|
||||
assert!(args.iter().any(|a| a == "--strict-mcp-config"));
|
||||
assert!(args.windows(2).any(|a| a == ["--mcp-config", "{\"mcpServers\":{}}"]));
|
||||
let requests = std::fs::read_to_string(root.join("requests")).unwrap();
|
||||
let subtypes: Vec<String> = requests
|
||||
.lines()
|
||||
@@ -1431,6 +1437,7 @@ async fn real_claude_task_in_a_worktree() {
|
||||
custom_agents_dir: None,
|
||||
home_dir: None,
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
);
|
||||
@@ -2265,6 +2272,7 @@ async fn reload_providers_updates_registry_install_clears_stale_probes_and_notif
|
||||
custom_agents_dir: Some(agents_dir.clone()),
|
||||
home_dir: None,
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
connector,
|
||||
@@ -2597,6 +2605,7 @@ async fn delete_running_agent(no_load: bool) {
|
||||
custom_agents_dir: None,
|
||||
home_dir: None,
|
||||
env_secrets: None,
|
||||
catalogs: Arc::default(),
|
||||
},
|
||||
tokio::runtime::Handle::current(),
|
||||
connector,
|
||||
@@ -2659,34 +2668,6 @@ fn mcp_servers_follow_server_state_and_agent_capabilities() {
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
|
||||
for discovery_first in [false, true] {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
|
||||
let options = |model: &str, effort: &str| {
|
||||
config::options_of(&json!({ "configOptions": [
|
||||
{ "id": "model", "category": "model", "type": "select", "currentValue": model,
|
||||
"options": [{ "value": "saved" }, { "value": "discovered" }] },
|
||||
{ "id": "effort", "category": "thought_level", "type": "select", "currentValue": effort,
|
||||
"options": [{ "value": "low" }, { "value": "high" }] }
|
||||
] }))
|
||||
};
|
||||
let saved = options("saved", "low");
|
||||
let discovered = options("discovered", "high");
|
||||
if discovery_first {
|
||||
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
|
||||
}
|
||||
mgr.shared.remember("claude", Some(&saved), None, Some("saved"));
|
||||
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
|
||||
let restored = manager(tmp.path(), Arc::new(FakeAgent::default()));
|
||||
for mgr in [&mgr, &restored] {
|
||||
assert_eq!(model_of(&mgr.shared.known_config("claude")).as_deref(), Some("saved"));
|
||||
assert_eq!(mgr.shared.known.lock()["claude"].thought_levels["saved"], saved[1..]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn provider_reload_does_not_start_acp_discovery() {
|
||||
let agent = Arc::new(FakeAgent::default());
|
||||
@@ -2773,31 +2754,6 @@ async fn discovery_times_out_when_session_new_hangs() {
|
||||
assert!(!cursor.config.iter().any(|o| o.category.as_deref() == Some("model")));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
|
||||
let effort = |values: &[&str], current: &str| {
|
||||
config::options_of(&json!({ "configOptions": [
|
||||
{ "id": "effort", "category": "thought_level", "type": "select", "currentValue": current,
|
||||
"options": values.iter().map(|v| json!({ "value": v })).collect::<Vec<_>>() }
|
||||
] }))
|
||||
};
|
||||
let model = config::options_of(&json!({ "configOptions": [
|
||||
{ "id": "model", "category": "model", "type": "select", "currentValue": "sonnet",
|
||||
"options": [{ "value": "sonnet" }, { "value": "opus" }] }
|
||||
] }));
|
||||
// A session already reported sonnet's levels; discovery adds opus's.
|
||||
let reported = [model.clone(), effort(&["low", "high"], "high")].concat();
|
||||
mgr.shared.remember("claude", Some(&reported), None, Some("sonnet"));
|
||||
let levels =
|
||||
[("sonnet".to_owned(), effort(&["low"], "low")), ("opus".to_owned(), effort(&["low", "max"], "default"))];
|
||||
mgr.shared.remember_discovered("claude", &model, &[], &levels);
|
||||
let known = mgr.shared.known.lock();
|
||||
assert_eq!(known["claude"].thought_levels["sonnet"], effort(&["low", "high"], "high"), "the session's report wins");
|
||||
assert_eq!(known["claude"].thought_levels["opus"], effort(&["low", "max"], "default"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn streamed_text_is_sent_as_appends() {
|
||||
let mut streamed = HashMap::new();
|
||||
|
||||
@@ -144,6 +144,13 @@ fn bracket_pos(s: &str) -> Option<(u32, Option<u32>, &str)> {
|
||||
Some((line, col, rest.trim_start()))
|
||||
}
|
||||
|
||||
/// `/C:/p/A.java` → `C:/p/A.java`: Maven and file URIs put a `/` before the drive letter on Windows.
|
||||
/// Joined as is, it would become the drive-relative `C:p/A.java`.
|
||||
fn drive_path(p: &str) -> &str {
|
||||
let b = p.as_bytes();
|
||||
if b.len() > 2 && b[0] == b'/' && b[1].is_ascii_alphabetic() && b[2] == b':' { &p[1..] } else { p }
|
||||
}
|
||||
|
||||
/// `file:///p/A.kt` → `/p/A.kt` (percent-decoded).
|
||||
fn from_file_uri(s: &str) -> String {
|
||||
let Some(rest) = s.strip_prefix("file://") else { return s.to_owned() };
|
||||
@@ -162,9 +169,7 @@ fn from_file_uri(s: &str) -> String {
|
||||
out.push(bytes[i]);
|
||||
i += 1;
|
||||
}
|
||||
let path = String::from_utf8_lossy(&out).into_owned();
|
||||
// `file:///C:/x` on Windows.
|
||||
if path.len() > 2 && path.as_bytes()[0] == b'/' && path.as_bytes()[2] == b':' { path[1..].to_owned() } else { path }
|
||||
drive_path(&String::from_utf8_lossy(&out)).to_owned()
|
||||
}
|
||||
|
||||
/// Kotlin: `file:///p/A.kt:12:5 msg` or `/p/A.kt: (12, 5): msg`, after the `e: ` / `w: ` prefix.
|
||||
@@ -222,7 +227,7 @@ fn maven_line(line: &str) -> Option<RawMessage> {
|
||||
&& looks_like_source_path(path)
|
||||
&& let Some((ln, col, msg)) = bracket_pos(&format!("[{pos}"))
|
||||
{
|
||||
return Some(RawMessage::new(severity, msg, ColUnit::Char).at(path, ln, col));
|
||||
return Some(RawMessage::new(severity, msg, ColUnit::Char).at(drive_path(path), ln, col));
|
||||
}
|
||||
if let Some(m) = kotlin_line(rest) {
|
||||
return Some(m);
|
||||
@@ -456,6 +461,20 @@ mod tests {
|
||||
assert_eq!(out[1].unit, ColUnit::Char);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn maven_windows_paths_drop_the_slash_before_the_drive() {
|
||||
let out = parse(
|
||||
Format::Maven,
|
||||
"[ERROR] /C:/dev/p/src/main/java/a/A.java:[18,41] package a.b does not exist\n\
|
||||
[ERROR] C:\\dev\\p\\B.java:[3,8] cannot find symbol\n\
|
||||
[ERROR] file:///C:/dev/p/K.kt:4:9 Unresolved reference 'x'.\n",
|
||||
);
|
||||
assert_eq!(out.len(), 3, "{out:?}");
|
||||
assert_eq!(pos(&out[0]), (Severity::Error, "C:/dev/p/src/main/java/a/A.java", 18, Some(41)));
|
||||
assert_eq!(pos(&out[1]), (Severity::Error, "C:\\dev\\p\\B.java", 3, Some(8)));
|
||||
assert_eq!(pos(&out[2]), (Severity::Error, "C:/dev/p/K.kt", 4, Some(9)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn maven_detail_lines_with_level_prefix_and_kotlin() {
|
||||
let out = parse(
|
||||
|
||||
@@ -291,6 +291,56 @@ pub enum DebugEvent {
|
||||
JvmBackend {
|
||||
state: JvmBackendState,
|
||||
},
|
||||
/// A step in the life of one request of a debugged page.
|
||||
JsNetwork {
|
||||
event: JsNetworkEvent,
|
||||
},
|
||||
/// The adapter does not report the requests of this page: there will be no `JsNetwork`.
|
||||
JsNetworkUnavailable,
|
||||
}
|
||||
|
||||
/// Which step of a request a [`JsNetworkEvent`] reports.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub enum JsNetworkKind {
|
||||
Request,
|
||||
Response,
|
||||
Finished,
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// The script frame that made a request, where the browser ran it (0-based line and column).
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct JsInitiator {
|
||||
pub function: String,
|
||||
pub url: String,
|
||||
pub line: u32,
|
||||
pub column: u32,
|
||||
}
|
||||
|
||||
/// What a debugged page's browser reports about a request, reduced to what the Network view
|
||||
/// shows. Events of one request share `request_id`; each kind fills its own fields.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct JsNetworkEvent {
|
||||
pub kind: JsNetworkKind,
|
||||
pub request_id: String,
|
||||
/// The browser's monotonic clock, comparable between the events of a page.
|
||||
pub timestamp_ms: u64,
|
||||
/// Request and response.
|
||||
pub url: Option<String>,
|
||||
pub method: Option<String>,
|
||||
/// `Fetch`, `XHR`, `Script`, `Document`, …
|
||||
pub resource_type: Option<String>,
|
||||
pub status: Option<u16>,
|
||||
pub mime_type: Option<String>,
|
||||
/// The request's headers, or the response's.
|
||||
pub headers: BTreeMap<String, String>,
|
||||
/// Bytes received over the network, once finished.
|
||||
pub encoded_length: Option<u64>,
|
||||
pub error_text: Option<String>,
|
||||
pub initiator: Option<JsInitiator>,
|
||||
}
|
||||
|
||||
/// The backend that reads a debugged JVM's objects (instances, fields, referrers).
|
||||
|
||||
@@ -291,7 +291,7 @@ pub enum NewTaskApprovals {
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
|
||||
#[serde(default, rename_all = "camelCase")]
|
||||
pub struct McpServerSettings {
|
||||
/// Serve the tools on 127.0.0.1 while the IDE runs.
|
||||
/// Serve the tools on [`Self::address`] while the IDE runs.
|
||||
pub enabled: bool,
|
||||
/// Run the tools that change the project or start processes without asking first
|
||||
/// (IDEA's brave mode). Read from the IDE's own settings only, never a project's.
|
||||
@@ -300,6 +300,9 @@ pub struct McpServerSettings {
|
||||
pub agent_tab_limit: u32,
|
||||
/// Archived agent tabs kept; older ones are closed.
|
||||
pub agent_archive_limit: u32,
|
||||
/// The IP address to listen on: `127.0.0.1` (this computer only, the default), another
|
||||
/// address of this computer, or `0.0.0.0` / `::` for every address.
|
||||
pub address: String,
|
||||
/// Listen on [`Self::port`] in every run, rather than a free port each time, so HTTP clients and
|
||||
/// port forwards keep working after a restart.
|
||||
pub fixed_port: bool,
|
||||
@@ -312,6 +315,8 @@ pub struct McpServerSettings {
|
||||
|
||||
/// The default of [`McpServerSettings::port`].
|
||||
pub const DEFAULT_MCP_PORT: u16 = 26371;
|
||||
/// The default of [`McpServerSettings::address`]: this computer only.
|
||||
pub const DEFAULT_MCP_ADDRESS: &str = "127.0.0.1";
|
||||
|
||||
impl Default for McpServerSettings {
|
||||
fn default() -> Self {
|
||||
@@ -320,6 +325,7 @@ impl Default for McpServerSettings {
|
||||
run_without_confirmation: false,
|
||||
agent_tab_limit: 1,
|
||||
agent_archive_limit: 999,
|
||||
address: DEFAULT_MCP_ADDRESS.to_owned(),
|
||||
fixed_port: true,
|
||||
port: DEFAULT_MCP_PORT,
|
||||
fixed_token: false,
|
||||
@@ -1153,6 +1159,9 @@ pub struct NodeSettings {
|
||||
pub run_target: String,
|
||||
/// `npm` | `pnpm` | `yarn`; empty = detect from the lock file / `packageManager`.
|
||||
pub package_manager: String,
|
||||
/// Runtime for running JavaScript / TypeScript files: `node` | `bun` | `deno`; empty = from the
|
||||
/// project (`deno.json` → Deno, `bun.lock` → Bun, else Node.js).
|
||||
pub runtime: String,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
|
||||
@@ -1349,11 +1358,15 @@ pub struct TypeScriptSettings {
|
||||
/// `bundled` (the server's own TypeScript), `workspace` (`node_modules/typescript`),
|
||||
/// or a path to a TypeScript `lib` directory.
|
||||
pub version: String,
|
||||
/// tsserver heap limit in MB. `0` = automatic: 8192 when vtsls hosts the Vue
|
||||
/// plugin (hybrid mode; a Vue project's program alone needs ~3 GiB, and the
|
||||
/// vtsls default 3072 dies in a GC death spiral), else unset (vtsls default).
|
||||
pub tsserver_max_memory: u32,
|
||||
}
|
||||
|
||||
impl Default for TypeScriptSettings {
|
||||
fn default() -> Self {
|
||||
Self { version: "bundled".into() }
|
||||
Self { version: "bundled".into(), tsserver_max_memory: 0 }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -260,18 +260,24 @@ pub struct DebuggerSettings {
|
||||
pub show_on_breakpoint: bool,
|
||||
pub focus_app_on_breakpoint: bool,
|
||||
pub hide_on_termination: bool,
|
||||
/// A breakpoint or exception hit brings the Debug tool window's Threads & Variables tab to the front.
|
||||
pub variables_on_breakpoint: bool,
|
||||
pub breakpoints_any_file: bool,
|
||||
/// While suspended, hovering the start of a line shows a Run to Cursor button.
|
||||
pub run_to_cursor_inlay: bool,
|
||||
pub inline_values: bool,
|
||||
/// Data Views: list variables by name instead of the adapter's order.
|
||||
pub sort_values: bool,
|
||||
/// Data Views: evaluate watches while the program runs, where the debugger can (JavaScript).
|
||||
pub live_watches: bool,
|
||||
pub value_tooltip: bool,
|
||||
/// While suspended, selecting an expression shows its value.
|
||||
pub value_tooltip_on_selection: bool,
|
||||
pub value_tooltip_delay: u32,
|
||||
/// Stepping: do not step into library code (debugpy `justMyCode`, js-debug `skipFiles`).
|
||||
pub skip_library_code: bool,
|
||||
/// JavaScript: debugging a dev server (`vite`, `next dev`) also opens its page in a browser session.
|
||||
pub debug_server_page: bool,
|
||||
pub python: PythonDebuggerSettings,
|
||||
}
|
||||
|
||||
@@ -281,14 +287,17 @@ impl Default for DebuggerSettings {
|
||||
show_on_breakpoint: true,
|
||||
focus_app_on_breakpoint: false,
|
||||
hide_on_termination: false,
|
||||
variables_on_breakpoint: true,
|
||||
breakpoints_any_file: false,
|
||||
run_to_cursor_inlay: true,
|
||||
inline_values: true,
|
||||
sort_values: false,
|
||||
live_watches: false,
|
||||
value_tooltip: true,
|
||||
value_tooltip_on_selection: true,
|
||||
value_tooltip_delay: 700,
|
||||
skip_library_code: true,
|
||||
debug_server_page: false,
|
||||
python: PythonDebuggerSettings::default(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
use ide_core::process::executable_names;
|
||||
use ide_core::{IdeError, IdeResult};
|
||||
use ide_language::{DebuggerConfig, LanguageRegistry, ProgramPin};
|
||||
use serde_json::Value;
|
||||
use std::future::Future;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::pin::Pin;
|
||||
@@ -212,6 +213,50 @@ pub fn tools_root() -> Option<PathBuf> {
|
||||
TOOLS_ROOT.read().clone()
|
||||
}
|
||||
|
||||
/// The browser js-debug launches for `ty`, or `None` when the launch starts no browser.
|
||||
fn launched_browser(ty: &str) -> Option<&'static str> {
|
||||
match ty {
|
||||
"pwa-chrome" | "chrome" => Some("chrome"),
|
||||
"pwa-msedge" | "msedge" => Some("msedge"),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Gives a browser launch that names no `userDataDir` a profile of `project`'s own in the
|
||||
/// IDE's local data (`cache/browser-profiles`), so sign-ins and site data are still there in
|
||||
/// the next session. js-debug's own default is one temporary profile for every project.
|
||||
pub(crate) fn set_browser_profile(arguments: &mut Value, project: &Path) {
|
||||
if let Some(local_data) = tools_root().and_then(|tools| tools.parent().map(Path::to_owned)) {
|
||||
set_browser_profile_in(arguments, project, &local_data);
|
||||
}
|
||||
}
|
||||
|
||||
fn set_browser_profile_in(arguments: &mut Value, project: &Path, local_data: &Path) {
|
||||
let Some(obj) = arguments.as_object_mut() else { return };
|
||||
let browser = obj.get("type").and_then(Value::as_str).and_then(launched_browser);
|
||||
let launch = obj.get("request").and_then(Value::as_str) == Some("launch");
|
||||
let (Some(browser), true, false) = (browser, launch, obj.contains_key("userDataDir")) else { return };
|
||||
let dir = local_data.join("cache").join("browser-profiles").join(profile_name(project)).join(browser);
|
||||
obj.insert("userDataDir".into(), dir.to_string_lossy().into_owned().into());
|
||||
}
|
||||
|
||||
/// `<folder name>-<hash of the path>`: readable, and distinct for projects whose folders share a name.
|
||||
fn profile_name(project: &Path) -> String {
|
||||
// FNV-1a: the name has to stay the same across toolchains, which `DefaultHasher` doesn't promise.
|
||||
let hash = project
|
||||
.to_string_lossy()
|
||||
.bytes()
|
||||
.fold(0xcbf2_9ce4_8422_2325_u64, |h, b| (h ^ u64::from(b)).wrapping_mul(0x0000_0100_0000_01b3));
|
||||
let name: String = project
|
||||
.file_name()
|
||||
.map(|n| n.to_string_lossy().into_owned())
|
||||
.unwrap_or_default()
|
||||
.chars()
|
||||
.map(|c| if c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.' { c } else { '_' })
|
||||
.collect();
|
||||
format!("{name}-{hash:016x}")
|
||||
}
|
||||
|
||||
/// Executable directories of managed installs, read at lookup time so tools installed
|
||||
/// after startup are found without a restart: `<tool>/bin`, `<tool>/venv/bin`, npm's `.bin`.
|
||||
fn managed_dirs(root: &Path) -> Vec<PathBuf> {
|
||||
@@ -1040,6 +1085,33 @@ mod tests {
|
||||
assert!(err.message.contains("runs inside the Java language server"), "{}", err.message);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_browser_launch_gets_a_profile_of_its_project() {
|
||||
let local = Path::new("/data/local");
|
||||
let profile = |args: serde_json::Value, project: &str| {
|
||||
let mut args = args;
|
||||
set_browser_profile_in(&mut args, Path::new(project), local);
|
||||
args.get("userDataDir").cloned()
|
||||
};
|
||||
let chrome = serde_json::json!({ "type": "pwa-chrome", "request": "launch" });
|
||||
let dir = profile(chrome.clone(), "/work/shop").unwrap();
|
||||
let dir = Path::new(dir.as_str().unwrap());
|
||||
assert!(dir.starts_with("/data/local/cache/browser-profiles"), "{dir:?}");
|
||||
assert!(dir.ends_with("chrome"), "{dir:?}");
|
||||
let name = dir.parent().unwrap().file_name().unwrap().to_string_lossy().into_owned();
|
||||
assert!(name.starts_with("shop-") && name.len() == "shop-".len() + 16, "{name}");
|
||||
// The same project again: the same profile. Another folder of that name, or Edge: another one.
|
||||
assert_eq!(profile(chrome.clone(), "/work/shop").unwrap(), dir.to_string_lossy().as_ref());
|
||||
assert_ne!(profile(chrome.clone(), "/other/shop").unwrap(), dir.to_string_lossy().as_ref());
|
||||
let edge = profile(serde_json::json!({ "type": "pwa-msedge", "request": "launch" }), "/work/shop").unwrap();
|
||||
assert!(Path::new(edge.as_str().unwrap()).ends_with("msedge"), "{edge}");
|
||||
// A chosen profile (or the default one, `false`) is kept; Node and attach requests get none.
|
||||
let own = serde_json::json!({ "type": "pwa-chrome", "request": "launch", "userDataDir": false });
|
||||
assert_eq!(profile(own, "/work/shop"), Some(false.into()));
|
||||
assert_eq!(profile(serde_json::json!({ "type": "pwa-node", "request": "launch" }), "/work/shop"), None);
|
||||
assert_eq!(profile(serde_json::json!({ "type": "pwa-chrome", "request": "attach" }), "/work/shop"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn port_conflicts_are_recognised_for_retry() {
|
||||
let in_use = IdeError::unavailable(
|
||||
|
||||
@@ -0,0 +1,283 @@
|
||||
//! Requests of vscode-js-debug's own, beyond the Debug Adapter Protocol: what a page in a
|
||||
//! browser offers a debugger besides stepping (breakpoints on DOM events and requests).
|
||||
//! Each is a typed method; a js-debug build that lacks one answers with an error, which is
|
||||
//! reported as the feature being unavailable.
|
||||
|
||||
use crate::session::Session;
|
||||
use crate::{DebugManager, DebugSessionId, REQUEST_TIMEOUT};
|
||||
use ide_core::event::{JsInitiator, JsNetworkEvent, JsNetworkKind};
|
||||
use ide_core::{IdeError, IdeResult};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{Value, json};
|
||||
use specta::Type;
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
pub(crate) const ADAPTER: &str = "js-debug";
|
||||
|
||||
/// Whether `session` debugs a page: the child js-debug starts for a browser launch or attach.
|
||||
/// The launch's own session only owns the browser.
|
||||
pub(crate) fn is_page(session: &Session) -> bool {
|
||||
session.adapter == ADAPTER
|
||||
&& session.parent.is_some()
|
||||
&& matches!(session.launch_type.get().map(String::as_str), Some("pwa-chrome" | "pwa-msedge"))
|
||||
}
|
||||
|
||||
/// Waits until the adapter answered the `launch` of `session`. js-debug sets a page's session
|
||||
/// up between `configurationDone` and that answer; a request of its own that arrives then is
|
||||
/// answered and has no effect.
|
||||
async fn launched(session: &Session) -> bool {
|
||||
let mut launched = session.launched.subscribe();
|
||||
let answered = tokio::time::timeout(REQUEST_TIMEOUT, launched.wait_for(|answered| *answered)).await;
|
||||
matches!(answered, Ok(Ok(_)))
|
||||
}
|
||||
|
||||
impl DebugManager {
|
||||
/// One of js-debug's requests for pages, sent to the page `s` once its session has started.
|
||||
async fn js_page_request(&self, s: DebugSessionId, command: &str, arguments: Value) -> IdeResult<Value> {
|
||||
let session = self.session(s)?;
|
||||
if !is_page(&session) {
|
||||
return Err(IdeError::unavailable("this debug session is not a page in a browser"));
|
||||
}
|
||||
if !launched(&session).await {
|
||||
return Err(IdeError::unavailable("the page's debug session did not finish starting"));
|
||||
}
|
||||
session
|
||||
.conn
|
||||
.request_timeout(command, arguments, REQUEST_TIMEOUT)
|
||||
.await
|
||||
.map_err(|e| IdeError::unavailable(format!("the JavaScript debugger refused `{command}`: {e}")))
|
||||
}
|
||||
|
||||
/// Breakpoints on events (`listener:click`, `instrumentation:setTimeout`) and on requests
|
||||
/// whose URL contains one of `xhr`. The lists replace the page's previous ones.
|
||||
pub async fn js_set_custom_breakpoints(
|
||||
&self,
|
||||
s: DebugSessionId,
|
||||
ids: Vec<String>,
|
||||
xhr: Vec<String>,
|
||||
) -> IdeResult<()> {
|
||||
self.js_page_request(s, "setCustomBreakpoints", json!({ "ids": ids, "xhr": xhr })).await?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// The browser events js-debug is asked to pass on (`enableNetworking`'s `mirrorEvents`).
|
||||
pub(crate) const NETWORK_EVENTS: [&str; 4] =
|
||||
["requestWillBeSent", "responseReceived", "loadingFinished", "loadingFailed"];
|
||||
|
||||
/// Response bodies shown are cut at this many bytes.
|
||||
pub const RESPONSE_BODY_LIMIT: usize = 1_048_576;
|
||||
|
||||
/// The body of a response, as text or as the browser's base64 for binary content.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct JsResponseBody {
|
||||
pub body: String,
|
||||
pub base64_encoded: bool,
|
||||
/// The body was longer than [`RESPONSE_BODY_LIMIT`] and is cut there.
|
||||
pub truncated: bool,
|
||||
}
|
||||
|
||||
fn text(v: &Value, key: &str) -> Option<String> {
|
||||
v.get(key).and_then(Value::as_str).map(str::to_owned)
|
||||
}
|
||||
|
||||
fn headers(v: &Value) -> BTreeMap<String, String> {
|
||||
v.get("headers")
|
||||
.and_then(Value::as_object)
|
||||
.map(|h| h.iter().filter_map(|(k, v)| Some((k.clone(), v.as_str()?.to_owned()))).collect())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// The frame a request is attributed to: the innermost one outside `node_modules`, looked
|
||||
/// for through the stacks that scheduled this one (an `await`, a timer), since a request sent
|
||||
/// through a library has the library innermost. A stack of library frames only gives its first.
|
||||
fn initiator(data: &Value) -> Option<JsInitiator> {
|
||||
let stack = data.pointer("/initiator/stack")?;
|
||||
let frames = |stack: &Value| stack.get("callFrames").and_then(Value::as_array).cloned().unwrap_or_default();
|
||||
let stacks = std::iter::successors(Some(stack), |stack| stack.get("parent")).take(32);
|
||||
let all: Vec<Value> = stacks.flat_map(frames).collect();
|
||||
let own = |frame: &&Value| {
|
||||
frame.get("url").and_then(Value::as_str).is_some_and(|url| !url.is_empty() && !url.contains("/node_modules/"))
|
||||
};
|
||||
let frame = all.iter().find(own).or(all.first())?;
|
||||
let number = |key: &str| frame.get(key).and_then(Value::as_u64).and_then(|n| u32::try_from(n).ok()).unwrap_or(0);
|
||||
Some(JsInitiator {
|
||||
function: text(frame, "functionName").unwrap_or_default(),
|
||||
url: text(frame, "url")?,
|
||||
line: number("lineNumber"),
|
||||
column: number("columnNumber"),
|
||||
})
|
||||
}
|
||||
|
||||
/// js-debug's `networkEvent` (`{event, data}`, `data` being the browser's own parameters of
|
||||
/// that event) as a [`JsNetworkEvent`]; `None` for events the Network view has no use for.
|
||||
pub(crate) fn network_event(body: &Value) -> Option<JsNetworkEvent> {
|
||||
let data = body.get("data")?;
|
||||
let kind = match body.get("event").and_then(Value::as_str)? {
|
||||
"requestWillBeSent" => JsNetworkKind::Request,
|
||||
"responseReceived" => JsNetworkKind::Response,
|
||||
"loadingFinished" => JsNetworkKind::Finished,
|
||||
"loadingFailed" => JsNetworkKind::Failed,
|
||||
_ => return None,
|
||||
};
|
||||
let part = match kind {
|
||||
JsNetworkKind::Request => data.get("request"),
|
||||
JsNetworkKind::Response => data.get("response"),
|
||||
_ => None,
|
||||
};
|
||||
let part = part.unwrap_or(&Value::Null);
|
||||
Some(JsNetworkEvent {
|
||||
kind,
|
||||
request_id: text(data, "requestId")?,
|
||||
// Seconds with a fraction.
|
||||
timestamp_ms: (data.get("timestamp").and_then(Value::as_f64).unwrap_or(0.0).max(0.0) * 1000.0).round() as u64,
|
||||
url: text(part, "url"),
|
||||
method: text(part, "method"),
|
||||
resource_type: text(data, "type"),
|
||||
status: part.get("status").and_then(Value::as_u64).and_then(|s| u16::try_from(s).ok()),
|
||||
mime_type: text(part, "mimeType"),
|
||||
headers: headers(part),
|
||||
encoded_length: data.get("encodedDataLength").and_then(Value::as_f64).map(|n| n.max(0.0) as u64),
|
||||
error_text: text(data, "errorText"),
|
||||
initiator: if kind == JsNetworkKind::Request { initiator(data) } else { None },
|
||||
})
|
||||
}
|
||||
|
||||
/// `networkCall getResponseBody`'s answer, cut at `limit` bytes on a character boundary.
|
||||
fn response_body(answer: &Value, limit: usize) -> JsResponseBody {
|
||||
let mut body = text(answer, "body").unwrap_or_default();
|
||||
let truncated = body.len() > limit;
|
||||
if truncated {
|
||||
let mut end = limit;
|
||||
while !body.is_char_boundary(end) {
|
||||
end -= 1;
|
||||
}
|
||||
body.truncate(end);
|
||||
}
|
||||
JsResponseBody {
|
||||
body,
|
||||
base64_encoded: answer.get("base64Encoded").and_then(Value::as_bool).unwrap_or(false),
|
||||
truncated,
|
||||
}
|
||||
}
|
||||
|
||||
/// The arguments of `enableNetworking`, which has a page report its requests
|
||||
/// ([`ide_core::event::DebugEvent::JsNetwork`]). A page session sends it as it starts.
|
||||
pub(crate) fn networking_arguments() -> Value {
|
||||
json!({ "mirrorEvents": NETWORK_EVENTS })
|
||||
}
|
||||
|
||||
impl DebugManager {
|
||||
/// The body of the response to `request_id`, while the browser still has it.
|
||||
pub async fn js_response_body(&self, s: DebugSessionId, request_id: String) -> IdeResult<JsResponseBody> {
|
||||
let call = json!({ "method": "getResponseBody", "params": { "requestId": request_id } });
|
||||
let answer = self.js_page_request(s, "networkCall", call).await?;
|
||||
Ok(response_body(&answer, RESPONSE_BODY_LIMIT))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn event(kind: &str, data: Value) -> Option<ide_core::event::JsNetworkEvent> {
|
||||
network_event(&json!({ "event": kind, "data": data }))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_request_carries_what_was_asked_and_who_asked() {
|
||||
let e = event(
|
||||
"requestWillBeSent",
|
||||
json!({
|
||||
"requestId": "76695.31", "timestamp": 611807.5, "type": "Fetch",
|
||||
"request": { "method": "GET", "url": "http://127.0.0.1:5195/src/main.ts", "headers": { "Accept": "*/*", "X-N": 1 } },
|
||||
"initiator": { "type": "script", "stack": { "callFrames": [
|
||||
{ "functionName": "load", "url": "http://127.0.0.1:5195/src/App.vue", "lineNumber": 9, "columnNumber": 2 }
|
||||
] } }
|
||||
}),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(e.kind, JsNetworkKind::Request);
|
||||
assert_eq!((e.request_id.as_str(), e.timestamp_ms), ("76695.31", 611_807_500));
|
||||
assert_eq!(e.url.as_deref(), Some("http://127.0.0.1:5195/src/main.ts"));
|
||||
assert_eq!((e.method.as_deref(), e.resource_type.as_deref()), (Some("GET"), Some("Fetch")));
|
||||
// Only text headers are kept.
|
||||
assert_eq!(e.headers.len(), 1);
|
||||
assert_eq!(e.headers["Accept"], "*/*");
|
||||
let from = e.initiator.unwrap();
|
||||
assert_eq!((from.function.as_str(), from.line, from.column), ("load", 9, 2));
|
||||
assert_eq!(from.url, "http://127.0.0.1:5195/src/App.vue");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_request_made_through_a_library_is_asked_by_the_code_that_called_the_library() {
|
||||
let frame = |function: &str, url: &str| json!({ "functionName": function, "url": url, "lineNumber": 1, "columnNumber": 0 });
|
||||
let lib = "http://localhost:5173/node_modules/.vite/deps/http-client.js";
|
||||
let asked_by = |stack: Value| {
|
||||
let data = json!({ "requestId": "1", "timestamp": 1.0, "request": { "method": "GET", "url": "http://localhost/api" },
|
||||
"initiator": { "type": "script", "stack": stack } });
|
||||
event("requestWillBeSent", data).unwrap().initiator.unwrap().function
|
||||
};
|
||||
// The innermost frame is the library's; the project's caller is further out.
|
||||
let direct = json!({ "callFrames": [frame("dispatch", lib), frame("request", lib), frame("loadItems", "http://localhost:5173/src/api.ts")] });
|
||||
assert_eq!(asked_by(direct), "loadItems");
|
||||
// Across an `await`, the caller is in the stack that scheduled this one.
|
||||
let scheduled = json!({ "callFrames": [frame("dispatch", lib)],
|
||||
"parent": { "callFrames": [frame("then", lib)], "parent": { "callFrames": [frame("onMounted", "http://localhost:5173/src/App.vue")] } } });
|
||||
assert_eq!(asked_by(scheduled), "onMounted");
|
||||
// A library on its own: the frame that sent the request.
|
||||
assert_eq!(asked_by(json!({ "callFrames": [frame("dispatch", lib), frame("poll", lib)] })), "dispatch");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_response_a_finish_and_a_failure_carry_their_outcome() {
|
||||
let response = event(
|
||||
"responseReceived",
|
||||
json!({ "requestId": "7", "timestamp": 2.0, "type": "Script", "response": {
|
||||
"url": "http://localhost/a.js", "status": 200, "mimeType": "text/javascript",
|
||||
"headers": { "Content-Type": "text/javascript" }
|
||||
} }),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(response.kind, JsNetworkKind::Response);
|
||||
assert_eq!((response.status, response.mime_type.as_deref()), (Some(200), Some("text/javascript")));
|
||||
assert_eq!(response.headers["Content-Type"], "text/javascript");
|
||||
|
||||
let finished =
|
||||
event("loadingFinished", json!({ "requestId": "7", "timestamp": 2.5, "encodedDataLength": 1234 })).unwrap();
|
||||
assert_eq!((finished.kind, finished.encoded_length), (JsNetworkKind::Finished, Some(1234)));
|
||||
|
||||
let failed =
|
||||
event("loadingFailed", json!({ "requestId": "8", "timestamp": 3.0, "errorText": "net::ERR_FAILED" }))
|
||||
.unwrap();
|
||||
assert_eq!((failed.kind, failed.error_text.as_deref()), (JsNetworkKind::Failed, Some("net::ERR_FAILED")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn events_that_name_no_request_or_are_not_mirrored_are_dropped() {
|
||||
assert!(event("requestWillBeSent", json!({ "timestamp": 1.0 })).is_none());
|
||||
assert!(event("dataReceived", json!({ "requestId": "7", "timestamp": 1.0 })).is_none());
|
||||
assert!(network_event(&json!({ "data": { "requestId": "7" } })).is_none());
|
||||
// A request made by the page's parser has no script behind it.
|
||||
let parser = event(
|
||||
"requestWillBeSent",
|
||||
json!({ "requestId": "9", "timestamp": 1.0, "request": { "method": "GET", "url": "http://localhost/" },
|
||||
"initiator": { "type": "parser" } }),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(parser.initiator.is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_response_body_is_cut_at_the_limit_on_a_character() {
|
||||
let whole = response_body(&json!({ "body": "héllo", "base64Encoded": false }), 100);
|
||||
assert_eq!((whole.body.as_str(), whole.base64_encoded, whole.truncated), ("héllo", false, false));
|
||||
// `é` is two bytes: a cut inside it backs off to the character before.
|
||||
let cut = response_body(&json!({ "body": "héllo" }), 2);
|
||||
assert_eq!((cut.body.as_str(), cut.truncated), ("h", true));
|
||||
let binary = response_body(&json!({ "body": "AAEC", "base64Encoded": true }), 100);
|
||||
assert!(binary.base64_encoded && !binary.truncated);
|
||||
assert_eq!(response_body(&json!({}), 100).body, "");
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,7 @@ pub mod attach;
|
||||
mod cargo;
|
||||
pub mod connection;
|
||||
pub mod jdwp;
|
||||
mod js_debug;
|
||||
mod jvm_memory;
|
||||
pub mod kotlin;
|
||||
pub mod kotlin_lsp;
|
||||
@@ -23,6 +24,7 @@ use connection::{Connection, DapError};
|
||||
use futures::future::join_all;
|
||||
use ide_core::{DebugSessionId, IdeError, IdeResult, SharedSink};
|
||||
use ide_language::LanguageRegistry;
|
||||
pub use js_debug::JsResponseBody;
|
||||
pub use jvm_memory::*;
|
||||
use parking_lot::Mutex;
|
||||
use serde::{Deserialize, Serialize};
|
||||
@@ -136,6 +138,11 @@ pub struct DebugCapabilities {
|
||||
pub jvm_instances: bool,
|
||||
/// The session can be asked for a heap dump ([`DebugManager::jvm_heap_dump`]).
|
||||
pub jvm_heap: bool,
|
||||
/// The session debugs a page in a browser: js-debug's requests for pages apply
|
||||
/// ([`DebugManager::js_set_custom_breakpoints`]).
|
||||
pub js_page: bool,
|
||||
/// The adapter evaluates expressions while the program runs, without a stack frame.
|
||||
pub live_evaluate: bool,
|
||||
}
|
||||
|
||||
/// Result of [`DebugManager::redefine_classes`].
|
||||
@@ -393,6 +400,16 @@ fn no_session(s: DebugSessionId) -> IdeError {
|
||||
IdeError::not_found(format!("no debug session {}", s.0))
|
||||
}
|
||||
|
||||
/// The name of a child session: the adapter's, or the parent's when it gives none. js-debug
|
||||
/// names a browser page's session after the page the browser starts on, before the URL
|
||||
/// loads (`about:blank`, or nothing in a new profile), which says nothing about what is debugged.
|
||||
fn child_name(configuration: &Value, parent: &str) -> String {
|
||||
match configuration.get("name").and_then(Value::as_str) {
|
||||
Some(name) if !name.trim().is_empty() && name != "about:blank" => name.to_owned(),
|
||||
_ => parent.to_owned(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds the `launch`/`attach` arguments: our fields first, then `extra_json` merged on top.
|
||||
pub fn launch_arguments(config: &LaunchConfig) -> IdeResult<Value> {
|
||||
// VS Code puts `type`, `request` and `name` in the arguments; js-debug picks launch
|
||||
@@ -580,6 +597,8 @@ pub(crate) fn capabilities_from(caps: &Value) -> DebugCapabilities {
|
||||
jvm_memory: false,
|
||||
jvm_instances: false,
|
||||
jvm_heap: false,
|
||||
js_page: false,
|
||||
live_evaluate: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -589,6 +608,8 @@ pub(crate) fn session_capabilities(session: &Session) -> DebugCapabilities {
|
||||
caps.jvm_memory = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
|
||||
caps.jvm_instances = session.jvm_inspector.lock().is_some();
|
||||
caps.jvm_heap = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
|
||||
caps.js_page = js_debug::is_page(session);
|
||||
caps.live_evaluate = session.adapter == js_debug::ADAPTER;
|
||||
caps
|
||||
}
|
||||
|
||||
@@ -744,6 +765,9 @@ impl DebugManager {
|
||||
.unwrap_or_else(|| std::env::current_dir().unwrap_or_else(|_| "/".into()));
|
||||
let dbg = adapter::find_config(&self.languages, &config.adapter)
|
||||
.ok_or_else(|| IdeError::unavailable(format!("unknown debug adapter `{}`", config.adapter)))?;
|
||||
if config.adapter == "js-debug" {
|
||||
adapter::set_browser_profile(&mut arguments, &cwd);
|
||||
}
|
||||
let mut spec = adapter::resolve(&dbg, cwd.clone());
|
||||
if config.adapter == kotlin::ADAPTER
|
||||
&& let Some(home) = arguments
|
||||
@@ -872,8 +896,7 @@ impl DebugManager {
|
||||
request: String,
|
||||
configuration: Value,
|
||||
) -> IdeResult<()> {
|
||||
let name =
|
||||
configuration.get("name").and_then(Value::as_str).map(str::to_owned).unwrap_or_else(|| parent.name.clone());
|
||||
let name = child_name(&configuration, &parent.name);
|
||||
let mut configuration = configuration;
|
||||
if let Some(obj) = configuration.as_object_mut() {
|
||||
obj.entry("request").or_insert_with(|| request.clone().into());
|
||||
@@ -921,6 +944,9 @@ impl DebugManager {
|
||||
.flatten();
|
||||
let session =
|
||||
Arc::new(Session::new(id, owner, name, adapter_name, parent, request, conn, spec, io.port, io.child));
|
||||
if let Some(launch_type) = arguments.get("type").and_then(Value::as_str) {
|
||||
let _ = session.launch_type.set(launch_type.to_owned());
|
||||
}
|
||||
// rdbg execs the debuggee in its own process and sends no DAP process event.
|
||||
if let Some(pid) = rdbg_pid {
|
||||
session.debuggee_pid.store(i64::from(pid), Ordering::Release);
|
||||
@@ -1155,6 +1181,22 @@ impl DebugManager {
|
||||
reply.await.unwrap_or(Err(DapError::Closed))?;
|
||||
}
|
||||
}
|
||||
s.launched.send_replace(true);
|
||||
// A page reports its requests from here on. This is the earliest js-debug takes the
|
||||
// request (before its answer to `launch` the page has no adapter and the request is
|
||||
// never answered), and the page is already loading: its document and first scripts
|
||||
// are not in the list.
|
||||
if js_debug::is_page(s) {
|
||||
let answer = s.conn.send_request("enableNetworking", js_debug::networking_arguments())?;
|
||||
let session = s.clone();
|
||||
// Not waited for here: the page runs on. A refusal is told to the Network view,
|
||||
// which would otherwise wait for requests that never come.
|
||||
tokio::spawn(async move {
|
||||
if matches!(answer.await, Ok(Err(_))) {
|
||||
session.emit(ide_core::event::DebugEvent::JsNetworkUnavailable);
|
||||
}
|
||||
});
|
||||
}
|
||||
if let Some(link) = s.jdwp_link.lock().as_ref() {
|
||||
link.release();
|
||||
}
|
||||
|
||||
@@ -57,6 +57,8 @@ pub(crate) struct Session {
|
||||
pub job: Mutex<Option<ide_core::process::Job>>,
|
||||
/// Set once `configurationDone` was sent; breakpoint updates go out live after that.
|
||||
pub configured: AtomicBool,
|
||||
/// Set once the adapter answered `launch` / `attach`, after the configuration.
|
||||
pub launched: watch::Sender<bool>,
|
||||
pub initialized: watch::Sender<bool>,
|
||||
pub startup_cancel: ide_core::CancellationToken,
|
||||
pub output_drained: watch::Sender<bool>,
|
||||
@@ -89,6 +91,9 @@ pub(crate) struct Session {
|
||||
progress: Mutex<HashMap<String, TaskId>>,
|
||||
/// Relay between a Kotlin adapter and its JVM. Dropping it closes both sockets.
|
||||
pub jdwp_link: parking_lot::Mutex<Option<ide_jdwp::JdwpLink>>,
|
||||
/// The `type` of the launch or attach request (js-debug's `pwa-node`, `pwa-chrome`); a child
|
||||
/// session's comes from the configuration the adapter started it with.
|
||||
pub launch_type: std::sync::OnceLock<String>,
|
||||
pub jvm_inspector: parking_lot::Mutex<Option<Arc<dyn ide_jvm_inspect::JvmInspector>>>,
|
||||
/// One inspector request per session at a time.
|
||||
pub inspect_lock: tokio::sync::Mutex<()>,
|
||||
@@ -230,6 +235,7 @@ impl Session {
|
||||
#[cfg(windows)]
|
||||
job: Mutex::new(job),
|
||||
configured: AtomicBool::new(false),
|
||||
launched: watch::channel(false).0,
|
||||
initialized: watch::channel(false).0,
|
||||
startup_cancel: ide_core::CancellationToken::new(),
|
||||
output_drained: watch::channel(true).0,
|
||||
@@ -248,6 +254,7 @@ impl Session {
|
||||
resolved_paths: Mutex::new(HashMap::new()),
|
||||
progress: Mutex::new(HashMap::new()),
|
||||
jdwp_link: parking_lot::Mutex::new(None),
|
||||
launch_type: std::sync::OnceLock::new(),
|
||||
jvm_inspector: parking_lot::Mutex::new(None),
|
||||
inspect_lock: tokio::sync::Mutex::default(),
|
||||
ended: ide_core::CancellationToken::new(),
|
||||
@@ -467,10 +474,7 @@ impl Session {
|
||||
let _ = c.start_kill();
|
||||
}
|
||||
if let Some(mut program) = self.program.lock().take() {
|
||||
let _ = program.child.kill();
|
||||
let _ = std::thread::Builder::new().name("openidea-debug-reap".into()).spawn(move || {
|
||||
let _ = program.child.wait();
|
||||
});
|
||||
program.kill();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -567,7 +571,7 @@ fn launch_groups(s: &Session) -> Vec<libc::pid_t> {
|
||||
for child in s.debuggees.lock().iter() {
|
||||
push(child.id());
|
||||
}
|
||||
push(s.program.lock().as_ref().and_then(|p| p.child.process_id()));
|
||||
push(s.program.lock().as_ref().and_then(ProgramTerminal::process_id));
|
||||
groups.retain(|g| *g > 0);
|
||||
groups
|
||||
}
|
||||
@@ -832,6 +836,12 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
|
||||
.map(|a| a.iter().filter_map(Value::as_str).map(str::to_owned).collect())
|
||||
.unwrap_or_default(),
|
||||
}),
|
||||
// js-debug, after `enableNetworking`.
|
||||
"networkEvent" => {
|
||||
if let Some(event) = crate::js_debug::network_event(&body) {
|
||||
s.emit(DebugEvent::JsNetwork { event });
|
||||
}
|
||||
}
|
||||
_ => tracing::trace!("dap: ignored event {event}"),
|
||||
}
|
||||
}
|
||||
@@ -861,7 +871,10 @@ fn on_reverse_request(mgr: &Arc<DebugManager>, s: &Arc<Session>, seq: i64, comma
|
||||
/// A `runInTerminal` program and the pseudo-terminal it runs in.
|
||||
pub(crate) struct ProgramTerminal {
|
||||
writer: ide_core::input::InputWriter,
|
||||
child: Box<dyn portable_pty::Child + Send + Sync>,
|
||||
killer: Box<dyn portable_pty::ChildKiller + Send + Sync>,
|
||||
pid: Option<u32>,
|
||||
/// Set once the program has exited and been reaped.
|
||||
exited: Arc<AtomicBool>,
|
||||
_master: Box<dyn portable_pty::MasterPty + Send>,
|
||||
}
|
||||
|
||||
@@ -869,6 +882,28 @@ impl ProgramTerminal {
|
||||
pub(crate) fn write(&mut self, text: &str) -> std::io::Result<()> {
|
||||
self.writer.write(text.as_bytes())
|
||||
}
|
||||
|
||||
/// The program's process id while it runs.
|
||||
pub(crate) fn process_id(&self) -> Option<u32> {
|
||||
if self.exited.load(Ordering::Acquire) { None } else { self.pid }
|
||||
}
|
||||
|
||||
pub(crate) fn kill(&mut self) {
|
||||
// After the reap the pid may belong to another process.
|
||||
if !self.exited.load(Ordering::Acquire) {
|
||||
let _ = self.killer.kill();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Reaps the program as soon as it exits. Until then it stays a zombie that `kill(pid, 0)` still
|
||||
/// finds: js-debug polls the `processId` it was given and keeps the session (and its terminal)
|
||||
/// open while that process looks alive, and Stop would wait for it as for a running program.
|
||||
fn reap_on_exit(mut child: Box<dyn portable_pty::Child + Send + Sync>, exited: Arc<AtomicBool>) {
|
||||
let _ = std::thread::Builder::new().name("openidea-debug-reap".into()).spawn(move || {
|
||||
let _ = child.wait();
|
||||
exited.store(true, Ordering::Release);
|
||||
});
|
||||
}
|
||||
|
||||
/// `runInTerminal`: starts the debuggee on a pseudo-terminal, as an IDE terminal would (CodeLLDB
|
||||
@@ -955,8 +990,12 @@ pub(crate) fn run_in_terminal(s: &Arc<Session>, args: &Value) -> Result<Value, S
|
||||
#[cfg(unix)]
|
||||
wait_readable,
|
||||
);
|
||||
if let Some(mut previous) = s.program.lock().replace(ProgramTerminal { writer, child, _master: pair.master }) {
|
||||
let _ = previous.child.kill();
|
||||
let killer = child.clone_killer();
|
||||
let exited = Arc::new(AtomicBool::new(false));
|
||||
reap_on_exit(child, exited.clone());
|
||||
let program = ProgramTerminal { writer, killer, pid, exited, _master: pair.master };
|
||||
if let Some(mut previous) = s.program.lock().replace(program) {
|
||||
previous.kill();
|
||||
}
|
||||
s.owner.sink.emit(IdeEvent::Debug { session: s.id, event: DebugEvent::ProgramInput });
|
||||
Ok(json!({ "processId": pid }))
|
||||
@@ -1325,4 +1364,29 @@ mod tests {
|
||||
assert_eq!(parse_jdwp_port("Listening for transport dt_socket at address: [::1]:51234"), Some(51234));
|
||||
assert_eq!(parse_jdwp_port("unrelated output"), None);
|
||||
}
|
||||
|
||||
/// js-debug polls the `runInTerminal` pid with `kill(pid, 0)`; an unreaped zombie answers it as
|
||||
/// a live process and keeps the session open after the program ends.
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn a_terminal_program_is_reaped_when_it_exits() {
|
||||
use portable_pty::{CommandBuilder, PtySize, native_pty_system};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
let pair =
|
||||
native_pty_system().openpty(PtySize { rows: 24, cols: 80, pixel_width: 0, pixel_height: 0 }).unwrap();
|
||||
let mut cmd = CommandBuilder::new("/bin/sh");
|
||||
cmd.args(["-c", "exit 0"]);
|
||||
let child = pair.slave.spawn_command(cmd).unwrap();
|
||||
let pid = child.process_id().unwrap() as libc::pid_t;
|
||||
let exited = Arc::new(AtomicBool::new(false));
|
||||
super::reap_on_exit(child, exited.clone());
|
||||
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
|
||||
while !exited.load(Ordering::Acquire) {
|
||||
assert!(std::time::Instant::now() < deadline, "program was not reaped");
|
||||
std::thread::sleep(std::time::Duration::from_millis(10));
|
||||
}
|
||||
// SAFETY: signal 0 only checks whether the process exists.
|
||||
assert_eq!(unsafe { libc::kill(pid, 0) }, -1);
|
||||
}
|
||||
}
|
||||
@@ -146,6 +146,10 @@ struct Script {
|
||||
delay_initialize: Option<Duration>,
|
||||
initialized_delay: Duration,
|
||||
launch_delay: Duration,
|
||||
/// Answer a child session's `launch` this long after its `configurationDone`, reading
|
||||
/// requests meanwhile. Like js-debug, one of `accept` that arrives in between is answered
|
||||
/// and has no effect: the log gets a `dropped` entry for it.
|
||||
child_launch_delay: Duration,
|
||||
/// Leave startup pending before `initialized` (false) or after configuration (true).
|
||||
pending_startup: Option<bool>,
|
||||
startup_error: Option<&'static str>,
|
||||
@@ -154,8 +158,15 @@ struct Script {
|
||||
/// `launch`, and a `configurationDone` sent while it runs is refused with `notStopped`.
|
||||
gdb15_starts_at_launch: bool,
|
||||
launch_after_configuration: bool,
|
||||
/// Requests of the adapter's own (js-debug's `setCustomBreakpoints`…) answered with an empty body.
|
||||
accept: &'static [&'static str],
|
||||
/// Requests of the adapter's own answered with a body and followed by events.
|
||||
custom: Option<fn(&Value) -> Option<CustomReply>>,
|
||||
}
|
||||
|
||||
/// The answer to a request of the adapter's own: the response body, then events (name, body).
|
||||
type CustomReply = (Value, Vec<(&'static str, Value)>);
|
||||
|
||||
impl Default for Script {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
@@ -181,11 +192,14 @@ impl Default for Script {
|
||||
delay_initialize: None,
|
||||
initialized_delay: Duration::ZERO,
|
||||
launch_delay: Duration::ZERO,
|
||||
child_launch_delay: Duration::ZERO,
|
||||
pending_startup: None,
|
||||
startup_error: None,
|
||||
close_during_startup: false,
|
||||
gdb15_starts_at_launch: false,
|
||||
launch_after_configuration: false,
|
||||
accept: &[],
|
||||
custom: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -218,10 +232,15 @@ impl AdapterLauncher for FakeLauncher {
|
||||
}
|
||||
|
||||
async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream: S, log: Log, script: Script) {
|
||||
let (r, mut w) = tokio::io::split(stream);
|
||||
let (r, w) = tokio::io::split(stream);
|
||||
// Shared with the task that answers a child's `launch` late (`child_launch_delay`).
|
||||
let w = Arc::new(tokio::sync::Mutex::new(w));
|
||||
let mut r = BufReader::new(r);
|
||||
let mut seq = 1000;
|
||||
let mut pending_launch: Option<i64> = None;
|
||||
let mut child = false;
|
||||
// Between `configurationDone` and the answer to `launch`.
|
||||
let starting = Arc::new(std::sync::atomic::AtomicBool::new(false));
|
||||
let mut held: Vec<Value> = Vec::new();
|
||||
let mut stopped = false;
|
||||
let mut running = false;
|
||||
@@ -230,7 +249,7 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
|
||||
let mut v: Value = $v;
|
||||
seq += 1;
|
||||
v["seq"] = seq.into();
|
||||
framing::write_message(&mut w, &serde_json::to_vec(&v).unwrap()).await.unwrap();
|
||||
framing::write_message(&mut *w.lock().await, &serde_json::to_vec(&v).unwrap()).await.unwrap();
|
||||
}};
|
||||
}
|
||||
macro_rules! ok {
|
||||
@@ -292,6 +311,7 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
|
||||
continue;
|
||||
}
|
||||
pending_launch = msg["seq"].as_i64();
|
||||
child = msg["arguments"]["isChild"] == true;
|
||||
if let Some(args) = &script.run_in_terminal {
|
||||
send!(
|
||||
json!({"type": "request", "command": "runInTerminal", "arguments": {"kind": "integrated", "args": args, "cwd": std::env::temp_dir()}})
|
||||
@@ -299,7 +319,8 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
|
||||
}
|
||||
if script.start_debugging && msg["arguments"]["isChild"] != true {
|
||||
send!(
|
||||
json!({"type": "request", "command": "startDebugging", "arguments": {"request": "launch", "configuration": {"name": "child", "isChild": true}}})
|
||||
// Like js-debug, the child's configuration repeats the launch's `type`.
|
||||
json!({"type": "request", "command": "startDebugging", "arguments": {"request": "launch", "configuration": {"name": "child", "isChild": true, "type": msg["arguments"]["type"]}}})
|
||||
);
|
||||
}
|
||||
if script.pending_startup != Some(false) {
|
||||
@@ -378,7 +399,21 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
|
||||
}
|
||||
tokio::time::sleep(script.launch_delay).await;
|
||||
if let Some(l) = pending_launch.take() {
|
||||
send!(json!({"type": "response", "request_seq": l, "command": "launch", "success": true}));
|
||||
let answer = json!({"type": "response", "request_seq": l, "command": "launch", "success": true});
|
||||
if child && !script.child_launch_delay.is_zero() {
|
||||
starting.store(true, Ordering::Release);
|
||||
let (w, starting, delay) = (w.clone(), starting.clone(), script.child_launch_delay);
|
||||
tokio::spawn(async move {
|
||||
tokio::time::sleep(delay).await;
|
||||
let mut answer = answer;
|
||||
answer["seq"] = 999_999.into();
|
||||
let _ = framing::write_message(&mut *w.lock().await, &serde_json::to_vec(&answer).unwrap())
|
||||
.await;
|
||||
starting.store(false, Ordering::Release);
|
||||
});
|
||||
} else {
|
||||
send!(answer);
|
||||
}
|
||||
}
|
||||
if let Some(pid) = script.debuggee_pid {
|
||||
let event = if script.java_process_id {
|
||||
@@ -474,9 +509,25 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
|
||||
}
|
||||
"disconnect" => {
|
||||
ok!(msg, Value::Null);
|
||||
let _ = w.shutdown().await;
|
||||
let _ = w.lock().await.shutdown().await;
|
||||
break;
|
||||
}
|
||||
other if script.accept.contains(&other) => {
|
||||
if starting.load(Ordering::Acquire) {
|
||||
log.lock().push(json!({"type": "dropped", "command": other}));
|
||||
}
|
||||
ok!(msg, json!({}))
|
||||
}
|
||||
other if script.custom.and_then(|custom| custom(&msg)).is_some() => {
|
||||
if starting.load(Ordering::Acquire) {
|
||||
log.lock().push(json!({"type": "dropped", "command": other}));
|
||||
}
|
||||
let (body, events) = script.custom.and_then(|custom| custom(&msg)).unwrap();
|
||||
ok!(msg, body);
|
||||
for (event, body) in events {
|
||||
send!(json!({"type": "event", "event": event, "body": body}));
|
||||
}
|
||||
}
|
||||
other => send!(
|
||||
json!({"type": "response", "request_seq": msg["seq"], "command": other, "success": false, "message": "unsupported"})
|
||||
),
|
||||
@@ -905,6 +956,162 @@ async fn blocked_program_input_does_not_hold_up_shutdown() {
|
||||
assert!(h.manager.send_input(session, "later").is_err());
|
||||
}
|
||||
|
||||
fn js_config(launch_type: &str) -> LaunchConfig {
|
||||
LaunchConfig {
|
||||
adapter: "js-debug".into(),
|
||||
program: None,
|
||||
args: Vec::new(),
|
||||
extra_json: Some(format!(r#"{{"type": "{launch_type}"}}"#)),
|
||||
..config()
|
||||
}
|
||||
}
|
||||
|
||||
/// The session the adapter started under `parent`.
|
||||
async fn child_of(h: &Harness, parent: DebugSessionId) -> DebugSessionId {
|
||||
for _ in 0..200 {
|
||||
if let Some(child) = h.manager.sessions(Some("main")).into_iter().find(|s| s.parent == Some(parent)) {
|
||||
return child.id;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
panic!("no child session of {parent:?}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_browser_page_session_has_the_page_capability() {
|
||||
let h = harness(Script { start_debugging: true, ..Script::default() });
|
||||
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
|
||||
let page = h.manager.capabilities(child_of(&h, browser).await).unwrap();
|
||||
assert!(page.js_page && page.live_evaluate);
|
||||
assert!(!h.manager.capabilities(browser).unwrap().js_page, "the launch itself is not a page");
|
||||
|
||||
let node = h.start(js_config("pwa-node")).await.unwrap();
|
||||
let process = h.manager.capabilities(child_of(&h, node).await).unwrap();
|
||||
assert!(!process.js_page && process.live_evaluate);
|
||||
|
||||
let python = h.start(config()).await.unwrap();
|
||||
let script = h.manager.capabilities(child_of(&h, python).await).unwrap();
|
||||
assert!(!script.js_page && !script.live_evaluate);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn custom_breakpoints_are_sent_to_a_page_and_refused_elsewhere() {
|
||||
let h = harness(Script { start_debugging: true, accept: &["setCustomBreakpoints"], ..Script::default() });
|
||||
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
|
||||
let page = child_of(&h, browser).await;
|
||||
h.manager.js_set_custom_breakpoints(page, vec!["listener:click".into()], vec!["api/".into()]).await.unwrap();
|
||||
assert_eq!(requests(&h, "setCustomBreakpoints"), [json!({"ids": ["listener:click"], "xhr": ["api/"]})]);
|
||||
|
||||
let not_a_page = h.manager.js_set_custom_breakpoints(browser, Vec::new(), Vec::new()).await.unwrap_err();
|
||||
assert_eq!(not_a_page.kind, ide_core::error::ErrorKind::Unavailable);
|
||||
|
||||
// A js-debug build without the request.
|
||||
let old = harness(Script { start_debugging: true, ..Script::default() });
|
||||
let browser = old.start(js_config("pwa-chrome")).await.unwrap();
|
||||
let page = child_of(&old, browser).await;
|
||||
let refused = old.manager.js_set_custom_breakpoints(page, Vec::new(), Vec::new()).await.unwrap_err();
|
||||
assert_eq!(refused.kind, ide_core::error::ErrorKind::Unavailable);
|
||||
}
|
||||
|
||||
/// js-debug answers a request of its own that reaches a page after `configurationDone` and
|
||||
/// before the page's `launch` is answered, and then does nothing with it.
|
||||
#[tokio::test]
|
||||
async fn a_page_gets_its_requests_once_its_session_has_started() {
|
||||
let h = harness(Script {
|
||||
start_debugging: true,
|
||||
accept: &["setCustomBreakpoints"],
|
||||
child_launch_delay: Duration::from_millis(150),
|
||||
..Script::default()
|
||||
});
|
||||
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
|
||||
let page = child_of(&h, browser).await;
|
||||
// Past the page's `configurationDone`, with its `launch` still unanswered.
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
h.manager.js_set_custom_breakpoints(page, vec!["listener:click".into()], Vec::new()).await.unwrap();
|
||||
assert_eq!(requests(&h, "setCustomBreakpoints"), [json!({"ids": ["listener:click"], "xhr": []})]);
|
||||
let dropped = h.log.lock().iter().filter(|m| m["type"] == "dropped").count();
|
||||
assert_eq!(dropped, 0, "the request reached the page before it took requests");
|
||||
}
|
||||
|
||||
/// js-debug's network requests: `enableNetworking` is followed by one mirrored browser event.
|
||||
fn network_adapter(msg: &Value) -> Option<CustomReply> {
|
||||
match msg["command"].as_str()? {
|
||||
"enableNetworking" => Some((
|
||||
json!({}),
|
||||
vec![(
|
||||
"networkEvent",
|
||||
json!({ "event": "requestWillBeSent", "data": {
|
||||
"requestId": "7", "timestamp": 1.5, "type": "Fetch",
|
||||
"request": { "method": "GET", "url": "http://localhost:5173/api/items", "headers": {} }
|
||||
} }),
|
||||
)],
|
||||
)),
|
||||
"networkCall" => Some((json!({ "body": "é".repeat(600_000), "base64Encoded": false }), Vec::new())),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_page_reports_its_requests_and_their_bodies() {
|
||||
let h = harness(Script {
|
||||
start_debugging: true,
|
||||
custom: Some(network_adapter),
|
||||
child_launch_delay: Duration::from_millis(100),
|
||||
..Script::default()
|
||||
});
|
||||
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
|
||||
let page = child_of(&h, browser).await;
|
||||
|
||||
// Recording is asked for by the session itself, as soon as js-debug takes the request:
|
||||
// once it has answered the page's `launch` (before that, the request is never answered).
|
||||
let DebugEvent::JsNetwork { event } = h.wait_event(|e| matches!(e, DebugEvent::JsNetwork { .. })).await else {
|
||||
unreachable!()
|
||||
};
|
||||
assert_eq!(
|
||||
requests(&h, "enableNetworking"),
|
||||
[json!({ "mirrorEvents": ["requestWillBeSent", "responseReceived", "loadingFinished", "loadingFailed"] })]
|
||||
);
|
||||
let dropped = h.log.lock().iter().filter(|m| m["type"] == "dropped").count();
|
||||
assert_eq!(dropped, 0, "recording was asked for before the page took requests");
|
||||
assert_eq!((event.request_id.as_str(), event.timestamp_ms), ("7", 1500));
|
||||
assert_eq!(event.url.as_deref(), Some("http://localhost:5173/api/items"));
|
||||
|
||||
let body = h.manager.js_response_body(page, "7".into()).await.unwrap();
|
||||
assert_eq!(requests(&h, "networkCall"), [json!({ "method": "getResponseBody", "params": { "requestId": "7" } })]);
|
||||
// 1.2 MB of two-byte characters: cut at the limit, on a character.
|
||||
assert!(body.truncated && !body.base64_encoded);
|
||||
assert_eq!(body.body.len(), 1_048_576);
|
||||
assert!(body.body.chars().all(|c| c == 'é'));
|
||||
|
||||
assert_eq!(
|
||||
h.manager.js_response_body(browser, "7".into()).await.unwrap_err().kind,
|
||||
ide_core::error::ErrorKind::Unavailable
|
||||
);
|
||||
|
||||
// A Node.js program is not a page: nothing is asked of it.
|
||||
let node = h.start(js_config("pwa-node")).await.unwrap();
|
||||
child_of(&h, node).await;
|
||||
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
|
||||
assert_eq!(requests(&h, "enableNetworking").len(), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_page_whose_requests_the_adapter_does_not_report_says_so() {
|
||||
// A js-debug build without `enableNetworking`.
|
||||
let h = harness(Script { start_debugging: true, ..Script::default() });
|
||||
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
|
||||
child_of(&h, browser).await;
|
||||
h.wait_event(|e| matches!(e, DebugEvent::JsNetworkUnavailable)).await;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_child_session_is_named_by_the_adapter_unless_it_is_a_blank_page() {
|
||||
assert_eq!(child_name(&json!({ "name": "server.js [4120]" }), "Server"), "server.js [4120]");
|
||||
assert_eq!(child_name(&json!({ "name": "about:blank" }), "Shop in Chrome"), "Shop in Chrome");
|
||||
assert_eq!(child_name(&json!({ "name": "" }), "Shop in Chrome"), "Shop in Chrome");
|
||||
assert_eq!(child_name(&json!({}), "Shop in Chrome"), "Shop in Chrome");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn launch_arguments_merge_and_defaults() {
|
||||
let mut c = config();
|
||||
|
||||
@@ -324,6 +324,17 @@ fn group_foreign_keys(rows: Vec<Vec<Option<String>>>, tables: &[CompletionTable]
|
||||
/// Connection target: a data source id (its configured database), or
|
||||
/// `<id>:<database>` for another database of the same server. Data source ids
|
||||
/// never contain `:`.
|
||||
/// Snowflake is HTTPS to a named host: a tunnel's local end cannot pass its certificate check.
|
||||
fn tunneled(driver: DbDriver) -> bool {
|
||||
!matches!(driver, DbDriver::Sqlite | DbDriver::Snowflake)
|
||||
}
|
||||
|
||||
/// The database host and port the SSH host forwards to.
|
||||
fn tunnel_target(source: &DataSource) -> (&str, u16) {
|
||||
let port = source.port.or(source.driver.default_port()).unwrap_or_default();
|
||||
(if source.host.trim().is_empty() { "localhost" } else { source.host.trim() }, port)
|
||||
}
|
||||
|
||||
pub fn split_target(target: &str) -> (&str, Option<&str>) {
|
||||
match target.split_once(':') {
|
||||
Some((id, db)) if !db.is_empty() => (id, Some(db)),
|
||||
@@ -655,9 +666,7 @@ impl DbManager {
|
||||
ssh_secret: Option<String>,
|
||||
shared: bool,
|
||||
) -> IdeResult<(DataSource, Option<Arc<Tunnel>>)> {
|
||||
// Snowflake is HTTPS to a named host: a tunnel's local end cannot pass its certificate check.
|
||||
let tunneled = !matches!(source.driver, DbDriver::Sqlite | DbDriver::Snowflake);
|
||||
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && tunneled) else {
|
||||
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && tunneled(source.driver)) else {
|
||||
return Ok((source.clone(), None));
|
||||
};
|
||||
let local = |t: &Tunnel| DataSource { host: "127.0.0.1".into(), port: Some(t.port()), ..source.clone() };
|
||||
@@ -668,8 +677,25 @@ impl DbManager {
|
||||
return Ok((local(&t), Some(t)));
|
||||
}
|
||||
}
|
||||
let session = self.ssh_session(source, ssh, ssh_secret).await?;
|
||||
let (target_host, target_port) = tunnel_target(source);
|
||||
let t = Arc::new(Tunnel::open(session, target_host, target_port).await?);
|
||||
if shared {
|
||||
self.tunnels.lock().insert(source.id.clone(), t.clone());
|
||||
}
|
||||
Ok((local(&t), Some(t)))
|
||||
}
|
||||
|
||||
/// The SSH session the tunnel of `source` goes through: its shared SSH configuration's, or one
|
||||
/// opened with the settings typed in the data source.
|
||||
async fn ssh_session(
|
||||
&self,
|
||||
source: &DataSource,
|
||||
ssh: &DbSsh,
|
||||
ssh_secret: Option<String>,
|
||||
) -> IdeResult<Arc<ide_ssh::Session>> {
|
||||
let config = ssh.config.trim();
|
||||
let session = if config.is_empty() {
|
||||
Ok(if config.is_empty() {
|
||||
let secret = match ssh_secret {
|
||||
Some(s) => Some(s),
|
||||
None => self.secret(source, secrets::ssh_key(&source.id), ssh.save_secret).await,
|
||||
@@ -688,14 +714,22 @@ impl DbManager {
|
||||
self.ssh.set_secret(config, Some(s), save);
|
||||
}
|
||||
self.ssh.session(config).await?
|
||||
};
|
||||
let target_port = source.port.or(source.driver.default_port()).unwrap_or_default();
|
||||
let target_host = if source.host.trim().is_empty() { "localhost" } else { source.host.trim() };
|
||||
let t = Arc::new(Tunnel::open(session, target_host, target_port).await?);
|
||||
if shared {
|
||||
self.tunnels.lock().insert(source.id.clone(), t.clone());
|
||||
}
|
||||
Ok((local(&t), Some(t)))
|
||||
})
|
||||
}
|
||||
|
||||
/// Data Source › SSH/SSL › Test SSH Tunnel: logs in to the SSH host and checks that it reaches
|
||||
/// the database host and port, without logging in to the database.
|
||||
pub async fn test_tunnel(&self, source: &DataSource, ssh_secret: Option<String>) -> IdeResult<String> {
|
||||
let source = &source.resolved()?;
|
||||
let ssh = source
|
||||
.ssh
|
||||
.as_ref()
|
||||
.filter(|s| s.enabled && tunneled(source.driver))
|
||||
.ok_or_else(|| IdeError::invalid("This data source does not use an SSH tunnel"))?;
|
||||
let session = self.ssh_session(source, ssh, ssh_secret).await?;
|
||||
let (host, port) = tunnel_target(source);
|
||||
drop(Tunnel::open(session.clone(), host, port).await?);
|
||||
Ok(format!("{} reaches {host}:{port}", session.label()))
|
||||
}
|
||||
|
||||
/// Connects with `source` as given (not necessarily saved): Test Connection.
|
||||
|
||||
@@ -117,6 +117,14 @@ pub(crate) fn oracle_err(e: &oracledb::Error) -> IdeError {
|
||||
// config first. Treat it as a logon failure, so the UI prompts for one.
|
||||
let kind = match e.kind() {
|
||||
oracledb::ErrorKind::NoCredentials => ErrorKind::PermissionDenied,
|
||||
// The thin driver speaks the protocol of Oracle Database 12c and later only.
|
||||
oracledb::ErrorKind::ServerVersionNotSupported => {
|
||||
return IdeError::new(
|
||||
ErrorKind::Unavailable,
|
||||
"This Oracle Database is older than 12c. OpenIDEA connects to Oracle Database 12c and later; \
|
||||
11g and earlier are not supported.",
|
||||
);
|
||||
}
|
||||
_ => ErrorKind::InvalidInput,
|
||||
};
|
||||
return IdeError::new(kind, e.to_string());
|
||||
@@ -571,6 +579,33 @@ mod tests {
|
||||
assert_eq!(connect_string_with(&s("PROD=MYDB"), false).unwrap(), "PROD=MYDB");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_server_older_than_12c_is_named() {
|
||||
use std::io::{Read, Write};
|
||||
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
let port = listener.local_addr().unwrap().port();
|
||||
let server = std::thread::spawn(move || {
|
||||
let (mut socket, _) = listener.accept().unwrap();
|
||||
let mut connect = [0u8; 1024];
|
||||
let _ = socket.read(&mut connect);
|
||||
// A TNS ACCEPT packet with protocol version 314, as Oracle Database 11g answers.
|
||||
let mut accept = vec![0u8; 32];
|
||||
accept[1] = 32;
|
||||
accept[4] = 2;
|
||||
accept[8..10].copy_from_slice(&314u16.to_be_bytes());
|
||||
let _ = socket.write_all(&accept);
|
||||
});
|
||||
let config = oracledb::Config::default()
|
||||
.set_credentials("scott", "tiger")
|
||||
.set_connect_string(&format!("//127.0.0.1:{port}/ORCL"))
|
||||
.unwrap()
|
||||
.set_transport_connect_timeout(Some(std::time::Duration::from_secs(5)));
|
||||
let err = oracle_err(&oracledb::connect(config).err().expect("an 11g server is refused"));
|
||||
assert_eq!(err.kind, ErrorKind::Unavailable, "{err:?}");
|
||||
assert!(err.message.contains("older than 12c"), "{}", err.message);
|
||||
server.join().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_tns_alias_resolves_when_tnsnames_exists() {
|
||||
let alias = DataSource {
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
//! Data Source › SSH/SSL › Test SSH Tunnel against an in-process SSH server: the SSH login and
|
||||
//! the SSH host reaching the database host and port, with no database login.
|
||||
|
||||
use ide_core::error::ErrorKind;
|
||||
use ide_db::{DataSource, DbDriver, DbManager, DbSsh, DbSshAuth};
|
||||
use ide_ssh::{SshFiles, SshManager};
|
||||
use std::sync::Arc;
|
||||
use tokio::net::TcpListener;
|
||||
|
||||
struct Env {
|
||||
_tmp: tempfile::TempDir,
|
||||
db: DbManager,
|
||||
ssh_port: u16,
|
||||
}
|
||||
|
||||
async fn env() -> Env {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = tmp.path().to_path_buf();
|
||||
let ssh_port = ide_ssh::testing::sshd(root.join("server"), Arc::new(|cmd| panic!("unexpected exec: {cmd}"))).await;
|
||||
let ssh = Arc::new(SshManager::with_files(
|
||||
root.join("ssh-secrets.json"),
|
||||
SshFiles { known_hosts: root.join("known_hosts"), config: None },
|
||||
));
|
||||
let db = DbManager::new(root.clone(), root.join("passwords.json"), root.join("history.json"), ssh);
|
||||
Env { _tmp: tmp, db, ssh_port }
|
||||
}
|
||||
|
||||
impl Env {
|
||||
/// A PostgreSQL source on `127.0.0.1:<db_port>` through the test server as `dev`.
|
||||
fn source(&self, db_port: u16) -> DataSource {
|
||||
DataSource {
|
||||
id: "pg".into(),
|
||||
name: "pg".into(),
|
||||
driver: DbDriver::Postgres,
|
||||
host: "127.0.0.1".into(),
|
||||
port: Some(db_port),
|
||||
user: "app".into(),
|
||||
ssh: Some(DbSsh {
|
||||
enabled: true,
|
||||
host: "127.0.0.1".into(),
|
||||
port: Some(self.ssh_port),
|
||||
user: "dev".into(),
|
||||
auth: DbSshAuth::Password,
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A port something listens on (and never speaks a database protocol).
|
||||
async fn listening() -> (TcpListener, u16) {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let port = listener.local_addr().unwrap().port();
|
||||
(listener, port)
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn the_tunnel_is_tested_without_a_database_login() {
|
||||
let env = env().await;
|
||||
let (_db, db_port) = listening().await;
|
||||
let source = env.source(db_port);
|
||||
|
||||
// First contact: the host key has to be trusted, as with Test Connection.
|
||||
let unknown = env.db.test_tunnel(&source, Some("secret".into())).await.unwrap_err();
|
||||
assert!(unknown.message.starts_with("Unknown SSH host key"), "{unknown:?}");
|
||||
env.db.trust_ssh_host("pg").await.unwrap();
|
||||
|
||||
let ok = env.db.test_tunnel(&source, Some("secret".into())).await.unwrap();
|
||||
assert_eq!(ok, format!("[email protected]:{} reaches 127.0.0.1:{db_port}", env.ssh_port));
|
||||
|
||||
let wrong = env.db.test_tunnel(&source, Some("nope".into())).await.unwrap_err();
|
||||
assert!(wrong.message.contains("SSH authentication failed"), "{}", wrong.message);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn a_database_port_the_ssh_host_cannot_reach_is_named() {
|
||||
let env = env().await;
|
||||
let (listener, closed) = listening().await;
|
||||
drop(listener);
|
||||
let source = env.source(closed);
|
||||
let _ = env.db.test_tunnel(&source, Some("secret".into())).await;
|
||||
env.db.trust_ssh_host("pg").await.unwrap();
|
||||
|
||||
let err = env.db.test_tunnel(&source, Some("secret".into())).await.unwrap_err();
|
||||
assert!(err.message.contains(&format!("cannot reach 127.0.0.1:{closed}")), "{}", err.message);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_source_without_a_tunnel_has_nothing_to_test() {
|
||||
let env = env().await;
|
||||
let mut source = env.source(5432);
|
||||
source.ssh.as_mut().unwrap().enabled = false;
|
||||
let err = env.db.test_tunnel(&source, None).await.unwrap_err();
|
||||
assert_eq!(err.kind, ErrorKind::InvalidInput, "{err:?}");
|
||||
}
|
||||
@@ -66,6 +66,7 @@ pub struct Host {
|
||||
pub mcp_requests: mcp::Requests,
|
||||
/// Commands agents run in the Run tool window (`run_command`).
|
||||
pub mcp_agent_runs: mcp::AgentRuns,
|
||||
pub(crate) agent_catalogs: Arc<ide_agent::AgentCatalogs>,
|
||||
pub trust: trust::TrustStore,
|
||||
pub(crate) mcp_approvals: Arc<trust::McpServerApprovals>,
|
||||
pub config_dir: PathBuf,
|
||||
@@ -152,6 +153,7 @@ impl Host {
|
||||
mcp_reports: Default::default(),
|
||||
mcp_requests: Default::default(),
|
||||
mcp_agent_runs: Default::default(),
|
||||
agent_catalogs: Arc::default(),
|
||||
trust: trust::TrustStore::load(&config_dir, &data_dir),
|
||||
mcp_approvals: Arc::new(trust::McpServerApprovals::load(&data_dir)),
|
||||
proxy_password: ide_ssh::secrets::SecretFile::new(data_dir.join("proxy-password.json")),
|
||||
|
||||
@@ -7,8 +7,8 @@ use crate::rpc::Dispatcher;
|
||||
use ide_agent::registry::{AgentRegistry, RegistryPaths};
|
||||
use ide_agent::{
|
||||
AcpImportOutcome, AgentAcceptOutcome, AgentChange, AgentChanges, AgentChangesView, AgentCommand, AgentEntry,
|
||||
AgentEvent, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask, CustomAgentInput, ExternalSession,
|
||||
McpScope, McpServer, McpServerInput, NewAgentTask,
|
||||
AgentEvent, AgentModelCatalog, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask, CustomAgentInput,
|
||||
ExternalSession, McpScope, McpServer, McpServerInput, NewAgentTask,
|
||||
};
|
||||
use ide_core::error::ErrorKind;
|
||||
use ide_core::{IdeError, IdeEvent, IdeResult, IpcPath, RequestId, SharedSink, TaskId};
|
||||
@@ -28,6 +28,12 @@ pub fn providers(host: &Host, window: &str) -> IdeResult<Vec<AgentProvider>> {
|
||||
Ok(providers)
|
||||
}
|
||||
|
||||
/// Refreshes the selected agent's model catalog without sending a prompt.
|
||||
pub fn refresh_models(host: &Host, window: &str, provider: String, force: bool) -> IdeResult<AgentModelCatalog> {
|
||||
host.require_trust(window)?;
|
||||
host.project(window)?.agents.refresh_models(&provider, force)
|
||||
}
|
||||
|
||||
/// Overrides built-in agent `id`'s command with `command` (its agent file keeps the
|
||||
/// built-in's driver and arguments), or returns it to auto-detection with `None`.
|
||||
pub fn set_command(host: &Host, window: &str, id: String, command: Option<String>) -> IdeResult<()> {
|
||||
@@ -557,6 +563,9 @@ pub fn mcp_import(host: &Host, window: &str) -> IdeResult<Vec<McpServer>> {
|
||||
pub(crate) fn register(d: &mut Dispatcher) {
|
||||
d.add("agent_subscribe", |c| async move { subscribe(&c.host, &c.window, c.channel("onEvent")?) });
|
||||
d.add("agent_providers", |c| async move { providers(&c.host, &c.window) });
|
||||
d.add("agent_refresh_models", |c| async move {
|
||||
refresh_models(&c.host, &c.window, c.arg("provider")?, c.arg("force")?)
|
||||
});
|
||||
d.add("agent_set_command", |c| async move { set_command(&c.host, &c.window, c.arg("id")?, c.arg("command")?) });
|
||||
d.add("agent_set_env", |c| async move { set_env(&c.host, &c.window, c.arg("id")?, c.arg("env")?) });
|
||||
d.add("agent_registry_list", |c| async move { registry_list(&c.host, &c.window, c.arg("request")?).await });
|
||||
|
||||
@@ -12,6 +12,7 @@ use ide_db::{
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use specta::Type;
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// One statement of a console run.
|
||||
#[derive(Clone, Serialize, Deserialize, Type)]
|
||||
@@ -42,6 +43,10 @@ pub(crate) fn register(d: &mut Dispatcher) {
|
||||
);
|
||||
test(&c.host, &c.window, source, password, ssh_secret).await
|
||||
});
|
||||
d.add("db_test_tunnel", |c| async move {
|
||||
let (source, ssh_secret) = (c.arg::<DataSource>("source")?, c.arg::<Option<String>>("sshSecret")?);
|
||||
test_tunnel(&c.host, &c.window, source, ssh_secret).await
|
||||
});
|
||||
d.add("db_set_ssh_secret", |c| async move {
|
||||
let (id, secret, save) = (c.arg::<String>("id")?, c.arg::<Option<String>>("secret")?, c.arg::<bool>("save")?);
|
||||
set_ssh_secret(&c.host, &c.window, &id, secret, save).await
|
||||
@@ -175,6 +180,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
|
||||
apply_edits(&c.host, &c.window, &id, &schema, &table, edits).await
|
||||
});
|
||||
d.add("db_console_path", |c| async move { console_path(&c.host, &c.arg::<String>("id")?) });
|
||||
d.add("db_consoles", |c| async move { consoles(&c.host, &c.arg::<String>("id")?, c.arg("create")?) });
|
||||
}
|
||||
|
||||
/// The data sources configured for the project (the tree's roots).
|
||||
@@ -212,6 +218,17 @@ pub async fn test(
|
||||
db.test(&source, password, ssh_secret).await
|
||||
}
|
||||
|
||||
/// Checks the SSH tunnel of `source` alone: SSH login and the SSH host reaching the database.
|
||||
pub async fn test_tunnel(
|
||||
host: &Host,
|
||||
window: &str,
|
||||
source: DataSource,
|
||||
ssh_secret: Option<String>,
|
||||
) -> IdeResult<String> {
|
||||
let db = host.project(window)?.db.clone();
|
||||
db.test_tunnel(&source, ssh_secret).await
|
||||
}
|
||||
|
||||
/// Remembers the SSH password / key passphrase of a data source (see `set_password`).
|
||||
pub async fn set_ssh_secret(host: &Host, window: &str, id: &str, secret: Option<String>, save: bool) -> IdeResult<()> {
|
||||
let db = host.project(window)?.db.clone();
|
||||
@@ -457,11 +474,52 @@ pub async fn apply_edits(
|
||||
/// created on first use. Consoles are files of the project, so a remote
|
||||
/// window resolves it on the host the project runs on.
|
||||
pub fn console_path(host: &Host, id: &str) -> IdeResult<IpcPath> {
|
||||
let file = console_dir(host, id)?.join("console.sql");
|
||||
if !file.exists() {
|
||||
std::fs::write(&file, "")?;
|
||||
}
|
||||
Ok(file.into())
|
||||
}
|
||||
|
||||
/// The query consoles of data source `id`: the default `console.sql`, then
|
||||
/// `console_1.sql`, `console_2.sql`… in order. `create` adds the next one first
|
||||
/// (DataGrip's New › Query Console).
|
||||
pub fn consoles(host: &Host, id: &str, create: bool) -> IdeResult<Vec<IpcPath>> {
|
||||
let dir = console_dir(host, id)?;
|
||||
let default = dir.join("console.sql");
|
||||
let mut numbered: Vec<(u32, PathBuf)> = std::fs::read_dir(&dir)?
|
||||
.filter_map(|e| {
|
||||
let path = e.ok()?.path();
|
||||
let n = path.file_name()?.to_str()?.strip_prefix("console_")?.strip_suffix(".sql")?.parse().ok()?;
|
||||
Some((n, path))
|
||||
})
|
||||
.collect();
|
||||
numbered.sort();
|
||||
if create {
|
||||
// The default console comes first; once it exists, the next number.
|
||||
if !default.exists() {
|
||||
std::fs::write(&default, "")?;
|
||||
} else {
|
||||
let n = numbered.last().map_or(1, |(n, _)| n + 1);
|
||||
let file = dir.join(format!("console_{n}.sql"));
|
||||
std::fs::write(&file, "")?;
|
||||
numbered.push((n, file));
|
||||
}
|
||||
}
|
||||
let mut out: Vec<IpcPath> = Vec::new();
|
||||
if default.exists() {
|
||||
out.push(default.into());
|
||||
}
|
||||
out.extend(numbered.into_iter().map(|(_, p)| p.into()));
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// `<dataDir>/consoles/db/<id>`: `<id>` for the default database, `<id>.<database>` for the others.
|
||||
fn console_dir(host: &Host, id: &str) -> IdeResult<PathBuf> {
|
||||
let (source, database) = ide_db::split_target(id);
|
||||
if source.is_empty() || !source.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_') {
|
||||
return Err(IdeError::invalid("invalid data source id"));
|
||||
}
|
||||
// One console per database: `<id>` for the default one, `<id>.<database>` for the others.
|
||||
let name = match database {
|
||||
Some(db) => {
|
||||
let safe: String =
|
||||
@@ -472,9 +530,5 @@ pub fn console_path(host: &Host, id: &str) -> IdeResult<IpcPath> {
|
||||
};
|
||||
let dir = host.data_dir.join("consoles").join("db").join(&name);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let file = dir.join("console.sql");
|
||||
if !file.exists() {
|
||||
std::fs::write(&file, "")?;
|
||||
}
|
||||
Ok(file.into())
|
||||
Ok(dir)
|
||||
}
|
||||
@@ -372,6 +372,25 @@ pub async fn debug_restart_frame(host: &Host, _window: &str, session: DebugSessi
|
||||
host.debugger.restart_frame(session, frame_id).await
|
||||
}
|
||||
|
||||
pub async fn debug_js_custom_breakpoints(
|
||||
host: &Host,
|
||||
_window: &str,
|
||||
session: DebugSessionId,
|
||||
ids: Vec<String>,
|
||||
xhr: Vec<String>,
|
||||
) -> IdeResult<()> {
|
||||
host.debugger.js_set_custom_breakpoints(session, ids, xhr).await
|
||||
}
|
||||
|
||||
pub async fn debug_js_response_body(
|
||||
host: &Host,
|
||||
_window: &str,
|
||||
session: DebugSessionId,
|
||||
request_id: String,
|
||||
) -> IdeResult<ide_dap::JsResponseBody> {
|
||||
host.debugger.js_response_body(session, request_id).await
|
||||
}
|
||||
|
||||
pub fn debug_capabilities(host: &Host, _window: &str, session: DebugSessionId) -> IdeResult<DebugCapabilities> {
|
||||
host.debugger.capabilities(session)
|
||||
}
|
||||
@@ -588,6 +607,12 @@ pub(crate) fn register(d: &mut Dispatcher) {
|
||||
debug_restart_frame(&c.host, &c.window, c.arg("session")?, c.arg("frameId")?).await
|
||||
});
|
||||
d.add("debug_capabilities", |c| async move { debug_capabilities(&c.host, &c.window, c.arg("session")?) });
|
||||
d.add("debug_js_response_body", |c| async move {
|
||||
debug_js_response_body(&c.host, &c.window, c.arg("session")?, c.arg("requestId")?).await
|
||||
});
|
||||
d.add("debug_js_custom_breakpoints", |c| async move {
|
||||
debug_js_custom_breakpoints(&c.host, &c.window, c.arg("session")?, c.arg("ids")?, c.arg("xhr")?).await
|
||||
});
|
||||
d.add("debug_set_function_breakpoints", |c| async move {
|
||||
debug_set_function_breakpoints(&c.host, &c.window, c.arg("breakpoints")?).await
|
||||
});
|
||||
|
||||
@@ -74,9 +74,15 @@ pub async fn lsp_completion(
|
||||
host.end_request(request);
|
||||
// Plugin providers (`[[contributes.providers]]`) add theirs.
|
||||
let Some((plugins, d)) = plugin_providers(host, window, doc, ProviderKind::Completion) else { return result };
|
||||
let items = plugin_call(move || plugins.provide_completion(&d, pos, trigger.as_deref())).await.unwrap_or_default();
|
||||
let (covered, items) = plugin_call(move || plugins.provide_completion_for_editor(&d, pos, trigger.as_deref()))
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
if items.is_empty() {
|
||||
return Ok(result.unwrap_or(CompletionList { is_incomplete: true, items: Vec::new() }));
|
||||
return if covered {
|
||||
Ok(result.unwrap_or(CompletionList { is_incomplete: true, items: Vec::new() }))
|
||||
} else {
|
||||
result
|
||||
};
|
||||
}
|
||||
let mut list = result.unwrap_or(CompletionList { is_incomplete: false, items: Vec::new() });
|
||||
list.items.extend(items.into_iter().map(|(_, item)| plugin_completion(item)));
|
||||
|
||||
@@ -31,6 +31,7 @@ pub mod scaffold;
|
||||
pub mod sdk;
|
||||
pub mod search;
|
||||
pub mod spell;
|
||||
mod ssh;
|
||||
pub mod syntax;
|
||||
pub mod term;
|
||||
pub mod tools;
|
||||
@@ -95,6 +96,7 @@ pub fn register_all(d: &mut crate::rpc::Dispatcher) {
|
||||
deploy::register(d);
|
||||
docker::register(d);
|
||||
db::register(d);
|
||||
ssh::register(d);
|
||||
}
|
||||
|
||||
impl Host {
|
||||
|
||||
@@ -2,7 +2,7 @@ use super::doc::blocking;
|
||||
use crate::Host;
|
||||
use crate::Project;
|
||||
use crate::rpc::Dispatcher;
|
||||
use ide_core::{DocId, IdeResult, IpcPath, Location, Pos};
|
||||
use ide_core::{DocId, IdeResult, IpcPath, LanguageId, Location, Pos};
|
||||
use ide_index::{ExtensionCount, FileMatch, IndexStatus, SymbolFilter, SymbolMatch};
|
||||
use ide_language::{FileReference, WebReference};
|
||||
use ide_workspace::file_refs::{self, PathCompletion};
|
||||
@@ -22,6 +22,23 @@ pub struct WebGoto {
|
||||
|
||||
const MAX_WEB_TARGETS: usize = 200;
|
||||
|
||||
/// A Python class definition and the base names written in its source. The UI builds a
|
||||
/// name → locations index from every class in the project (attached dependency roots
|
||||
/// included) to compute ↑ overrides / ↓ overridden gutter markers without a language
|
||||
/// server's type-hierarchy provider.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct ClassInheritance {
|
||||
/// Start of the class definition's selection range (0-based line, UTF-16 column).
|
||||
pub path: IpcPath,
|
||||
pub line: u32,
|
||||
pub col: u32,
|
||||
pub name: String,
|
||||
/// Base names as written in source (bare name, e.g. `AllToAllTokenDispatcher`; a
|
||||
/// qualified name keeps only its last segment).
|
||||
pub bases: Vec<String>,
|
||||
}
|
||||
|
||||
/// The project root and its resource roots (served at `/`), for file references.
|
||||
fn reference_roots(project: &Project) -> (PathBuf, Vec<PathBuf>) {
|
||||
let resources = project.roots().resource_dirs().iter().map(|d| project.root.join(d)).collect();
|
||||
@@ -174,6 +191,74 @@ pub async fn nav_index_definitions(
|
||||
.await
|
||||
}
|
||||
|
||||
/// Every Python class in the project (attached dependency roots included) and its base
|
||||
/// names. The index covers content roots, so this is the whole-project answer the UI uses
|
||||
/// for inheritance gutter markers when no type-hierarchy provider exists.
|
||||
pub async fn nav_inheritance(host: &Host, window: &str) -> IdeResult<Vec<ClassInheritance>> {
|
||||
let project = host.project(window)?;
|
||||
let syntax = host.syntax();
|
||||
blocking(move || {
|
||||
let python = LanguageId::new("python");
|
||||
let files = project.index.files_with_extensions(&["py".into(), "pyi".into()], 20_000);
|
||||
let mut cache = CLASS_BASES.lock();
|
||||
let mut out = Vec::new();
|
||||
for path in files {
|
||||
// The open (possibly unsaved) buffer is keyed by its text, a file on disk by its
|
||||
// size and modification time, so only changed files are parsed again.
|
||||
let open = project.documents.text_of(&path);
|
||||
let stamp = match &open {
|
||||
Some(text) => Stamp::Text(text_hash(text)),
|
||||
None => match std::fs::metadata(&path) {
|
||||
Ok(m) => Stamp::Disk(m.len(), m.modified().ok()),
|
||||
Err(_) => continue,
|
||||
},
|
||||
};
|
||||
let classes = match cache.get(&path) {
|
||||
Some((cached, classes)) if *cached == stamp => classes,
|
||||
_ => {
|
||||
let text = match open {
|
||||
Some(text) => text,
|
||||
None => match project.text_of(&path) {
|
||||
Ok(text) => text,
|
||||
Err(_) => continue,
|
||||
},
|
||||
};
|
||||
let classes = syntax.class_bases(&python, &text);
|
||||
&cache.entry(path.clone()).insert_entry((stamp, classes)).into_mut().1
|
||||
}
|
||||
};
|
||||
out.extend(classes.iter().map(|class| ClassInheritance {
|
||||
path: IpcPath::from(path.clone()),
|
||||
line: class.line,
|
||||
col: class.col,
|
||||
name: class.name.clone(),
|
||||
bases: class.bases.clone(),
|
||||
}));
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
#[derive(Clone, PartialEq, Eq)]
|
||||
enum Stamp {
|
||||
Text(u64),
|
||||
Disk(u64, Option<std::time::SystemTime>),
|
||||
}
|
||||
|
||||
fn text_hash(text: &str) -> u64 {
|
||||
use std::hash::{Hash, Hasher};
|
||||
let mut h = std::collections::hash_map::DefaultHasher::new();
|
||||
text.hash(&mut h);
|
||||
h.finish()
|
||||
}
|
||||
|
||||
type ClassBasesCache = std::collections::HashMap<PathBuf, (Stamp, Vec<ide_language::ClassBases>)>;
|
||||
|
||||
/// Class bases per Python file, reused while the file is unchanged.
|
||||
static CLASS_BASES: std::sync::LazyLock<parking_lot::Mutex<ClassBasesCache>> =
|
||||
std::sync::LazyLock::new(Default::default);
|
||||
|
||||
/// The existing file or directory a path written in `from` names (`href`, `url()`, a string literal).
|
||||
pub async fn fs_resolve_reference(
|
||||
host: &Host,
|
||||
@@ -249,6 +334,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
|
||||
d.add("nav_index_definitions", |c| async move {
|
||||
nav_index_definitions(&c.host, &c.window, c.arg("name")?, c.arg("from")?).await
|
||||
});
|
||||
d.add("nav_inheritance", |c| async move { nav_inheritance(&c.host, &c.window).await });
|
||||
d.add("fs_resolve_reference", |c| async move {
|
||||
fs_resolve_reference(&c.host, &c.window, c.arg("from")?, c.arg("value")?).await
|
||||
});
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
//! SSH configuration commands shared by the desktop shell's IPC commands and the remote
|
||||
//! server ([`crate::rpc`]). A remote window connects from the server (deployment, database
|
||||
//! tunnels, run targets, SSH sessions), so the host keys it trusts and the passwords it is
|
||||
//! given belong to the server's [`ide_ssh::SshManager`], not the desktop's.
|
||||
use crate::rpc::Dispatcher;
|
||||
use ide_core::remote_settings::SshConfiguration;
|
||||
|
||||
pub(crate) fn register(d: &mut Dispatcher) {
|
||||
d.add("ssh_test", |c| async move {
|
||||
let (config, secret) = (c.arg::<SshConfiguration>("config")?, c.arg::<Option<String>>("secret")?);
|
||||
c.host.ssh.test(&config, secret).await
|
||||
});
|
||||
d.add("ssh_set_secret", |c| async move {
|
||||
let (id, secret, save) = (c.arg::<String>("id")?, c.arg::<Option<String>>("secret")?, c.arg::<bool>("save")?);
|
||||
c.host.ssh.set_secret(&id, secret, save);
|
||||
Ok(())
|
||||
});
|
||||
d.add("ssh_password_saved", |c| async move { Ok(c.host.ssh.password_saved(&c.arg::<String>("key")?)) });
|
||||
d.add("ssh_needs_secret", |c| async move {
|
||||
let ssh = &c.host.ssh;
|
||||
Ok(ssh.needs_secret(&ssh.config(&c.arg::<String>("id")?)?))
|
||||
});
|
||||
d.add("ssh_trust_host", |c| async move { c.host.ssh.trust_host(&c.arg::<String>("id")?).await });
|
||||
d.add("ssh_connected", |c| async move { Ok(c.host.ssh.connected()) });
|
||||
d.add("ssh_disconnect", |c| async move {
|
||||
c.host.ssh.disconnect(&c.arg::<String>("id")?);
|
||||
Ok(())
|
||||
});
|
||||
}
|
||||
@@ -22,6 +22,27 @@ pub async fn status(host: &Host, window: &str) -> IdeResult<Vec<FileStatus>> {
|
||||
.await
|
||||
}
|
||||
|
||||
/// Changes of `paths` only: the dirty-scope refresh after a watcher batch. `paths` that belong
|
||||
/// to no repository are ignored, and an empty list is the full status.
|
||||
pub async fn status_paths(host: &Host, window: &str, paths: Vec<std::path::PathBuf>) -> IdeResult<Vec<FileStatus>> {
|
||||
let project = host.project(window)?;
|
||||
host.blocking(move || {
|
||||
let settings = project.settings.effective();
|
||||
if project.is_trusted() {
|
||||
apply_git_executable(&settings);
|
||||
}
|
||||
let repos = project.repos();
|
||||
if repos.is_empty() {
|
||||
return Err(ide_core::IdeError::unavailable("project is not a git repository"));
|
||||
}
|
||||
for r in repos.all() {
|
||||
r.set_ignored_names(settings.ignored_names());
|
||||
}
|
||||
repos.status_paths(&paths)
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
mod commands;
|
||||
pub use commands::*;
|
||||
|
||||
|
||||
@@ -585,6 +585,21 @@ pub async fn vcs_stage(host: &Host, window: &str, paths: Vec<PathBuf>) -> IdeRes
|
||||
blocking(move || per_backend(&project, &paths, None, |r, paths| r.stage(paths))).await
|
||||
}
|
||||
|
||||
/// Adds `commit` to annotate's ignore list in `path`'s repository (the file `blame.ignoreRevsFile`
|
||||
/// names, else `.git-blame-ignore-revs`, which the local config then names).
|
||||
pub async fn vcs_ignore_blame_revision(host: &Host, window: &str, path: PathBuf, commit: String) -> IdeResult<()> {
|
||||
host.require_trust(window)?;
|
||||
let project = host.project(window)?;
|
||||
blocking(move || {
|
||||
let result = project.with_disk_sync(|| project.with_repo_of(&path, |r| r.ignore_blame_revision(&commit)));
|
||||
if result.is_ok() {
|
||||
project.sink.emit(ide_core::IdeEvent::VcsChanged { root: project.root.clone().into() });
|
||||
}
|
||||
result
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
/// Project-host operation (also registered with the remote dispatcher).
|
||||
/// Returns the ignore files that were updated, so the UI can show them.
|
||||
pub async fn vcs_add_to_ignore(
|
||||
@@ -792,12 +807,19 @@ pub async fn vcs_stash_push(
|
||||
message: Option<String>,
|
||||
include_untracked: bool,
|
||||
keep_index: bool,
|
||||
paths: Option<Vec<PathBuf>>,
|
||||
) -> IdeResult<()> {
|
||||
host.require_trust(window)?;
|
||||
let project = host.project(window)?;
|
||||
let paths = paths.unwrap_or_default();
|
||||
blocking(move || {
|
||||
project.with_repo_recorded(repo.as_deref(), Some("Git stash"), |r| {
|
||||
r.stash_push(message.as_deref(), include_untracked, keep_index)
|
||||
if paths.is_empty() {
|
||||
return project.with_repo_recorded(repo.as_deref(), Some("Git stash"), |r| {
|
||||
r.stash_push(message.as_deref(), include_untracked, keep_index, &[])
|
||||
});
|
||||
}
|
||||
per_repo(&project, &paths, Some("Git stash"), |r, paths| {
|
||||
r.stash_push(message.as_deref(), include_untracked, keep_index, paths)
|
||||
})
|
||||
})
|
||||
.await
|
||||
@@ -1462,6 +1484,9 @@ pub(crate) fn register(d: &mut Dispatcher) {
|
||||
d.add("vcs_add_to_ignore", |c| async move {
|
||||
vcs_add_to_ignore(&c.host, &c.window, c.arg("paths")?, c.arg("destination")?).await
|
||||
});
|
||||
d.add("vcs_ignore_blame_revision", |c| async move {
|
||||
vcs_ignore_blame_revision(&c.host, &c.window, c.arg("path")?, c.arg("commit")?).await
|
||||
});
|
||||
d.add("vcs_unstage", |c| async move { vcs_unstage(&c.host, &c.window, c.arg("paths")?).await });
|
||||
d.add("vcs_stage_patch", |c| async move { vcs_stage_patch(&c.host, &c.window, c.arg("patch")?).await });
|
||||
d.add("vcs_discard", |c| async move { vcs_discard(&c.host, &c.window, c.arg("paths")?).await });
|
||||
@@ -1520,6 +1545,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
|
||||
c.arg("message")?,
|
||||
c.arg("includeUntracked")?,
|
||||
c.arg("keepIndex")?,
|
||||
c.arg("paths")?,
|
||||
)
|
||||
.await
|
||||
});
|
||||
|
||||
@@ -2,10 +2,106 @@
|
||||
|
||||
use crate::Host;
|
||||
use ide_core::IdeResult;
|
||||
use ide_core::proxy_settings::{ProxyMode, ProxySettings};
|
||||
use std::path::Path;
|
||||
use std::time::Duration;
|
||||
|
||||
/// Whether `url` names this machine (`localhost` or a loopback address).
|
||||
fn is_loopback(url: &reqwest::Url) -> bool {
|
||||
let host = url.host_str().unwrap_or_default().trim_start_matches('[').trim_end_matches(']');
|
||||
host.eq_ignore_ascii_case("localhost") || host.parse::<std::net::IpAddr>().is_ok_and(|ip| ip.is_loopback())
|
||||
}
|
||||
|
||||
fn is_html(content_type: &str) -> bool {
|
||||
let kind = content_type.split(';').next().unwrap_or_default().trim();
|
||||
kind.eq_ignore_ascii_case("text/html") || kind.eq_ignore_ascii_case("application/xhtml+xml")
|
||||
}
|
||||
|
||||
/// A client that reaches this machine only: no proxy, and a redirect is followed while it
|
||||
/// stays here (a dev server sending `/` to `/app/`).
|
||||
fn loopback_client() -> IdeResult<reqwest::Client> {
|
||||
let direct = ProxySettings { mode: ProxyMode::None, ..ProxySettings::default() };
|
||||
let stay = reqwest::redirect::Policy::custom(|attempt| {
|
||||
if is_loopback(attempt.url()) && attempt.previous().len() < 5 { attempt.follow() } else { attempt.stop() }
|
||||
});
|
||||
ide_http::client::build_with_redirect(&direct, stay)
|
||||
}
|
||||
|
||||
/// Whether `url`, a server on this machine, answers a browser with an HTML document: a page
|
||||
/// to open, as a dev server has, and not an API endpoint. Other hosts are never asked.
|
||||
pub async fn serves_page(url: &str) -> bool {
|
||||
let Ok(url) = reqwest::Url::parse(url) else { return false };
|
||||
if !matches!(url.scheme(), "http" | "https") || !is_loopback(&url) {
|
||||
return false;
|
||||
}
|
||||
let Ok(client) = loopback_client() else { return false };
|
||||
// A dev server may compile the page on this first request.
|
||||
let request = client.get(url).header(reqwest::header::ACCEPT, "text/html").timeout(Duration::from_secs(10));
|
||||
let Ok(response) = request.send().await else { return false };
|
||||
let html = response
|
||||
.headers()
|
||||
.get(reqwest::header::CONTENT_TYPE)
|
||||
.and_then(|value| value.to_str().ok())
|
||||
.is_some_and(is_html);
|
||||
response.status().is_success() && html
|
||||
}
|
||||
|
||||
/// The URL serving `file`, under the window's project when it holds the file.
|
||||
pub async fn url(host: &Host, window: &str, file: &Path) -> IdeResult<String> {
|
||||
let root = host.project(window).ok().map(|p| p.root.clone());
|
||||
host.web.url_for(root.as_deref(), file).await
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
|
||||
/// A server on a free port that answers every request with `status` and `header`. It
|
||||
/// listens on every interface, so that a test can reach it under a name that is not loopback.
|
||||
async fn answering(status: &'static str, header: String) -> String {
|
||||
let listener = tokio::net::TcpListener::bind("0.0.0.0:0").await.unwrap();
|
||||
let port = listener.local_addr().unwrap().port();
|
||||
tokio::spawn(async move {
|
||||
while let Ok((mut socket, _)) = listener.accept().await {
|
||||
let mut request = [0u8; 2048];
|
||||
let _ = socket.read(&mut request).await;
|
||||
let response =
|
||||
format!("HTTP/1.1 {status}\r\n{header}\r\nContent-Length: 2\r\nConnection: close\r\n\r\nok");
|
||||
let _ = socket.write_all(response.as_bytes()).await;
|
||||
}
|
||||
});
|
||||
format!("http://127.0.0.1:{port}/")
|
||||
}
|
||||
|
||||
async fn server(status: &'static str, content_type: &'static str) -> String {
|
||||
answering(status, format!("Content-Type: {content_type}")).await
|
||||
}
|
||||
|
||||
async fn redirecting_to(location: &str) -> String {
|
||||
answering("302 Found", format!("Location: {location}")).await
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn a_redirect_is_followed_on_this_machine_only() {
|
||||
let page = server("200 OK", "text/html").await;
|
||||
assert!(serves_page(&redirecting_to(&page).await).await);
|
||||
// The same page under an address that is not loopback: it is not asked.
|
||||
let elsewhere = page.replace("127.0.0.1", "0.0.0.0");
|
||||
assert!(!serves_page(&redirecting_to(&elsewhere).await).await);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn a_page_is_an_html_answer_from_this_machine() {
|
||||
assert!(serves_page(&server("200 OK", "text/html; charset=utf-8").await).await);
|
||||
assert!(serves_page(&server("200 OK", "Text/HTML").await.replace("127.0.0.1", "localhost")).await);
|
||||
// An API, an error page and a port nothing listens on are not pages.
|
||||
assert!(!serves_page(&server("200 OK", "application/json").await).await);
|
||||
assert!(!serves_page(&server("404 Not Found", "text/html").await).await);
|
||||
assert!(!serves_page("http://127.0.0.1:1/").await);
|
||||
// Other machines are not asked at all.
|
||||
assert!(!serves_page("http://example.com/").await);
|
||||
assert!(!serves_page("ftp://localhost/").await);
|
||||
assert!(!serves_page("not a url").await);
|
||||
}
|
||||
}
|
||||
@@ -396,15 +396,37 @@ impl PluginHost {
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn has_document_providers(&self, kind: ProviderKind, doc: &ProviderDocument) -> bool {
|
||||
self.provider_plugins(kind, &doc.language)
|
||||
.into_iter()
|
||||
.any(|p| p.dev_root.as_ref().is_none_or(|r| doc.path.starts_with(r)))
|
||||
}
|
||||
|
||||
pub fn provide_completion(
|
||||
&self,
|
||||
doc: &ProviderDocument,
|
||||
at: ide_core::Pos,
|
||||
trigger: Option<&str>,
|
||||
) -> Vec<(String, CompletionItem)> {
|
||||
self.feature_documents(doc, Some(at))
|
||||
self.provide_completion_for_editor(doc, at, trigger).1
|
||||
}
|
||||
|
||||
pub(crate) fn provide_completion_for_editor(
|
||||
&self,
|
||||
doc: &ProviderDocument,
|
||||
at: ide_core::Pos,
|
||||
trigger: Option<&str>,
|
||||
) -> (bool, Vec<(String, CompletionItem)>) {
|
||||
// An injection-capable outer document (e.g. SQL embedded in text) may complete at other
|
||||
// positions even when `at` is outside an injection, so avoid marking document-wide backoff.
|
||||
let mut covered = self.has_document_providers(ProviderKind::Injections, doc);
|
||||
let items = self
|
||||
.feature_documents(doc, Some(at))
|
||||
.into_iter()
|
||||
.flat_map(|(doc, bounds)| {
|
||||
if self.has_document_providers(ProviderKind::Completion, &doc) {
|
||||
covered = true;
|
||||
}
|
||||
let (d, t) = (Arc::new(doc.clone()), trigger.map(str::to_owned));
|
||||
self.document_calls(ProviderKind::Completion, &doc, move |rt, timeout| {
|
||||
rt.provide_completion(&d, at, t.as_deref(), timeout)
|
||||
@@ -419,7 +441,8 @@ impl PluginHost {
|
||||
Some((p, i))
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
.collect();
|
||||
(covered, items)
|
||||
}
|
||||
|
||||
pub fn provide_code_actions(
|
||||
|
||||
@@ -447,6 +447,11 @@ impl Host {
|
||||
.collect();
|
||||
matches!(rest.as_slice(), [first, ..] if first == "worktrees" && rest.len() <= 2)
|
||||
});
|
||||
// A repository folder moved or restored into the project: no `.git` event is
|
||||
// written inside it before the watcher sees the new directory, its creation is
|
||||
// the signal. A `git clone` / `git init` inside the project already arrives
|
||||
// through the `.git` paths above; this covers the whole tree being brought in.
|
||||
git_dirs |= batch.created.iter().any(|p| p.join(".git").exists());
|
||||
if git_dirs {
|
||||
scan_repositories(&root, &repos, &settings, &sink, &build_cache);
|
||||
}
|
||||
@@ -659,6 +664,7 @@ impl Host {
|
||||
.join(project_key)
|
||||
}),
|
||||
models_file: self.data_dir.join("agents").join("models.json"),
|
||||
catalogs: self.agent_catalogs.clone(),
|
||||
custom_agents_dir: Some(self.config_dir.join("agents")),
|
||||
home_dir: dirs::home_dir(),
|
||||
env_secrets: Some(Arc::new(AgentEnvSecrets {
|
||||
@@ -864,6 +870,20 @@ fn scan_repositories(
|
||||
sink: &SharedSink,
|
||||
build_cache: &Path,
|
||||
) {
|
||||
match discover_repositories(root, settings, build_cache) {
|
||||
Ok(found) => {
|
||||
if found.roots() != repos.read().roots() {
|
||||
*repos.write() = found;
|
||||
sink.emit(IdeEvent::VcsChanged { root: root.to_path_buf().into() });
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::warn!("looking for git repositories: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The repository containing `root` and those nested below it or below a content root.
|
||||
/// The project-open scan and the watcher's rescan share these skip rules.
|
||||
fn discover_repositories(root: &Path, settings: &SettingsStore, build_cache: &Path) -> IdeResult<Repos> {
|
||||
let ignored = settings.effective().ignored_names();
|
||||
let skip = |p: &Path| {
|
||||
p.file_name().is_some_and(|n| {
|
||||
@@ -875,15 +895,7 @@ fn scan_repositories(
|
||||
};
|
||||
let mut roots = vec![root.to_path_buf()];
|
||||
roots.extend(content_roots(&settings.effective(), root, build_cache));
|
||||
match Repos::discover_roots(&roots, &skip) {
|
||||
Ok(found) => {
|
||||
if found.roots() != repos.read().roots() {
|
||||
*repos.write() = found;
|
||||
sink.emit(IdeEvent::VcsChanged { root: root.to_path_buf().into() });
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::warn!("looking for git repositories: {e}"),
|
||||
}
|
||||
Repos::discover_roots(&roots, &skip)
|
||||
}
|
||||
|
||||
impl Project {
|
||||
|
||||
@@ -270,7 +270,8 @@ impl Dispatcher {
|
||||
.add("run_resize", |c| async move { ops::run::resize(&c.host, c.arg("id")?, c.arg("cols")?, c.arg("rows")?) })
|
||||
.add("run_close_stdin", |c| async move { ops::run::close_stdin(&c.host, c.arg("id")?) })
|
||||
.add("run_list", |c| async move { Ok(ops::run::list(&c.host)) })
|
||||
.add("vcs_status", |c| async move { ops::vcs::status(&c.host, &c.window).await });
|
||||
.add("vcs_status", |c| async move { ops::vcs::status(&c.host, &c.window).await })
|
||||
.add("vcs_status_paths", |c| async move { ops::vcs::status_paths(&c.host, &c.window, c.arg("paths")?).await });
|
||||
ops::register_all(&mut d);
|
||||
d
|
||||
}
|
||||
|
||||
@@ -357,6 +357,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
|
||||
| "docker_set_registry_password"
|
||||
| "docker_run_env"
|
||||
| "db_console_path"
|
||||
| "db_consoles"
|
||||
| "db_export_file"
|
||||
| "vcs_write_patch"
|
||||
| "app_metrics"
|
||||
@@ -475,6 +476,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
|
||||
| "vcs_clone"
|
||||
| "vcs_init"
|
||||
| "vcs_status"
|
||||
| "vcs_status_paths"
|
||||
| "vcs_roots"
|
||||
| "vcs_branch_info"
|
||||
| "vcs_line_hunks"
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
//! Separate project windows share discovery and its backend-enforced cooldown.
|
||||
#![cfg(unix)]
|
||||
|
||||
use ide_agent::AgentEvent;
|
||||
use ide_core::NullSink;
|
||||
use ide_host::{Host, HostConfig, ops};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn windows_share_model_discovery_and_manual_refresh_cooldown() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = tmp.path().join("project");
|
||||
let config = tmp.path().join("config");
|
||||
std::fs::create_dir_all(&root).unwrap();
|
||||
std::fs::create_dir_all(config.join("agents")).unwrap();
|
||||
let script = tmp.path().join("catalog.sh");
|
||||
std::fs::write(
|
||||
&script,
|
||||
r#"#!/bin/sh
|
||||
printf 'launch\n' >> "${0%/*}/launches"
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
*'"id":'*) rest=${line#*\"id\":}; id=${rest%%,*}; id=${id%%\}*} ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
case "$line" in
|
||||
*'"method":"initialize"'*)
|
||||
printf '{"jsonrpc":"2.0","id":%s,"result":{"protocolVersion":1,"agentCapabilities":{}}}\n' "$id" ;;
|
||||
*'"method":"session/new"'*)
|
||||
while [ ! -f "${0%/*}/release" ]; do sleep 0.01; done
|
||||
printf '{"jsonrpc":"2.0","id":%s,"result":{"sessionId":"catalog","configOptions":[{"id":"model","name":"Model","category":"model","type":"select","currentValue":"new-model","options":[{"value":"new-model","name":"New model"}]}]}}\n' "$id" ;;
|
||||
esac
|
||||
done
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
std::fs::write(
|
||||
config.join("agents/catalog-test.toml"),
|
||||
format!(
|
||||
"name = \"Catalog test\"\ncommand = \"/bin/sh\"\nargs = [{}]\n",
|
||||
serde_json::to_string(&script).unwrap()
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
let host = Host::new(HostConfig {
|
||||
config_dir: config,
|
||||
data_dir: tmp.path().join("data"),
|
||||
local_data_dir: tmp.path().join("local"),
|
||||
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
|
||||
syntax: Arc::default(),
|
||||
sink_for: Box::new(|_| Arc::new(NullSink)),
|
||||
owner: |window| window,
|
||||
runtime: tokio::runtime::Handle::current(),
|
||||
project_environment: false,
|
||||
});
|
||||
host.trust.decide(&root, true, false).unwrap();
|
||||
host.open_project("first", &root).unwrap();
|
||||
host.open_project("second", &root).unwrap();
|
||||
let (send, mut events) = tokio::sync::mpsc::unbounded_channel();
|
||||
for window in ["first", "second"] {
|
||||
let send = send.clone();
|
||||
host.project(window).unwrap().agents.subscribe(Arc::new(move |event| {
|
||||
if let AgentEvent::ModelCatalog { catalog } = event {
|
||||
let _ = send.send((window, catalog));
|
||||
}
|
||||
}));
|
||||
}
|
||||
for _ in 0..20 {
|
||||
for window in ["first", "second"] {
|
||||
let state = ops::agent::refresh_models(&host, window, "catalog-test".into(), true).unwrap();
|
||||
assert!(state.refreshing);
|
||||
}
|
||||
}
|
||||
std::fs::write(tmp.path().join("release"), "").unwrap();
|
||||
let completed = tokio::time::timeout(Duration::from_secs(10), async {
|
||||
let mut windows = std::collections::HashSet::new();
|
||||
while windows.len() < 2 {
|
||||
let (window, catalog) = events.recv().await.expect("catalog event stream closed");
|
||||
if !catalog.refreshing && catalog.refreshed_at_ms.is_some() {
|
||||
assert!(catalog.error.is_none(), "{:?}", catalog.error);
|
||||
windows.insert(window);
|
||||
}
|
||||
}
|
||||
})
|
||||
.await;
|
||||
assert!(completed.is_ok(), "both windows must receive the completed catalog");
|
||||
for _ in 0..20 {
|
||||
for window in ["first", "second"] {
|
||||
let catalog = ops::agent::refresh_models(&host, window, "catalog-test".into(), true).unwrap();
|
||||
assert!(!catalog.refreshing);
|
||||
assert!(catalog.retry_at_ms.is_some());
|
||||
let models = catalog.config.iter().find(|o| o.category.as_deref() == Some("model")).unwrap();
|
||||
assert_eq!(models.values[0].value, "new-model");
|
||||
}
|
||||
}
|
||||
assert_eq!(std::fs::read_to_string(tmp.path().join("launches")).unwrap(), "launch\n");
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
use ide_core::NullSink;
|
||||
use ide_host::ops::db;
|
||||
use ide_host::{Host, HostConfig};
|
||||
use std::sync::Arc;
|
||||
|
||||
#[tokio::test]
|
||||
async fn new_query_consoles_are_numbered_after_the_default_one() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let base = tmp.path();
|
||||
let host = Arc::new(Host::new(HostConfig {
|
||||
config_dir: base.join("config"),
|
||||
data_dir: base.join("data"),
|
||||
local_data_dir: base.join("data"),
|
||||
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
|
||||
syntax: Arc::default(),
|
||||
sink_for: Box::new(|_| Arc::new(NullSink)),
|
||||
owner: |w| w,
|
||||
runtime: tokio::runtime::Handle::current(),
|
||||
project_environment: false,
|
||||
}));
|
||||
let names = |paths: Vec<ide_core::IpcPath>| -> Vec<String> {
|
||||
paths.iter().map(|p| p.0.file_name().unwrap().to_string_lossy().into_owned()).collect()
|
||||
};
|
||||
assert!(db::consoles(&host, "pg", false).unwrap().is_empty());
|
||||
// The first new console is the default one, then console_1, console_2…
|
||||
assert_eq!(names(db::consoles(&host, "pg", true).unwrap()), ["console.sql"]);
|
||||
assert_eq!(names(db::consoles(&host, "pg", true).unwrap()), ["console.sql", "console_1.sql"]);
|
||||
let all = db::consoles(&host, "pg", true).unwrap();
|
||||
assert_eq!(names(all.clone()), ["console.sql", "console_1.sql", "console_2.sql"]);
|
||||
// Numbers sort numerically, and the default console path is the first one.
|
||||
std::fs::write(all[0].0.with_file_name("console_10.sql"), "").unwrap();
|
||||
assert_eq!(names(db::consoles(&host, "pg", false).unwrap())[3], "console_10.sql");
|
||||
assert_eq!(db::console_path(&host, "pg").unwrap().0, all[0].0);
|
||||
// Each database of a source has its own consoles.
|
||||
assert!(db::consoles(&host, "pg:other", false).unwrap().is_empty());
|
||||
assert!(db::consoles(&host, "../x", true).is_err());
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
//! The native `nav_inheritance` command: it must return every Python class definition
|
||||
//! in the project — attached dependency roots included — together with the base names
|
||||
//! written in its source, so the UI can compute inheritance gutter markers without a
|
||||
//! language server's type-hierarchy provider.
|
||||
|
||||
use ide_host::{Host, HostConfig, ops};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
const WINDOW: &str = "main";
|
||||
|
||||
struct Opened {
|
||||
host: Arc<Host>,
|
||||
root: PathBuf,
|
||||
_tmp: tempfile::TempDir,
|
||||
}
|
||||
|
||||
/// Opens a synthetic project whose Python files carry known class / base shapes, then
|
||||
/// waits for the index to settle (it walks content roots, so dependency roots are covered).
|
||||
async fn open() -> Opened {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = dunce::canonicalize(tmp.path()).unwrap().join("proj");
|
||||
let src = root.join("src");
|
||||
let dep = root.join("vendor");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::create_dir_all(&dep).unwrap();
|
||||
|
||||
// Own root: a plain class, a class with two bases + a keyword arg, a qualified base,
|
||||
// and a subscripted generic base. A comment / docstring that mentions `class X(Y)`
|
||||
// must not be reported.
|
||||
std::fs::write(
|
||||
src.join("app.py"),
|
||||
concat!(
|
||||
"class Base:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"class Mixin:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"class Child(Base, Mixin, metaclass=Meta):\n",
|
||||
" # class Ghost(Spook): must be ignored\n",
|
||||
" \"\"\"class Phantom(Apparition): must be ignored\"\"\"\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"class Qualified(pkg.mod.Base):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"class Generic(Generic[T]):\n",
|
||||
" pass\n",
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
// A second own-root file to prove the command scans more than one file.
|
||||
std::fs::write(src.join("extra.pyi"), "class Stub(Base):\n ...\n").unwrap();
|
||||
// An attached dependency root (content root) — must be covered by the index.
|
||||
std::fs::write(dep.join("dep.py"), "class DepBase:\n pass\n").unwrap();
|
||||
|
||||
let app = tmp.path().join("app");
|
||||
let host = Arc::new(Host::new(HostConfig {
|
||||
config_dir: app.join("config"),
|
||||
data_dir: app.join("data"),
|
||||
local_data_dir: app.join("data"),
|
||||
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
|
||||
syntax: Arc::default(),
|
||||
sink_for: Box::new(|_| Arc::new(|_| {})),
|
||||
owner: |w| w,
|
||||
runtime: tokio::runtime::Handle::current(),
|
||||
project_environment: false,
|
||||
}));
|
||||
host.trust.decide(&root, true, false).unwrap();
|
||||
ops::project::open(&host, WINDOW, &root).await.unwrap();
|
||||
|
||||
let deadline = Instant::now() + Duration::from_secs(30);
|
||||
loop {
|
||||
let s = ops::nav::index_status(&host, WINDOW).unwrap();
|
||||
if !s.indexing && s.files > 0 {
|
||||
break;
|
||||
}
|
||||
assert!(Instant::now() < deadline, "indexing timed out; status: {s:?}");
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
Opened { host, root, _tmp: tmp }
|
||||
}
|
||||
|
||||
fn find<'a>(all: &'a [ide_host::ops::nav::ClassInheritance], name: &str) -> &'a ide_host::ops::nav::ClassInheritance {
|
||||
all.iter().find(|c| c.name == name).unwrap_or_else(|| panic!("no class {name:?} in {all:#?}"))
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn returns_all_python_classes_and_bases() {
|
||||
let p = open().await;
|
||||
let all = ops::nav::nav_inheritance(&p.host, WINDOW).await.unwrap();
|
||||
|
||||
// Both own-root Python files are scanned (src/app.py + src/extra.pyi).
|
||||
let base = find(&all, "Base");
|
||||
assert!(base.path.0.ends_with("src/app.py"), "Base path: {:?}", base.path);
|
||||
// The location is the class name identifier (selection-range start), i.e. after `class `.
|
||||
assert_eq!(base.line, 0);
|
||||
assert_eq!(base.col, 6);
|
||||
assert!(base.bases.is_empty(), "a bare class has no bases: {:?}", base.bases);
|
||||
|
||||
let child = find(&all, "Child");
|
||||
assert_eq!(child.line, 6);
|
||||
assert_eq!(child.col, 6);
|
||||
// The `metaclass=Meta` keyword argument is not a base.
|
||||
assert!(child.bases.contains(&"Base".to_string()), "Child bases: {:?}", child.bases);
|
||||
assert!(child.bases.contains(&"Mixin".to_string()), "Child bases: {:?}", child.bases);
|
||||
assert!(!child.bases.iter().any(|b| b == "Meta"), "metaclass must be skipped: {:?}", child.bases);
|
||||
|
||||
// A qualified base keeps only its last segment.
|
||||
let qualified = find(&all, "Qualified");
|
||||
assert_eq!(qualified.bases, vec!["Base".to_string()], "qualified base: {:?}", qualified.bases);
|
||||
|
||||
// A subscripted generic base keeps the identifier.
|
||||
let generic = find(&all, "Generic");
|
||||
assert_eq!(generic.bases, vec!["Generic".to_string()], "generic base: {:?}", generic.bases);
|
||||
|
||||
// Comments / docstrings are not mistaken for class definitions.
|
||||
assert!(!all.iter().any(|c| c.name == "Ghost" || c.name == "Phantom"), "parsed a comment: {all:#?}");
|
||||
|
||||
// The .pyi stub in the second own-root file is included.
|
||||
let stub = find(&all, "Stub");
|
||||
assert!(stub.path.0.ends_with("src/extra.pyi"), "Stub path: {:?}", stub.path);
|
||||
assert_eq!(stub.bases, vec!["Base".to_string()]);
|
||||
|
||||
// The attached dependency root is covered by the project index.
|
||||
let dep = find(&all, "DepBase");
|
||||
assert!(dep.path.0.starts_with(&p.root), "DepBase path: {:?}", dep.path);
|
||||
|
||||
// Every returned entry is a Python file.
|
||||
assert!(all.iter().all(|c| { matches!(c.path.0.extension().and_then(|e| e.to_str()), Some("py" | "pyi")) }));
|
||||
|
||||
// A file changed on disk is parsed again; the others come from the cache.
|
||||
let extra = p.root.join("src/extra.pyi");
|
||||
std::fs::write(&extra, "class Stub(Child): ...\nclass Added(Base): ...\n").unwrap();
|
||||
let again = ops::nav::nav_inheritance(&p.host, WINDOW).await.unwrap();
|
||||
assert_eq!(find(&again, "Stub").bases, vec!["Child".to_string()]);
|
||||
assert_eq!(find(&again, "Added").bases, vec!["Base".to_string()]);
|
||||
assert_eq!(find(&again, "Child").bases, child.bases);
|
||||
}
|
||||
@@ -99,6 +99,21 @@ async fn editor_ops_merge_the_fixture_format_selection_and_execute_owned_command
|
||||
let path = root.join("a.txt");
|
||||
std::fs::write(&path, "value\nnext\nend").unwrap();
|
||||
let doc = ops::doc::open(&host, "main", path.clone()).await.unwrap();
|
||||
// `a.toml` has neither a language server, a completion plugin, nor an injection plugin, so `Unavailable` is preserved.
|
||||
let toml_path = root.join("a.toml");
|
||||
std::fs::write(&toml_path, "k = 1\n").unwrap();
|
||||
let toml_doc = ops::doc::open(&host, "main", toml_path).await.unwrap();
|
||||
assert_eq!(
|
||||
ops::lsp::lsp_completion(&host, "main", RequestId(89), toml_doc.id, Pos::default(), None)
|
||||
.await
|
||||
.unwrap_err()
|
||||
.kind,
|
||||
ide_core::error::ErrorKind::Unavailable
|
||||
);
|
||||
let sql_path = root.join("q.sql");
|
||||
std::fs::write(&sql_path, "SELECT 1;").unwrap();
|
||||
let sql_doc = ops::doc::open(&host, "main", sql_path).await.unwrap();
|
||||
assert!(ops::lsp::lsp_completion(&host, "main", RequestId(90), sql_doc.id, Pos::default(), None).await.is_ok());
|
||||
assert_eq!(
|
||||
ops::lsp::lsp_signature_help(&host, "main", doc.id, Pos::default()).await.unwrap().unwrap().signatures[0]
|
||||
.parameters,
|
||||
|
||||
@@ -821,8 +821,10 @@ fn a_reporting_task_outlives_its_background_budget() {
|
||||
|
||||
#[test]
|
||||
fn background_tasks_scale_and_renew_cpu_fuel_budget() {
|
||||
// Room for activation (about 3_100 on macOS, more on Windows) but not for `patient`'s
|
||||
// whole run (about 9_000) without the refill on each report.
|
||||
let f = fixture(Limits {
|
||||
fuel: 3_500,
|
||||
fuel: 6_000,
|
||||
timeout: Duration::from_millis(500),
|
||||
background: Duration::from_millis(5_000),
|
||||
..Limits::default()
|
||||
@@ -831,14 +833,14 @@ fn background_tasks_scale_and_renew_cpu_fuel_budget() {
|
||||
if f.host.set_capability("misbehave", "process:git", true).is_err() {
|
||||
return;
|
||||
}
|
||||
// Warm up activation in its own call so `patient` starts with a fresh 3_500-fuel budget.
|
||||
// Warm up activation in its own call so `patient` starts with a fresh budget.
|
||||
assert_eq!(f.run("misbehave", "echo", Some("warm")).unwrap().as_deref(), Some("warm"));
|
||||
|
||||
let out = f.run("misbehave", "patient", None).expect("background task succeeds").expect("output");
|
||||
assert_eq!(out, "still here");
|
||||
|
||||
let err = f.run("misbehave", "spin", None).unwrap_err();
|
||||
assert!(err.contains("exceeded its CPU budget (3500 fuel per call)"), "{err}");
|
||||
assert!(err.contains("exceeded its CPU budget (6000 fuel per call)"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -82,7 +82,9 @@ async fn serves_projects_files_documents_search_and_runs() {
|
||||
invoke(&peer, "lsp_resend_diagnostics", json!({})).await.unwrap();
|
||||
let files = invoke(&peer, "find_files", json!({"query": "main", "limit": 10})).await.unwrap();
|
||||
assert!(files.is_array());
|
||||
for command in ["build_sync", "run_start", "sdk_locate_tools", "notebook_execute", "debug_start"] {
|
||||
for command in
|
||||
["build_sync", "run_start", "sdk_locate_tools", "notebook_execute", "debug_start", "agent_refresh_models"]
|
||||
{
|
||||
let error = invoke(&peer, command, json!({})).await.unwrap_err();
|
||||
assert!(error.message.contains("Restricted Mode"), "{command}: {error:?}");
|
||||
}
|
||||
@@ -341,6 +343,7 @@ async fn local_deployment_runs_on_the_project_host() {
|
||||
"db_save_sources",
|
||||
"db_set_password",
|
||||
"db_test",
|
||||
"db_test_tunnel",
|
||||
"db_set_ssh_secret",
|
||||
"db_trust_ssh_host",
|
||||
"db_connected",
|
||||
@@ -364,6 +367,14 @@ async fn local_deployment_runs_on_the_project_host() {
|
||||
"db_table_page",
|
||||
"db_apply_edits",
|
||||
"db_console_path",
|
||||
"db_consoles",
|
||||
"ssh_test",
|
||||
"ssh_set_secret",
|
||||
"ssh_password_saved",
|
||||
"ssh_needs_secret",
|
||||
"ssh_trust_host",
|
||||
"ssh_connected",
|
||||
"ssh_disconnect",
|
||||
] {
|
||||
assert!(dispatcher.commands().contains(&cmd), "missing host command {cmd}");
|
||||
}
|
||||
@@ -372,6 +383,12 @@ async fn local_deployment_runs_on_the_project_host() {
|
||||
tokio::spawn(rpc::serve(host, dispatcher, slot, sr, sw));
|
||||
let (cr, cw) = tokio::io::split(client_io);
|
||||
let peer = Peer::connect(cr, cw, Arc::new(Client::default()));
|
||||
// A remote window's host-key and password remedies reach the server's SSH manager,
|
||||
// the one its deployment / database connections failed in.
|
||||
let trust = invoke(&peer, "ssh_trust_host", json!({"id": "nothing-pending"})).await.unwrap_err();
|
||||
assert!(trust.message.contains("no host key to trust"), "{trust:?}");
|
||||
invoke(&peer, "ssh_set_secret", json!({"id": "deploy", "secret": "pw", "save": false})).await.unwrap();
|
||||
assert_eq!(invoke(&peer, "ssh_connected", json!({})).await.unwrap(), json!([]));
|
||||
invoke(&peer, "project_trust_set", json!({"path": project, "trusted": true, "folder": false})).await.unwrap();
|
||||
invoke(&peer, "open_project", json!({"path": project})).await.unwrap();
|
||||
let deployment = json!({"servers": [{"id":"local", "kind":"local", "rootPath":target}], "defaultServer":"local"});
|
||||
|
||||
@@ -66,6 +66,7 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
|
||||
"docker_set_registry_password",
|
||||
"docker_run_env",
|
||||
"db_console_path",
|
||||
"db_consoles",
|
||||
"db_export_file",
|
||||
"vcs_write_patch",
|
||||
"remote_bootstrap_list",
|
||||
@@ -92,6 +93,7 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
|
||||
"vcs_git_version",
|
||||
"docker_compose_op",
|
||||
"db_execute",
|
||||
"agent_refresh_models",
|
||||
"remote_delete",
|
||||
"remote_mkdir",
|
||||
"remote_rename",
|
||||
|
||||
@@ -172,6 +172,7 @@ async fn restricted_vcs_commands_run_no_program_of_the_repository() {
|
||||
}
|
||||
let calls: Vec<Call> = vec![
|
||||
call!("vcs_status", "", ops::vcs::status(host, w)),
|
||||
call!("vcs_status_paths", "", ops::vcs::status_paths(host, w, vec![file.clone()])),
|
||||
sync!("vcs_roots", "", vcs::vcs_roots(host, w)),
|
||||
call!("vcs_branch_info", "", vcs::vcs_branch_info(host, w, root())),
|
||||
call!("vcs_line_hunks", "", vcs::vcs_line_hunks(host, w, file.clone())),
|
||||
|
||||
@@ -102,7 +102,7 @@ async fn stash_and_unstash_reload_clean_open_documents_before_returning() {
|
||||
let f = Fixture::new().await;
|
||||
let edited = format!("{ORIGINAL}\n");
|
||||
f.saved_edit(&edited);
|
||||
vcs::vcs_stash_push(&f.host, "main", None, None, false, false).await.unwrap();
|
||||
vcs::vcs_stash_push(&f.host, "main", None, None, false, false, None).await.unwrap();
|
||||
f.assert_clean(ORIGINAL);
|
||||
vcs::vcs_stash_apply(&f.host, "main", None, 0, true, false, None).await.unwrap();
|
||||
f.assert_clean(&edited);
|
||||
|
||||
@@ -168,7 +168,7 @@ async fn change_commit_branch_stash_and_merge_a_conflict() {
|
||||
let wip = format!("{}// work in progress\n", f.text(main_rs).await);
|
||||
f.host.project(WINDOW).unwrap().documents.replace_all(main_rs, &wip).unwrap();
|
||||
f.save(main_rs).await;
|
||||
ops::vcs::vcs_stash_push(&f.host, WINDOW, None, Some("wip".into()), false, false).await.unwrap();
|
||||
ops::vcs::vcs_stash_push(&f.host, WINDOW, None, Some("wip".into()), false, false, None).await.unwrap();
|
||||
assert!(f.status().await.is_empty());
|
||||
assert!(!f.text(main_rs).await.contains("work in progress"));
|
||||
let stashes = ops::vcs::vcs_stashes(&f.host, WINDOW, None).await.unwrap();
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
//! A repository folder moved into an open project is registered while the project is open:
|
||||
//! the folder brings no `.git` event of its own, so the watcher's "created path" is what the
|
||||
//! rescan has to answer to (the open scan already ran).
|
||||
|
||||
use ide_core::NullSink;
|
||||
use ide_host::{Host, HostConfig};
|
||||
use std::{path::Path, sync::Arc, time::Duration};
|
||||
|
||||
fn git_init(dir: &Path) {
|
||||
std::fs::create_dir_all(dir).unwrap();
|
||||
let status = std::process::Command::new("git").args(["init", "-q"]).current_dir(dir).status();
|
||||
assert!(status.is_ok_and(|status| status.success()), "git init in {}", dir.display());
|
||||
}
|
||||
|
||||
async fn wait_until(what: &str, mut done: impl FnMut() -> bool) {
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(15);
|
||||
while !done() {
|
||||
assert!(tokio::time::Instant::now() < deadline, "timed out waiting for {what}");
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn moving_a_repository_into_the_project_registers_it() {
|
||||
let temp = tempfile::tempdir().unwrap();
|
||||
let base = dunce::canonicalize(temp.path()).unwrap();
|
||||
let root = base.join("project");
|
||||
git_init(&root);
|
||||
// Prepared outside the project, as a restored or moved folder would be.
|
||||
let prepared = base.join("prepared");
|
||||
git_init(&prepared);
|
||||
|
||||
let host = Host::new(HostConfig {
|
||||
config_dir: base.join("config"),
|
||||
data_dir: base.join("data"),
|
||||
local_data_dir: base.join("data"),
|
||||
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
|
||||
syntax: Arc::default(),
|
||||
sink_for: Box::new(|_| Arc::new(NullSink)),
|
||||
owner: |w| w,
|
||||
runtime: tokio::runtime::Handle::current(),
|
||||
project_environment: false,
|
||||
});
|
||||
host.trust.decide(&root, true, false).unwrap();
|
||||
let project = host.open_project("main", &root).unwrap();
|
||||
// The scan on project open runs in the background: wait for it before moving anything in.
|
||||
wait_until("the project's own repository", || project.repos().roots().len() == 1).await;
|
||||
|
||||
std::fs::rename(&prepared, root.join("moved")).unwrap();
|
||||
wait_until("the moved repository to be registered", || {
|
||||
project.repos().roots().iter().any(|r| r.ends_with("moved"))
|
||||
})
|
||||
.await;
|
||||
}
|
||||
@@ -9,7 +9,7 @@ license.workspace = true
|
||||
ide-core.workspace = true
|
||||
ide-vfs.workspace = true
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
serde_json = { workspace = true, features = ["raw_value"] }
|
||||
specta.workspace = true
|
||||
tokio.workspace = true
|
||||
tracing.workspace = true
|
||||
|
||||
@@ -52,26 +52,35 @@ fn search_dirs(file: Option<&Path>, root: Option<&Path>) -> Vec<PathBuf> {
|
||||
}
|
||||
|
||||
fn parse_env(text: &str) -> IdeResult<EnvMap> {
|
||||
use serde_json::value::RawValue;
|
||||
|
||||
let invalid = |e| IdeError::invalid(format!("invalid environment file: {e}"));
|
||||
// Editors on Windows (Notepad) save UTF-8 with a byte order mark, which serde_json rejects.
|
||||
let text = text.strip_prefix('\u{feff}').unwrap_or(text);
|
||||
let value: serde_json::Value =
|
||||
serde_json::from_str(text).map_err(|e| IdeError::invalid(format!("invalid environment file: {e}")))?;
|
||||
let value: Box<RawValue> = serde_json::from_str(text).map_err(invalid)?;
|
||||
let mut out = EnvMap::new();
|
||||
let Some(envs) = value.as_object() else { return Ok(out) };
|
||||
if !value.get().starts_with('{') {
|
||||
return Ok(out);
|
||||
}
|
||||
let envs: BTreeMap<String, Box<RawValue>> = serde_json::from_str(value.get()).map_err(invalid)?;
|
||||
for (env, vars) in envs {
|
||||
let Some(vars) = vars.as_object() else { continue };
|
||||
if !vars.get().starts_with('{') {
|
||||
continue;
|
||||
}
|
||||
let vars: BTreeMap<String, Box<RawValue>> = serde_json::from_str(vars.get()).map_err(invalid)?;
|
||||
let mut map = BTreeMap::new();
|
||||
for (k, v) in vars {
|
||||
let s = match v {
|
||||
serde_json::Value::String(s) => s.clone(),
|
||||
serde_json::Value::Number(n) => n.to_string(),
|
||||
serde_json::Value::Bool(b) => b.to_string(),
|
||||
let raw = v.get();
|
||||
let s = match raw.as_bytes().first() {
|
||||
Some(b'"') => serde_json::from_str::<String>(raw).map_err(invalid)?,
|
||||
// Keep numeric tokens: Value::Number would round decimals and large integers.
|
||||
Some(b'-' | b'0'..=b'9' | b't' | b'f') => raw.to_owned(),
|
||||
// Objects configure SSL / auth in IDEA; not variables.
|
||||
_ => continue,
|
||||
};
|
||||
map.insert(k.clone(), s);
|
||||
map.insert(k, s);
|
||||
}
|
||||
out.insert(env.clone(), map);
|
||||
out.insert(env, map);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
@@ -304,6 +313,85 @@ impl Resolver<'_> {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn preserves_exact_numeric_environment_tokens() {
|
||||
for token in [
|
||||
"0",
|
||||
"-0",
|
||||
"0.0",
|
||||
"-0.000",
|
||||
"1",
|
||||
"-1",
|
||||
"9007199254740991",
|
||||
"9007199254740992",
|
||||
"9007199254740993",
|
||||
"9223372036854775807",
|
||||
"-9223372036854775808",
|
||||
"18446744073709551615",
|
||||
"18446744073709551616",
|
||||
"123456789012345678901234567890",
|
||||
"-123456789012345678901234567890",
|
||||
"1.0000000000000000002",
|
||||
"-1.0000000000000000002",
|
||||
"0.12345678901234567890123456789",
|
||||
"-0.0000000000000000000000000001",
|
||||
"1e-400",
|
||||
"-1e-400",
|
||||
"1e400",
|
||||
"-1e400",
|
||||
"1.2300e+20",
|
||||
"1.2300E+020",
|
||||
] {
|
||||
let json = format!(r#"{{"dev":{{"value":{token}}}}}"#);
|
||||
let envs = Environments::from_json(&json, "").unwrap();
|
||||
let vars = envs.variables(Some("dev"));
|
||||
assert_eq!(vars["value"], token);
|
||||
let resolver = Resolver { file_vars: &[], env: vars };
|
||||
let mut missing = BTreeSet::new();
|
||||
assert_eq!(resolver.substitute("{\"value\":{{value}}}", &mut missing), format!(r#"{{"value":{token}}}"#));
|
||||
assert!(missing.is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_malformed_numeric_environment_tokens() {
|
||||
for token in ["01", "+1", "1.", ".1", "NaN", "Infinity", "--1", "1e", "1e+", "1e--2", "0xFF"] {
|
||||
let json = format!(r#"{{"dev":{{"value":{token}}}}}"#);
|
||||
assert!(Environments::from_json(&json, "").is_err(), "accepted {token}");
|
||||
}
|
||||
assert!(Environments::from_json(r#"{"dev":{"ssl":{"invalid":01}}}"#, "").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reads_numeric_tokens_with_whitespace_and_escaped_names() {
|
||||
let envs = Environments::from_json(
|
||||
"\u{feff} \n { \"de\\u0076\": \n {\"va\\u006cue\": \t 1.2300E+020, \"value\": 1.0000000000000000002 }} \n",
|
||||
"",
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(envs.names(), vec!["dev"]);
|
||||
assert_eq!(envs.variables(Some("dev"))["value"], "1.0000000000000000002");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn exact_environment_parsing_keeps_types_and_precedence() {
|
||||
let envs = Environments::from_json(
|
||||
r#"{"$shared":{"shared":1.0000000000000000002},"dev":{"v":1,"v":9007199254740993,"enabled":true,"disabled":false,"s":"a\n\"b","null":null,"ssl":{},"arr":[]},"other":42}"#,
|
||||
r#"{"dev":{"v":9223372036854775807}}"#,
|
||||
).unwrap();
|
||||
let vars = envs.variables(Some("dev"));
|
||||
assert_eq!(vars["v"], "9223372036854775807");
|
||||
assert_eq!(vars["shared"], "1.0000000000000000002");
|
||||
assert_eq!(vars["enabled"], "true");
|
||||
assert_eq!(vars["disabled"], "false");
|
||||
assert_eq!(vars["s"], "a\n\"b");
|
||||
assert!(!vars.contains_key("null"));
|
||||
assert!(!vars.contains_key("ssl"));
|
||||
assert!(!vars.contains_key("arr"));
|
||||
assert_eq!(Environments::from_json("[]", "").unwrap().names(), Vec::<String>::new());
|
||||
assert!(Environments::from_json(r#"{"dev":{"v":01}}"#, "").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn finds_variable_keys_in_every_environment_private_first() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
|
||||
@@ -249,6 +249,68 @@ mod tests {
|
||||
assert_eq!(url("GET https://api.test/cb?u=http://x\n"), "https://api.test/cb?u=http://x");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn sends_exact_environment_numbers_to_a_local_server() {
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
let server = tokio::spawn(async move {
|
||||
let (mut socket, _) = listener.accept().await.unwrap();
|
||||
let mut request = Vec::new();
|
||||
let (headers, body) = loop {
|
||||
let mut buffer = [0u8; 4096];
|
||||
let read = socket.read(&mut buffer).await.unwrap();
|
||||
assert!(read > 0, "request ended before its body");
|
||||
request.extend_from_slice(&buffer[..read]);
|
||||
let Some(end) = request.windows(4).position(|w| w == b"\r\n\r\n") else { continue };
|
||||
let headers = String::from_utf8_lossy(&request[..end]);
|
||||
let length: usize = headers
|
||||
.lines()
|
||||
.find_map(|line| {
|
||||
let (name, value) = line.split_once(':')?;
|
||||
name.eq_ignore_ascii_case("content-length").then(|| value.trim().parse().unwrap())
|
||||
})
|
||||
.unwrap();
|
||||
if request.len() >= end + 4 + length {
|
||||
break (
|
||||
String::from_utf8(request[..end].to_vec()).unwrap(),
|
||||
String::from_utf8(request[end + 4..end + 4 + length].to_vec()).unwrap(),
|
||||
);
|
||||
}
|
||||
};
|
||||
let response = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{body}",
|
||||
body.len()
|
||||
);
|
||||
socket.write_all(response.as_bytes()).await.unwrap();
|
||||
(headers, body)
|
||||
});
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
std::fs::write(
|
||||
tmp.path().join(crate::env::PUBLIC_FILE),
|
||||
r#"{"dev":{"id":123456789012345678901234567890,"amount":1.0000000000000000002,"tiny":1e-400}}"#,
|
||||
)
|
||||
.unwrap();
|
||||
let file = tmp.path().join("precision.http");
|
||||
let text = format!(
|
||||
"POST http://{addr}/echo?id={{{{id}}}}\nContent-Type: application/json\nX-Exact-Amount: {{{{amount}}}}\n\n{{\"id\":{{{{id}}}},\"amount\":{{{{amount}}}},\"tiny\":{{{{tiny}}}}}}\n"
|
||||
);
|
||||
let input =
|
||||
crate::RunInput { text: &text, index: 0, file: Some(&file), root: Some(tmp.path()), env: Some("dev") };
|
||||
let exchange =
|
||||
tokio::time::timeout(std::time::Duration::from_secs(5), crate::run(input, &CancellationToken::new()))
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let expected = r#"{"id":123456789012345678901234567890,"amount":1.0000000000000000002,"tiny":1e-400}"#;
|
||||
let (headers, body) = server.await.unwrap();
|
||||
assert_eq!(headers.lines().next().unwrap(), "POST /echo?id=123456789012345678901234567890 HTTP/1.1");
|
||||
assert!(headers.lines().any(|line| line.eq_ignore_ascii_case("x-exact-amount: 1.0000000000000000002")));
|
||||
assert_eq!(body, expected);
|
||||
assert_eq!(exchange.request.body.as_deref(), Some(expected));
|
||||
assert_eq!(exchange.response.unwrap().body.as_deref(), Some(expected));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn executes_against_a_local_server() {
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
|
||||
@@ -31,3 +31,7 @@ criterion = { version = "0.5", default-features = false }
|
||||
[[bench]]
|
||||
name = "index"
|
||||
harness = false
|
||||
|
||||
[[bench]]
|
||||
name = "startup"
|
||||
harness = false
|
||||
@@ -0,0 +1,64 @@
|
||||
//! `cargo bench -p ide-index --bench startup`: open a persisted 20k-file
|
||||
//! project and wait for the index to become idle. This is the full warm-start
|
||||
//! path (columnar snapshot, directory/file manifest, lazy redb).
|
||||
|
||||
use criterion::{Criterion, black_box, criterion_group, criterion_main};
|
||||
use ide_core::NullSink;
|
||||
use ide_index::ProjectIndex;
|
||||
use ide_language::{LanguageRegistry, Syntax};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
const FILES: usize = 20_000;
|
||||
|
||||
fn project(root: &Path) {
|
||||
for i in 0..FILES {
|
||||
let dir = root.join(format!("mod{}/src", i % 50));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join(format!("File{i}.rs")), format!("fn symbol{i}() {{}}\n")).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
fn bench(c: &mut Criterion) {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = tmp.path().join("project");
|
||||
let cache = tmp.path().join("cache");
|
||||
project(&root);
|
||||
|
||||
let languages = Arc::new(LanguageRegistry::load(None).unwrap());
|
||||
let syntax = Arc::new(Syntax::new());
|
||||
let open = || {
|
||||
ProjectIndex::open(
|
||||
&root,
|
||||
&cache,
|
||||
Vec::<PathBuf>::new(),
|
||||
vec![],
|
||||
languages.clone(),
|
||||
syntax.clone(),
|
||||
Arc::new(NullSink),
|
||||
)
|
||||
.unwrap()
|
||||
};
|
||||
let first = open();
|
||||
assert!(first.wait_idle(Duration::from_secs(30)));
|
||||
drop(first);
|
||||
|
||||
let mut group = c.benchmark_group("startup_20k");
|
||||
group.sample_size(10);
|
||||
group.bench_function("reopen_wait_idle", |b| {
|
||||
b.iter(|| {
|
||||
let index = open();
|
||||
assert!(index.wait_idle(Duration::from_secs(30)));
|
||||
black_box(index.status().files)
|
||||
})
|
||||
});
|
||||
group.finish();
|
||||
}
|
||||
|
||||
criterion_group! {
|
||||
name = benches;
|
||||
config = Criterion::default().measurement_time(Duration::from_secs(3));
|
||||
targets = bench
|
||||
}
|
||||
criterion_main!(benches);
|
||||
@@ -24,6 +24,19 @@ pub(crate) fn char_mask(s: &str) -> u64 {
|
||||
m
|
||||
}
|
||||
|
||||
/// A relative index key must stay below the project root.
|
||||
pub(crate) fn safe_rel(rel: &str) -> bool {
|
||||
let mut any = false;
|
||||
for component in Path::new(rel).components() {
|
||||
if matches!(component, std::path::Component::Normal(_)) {
|
||||
any = true;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
any
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub(crate) fn mask_bit(b: u8) -> u64 {
|
||||
match b.to_ascii_lowercase() {
|
||||
@@ -553,6 +566,67 @@ impl IndexData {
|
||||
self.by_path.get(path).copied()
|
||||
}
|
||||
|
||||
/// Reserves the flat tables for a bulk warm-start hydration.
|
||||
pub(crate) fn reserve(&mut self, files: usize, symbols: usize, dirs: usize) {
|
||||
self.files.reserve(files);
|
||||
self.by_path.reserve(files);
|
||||
self.dirs.reserve(dirs);
|
||||
self.hot.rows.reserve(files);
|
||||
self.dirs_hot.rows.reserve(dirs);
|
||||
self.syms.head.reserve(symbols);
|
||||
self.syms.container.reserve(symbols);
|
||||
self.syms.kind.reserve(symbols);
|
||||
self.syms.range.reserve(symbols);
|
||||
}
|
||||
|
||||
/// Hydrates a validated warm-start snapshot in one pass, without per-file
|
||||
/// duplicate checks, ancestor walks or compaction checks.
|
||||
pub(crate) fn load_bulk(
|
||||
&mut self,
|
||||
entries: Vec<(FileEntry, Vec<StoredSymbol>)>,
|
||||
dirs: Vec<(Box<str>, PathBuf, u32)>,
|
||||
) {
|
||||
self.files = Vec::with_capacity(entries.len());
|
||||
self.by_path = HashMap::with_capacity(entries.len());
|
||||
self.hot = HotFiles::default();
|
||||
self.dirs_hot = HotDirs::default();
|
||||
self.syms = SymTable::default();
|
||||
self.dirs = HashMap::with_capacity(dirs.len());
|
||||
let symbols: usize = entries.iter().map(|(_, symbols)| symbols.len()).sum();
|
||||
self.reserve(entries.len(), symbols, dirs.len());
|
||||
self.symbol_count = 0;
|
||||
for (slot, (mut entry, symbols)) in entries.into_iter().enumerate() {
|
||||
let (sym_start, sym_mask) = self.syms.push(&symbols);
|
||||
entry.sym_start = sym_start;
|
||||
entry.sym_len = symbols.len() as u32;
|
||||
self.symbol_count += entry.sym_len;
|
||||
let rel = self.hot.append(&entry.rel);
|
||||
self.hot.set(
|
||||
slot as u32,
|
||||
FileRow {
|
||||
rel,
|
||||
name_start: entry.name_start,
|
||||
sym_start,
|
||||
sym_len: entry.sym_len,
|
||||
sym_mask,
|
||||
rel_mask: entry.rel_mask,
|
||||
name_mask: entry.name_mask,
|
||||
dir_mask: char_mask(entry.dir()),
|
||||
ascii: entry.rel.is_ascii(),
|
||||
live: true,
|
||||
},
|
||||
);
|
||||
self.by_path.insert(entry.path.clone(), slot as u32);
|
||||
self.files.push(Some(entry));
|
||||
}
|
||||
self.file_count = self.files.len() as u32;
|
||||
for (rel, path, count) in dirs {
|
||||
let row = self.dirs_hot.push(&rel, &path);
|
||||
self.dirs.insert(rel, (path, count, row));
|
||||
}
|
||||
self.free.clear();
|
||||
}
|
||||
|
||||
/// The persisted form of a file, rebuilt from [`IndexData::syms`].
|
||||
pub(crate) fn record(&self, e: &FileEntry) -> FileRecord {
|
||||
FileRecord {
|
||||
@@ -793,6 +867,23 @@ mod tests {
|
||||
assert_eq!(d.symbol_count, d.files.iter().flatten().map(|e| e.sym_len).sum::<u32>());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bulk_load_matches_incremental_insert() {
|
||||
let mut incremental = IndexData::default();
|
||||
insert(&mut incremental, "a/b.rs", 2);
|
||||
insert(&mut incremental, "c.rs", 1);
|
||||
assert_consistent(&incremental);
|
||||
|
||||
let mut bulk = IndexData::default();
|
||||
bulk.load_bulk(vec![entry("a/b.rs", 2), entry("c.rs", 1)], vec![("a/".into(), PathBuf::from("/r/a"), 1)]);
|
||||
assert_consistent(&bulk);
|
||||
assert_eq!((bulk.file_count, bulk.symbol_count), (2, 3));
|
||||
assert_eq!(bulk.hot.text(), incremental.hot.text());
|
||||
assert_eq!(bulk.files.len(), incremental.files.len());
|
||||
assert_eq!(bulk.slot(Path::new("/r/a/b.rs")), incremental.slot(Path::new("/r/a/b.rs")));
|
||||
assert_eq!(bulk.dirs["a/"].1, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dirs_follow_inserts_and_removals() {
|
||||
let mut d = IndexData::default();
|
||||
|
||||
@@ -138,6 +138,22 @@ impl PathFilter {
|
||||
&self.config.0 == extra_roots && self.config.1 == excludes
|
||||
}
|
||||
|
||||
/// Stable fingerprint of the persisted-scope configuration; a start-up
|
||||
/// snapshot is only trusted when this still matches.
|
||||
pub(crate) fn stamp(&self) -> String {
|
||||
let mut stamp = format!("include_root={}\n", self.config.0.include_root);
|
||||
for folder in &self.config.0.folders {
|
||||
stamp.push_str(&folder.to_string_lossy());
|
||||
stamp.push('\n');
|
||||
}
|
||||
stamp.push_str("--\n");
|
||||
for exclude in &self.config.1 {
|
||||
stamp.push_str(exclude);
|
||||
stamp.push('\n');
|
||||
}
|
||||
stamp
|
||||
}
|
||||
|
||||
pub(crate) fn reload(&self) -> Self {
|
||||
Self::new(&self.root, &self.config.0, &self.config.1)
|
||||
}
|
||||
@@ -146,6 +162,11 @@ impl PathFilter {
|
||||
&self.root
|
||||
}
|
||||
|
||||
/// Whether content roots outside the project root are indexed with it.
|
||||
pub(crate) fn has_extra_roots(&self) -> bool {
|
||||
!self.extra.is_empty()
|
||||
}
|
||||
|
||||
/// The project root, then the extra content roots.
|
||||
pub(crate) fn roots(&self) -> impl Iterator<Item = &Path> {
|
||||
std::iter::once(self.root.as_path())
|
||||
@@ -227,6 +248,30 @@ impl PathFilter {
|
||||
dir.is_some_and(|d| gitignore.matched_path_or_any_parents(d, true).is_ignore())
|
||||
}
|
||||
|
||||
/// All directories at or below `dir` that the index would walk, in walk order.
|
||||
pub(crate) fn dirs(&self, dir: &Path) -> Vec<PathBuf> {
|
||||
let Some((root, _, _)) = self.base_of(dir) else { return Vec::new() };
|
||||
let root = root.to_path_buf();
|
||||
let excludes = self.excludes.clone();
|
||||
let walk_root = root.clone();
|
||||
let walk_excludes = excludes.clone();
|
||||
let walker = WalkBuilder::new(dir)
|
||||
.hidden(false)
|
||||
.parents(true)
|
||||
.ignore(true)
|
||||
.git_ignore(true)
|
||||
.git_global(true)
|
||||
.git_exclude(true)
|
||||
.require_git(ide_core::ignored::inside_git_repo(dir))
|
||||
.follow_links(false)
|
||||
.filter_entry(move |e| {
|
||||
let name = e.file_name().to_string_lossy();
|
||||
!Self::skip_name(&name) && !glob_excluded(&walk_excludes, &walk_root, e.path())
|
||||
})
|
||||
.build();
|
||||
walker.flatten().filter(|e| e.file_type().is_some_and(|t| t.is_dir())).map(|e| e.into_path()).collect()
|
||||
}
|
||||
|
||||
/// All indexable files at or below `dir`, walked in parallel.
|
||||
/// Stops early (returning what was found so far) once `cancel` is set.
|
||||
pub(crate) fn walk(&self, dir: &Path, cancel: &AtomicBool) -> Vec<Found> {
|
||||
|
||||
@@ -1,75 +1,429 @@
|
||||
//! Persistence of file records in redb, so a restart only re-parses changed files.
|
||||
|
||||
use crate::data::FileRecord;
|
||||
use crate::data::{FileEntry, FileRecord, IndexData, StoredSymbol};
|
||||
use crate::filter::{PathFilter, mtime_of};
|
||||
use ide_core::{Range, SymbolKind};
|
||||
use parking_lot::RwLock;
|
||||
use redb::{Database, Durability, ReadableDatabase, ReadableTable, TableDefinition, TableError};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::Path;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
const FILES: TableDefinition<&str, &[u8]> = TableDefinition::new("files");
|
||||
const META: TableDefinition<&str, &[u8]> = TableDefinition::new("meta");
|
||||
/// Bump when [`FileRecord`] or its encoding changes.
|
||||
const SCHEMA: &str = "openidea-index-3";
|
||||
|
||||
/// The real Git directory for a checkout/worktree (`root/.git` may be a file
|
||||
/// containing `gitdir: ...`).
|
||||
fn git_dir(root: &Path) -> Option<std::path::PathBuf> {
|
||||
let dot_git = root.join(".git");
|
||||
if dot_git.is_dir() {
|
||||
return Some(dot_git);
|
||||
}
|
||||
let text = std::fs::read_to_string(&dot_git).ok()?;
|
||||
let target = text.strip_prefix("gitdir:")?.trim();
|
||||
let path = Path::new(target);
|
||||
Some(if path.is_absolute() { path.to_owned() } else { root.join(path) })
|
||||
}
|
||||
|
||||
/// The config and exclude file of the Git directory `git`; a linked worktree's directory
|
||||
/// (`.git/worktrees/<name>`) reads the shared ones its `commondir` names as well.
|
||||
fn insert_git_inputs(out: &mut std::collections::BTreeSet<std::path::PathBuf>, git: &Path) {
|
||||
out.insert(git.join("config"));
|
||||
out.insert(git.join("info/exclude"));
|
||||
if let Ok(common) = std::fs::read_to_string(git.join("commondir")) {
|
||||
let common = Path::new(common.trim());
|
||||
let common = if common.is_absolute() { common.to_owned() } else { git.join(common) };
|
||||
out.insert(common.join("config"));
|
||||
out.insert(common.join("info/exclude"));
|
||||
}
|
||||
}
|
||||
|
||||
/// External ignore/config inputs, including candidates that do not exist yet.
|
||||
/// These are outside the directory manifest (`.git` is skipped), so the full
|
||||
/// path set must match at warm start.
|
||||
pub(crate) fn external_ignore_candidates(root: &Path) -> Vec<std::path::PathBuf> {
|
||||
let mut out = std::collections::BTreeSet::new();
|
||||
if let Some(git) = git_dir(root) {
|
||||
insert_git_inputs(&mut out, &git);
|
||||
}
|
||||
out.insert(root.join(".git/config"));
|
||||
out.insert(root.join(".git/info/exclude"));
|
||||
// The walker also reads the ignore files above the root (`parents`), and a project nested
|
||||
// in a repository uses that repository's exclude file and config.
|
||||
for dir in root.ancestors().skip(1) {
|
||||
out.insert(dir.join(".gitignore"));
|
||||
out.insert(dir.join(".ignore"));
|
||||
if let Some(git) = git_dir(dir) {
|
||||
insert_git_inputs(&mut out, &git);
|
||||
}
|
||||
}
|
||||
if let Some(path) = ignore::gitignore::gitconfig_excludes_path() {
|
||||
out.insert(path);
|
||||
}
|
||||
out.insert(std::path::PathBuf::from("/etc/gitconfig"));
|
||||
if let Some(path) = std::env::var_os("GIT_CONFIG_GLOBAL") {
|
||||
out.insert(path.into());
|
||||
}
|
||||
if let Some(home) = std::env::var_os("HOME") {
|
||||
let home = std::path::PathBuf::from(home);
|
||||
out.insert(home.join(".gitconfig"));
|
||||
out.insert(
|
||||
std::env::var_os("XDG_CONFIG_HOME")
|
||||
.map(std::path::PathBuf::from)
|
||||
.unwrap_or_else(|| home.join(".config"))
|
||||
.join("git/config"),
|
||||
);
|
||||
}
|
||||
out.into_iter().collect()
|
||||
}
|
||||
|
||||
/// Existing per-directory ignore files. Adding/removing one changes the
|
||||
/// directory mtime, which the manifest already validates before this list is used.
|
||||
fn per_dir_ignore_candidates(root: &Path, dirs: &[String]) -> Vec<std::path::PathBuf> {
|
||||
let mut out = std::collections::BTreeSet::new();
|
||||
for rel in dirs {
|
||||
let dir = if rel.is_empty() { root.to_owned() } else { root.join(rel) };
|
||||
for name in [".gitignore", ".ignore"] {
|
||||
let path = dir.join(name);
|
||||
if path.is_file() {
|
||||
out.insert(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
out.into_iter().collect()
|
||||
}
|
||||
|
||||
/// One file's fixed-size columns in the start-up snapshot.
|
||||
#[derive(Clone, Copy, Serialize, Deserialize, Default)]
|
||||
struct FileMeta {
|
||||
mtime: u64,
|
||||
size: u64,
|
||||
hash: [u8; 32],
|
||||
ignored: bool,
|
||||
/// End of this file's symbols in [`Snapshot::symbols`].
|
||||
sym_end: u32,
|
||||
}
|
||||
|
||||
/// One symbol's fixed-size columns; names and containers share [`Snapshot::text`].
|
||||
#[derive(Clone, Copy, Serialize, Deserialize)]
|
||||
struct FlatSymbol {
|
||||
name_start: u32,
|
||||
name_len: u32,
|
||||
container_start: u32,
|
||||
container_len: u32,
|
||||
kind: SymbolKind,
|
||||
range: Range,
|
||||
}
|
||||
|
||||
/// A columnar copy of the index for fast start-up: a handful of contiguous
|
||||
/// arrays and two string arenas instead of one postcard record per redb key.
|
||||
#[derive(Serialize, Deserialize, Default)]
|
||||
pub(crate) struct Snapshot {
|
||||
rels: String,
|
||||
rel_ends: Vec<u32>,
|
||||
meta: Vec<FileMeta>,
|
||||
symbols: Vec<FlatSymbol>,
|
||||
text: String,
|
||||
dirs: String,
|
||||
dir_ends: Vec<u32>,
|
||||
dir_mtimes: Vec<u64>,
|
||||
dir_counts: Vec<u32>,
|
||||
filter: String,
|
||||
stamp: String,
|
||||
ignore_paths: String,
|
||||
ignore_ends: Vec<u32>,
|
||||
ignore_mtimes: Vec<u64>,
|
||||
ignore_sizes: Vec<u64>,
|
||||
ignore_present: Vec<bool>,
|
||||
ignore_external: Vec<bool>,
|
||||
}
|
||||
|
||||
impl Snapshot {
|
||||
pub(crate) fn len(&self) -> usize {
|
||||
self.meta.len()
|
||||
}
|
||||
|
||||
pub(crate) fn filter(&self) -> &str {
|
||||
&self.filter
|
||||
}
|
||||
|
||||
pub(crate) fn dirs_len(&self) -> usize {
|
||||
self.dir_mtimes.len()
|
||||
}
|
||||
|
||||
pub(crate) fn dir(&self, dir: usize) -> &str {
|
||||
let start = if dir == 0 { 0 } else { self.dir_ends[dir - 1] as usize };
|
||||
&self.dirs[start..self.dir_ends[dir] as usize]
|
||||
}
|
||||
|
||||
pub(crate) fn dir_mtime(&self, dir: usize) -> u64 {
|
||||
self.dir_mtimes[dir]
|
||||
}
|
||||
|
||||
pub(crate) fn dir_count(&self, dir: usize) -> u32 {
|
||||
self.dir_counts[dir]
|
||||
}
|
||||
|
||||
pub(crate) fn ignore_len(&self) -> usize {
|
||||
self.ignore_mtimes.len()
|
||||
}
|
||||
|
||||
pub(crate) fn ignore_path(&self, i: usize) -> &str {
|
||||
let start = if i == 0 { 0 } else { self.ignore_ends[i - 1] as usize };
|
||||
&self.ignore_paths[start..self.ignore_ends[i] as usize]
|
||||
}
|
||||
|
||||
pub(crate) fn ignore_mtime(&self, i: usize) -> u64 {
|
||||
self.ignore_mtimes[i]
|
||||
}
|
||||
|
||||
pub(crate) fn ignore_size(&self, i: usize) -> u64 {
|
||||
self.ignore_sizes[i]
|
||||
}
|
||||
|
||||
pub(crate) fn ignore_present(&self, i: usize) -> bool {
|
||||
self.ignore_present[i]
|
||||
}
|
||||
|
||||
pub(crate) fn ignore_external(&self, i: usize) -> bool {
|
||||
self.ignore_external[i]
|
||||
}
|
||||
|
||||
pub(crate) fn mtime(&self, file: usize) -> u64 {
|
||||
self.meta[file].mtime
|
||||
}
|
||||
|
||||
pub(crate) fn size(&self, file: usize) -> u64 {
|
||||
self.meta[file].size
|
||||
}
|
||||
|
||||
/// Rejects a corrupt snapshot before any offsets are used for slicing.
|
||||
fn valid(&self) -> bool {
|
||||
let max = u32::MAX as usize;
|
||||
if self.rels.len() > max || self.text.len() > max || self.dirs.len() > max || self.ignore_paths.len() > max {
|
||||
return false;
|
||||
}
|
||||
if self.rel_ends.len() != self.meta.len() {
|
||||
return false;
|
||||
}
|
||||
let boundary = |text: &str, end: u32| end as usize <= text.len() && text.is_char_boundary(end as usize);
|
||||
let mut rel_end = 0;
|
||||
for end in &self.rel_ends {
|
||||
if *end < rel_end || !boundary(&self.rels, *end) {
|
||||
return false;
|
||||
}
|
||||
rel_end = *end;
|
||||
}
|
||||
if rel_end as usize != self.rels.len() {
|
||||
return false;
|
||||
}
|
||||
let mut start = 0;
|
||||
for end in &self.rel_ends {
|
||||
let rel = &self.rels[start..*end as usize];
|
||||
if !crate::data::safe_rel(rel) {
|
||||
return false;
|
||||
}
|
||||
start = *end as usize;
|
||||
}
|
||||
let mut sym_end = 0;
|
||||
for meta in &self.meta {
|
||||
if meta.sym_end < sym_end || meta.sym_end as usize > self.symbols.len() {
|
||||
return false;
|
||||
}
|
||||
sym_end = meta.sym_end;
|
||||
}
|
||||
if sym_end as usize != self.symbols.len() {
|
||||
return false;
|
||||
}
|
||||
if !self.symbols.iter().all(|s| {
|
||||
let Some(name_end) = (s.name_start as usize).checked_add(s.name_len as usize) else {
|
||||
return false;
|
||||
};
|
||||
let Some(container_end) = (s.container_start as usize).checked_add(s.container_len as usize) else {
|
||||
return false;
|
||||
};
|
||||
name_end <= self.text.len()
|
||||
&& self.text.is_char_boundary(s.name_start as usize)
|
||||
&& self.text.is_char_boundary(name_end)
|
||||
&& (s.container_start == u32::MAX
|
||||
|| (s.container_start as usize <= self.text.len()
|
||||
&& container_end <= self.text.len()
|
||||
&& self.text.is_char_boundary(s.container_start as usize)
|
||||
&& self.text.is_char_boundary(container_end)))
|
||||
}) {
|
||||
return false;
|
||||
}
|
||||
if self.dir_ends.len() != self.dir_mtimes.len() || self.dir_counts.len() != self.dir_mtimes.len() {
|
||||
return false;
|
||||
}
|
||||
if self.dir_ends.first() != Some(&0) {
|
||||
return false;
|
||||
}
|
||||
let mut dir_end = 0;
|
||||
for end in &self.dir_ends {
|
||||
if *end < dir_end || !boundary(&self.dirs, *end) {
|
||||
return false;
|
||||
}
|
||||
dir_end = *end;
|
||||
}
|
||||
if dir_end as usize != self.dirs.len() {
|
||||
return false;
|
||||
}
|
||||
let mut start = 0;
|
||||
for end in &self.dir_ends {
|
||||
let dir = &self.dirs[start..*end as usize];
|
||||
if !dir.is_empty() && !crate::data::safe_rel(dir) {
|
||||
return false;
|
||||
}
|
||||
start = *end as usize;
|
||||
}
|
||||
if self.ignore_ends.len() != self.ignore_mtimes.len()
|
||||
|| self.ignore_sizes.len() != self.ignore_mtimes.len()
|
||||
|| self.ignore_present.len() != self.ignore_mtimes.len()
|
||||
|| self.ignore_external.len() != self.ignore_mtimes.len()
|
||||
{
|
||||
return false;
|
||||
}
|
||||
let mut ignore_end = 0;
|
||||
for end in &self.ignore_ends {
|
||||
if *end < ignore_end || !boundary(&self.ignore_paths, *end) {
|
||||
return false;
|
||||
}
|
||||
ignore_end = *end;
|
||||
}
|
||||
if ignore_end as usize != self.ignore_paths.len() {
|
||||
return false;
|
||||
}
|
||||
let mut start = 0;
|
||||
for end in &self.ignore_ends {
|
||||
if start == *end as usize {
|
||||
return false;
|
||||
}
|
||||
start = *end as usize;
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
pub(crate) fn rel(&self, file: usize) -> &str {
|
||||
let start = if file == 0 { 0 } else { self.rel_ends[file - 1] as usize };
|
||||
&self.rels[start..self.rel_ends[file] as usize]
|
||||
}
|
||||
|
||||
pub(crate) fn entry(&self, file: usize, root: &Path) -> (FileEntry, Vec<StoredSymbol>) {
|
||||
let meta = self.meta[file];
|
||||
let sym_start = if file == 0 { 0 } else { self.meta[file - 1].sym_end as usize };
|
||||
let symbols = (sym_start..meta.sym_end as usize)
|
||||
.map(|i| {
|
||||
let s = self.symbols[i];
|
||||
let name_start = s.name_start as usize;
|
||||
let name_end = name_start + s.name_len as usize;
|
||||
let name = &self.text[name_start..name_end];
|
||||
let container = (s.container_start != u32::MAX).then(|| {
|
||||
let start = s.container_start as usize;
|
||||
&self.text[start..start + s.container_len as usize]
|
||||
});
|
||||
StoredSymbol {
|
||||
name: name.to_owned(),
|
||||
kind: s.kind,
|
||||
container: container.map(str::to_owned),
|
||||
range: s.range,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
let rel = self.rel(file);
|
||||
FileEntry::new(
|
||||
root.join(rel),
|
||||
rel.to_owned(),
|
||||
FileRecord { mtime: meta.mtime, size: meta.size, hash: meta.hash, symbols, ignored: meta.ignored },
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
type Result<T> = std::result::Result<T, String>;
|
||||
|
||||
fn err(e: impl std::fmt::Display) -> String {
|
||||
e.to_string()
|
||||
}
|
||||
|
||||
fn fit(n: usize) -> Result<u32> {
|
||||
u32::try_from(n).map_err(|_| "index snapshot exceeds the 4 GiB span limit".to_owned())
|
||||
}
|
||||
|
||||
pub(crate) struct Store {
|
||||
/// Write-locked only for [`Store::compact`].
|
||||
db: RwLock<Database>,
|
||||
/// Opened lazily: a valid warm-start snapshot serves start-up without
|
||||
/// touching redb. Write-locked only for [`Store::compact`].
|
||||
db: RwLock<Option<Database>>,
|
||||
/// Columnar warm-start snapshot, kept beside the database.
|
||||
snapshot: std::path::PathBuf,
|
||||
/// Recent-file list, readable without opening redb.
|
||||
recent_file: std::path::PathBuf,
|
||||
stamp: String,
|
||||
}
|
||||
|
||||
impl Store {
|
||||
/// Opens (or creates) `cache_dir/index.redb`. Records written by another
|
||||
/// schema / extractor version or for another root are discarded. Returns
|
||||
/// `None` when the database cannot be used (e.g. locked by another window);
|
||||
/// the index then runs in memory only.
|
||||
/// Opens (or creates) the cache directory. The redb database itself is opened
|
||||
/// lazily so a valid columnar snapshot can serve start-up without touching it.
|
||||
/// Returns `None` only when the cache directory cannot be created; the index
|
||||
/// then runs in memory only.
|
||||
pub(crate) fn open(cache_dir: &Path, root: &Path, fingerprint: &str) -> Option<Self> {
|
||||
if let Err(e) = std::fs::create_dir_all(cache_dir) {
|
||||
tracing::warn!("index cache {}: {e}", cache_dir.display());
|
||||
return None;
|
||||
}
|
||||
let file = cache_dir.join("index.redb");
|
||||
let stamp = format!("{SCHEMA}\n{}\n{fingerprint}\n{}", env!("INDEX_SOURCE_HASH"), root.display());
|
||||
Some(Self {
|
||||
db: RwLock::new(None),
|
||||
snapshot: cache_dir.join("index.snapshot"),
|
||||
recent_file: cache_dir.join("recent.snapshot"),
|
||||
stamp,
|
||||
})
|
||||
}
|
||||
|
||||
fn open_db(&self) -> Result<Database> {
|
||||
let file = self.snapshot.with_file_name("index.redb");
|
||||
let db = match Database::create(&file) {
|
||||
Ok(db) => db,
|
||||
Err(redb::DatabaseError::DatabaseAlreadyOpen) => {
|
||||
tracing::warn!("index cache {} is in use; indexing in memory only", file.display());
|
||||
return None;
|
||||
return Err(format!("{} is in use", file.display()));
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!("index cache {} unreadable ({e}); rebuilding", file.display());
|
||||
let _ = std::fs::remove_file(&file);
|
||||
Database::create(&file).map_err(|e| tracing::warn!("index cache: {e}")).ok()?
|
||||
let _ = std::fs::remove_file(&self.snapshot);
|
||||
Database::create(&file).map_err(err)?
|
||||
}
|
||||
};
|
||||
let store = Self { db: RwLock::new(db) };
|
||||
let stamp = format!("{SCHEMA}\n{}\n{fingerprint}\n{}", env!("INDEX_SOURCE_HASH"), root.display());
|
||||
match store.meta("stamp") {
|
||||
Ok(Some(v)) if v == stamp.as_bytes() => {}
|
||||
_ => {
|
||||
if let Err(e) = store.reset(&stamp) {
|
||||
tracing::warn!("index cache reset failed: {e}");
|
||||
return None;
|
||||
}
|
||||
let valid = {
|
||||
let txn = db.begin_read().map_err(err)?;
|
||||
match txn.open_table(META) {
|
||||
Ok(table) => table.get("stamp").map_err(err)?.is_some_and(|v| v.value() == self.stamp.as_bytes()),
|
||||
Err(TableError::TableDoesNotExist(_)) => false,
|
||||
Err(e) => return Err(err(e)),
|
||||
}
|
||||
};
|
||||
if !valid {
|
||||
let _ = std::fs::remove_file(&self.snapshot);
|
||||
let _ = std::fs::remove_file(&self.recent_file);
|
||||
Self::reset_db(&db, &self.stamp)?;
|
||||
}
|
||||
Some(store)
|
||||
Ok(db)
|
||||
}
|
||||
|
||||
fn meta(&self, key: &str) -> Result<Option<Vec<u8>>> {
|
||||
let txn = self.db.read().begin_read().map_err(err)?;
|
||||
let table = match txn.open_table(META) {
|
||||
Ok(t) => t,
|
||||
Err(TableError::TableDoesNotExist(_)) => return Ok(None),
|
||||
Err(e) => return Err(err(e)),
|
||||
};
|
||||
Ok(table.get(key).map_err(err)?.map(|v| v.value().to_vec()))
|
||||
fn ensure_db(&self) -> Result<()> {
|
||||
if self.db.read().is_some() {
|
||||
return Ok(());
|
||||
}
|
||||
let mut slot = self.db.write();
|
||||
if slot.is_none() {
|
||||
*slot = Some(self.open_db()?);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn reset(&self, stamp: &str) -> Result<()> {
|
||||
let txn = self.db.read().begin_write().map_err(err)?;
|
||||
fn reset_db(db: &Database, stamp: &str) -> Result<()> {
|
||||
let txn = db.begin_write().map_err(err)?;
|
||||
match txn.delete_table(FILES) {
|
||||
Ok(_) | Err(TableError::TableDoesNotExist(_)) => {}
|
||||
Err(e) => return Err(err(e)),
|
||||
@@ -82,16 +436,31 @@ impl Store {
|
||||
txn.commit().map_err(err)
|
||||
}
|
||||
|
||||
fn meta(&self, key: &str) -> Result<Option<Vec<u8>>> {
|
||||
self.ensure_db()?;
|
||||
let db = self.db.read();
|
||||
let txn = db.as_ref().expect("ensured").begin_read().map_err(err)?;
|
||||
let table = match txn.open_table(META) {
|
||||
Ok(t) => t,
|
||||
Err(TableError::TableDoesNotExist(_)) => return Ok(None),
|
||||
Err(e) => return Err(err(e)),
|
||||
};
|
||||
Ok(table.get(key).map_err(err)?.map(|v| v.value().to_vec()))
|
||||
}
|
||||
|
||||
/// All persisted records; undecodable entries are skipped (and re-indexed).
|
||||
pub(crate) fn load(&self) -> Result<Vec<(String, FileRecord)>> {
|
||||
let txn = self.db.read().begin_read().map_err(err)?;
|
||||
self.ensure_db()?;
|
||||
let db = self.db.read();
|
||||
let txn = db.as_ref().expect("ensured").begin_read().map_err(err)?;
|
||||
let table = match txn.open_table(FILES) {
|
||||
Ok(t) => t,
|
||||
Err(TableError::TableDoesNotExist(_)) => return Ok(Vec::new()),
|
||||
Err(e) => return Err(err(e)),
|
||||
};
|
||||
let mut out = Vec::new();
|
||||
for item in table.iter().map_err(err)? {
|
||||
let iter = table.iter().map_err(err)?;
|
||||
let mut out = Vec::with_capacity(iter.size_hint().0);
|
||||
for item in iter {
|
||||
let (k, v) = item.map_err(err)?;
|
||||
if let Ok(rec) = postcard::from_bytes::<FileRecord>(v.value()) {
|
||||
out.push((k.value().to_owned(), rec));
|
||||
@@ -100,10 +469,115 @@ impl Store {
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Writes the current index as one columnar snapshot for fast next start-up.
|
||||
/// Only records persisted below `root` are included; content roots outside it
|
||||
/// are rescanned, as they are for the per-file records.
|
||||
pub(crate) fn save_snapshot(&self, data: &IndexData, filter: &PathFilter, not_after: SystemTime) -> Result<()> {
|
||||
let guard = not_after.duration_since(UNIX_EPOCH).map_or(0, |d| d.as_nanos() as u64);
|
||||
let root = filter.root();
|
||||
let mut snap = Snapshot { filter: filter.stamp(), stamp: self.stamp.clone(), ..Default::default() };
|
||||
let mut dirs: std::collections::BTreeSet<String> = std::collections::BTreeSet::new();
|
||||
dirs.insert(String::new());
|
||||
for dir in filter.dirs(root) {
|
||||
if let Ok(rel) = dir.strip_prefix(root) {
|
||||
dirs.insert(rel.to_string_lossy().replace('\\', "/").trim_end_matches('/').to_owned());
|
||||
}
|
||||
}
|
||||
for e in data.files.iter().flatten().filter(|e| e.path.starts_with(root)) {
|
||||
dirs.insert(e.dir().trim_end_matches('/').to_owned());
|
||||
snap.rels.push_str(&e.rel);
|
||||
snap.rel_ends.push(fit(snap.rels.len())?);
|
||||
for s in data.symbols_of(e) {
|
||||
let name_start = fit(snap.text.len())?;
|
||||
snap.text.push_str(s.name);
|
||||
let (container_start, container_len) = match s.container {
|
||||
Some(c) => {
|
||||
let start = fit(snap.text.len())?;
|
||||
snap.text.push_str(c);
|
||||
(start, fit(c.len())?)
|
||||
}
|
||||
None => (u32::MAX, 0),
|
||||
};
|
||||
snap.symbols.push(FlatSymbol {
|
||||
name_start,
|
||||
name_len: fit(s.name.len())?,
|
||||
container_start,
|
||||
container_len,
|
||||
kind: s.kind,
|
||||
range: s.range,
|
||||
});
|
||||
}
|
||||
snap.meta.push(FileMeta {
|
||||
mtime: e.mtime,
|
||||
size: e.size,
|
||||
hash: e.hash,
|
||||
ignored: e.ignored,
|
||||
sym_end: fit(snap.symbols.len())?,
|
||||
});
|
||||
}
|
||||
let dir_list: Vec<String> = dirs.into_iter().collect();
|
||||
for rel in &dir_list {
|
||||
let path = if rel.is_empty() { root.to_owned() } else { root.join(rel) };
|
||||
let mtime = std::fs::symlink_metadata(&path).ok().map(|m| mtime_of(&m)).unwrap_or(0);
|
||||
if mtime > guard {
|
||||
let _ = std::fs::remove_file(&self.snapshot);
|
||||
return Ok(());
|
||||
}
|
||||
let count = if rel.is_empty() {
|
||||
0
|
||||
} else {
|
||||
let key = format!("{rel}/");
|
||||
data.dirs.get(key.as_str()).map_or(0, |d| d.1)
|
||||
};
|
||||
snap.dirs.push_str(rel);
|
||||
snap.dir_ends.push(fit(snap.dirs.len())?);
|
||||
snap.dir_mtimes.push(mtime);
|
||||
snap.dir_counts.push(count);
|
||||
}
|
||||
let external: std::collections::BTreeSet<std::path::PathBuf> =
|
||||
external_ignore_candidates(root).into_iter().collect();
|
||||
let mut candidates: std::collections::BTreeSet<std::path::PathBuf> =
|
||||
per_dir_ignore_candidates(root, &dir_list).into_iter().collect();
|
||||
candidates.extend(external.iter().cloned());
|
||||
for path in candidates {
|
||||
let meta = std::fs::metadata(&path).ok();
|
||||
let present = meta.is_some();
|
||||
let mtime = meta.as_ref().map_or(0, mtime_of);
|
||||
if mtime > guard {
|
||||
let _ = std::fs::remove_file(&self.snapshot);
|
||||
return Ok(());
|
||||
}
|
||||
snap.ignore_paths.push_str(&path.to_string_lossy());
|
||||
snap.ignore_ends.push(fit(snap.ignore_paths.len())?);
|
||||
snap.ignore_mtimes.push(if present { mtime } else { 0 });
|
||||
snap.ignore_sizes.push(if present { meta.as_ref().map_or(0, |m| m.len()) } else { 0 });
|
||||
snap.ignore_present.push(present);
|
||||
snap.ignore_external.push(external.contains(&path));
|
||||
}
|
||||
fit(snap.rels.len())?;
|
||||
fit(snap.text.len())?;
|
||||
fit(snap.dirs.len())?;
|
||||
fit(snap.ignore_paths.len())?;
|
||||
let bytes = postcard::to_allocvec(&snap).map_err(err)?;
|
||||
let tmp = self.snapshot.with_extension("snapshot.tmp");
|
||||
std::fs::write(&tmp, &bytes).map_err(err)?;
|
||||
let _ = std::fs::remove_file(&self.snapshot);
|
||||
std::fs::rename(&tmp, &self.snapshot).map_err(err)
|
||||
}
|
||||
|
||||
/// The columnar start-up snapshot, if it was written and still decodes.
|
||||
pub(crate) fn load_snapshot(&self) -> Option<Snapshot> {
|
||||
let bytes = std::fs::read(&self.snapshot).ok()?;
|
||||
let snapshot: Snapshot = postcard::from_bytes(&bytes).ok()?;
|
||||
(snapshot.stamp == self.stamp && snapshot.valid()).then_some(snapshot)
|
||||
}
|
||||
|
||||
/// Applies upserts and deletions in one transaction. Non-durable commits
|
||||
/// are cheap (no fsync) and become durable with the next durable one.
|
||||
pub(crate) fn apply(&self, puts: &[(String, FileRecord)], deletes: &[String], durable: bool) -> Result<()> {
|
||||
let mut txn = self.db.read().begin_write().map_err(err)?;
|
||||
self.ensure_db()?;
|
||||
let db = self.db.read();
|
||||
let mut txn = db.as_ref().expect("ensured").begin_write().map_err(err)?;
|
||||
if !durable {
|
||||
txn.set_durability(Durability::None).map_err(err)?;
|
||||
}
|
||||
@@ -120,25 +594,47 @@ impl Store {
|
||||
table.remove(rel.as_str()).map_err(err)?;
|
||||
}
|
||||
}
|
||||
// Any file change makes the columnar start-up snapshot stale.
|
||||
let _ = std::fs::remove_file(&self.snapshot);
|
||||
txn.commit().map_err(err)
|
||||
}
|
||||
|
||||
/// Reclaims free pages in the database file; `true` if anything was compacted.
|
||||
pub(crate) fn compact(&self) -> Result<bool> {
|
||||
self.db.write().compact().map_err(err)
|
||||
self.ensure_db()?;
|
||||
let mut db = self.db.write();
|
||||
db.as_mut().expect("ensured").compact().map_err(err)
|
||||
}
|
||||
|
||||
pub(crate) fn load_recent(&self) -> Vec<String> {
|
||||
if let Some(recent) = self.load_recent_file() {
|
||||
return recent;
|
||||
}
|
||||
self.meta("recent").ok().flatten().and_then(|v| postcard::from_bytes(&v).ok()).unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Recent files without opening redb (the warm-start path).
|
||||
pub(crate) fn load_recent_file(&self) -> Option<Vec<String>> {
|
||||
let bytes = std::fs::read(&self.recent_file).ok()?;
|
||||
postcard::from_bytes(&bytes).ok()
|
||||
}
|
||||
|
||||
pub(crate) fn save_recent(&self, recent: &[String]) -> Result<()> {
|
||||
self.save_recent_file(recent)?;
|
||||
let bytes = postcard::to_allocvec(recent).map_err(err)?;
|
||||
self.set_meta("recent", &bytes)
|
||||
}
|
||||
|
||||
/// Rewrites only the recent sidecar, without opening redb.
|
||||
pub(crate) fn save_recent_file(&self, recent: &[String]) -> Result<()> {
|
||||
let bytes = postcard::to_allocvec(recent).map_err(err)?;
|
||||
std::fs::write(&self.recent_file, &bytes).map_err(err)
|
||||
}
|
||||
|
||||
fn set_meta(&self, key: &str, value: &[u8]) -> Result<()> {
|
||||
let txn = self.db.read().begin_write().map_err(err)?;
|
||||
self.ensure_db()?;
|
||||
let db = self.db.read();
|
||||
let txn = db.as_ref().expect("ensured").begin_write().map_err(err)?;
|
||||
txn.open_table(META).map_err(err)?.insert(key, value).map_err(err)?;
|
||||
txn.commit().map_err(err)
|
||||
}
|
||||
@@ -183,4 +679,58 @@ mod tests {
|
||||
let s = Store::open(dir.path(), Path::new("/p"), "fp").unwrap();
|
||||
assert!(s.load().unwrap().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn snapshot_roundtrips_and_is_invalidated_by_changes() {
|
||||
use crate::data::{FileEntry, IndexData, StoredSymbol};
|
||||
use ide_core::{Range, SymbolKind};
|
||||
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let store = Store::open(dir.path(), Path::new("/p"), "fp").unwrap();
|
||||
let record = FileRecord {
|
||||
mtime: 7,
|
||||
size: 3,
|
||||
symbols: vec![StoredSymbol {
|
||||
name: "OrderController".into(),
|
||||
kind: SymbolKind::Class,
|
||||
container: Some("Order".into()),
|
||||
range: Range::default(),
|
||||
}],
|
||||
..Default::default()
|
||||
};
|
||||
let mut data = IndexData::default();
|
||||
let (entry, symbols) =
|
||||
FileEntry::new(std::path::PathBuf::from("/p/src/Order.rs"), "src/Order.rs".into(), record);
|
||||
data.insert(entry, symbols);
|
||||
// Content roots outside the project root are rescanned, not snapshotted.
|
||||
let (entry, symbols) =
|
||||
FileEntry::new(std::path::PathBuf::from("/elsewhere/x.rs"), "x.rs".into(), FileRecord::default());
|
||||
data.insert(entry, symbols);
|
||||
store
|
||||
.save_snapshot(&data, &PathFilter::new(Path::new("/p"), &Vec::new().into(), &[]), SystemTime::now())
|
||||
.unwrap();
|
||||
|
||||
let snapshot = store.load_snapshot().expect("snapshot");
|
||||
assert_eq!(snapshot.len(), 1);
|
||||
assert_eq!(snapshot.rel(0), "src/Order.rs");
|
||||
let (entry, symbols) = snapshot.entry(0, Path::new("/p"));
|
||||
assert_eq!(&*entry.rel, "src/Order.rs");
|
||||
assert_eq!(symbols.len(), 1);
|
||||
assert_eq!(symbols[0].name, "OrderController");
|
||||
assert_eq!(symbols[0].container.as_deref(), Some("Order"));
|
||||
|
||||
store.apply(&[], &[], false).unwrap();
|
||||
assert!(store.load_snapshot().is_none(), "a file change invalidates the snapshot");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn snapshot_validation_rejects_bad_paths_and_split_utf8() {
|
||||
let escaping =
|
||||
Snapshot { rels: "../x".into(), rel_ends: vec![4], meta: vec![FileMeta::default()], ..Default::default() };
|
||||
assert!(!escaping.valid());
|
||||
|
||||
let split =
|
||||
Snapshot { rels: "é".into(), rel_ends: vec![1], meta: vec![FileMeta::default()], ..Default::default() };
|
||||
assert!(!split.valid());
|
||||
}
|
||||
}
|
||||
@@ -772,6 +772,95 @@ fn persisted_symbols_do_not_load_through_escaping_symlink_ancestors() {
|
||||
assert_eq!(initial.lock()[0], 0, "unsafe cached symbols must never be published");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn incremental_changes_refresh_the_warm_start_snapshot() {
|
||||
let f = Fixture::new(&[("src/a.rs", "fn a() {}\n")]);
|
||||
let idx = f.open();
|
||||
let changed = f.write("src/a.rs", "fn renamed() {}\n");
|
||||
idx.update(&[changed], &[]);
|
||||
assert!(idx.wait_idle(WAIT));
|
||||
std::thread::sleep(Duration::from_millis(150));
|
||||
assert!(f.cache.join("index.snapshot").exists(), "a watcher update must regenerate the snapshot");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_detects_changed_ignore_rules() {
|
||||
let f = Fixture::new(&[("sub/generated/a.rs", "fn a() {}\n"), (".git/HEAD", "ref: refs/heads/main\n")]);
|
||||
f.write("sub/.gitignore", "");
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 2, "generated/a.rs plus .gitignore itself");
|
||||
drop(idx);
|
||||
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
f.write("sub/.gitignore", "generated/\n");
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 1, "an updated ignore rule invalidates the warm snapshot");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_detects_changed_ignore_rules_above_the_project() {
|
||||
// The project is a folder of a repository whose ignore files sit above it.
|
||||
let f = Fixture::new(&[("generated/a.rs", "fn a() {}\n"), ("src/b.rs", "fn b() {}\n")]);
|
||||
let repo = f.root.parent().unwrap().to_owned();
|
||||
std::fs::create_dir_all(repo.join(".git/info")).unwrap();
|
||||
std::fs::write(repo.join(".git/HEAD"), "ref: refs/heads/main\n").unwrap();
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 2);
|
||||
drop(idx);
|
||||
|
||||
std::fs::write(repo.join(".gitignore"), "proj/generated/\n").unwrap();
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 1, "a new ignore file above the project invalidates the warm snapshot");
|
||||
drop(idx);
|
||||
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
std::fs::write(repo.join(".gitignore"), "").unwrap();
|
||||
std::fs::write(repo.join(".git/info/exclude"), "proj/src/\n").unwrap();
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 1, "so do the enclosing repository's changed rules");
|
||||
assert!(!symbols(&idx, "a", SymbolFilter::All).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_detects_a_changed_exclude_file_of_a_linked_worktree() {
|
||||
// The project is a linked worktree: its `.git` file points into the main repository,
|
||||
// whose shared `info/exclude` the walker reads through `commondir`.
|
||||
let f = Fixture::new(&[("generated/a.rs", "fn a() {}\n"), ("src/b.rs", "fn b() {}\n")]);
|
||||
let main = f.root.parent().unwrap().join("main/.git");
|
||||
let worktree = main.join("worktrees/proj");
|
||||
std::fs::create_dir_all(main.join("info")).unwrap();
|
||||
std::fs::create_dir_all(&worktree).unwrap();
|
||||
std::fs::write(main.join("HEAD"), "ref: refs/heads/main\n").unwrap();
|
||||
std::fs::write(worktree.join("HEAD"), "ref: refs/heads/wt\n").unwrap();
|
||||
std::fs::write(worktree.join("commondir"), "../..\n").unwrap();
|
||||
std::fs::write(f.root.join(".git"), format!("gitdir: {}\n", worktree.display())).unwrap();
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 2);
|
||||
drop(idx);
|
||||
|
||||
std::fs::write(main.join("info/exclude"), "generated/\n").unwrap();
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 1, "the shared exclude file invalidates the warm snapshot");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_detects_a_new_file_in_an_existing_empty_directory() {
|
||||
let f = Fixture::new(&[("src/a.rs", "fn a() {}\n")]);
|
||||
std::fs::create_dir(f.root.join("empty")).unwrap();
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 1);
|
||||
assert!(idx.find_dirs("empty", 10).is_empty());
|
||||
drop(idx);
|
||||
let idx = f.open();
|
||||
assert!(idx.find_dirs("empty", 10).is_empty(), "empty folders are not searchable after warm start");
|
||||
drop(idx);
|
||||
|
||||
f.write("empty/new.rs", "fn new_symbol() {}\n");
|
||||
let idx = f.open();
|
||||
assert_eq!(idx.status().files, 2);
|
||||
assert_eq!(symbols(&idx, "new_symbol", SymbolFilter::All), ["new_symbol"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rule_changes_refresh_only_the_affected_index_subtree() {
|
||||
for (rule, subtree) in [(".git/info/exclude", ""), ("sub/.gitignore", "sub"), ("sub/.ignore", "sub")] {
|
||||
@@ -834,3 +923,42 @@ fn repository_rule_changes_refresh_members_of_an_unindexed_container() {
|
||||
assert!(idx.wait_idle(WAIT));
|
||||
assert_eq!(files(&idx, "item.txt"), ["member/generated/item.txt"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_sees_edits_deletions_and_renames_made_while_closed() {
|
||||
let f = Fixture::new(&[
|
||||
("src/a.rs", "fn aaaa() {}\n"),
|
||||
("src/deep/b.rs", "fn bbbb() {}\n"),
|
||||
("c.rs", "fn cccc() {}\n"),
|
||||
]);
|
||||
let idx = f.open();
|
||||
assert!(idx.wait_idle(WAIT));
|
||||
drop(idx);
|
||||
std::thread::sleep(Duration::from_millis(200));
|
||||
// Warm start of an unchanged tree keeps everything.
|
||||
let idx = f.open();
|
||||
assert!(idx.wait_idle(WAIT));
|
||||
assert_eq!(symbols(&idx, "aaaa", SymbolFilter::All), ["aaaa"]);
|
||||
drop(idx);
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
// Same size, new content; a deletion; a new file two levels down; a rename.
|
||||
f.write("src/a.rs", "fn zzzz() {}\n");
|
||||
std::fs::remove_file(f.root.join("c.rs")).unwrap();
|
||||
f.write("src/deep/deeper/n.rs", "fn nnnn() {}\n");
|
||||
std::fs::rename(f.root.join("src/deep/b.rs"), f.root.join("src/deep/r.rs")).unwrap();
|
||||
let idx = f.open();
|
||||
assert!(idx.wait_idle(WAIT));
|
||||
assert!(symbols(&idx, "aaaa", SymbolFilter::All).is_empty(), "stale content");
|
||||
assert_eq!(symbols(&idx, "zzzz", SymbolFilter::All), ["zzzz"]);
|
||||
assert!(symbols(&idx, "cccc", SymbolFilter::All).is_empty(), "deleted file");
|
||||
assert_eq!(symbols(&idx, "nnnn", SymbolFilter::All), ["nnnn"]);
|
||||
assert_eq!(symbols(&idx, "bbbb", SymbolFilter::All), ["bbbb"]);
|
||||
assert_eq!(idx.status().files, 3);
|
||||
drop(idx);
|
||||
// After the cold rescan, the next warm start agrees.
|
||||
std::thread::sleep(Duration::from_millis(200));
|
||||
let idx = f.open();
|
||||
assert!(idx.wait_idle(WAIT));
|
||||
assert_eq!(idx.status().files, 3);
|
||||
assert_eq!(symbols(&idx, "zzzz", SymbolFilter::All), ["zzzz"]);
|
||||
}
|
||||
@@ -1,8 +1,8 @@
|
||||
//! The background indexing thread: initial load + full scan, then incremental
|
||||
//! updates queued by the watcher.
|
||||
|
||||
use crate::data::{FileEntry, FileRecord, IndexData, StoredSymbol};
|
||||
use crate::filter::{Found, PathFilter};
|
||||
use crate::data::{FileEntry, FileRecord, IndexData, StoredSymbol, safe_rel};
|
||||
use crate::filter::{Found, PathFilter, mtime_of};
|
||||
use crate::store::Store;
|
||||
use ide_core::{IdeEvent, SharedSink, TaskId};
|
||||
use ide_language::{LanguageRegistry, Syntax};
|
||||
@@ -12,12 +12,16 @@ use std::collections::{HashMap, HashSet, VecDeque};
|
||||
use std::io::Read;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
use std::time::{Duration, Instant, SystemTime};
|
||||
|
||||
/// Files larger than this are listed but not parsed for symbols.
|
||||
pub const MAX_SYMBOL_FILE: u64 = 2 * 1024 * 1024;
|
||||
const CHUNK: usize = 512;
|
||||
const PROGRESS_EVERY: Duration = Duration::from_millis(250);
|
||||
#[cfg(not(test))]
|
||||
const SNAPSHOT_DELAY: Duration = Duration::from_secs(2);
|
||||
#[cfg(test)]
|
||||
const SNAPSHOT_DELAY: Duration = Duration::from_millis(50);
|
||||
pub(crate) const MAX_RECENT: usize = 50;
|
||||
|
||||
pub(crate) enum Job {
|
||||
@@ -55,6 +59,11 @@ pub(crate) struct Shared {
|
||||
queue: Mutex<Queue>,
|
||||
wake: Condvar,
|
||||
idle: Condvar,
|
||||
/// Coalesced post-update snapshot refresh.
|
||||
snapshot_due: Mutex<Option<Instant>>,
|
||||
/// Directories whose watcher events still need reconciliation before a
|
||||
/// refreshed snapshot can be trusted.
|
||||
pending_dirs: Mutex<HashSet<PathBuf>>,
|
||||
}
|
||||
|
||||
impl Shared {
|
||||
@@ -79,6 +88,8 @@ impl Shared {
|
||||
queue: Mutex::default(),
|
||||
wake: Condvar::new(),
|
||||
idle: Condvar::new(),
|
||||
snapshot_due: Mutex::default(),
|
||||
pending_dirs: Mutex::default(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -135,8 +146,15 @@ impl Shared {
|
||||
q.busy = true;
|
||||
}
|
||||
if scan {
|
||||
self.load_persisted();
|
||||
self.full_scan();
|
||||
// A warm start can trust the snapshot when neither the scope nor any
|
||||
// directory/file metadata changed. Rebuilding the walk's file list is
|
||||
// the expensive part; validate the flat manifest first.
|
||||
if !self.try_fast_start() {
|
||||
let snapshot_guard = SystemTime::now();
|
||||
let found = self.walk_roots(None);
|
||||
self.load_persisted(&found);
|
||||
self.full_scan(Some(found), snapshot_guard);
|
||||
}
|
||||
} else {
|
||||
self.indexing.store(false, Ordering::SeqCst);
|
||||
}
|
||||
@@ -147,17 +165,38 @@ impl Shared {
|
||||
self.idle.notify_all();
|
||||
}
|
||||
loop {
|
||||
let mut save_due = false;
|
||||
let jobs: Vec<Job> = {
|
||||
let mut q = self.queue.lock();
|
||||
while q.jobs.is_empty() && !self.is_cancelled() {
|
||||
self.wake.wait(&mut q);
|
||||
loop {
|
||||
if self.is_cancelled() {
|
||||
return;
|
||||
}
|
||||
if !q.jobs.is_empty() {
|
||||
break;
|
||||
}
|
||||
let due = *self.snapshot_due.lock();
|
||||
if let Some(due) = due {
|
||||
if Instant::now() >= due {
|
||||
save_due = true;
|
||||
break;
|
||||
}
|
||||
self.wake.wait_until(&mut q, due);
|
||||
} else {
|
||||
self.wake.wait(&mut q);
|
||||
}
|
||||
}
|
||||
if self.is_cancelled() {
|
||||
return;
|
||||
if save_due {
|
||||
Vec::new()
|
||||
} else {
|
||||
q.busy = true;
|
||||
q.jobs.drain(..).collect()
|
||||
}
|
||||
q.busy = true;
|
||||
q.jobs.drain(..).collect()
|
||||
};
|
||||
if save_due {
|
||||
self.save_snapshot_now();
|
||||
continue;
|
||||
}
|
||||
self.process(jobs);
|
||||
let mut q = self.queue.lock();
|
||||
q.busy = false;
|
||||
@@ -167,27 +206,114 @@ impl Shared {
|
||||
}
|
||||
}
|
||||
|
||||
fn load_persisted(&self) {
|
||||
let Some(store) = &self.store else { return };
|
||||
/// Warm start: validate the snapshot's flat directory/file manifest and hydrate
|
||||
/// without walking the tree. A changed scope, symlink, directory mtime, file
|
||||
/// mtime or file size falls back to the full walk.
|
||||
fn try_fast_start(&self) -> bool {
|
||||
let started = Instant::now();
|
||||
let Some(store) = &self.store else { return false };
|
||||
let load_started = Instant::now();
|
||||
let Some(snapshot) = store.load_snapshot() else { return false };
|
||||
let loaded = load_started.elapsed();
|
||||
let filter = self.filter();
|
||||
let root = filter.root().to_owned();
|
||||
match store.load() {
|
||||
Ok(records) => {
|
||||
let entries: Vec<(FileEntry, Vec<StoredSymbol>)> = records
|
||||
.into_par_iter()
|
||||
.filter(|(rel, _)| {
|
||||
let path = root.join(rel);
|
||||
!filter.is_excluded(&path, false) && dunce::canonicalize(&path).is_ok_and(|real| real == path)
|
||||
})
|
||||
.map(|(rel, rec)| FileEntry::new(root.join(&rel), rel, rec))
|
||||
.collect();
|
||||
let mut d = self.data.write();
|
||||
for (e, symbols) in entries {
|
||||
d.insert(e, symbols);
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::warn!("index cache load failed: {e}"),
|
||||
if filter.has_extra_roots() || snapshot.filter() != filter.stamp() {
|
||||
return false;
|
||||
}
|
||||
let root = filter.root();
|
||||
for i in 0..snapshot.dirs_len() {
|
||||
let rel = snapshot.dir(i);
|
||||
let path = if rel.is_empty() { root.to_owned() } else { root.join(rel) };
|
||||
let Ok(meta) = std::fs::symlink_metadata(&path) else { return false };
|
||||
if !meta.is_dir() || mtime_of(&meta) != snapshot.dir_mtime(i) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
let external = crate::store::external_ignore_candidates(root);
|
||||
let saved_external: Vec<&str> = (0..snapshot.ignore_len())
|
||||
.filter(|&i| snapshot.ignore_external(i))
|
||||
.map(|i| snapshot.ignore_path(i))
|
||||
.collect();
|
||||
if external.len() != saved_external.len()
|
||||
|| external.iter().zip(&saved_external).any(|(path, saved)| path.to_string_lossy() != **saved)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
for i in 0..snapshot.ignore_len() {
|
||||
let path = Path::new(snapshot.ignore_path(i));
|
||||
let meta = std::fs::metadata(path).ok();
|
||||
let present = meta.is_some();
|
||||
if present != snapshot.ignore_present(i) {
|
||||
return false;
|
||||
}
|
||||
if let Some(meta) = meta
|
||||
&& (mtime_of(&meta) != snapshot.ignore_mtime(i) || meta.len() != snapshot.ignore_size(i))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
let build_started = Instant::now();
|
||||
let entries: Vec<Option<(FileEntry, Vec<StoredSymbol>)>> = (0..snapshot.len())
|
||||
.into_par_iter()
|
||||
.map(|i| {
|
||||
let path = root.join(snapshot.rel(i));
|
||||
let meta = std::fs::symlink_metadata(&path).ok()?;
|
||||
if !meta.is_file() || mtime_of(&meta) != snapshot.mtime(i) || meta.len() != snapshot.size(i) {
|
||||
return None;
|
||||
}
|
||||
Some(snapshot.entry(i, root))
|
||||
})
|
||||
.collect();
|
||||
let built = build_started.elapsed();
|
||||
if entries.iter().any(Option::is_none) {
|
||||
return false;
|
||||
}
|
||||
let validated = entries;
|
||||
let mut entries = Vec::with_capacity(validated.len());
|
||||
for entry in validated {
|
||||
entries.push(entry.expect("validated above"));
|
||||
}
|
||||
let dirs: Vec<(Box<str>, PathBuf, u32)> = (0..snapshot.dirs_len())
|
||||
.filter_map(|i| {
|
||||
let rel = snapshot.dir(i);
|
||||
(!rel.is_empty() && snapshot.dir_count(i) > 0).then(|| {
|
||||
let key = format!("{rel}/");
|
||||
(key.into_boxed_str(), root.join(rel), snapshot.dir_count(i))
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let insert_started = Instant::now();
|
||||
let mut d = self.data.write();
|
||||
d.load_bulk(entries, dirs);
|
||||
drop(d);
|
||||
tracing::info!(
|
||||
"index snapshot: warm start in {:?} (load {loaded:?}, validate/build {built:?}, insert {:?})",
|
||||
started.elapsed(),
|
||||
insert_started.elapsed()
|
||||
);
|
||||
#[cfg(test)]
|
||||
eprintln!(
|
||||
"index snapshot: warm start load {loaded:?}, validate/build {built:?}, insert {:?}, total {:?}",
|
||||
insert_started.elapsed(),
|
||||
started.elapsed()
|
||||
);
|
||||
self.indexing.store(false, Ordering::SeqCst);
|
||||
self.emit_state();
|
||||
let recent = store.load_recent_file().unwrap_or_else(|| {
|
||||
let recent = store.load_recent();
|
||||
let _ = store.save_recent_file(&recent);
|
||||
recent
|
||||
});
|
||||
let mut r = self.recent.lock();
|
||||
for p in recent.into_iter().map(|rel| root.join(rel)) {
|
||||
if !r.contains(&p) {
|
||||
r.push(p);
|
||||
}
|
||||
}
|
||||
r.truncate(MAX_RECENT);
|
||||
true
|
||||
}
|
||||
|
||||
fn load_recent(&self, store: &Store, root: &Path) {
|
||||
let recent: Vec<PathBuf> = store.load_recent().into_iter().map(|rel| root.join(rel)).collect();
|
||||
let mut r = self.recent.lock();
|
||||
for p in recent {
|
||||
@@ -198,6 +324,72 @@ impl Shared {
|
||||
r.truncate(MAX_RECENT);
|
||||
}
|
||||
|
||||
fn load_persisted(&self, found: &[Found]) {
|
||||
let Some(store) = &self.store else { return };
|
||||
let root = self.filter().root().to_owned();
|
||||
let seen: HashSet<&Path> = found.iter().map(|f| f.path.as_path()).collect();
|
||||
let started = Instant::now();
|
||||
match store.load_snapshot() {
|
||||
Some(snapshot) => {
|
||||
let entries: Vec<(FileEntry, Vec<StoredSymbol>)> = (0..snapshot.len())
|
||||
.into_par_iter()
|
||||
.filter_map(|i| {
|
||||
let rel = snapshot.rel(i);
|
||||
if !safe_rel(rel) {
|
||||
return None;
|
||||
}
|
||||
let path = root.join(rel);
|
||||
seen.contains(path.as_path()).then(|| snapshot.entry(i, &root))
|
||||
})
|
||||
.collect();
|
||||
let mut d = self.data.write();
|
||||
let symbols = entries.iter().map(|(_, symbols)| symbols.len()).sum();
|
||||
d.reserve(entries.len(), symbols, 0);
|
||||
for (e, symbols) in entries {
|
||||
d.insert(e, symbols);
|
||||
}
|
||||
tracing::info!(
|
||||
"index snapshot: hydrated {} files, {} symbols in {:?}",
|
||||
d.file_count,
|
||||
d.symbol_count,
|
||||
started.elapsed()
|
||||
);
|
||||
#[cfg(test)]
|
||||
eprintln!("index snapshot: hydrated {} files in {:?}", d.file_count, started.elapsed());
|
||||
}
|
||||
None => match store.load() {
|
||||
Ok(records) => {
|
||||
// Fallback for an index that changed since the snapshot: redb
|
||||
// records are decoded one file at a time and filtered by the walk.
|
||||
let entries: Vec<(FileEntry, Vec<StoredSymbol>)> = records
|
||||
.into_par_iter()
|
||||
.filter(|(rel, _)| {
|
||||
safe_rel(rel) && {
|
||||
let path = root.join(rel);
|
||||
seen.contains(path.as_path())
|
||||
}
|
||||
})
|
||||
.map(|(rel, rec)| FileEntry::new(root.join(&rel), rel, rec))
|
||||
.collect();
|
||||
let mut d = self.data.write();
|
||||
let symbols = entries.iter().map(|(_, symbols)| symbols.len()).sum();
|
||||
d.reserve(entries.len(), symbols, 0);
|
||||
for (e, symbols) in entries {
|
||||
d.insert(e, symbols);
|
||||
}
|
||||
tracing::info!(
|
||||
"index cache: hydrated {} files, {} symbols in {:?}",
|
||||
d.file_count,
|
||||
d.symbol_count,
|
||||
started.elapsed()
|
||||
);
|
||||
}
|
||||
Err(e) => tracing::warn!("index cache load failed: {e}"),
|
||||
},
|
||||
}
|
||||
self.load_recent(store, &root);
|
||||
}
|
||||
|
||||
/// A cancelled scan (project closed) still ends its progress task and the indexing state.
|
||||
fn end_scan(&self, task: TaskId) {
|
||||
self.indexing.store(false, Ordering::SeqCst);
|
||||
@@ -212,11 +404,24 @@ impl Shared {
|
||||
self.emit_state();
|
||||
}
|
||||
|
||||
fn full_scan(&self) {
|
||||
self.scan(None);
|
||||
fn walk_roots(&self, dirs: Option<&[PathBuf]>) -> Vec<Found> {
|
||||
let filter = self.filter();
|
||||
let roots: Vec<&Path> = match dirs {
|
||||
Some(dirs) => dirs.iter().map(PathBuf::as_path).collect(),
|
||||
None => filter.roots().collect(),
|
||||
};
|
||||
let mut found = Vec::new();
|
||||
for root in roots {
|
||||
found.extend(filter.walk(root, &self.cancelled));
|
||||
}
|
||||
found
|
||||
}
|
||||
|
||||
fn scan(&self, dirs: Option<&[PathBuf]>) {
|
||||
fn full_scan(&self, found: Option<Vec<Found>>, guard: SystemTime) {
|
||||
self.scan(None, found, guard);
|
||||
}
|
||||
|
||||
fn scan(&self, dirs: Option<&[PathBuf]>, prewalked: Option<Vec<Found>>, guard: SystemTime) {
|
||||
let started = Instant::now();
|
||||
self.indexing.store(true, Ordering::SeqCst);
|
||||
self.emit_state();
|
||||
@@ -231,14 +436,7 @@ impl Shared {
|
||||
});
|
||||
|
||||
let filter = self.filter();
|
||||
let mut found = Vec::new();
|
||||
let roots: Vec<&Path> = match dirs {
|
||||
Some(dirs) => dirs.iter().map(PathBuf::as_path).collect(),
|
||||
None => filter.roots().collect(),
|
||||
};
|
||||
for root in roots {
|
||||
found.extend(filter.walk(root, &self.cancelled));
|
||||
}
|
||||
let found = prewalked.unwrap_or_else(|| self.walk_roots(dirs));
|
||||
if self.is_cancelled() {
|
||||
return self.end_scan(task);
|
||||
}
|
||||
@@ -285,6 +483,17 @@ impl Shared {
|
||||
}
|
||||
}
|
||||
self.persist(&[], &[], true);
|
||||
if let Some(store) = &self.store {
|
||||
let recent: Vec<String> = self.recent.lock().iter().filter_map(|p| filter.store_key(p)).collect();
|
||||
let d = self.data.read();
|
||||
if let Err(e) = store.save_snapshot(&d, &filter, guard) {
|
||||
tracing::warn!("index snapshot: {e}");
|
||||
}
|
||||
if let Err(e) = store.save_recent_file(&recent) {
|
||||
tracing::warn!("index recent snapshot: {e}");
|
||||
}
|
||||
}
|
||||
*self.snapshot_due.lock() = None;
|
||||
self.indexing.store(false, Ordering::SeqCst);
|
||||
self.sink.emit(IdeEvent::Task {
|
||||
id: task,
|
||||
@@ -347,6 +556,36 @@ impl Shared {
|
||||
self.persist(&puts, &[], false);
|
||||
}
|
||||
|
||||
fn schedule_snapshot(&self) {
|
||||
if self.store.is_none() {
|
||||
return;
|
||||
}
|
||||
let mut due = self.snapshot_due.lock();
|
||||
if due.is_none() {
|
||||
*due = Some(Instant::now() + SNAPSHOT_DELAY);
|
||||
}
|
||||
}
|
||||
|
||||
fn save_snapshot_now(&self) {
|
||||
let dirs: Vec<PathBuf> = self.pending_dirs.lock().drain().collect();
|
||||
if !dirs.is_empty() {
|
||||
// Reconcile the directories touched since the last scan before
|
||||
// recording their mtimes; otherwise a missed watcher event could be
|
||||
// blessed into the manifest.
|
||||
self.scan(Some(&dirs), None, SystemTime::now());
|
||||
return;
|
||||
}
|
||||
if let Some(store) = &self.store {
|
||||
let filter = self.filter();
|
||||
let guard = SystemTime::now();
|
||||
let d = self.data.read();
|
||||
if let Err(e) = store.save_snapshot(&d, &filter, guard) {
|
||||
tracing::warn!("index snapshot: {e}");
|
||||
}
|
||||
}
|
||||
*self.snapshot_due.lock() = None;
|
||||
}
|
||||
|
||||
fn persist(&self, puts: &[(String, FileRecord)], deletes: &[String], durable: bool) {
|
||||
if let Some(store) = &self.store
|
||||
&& let Err(e) = store.apply(puts, deletes, durable)
|
||||
@@ -406,6 +645,17 @@ impl Shared {
|
||||
Job::Reparse(before) => reparse.push(before),
|
||||
}
|
||||
}
|
||||
{
|
||||
let mut pending = self.pending_dirs.lock();
|
||||
for path in changed.iter().chain(removed.iter()) {
|
||||
let dir = if path.is_dir() {
|
||||
path.clone()
|
||||
} else {
|
||||
path.parent().map_or_else(|| path.clone(), Path::to_owned)
|
||||
};
|
||||
pending.insert(dir);
|
||||
}
|
||||
}
|
||||
if !rule_dirs.is_empty() {
|
||||
{
|
||||
let mut filter = self.filter.write();
|
||||
@@ -420,14 +670,14 @@ impl Shared {
|
||||
}
|
||||
}
|
||||
if !rescan {
|
||||
self.scan(Some(&roots));
|
||||
self.scan(Some(&roots), None, SystemTime::now());
|
||||
changed.retain(|p| !roots.iter().any(|root| p.starts_with(root)));
|
||||
removed.retain(|p| !roots.iter().any(|root| p.starts_with(root)));
|
||||
}
|
||||
}
|
||||
if rescan {
|
||||
// The scan also picks up whatever `changed` / `removed` reported.
|
||||
self.full_scan();
|
||||
self.full_scan(None, SystemTime::now());
|
||||
changed.clear();
|
||||
removed.clear();
|
||||
}
|
||||
@@ -522,6 +772,7 @@ impl Shared {
|
||||
if touched {
|
||||
self.persist(&[], &[], true);
|
||||
self.emit_state();
|
||||
self.schedule_snapshot();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -541,3 +792,17 @@ enum Extracted {
|
||||
Unchanged,
|
||||
New(FileRecord),
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use crate::data::safe_rel;
|
||||
|
||||
#[test]
|
||||
fn cache_keys_stay_below_the_root() {
|
||||
assert!(safe_rel("src/main.rs"));
|
||||
assert!(safe_rel("a/b/c.rs"));
|
||||
for bad in ["", "/etc/passwd", "../out.rs", "a/../../b.rs", "./a.rs"] {
|
||||
assert!(!safe_rel(bad), "{bad:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -22,11 +22,11 @@ pub use markers::{arduino_sketch_root, is_arduino_sketch_dir, marked_root};
|
||||
pub use plugin_point::PluginLanguages;
|
||||
pub use registry::{FileTypes, LanguageRegistry};
|
||||
pub use syntax::{
|
||||
CSS_CLASS, CSS_ID, CSS_VARIABLE, ConfigurationKey, ExternalFold, ExternalSymbol, ExternalSyntax, ExternalToken,
|
||||
FileReference, Literal, LiteralKind, MAX_CACHED_TREES, MAX_PARSE_BYTES, ParsedTree, SpellingKind, SpellingRegion,
|
||||
Syntax, SyntaxCache, SyntaxCacheStats, SyntaxHighlightKind, SyntaxHighlights, TreeError, VueComponentImport,
|
||||
VueRefAccess, VueRefElement, WebReference, WebReferenceKind, merge_visibility, set_external_syntax, shift_pos,
|
||||
shift_range,
|
||||
CSS_CLASS, CSS_ID, CSS_VARIABLE, ClassBases, ConfigurationKey, ExternalFold, ExternalSymbol, ExternalSyntax,
|
||||
ExternalToken, FileReference, Literal, LiteralKind, MAX_CACHED_TREES, MAX_PARSE_BYTES, ParsedTree, SpellingKind,
|
||||
SpellingRegion, Syntax, SyntaxCache, SyntaxCacheStats, SyntaxHighlightKind, SyntaxHighlights, TreeError,
|
||||
VueComponentImport, VueRefAccess, VueRefElement, WebReference, WebReferenceKind, merge_visibility,
|
||||
set_external_syntax, shift_pos, shift_range,
|
||||
};
|
||||
|
||||
/// Parsed form of `languages/<id>.toml`. Backend-only; the UI gets [`LanguageInfo`].
|
||||
|
||||
@@ -40,6 +40,7 @@ const BUILTIN: &[(&str, &str)] = &[
|
||||
("groovy", include_str!("../../../languages/groovy.toml")),
|
||||
("haskell", include_str!("../../../languages/haskell.toml")),
|
||||
("hcl", include_str!("../../../languages/hcl.toml")),
|
||||
("helm-template", include_str!("../../../languages/helm-template.toml")),
|
||||
("html", include_str!("../../../languages/html.toml")),
|
||||
("http", include_str!("../../../languages/http.toml")),
|
||||
("ignore", include_str!("../../../languages/ignore.toml")),
|
||||
@@ -548,6 +549,7 @@ mod tests {
|
||||
"Groovy",
|
||||
"Haskell",
|
||||
"Haxe",
|
||||
"Helm Template",
|
||||
"HXML",
|
||||
"HTTP",
|
||||
"IDL",
|
||||
@@ -841,6 +843,30 @@ mod tests {
|
||||
assert!(reg.is_framework_marker(&root.join("src/manage.py")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn helm_charts_make_yaml_and_notes_in_templates_folders_helm_templates() {
|
||||
let reg = reg();
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = tmp.path();
|
||||
let detect = |file: &str| reg.detect(&root.join(file), None).as_str().to_owned();
|
||||
std::fs::create_dir_all(root.join("charts/podinfo/templates/tests")).unwrap();
|
||||
std::fs::create_dir_all(root.join("k8s/templates")).unwrap();
|
||||
std::fs::write(root.join("charts/podinfo/Chart.yaml"), "apiVersion: v2\nname: podinfo\n").unwrap();
|
||||
assert_eq!(detect("charts/podinfo/Chart.yaml"), "yaml");
|
||||
assert_eq!(detect("charts/podinfo/values.yaml"), "yaml");
|
||||
assert_eq!(detect("charts/podinfo/templates/deployment.yaml"), "helm-template");
|
||||
assert_eq!(detect("charts/podinfo/templates/tests/connection.yml"), "helm-template");
|
||||
assert_eq!(detect("charts/podinfo/templates/_helpers.tpl"), "helm-template");
|
||||
assert_eq!(detect("charts/podinfo/templates/NOTES.txt"), "helm-template");
|
||||
// No `Chart.yaml` / `Chart.yml` above this `templates` folder.
|
||||
assert_eq!(detect("k8s/templates/deployment.yaml"), "yaml");
|
||||
assert_eq!(detect("common.gotmpl"), "helm-template");
|
||||
// Outside a chart, `.tpl` belongs to other template engines (Smarty, Terraform).
|
||||
assert_ne!(detect("site/index.tpl"), "helm-template");
|
||||
assert!(reg.is_framework_marker(&root.join("charts/podinfo/Chart.yaml")));
|
||||
assert!(reg.is_framework_marker(&root.join("charts/podinfo/Chart.yml")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_django_fixture_has_templates_and_a_page() {
|
||||
let reg = reg();
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
//! Extraction of Python class base names, used to build an inheritance graph
|
||||
//! without an LSP `typeHierarchyProvider`.
|
||||
|
||||
use super::lines::LineIndex;
|
||||
use crate::{LanguageId, Syntax};
|
||||
use tree_sitter::{Node, Tree};
|
||||
|
||||
/// A Python class definition and the base names written in its source.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct ClassBases {
|
||||
/// The class name (bare identifier).
|
||||
pub name: String,
|
||||
/// 0-based line of the class name (selection-range start).
|
||||
pub line: u32,
|
||||
/// 0-based UTF-16 column of the class name (selection-range start).
|
||||
pub col: u32,
|
||||
/// Source-written base names (last segment of a qualified name).
|
||||
pub bases: Vec<String>,
|
||||
}
|
||||
|
||||
impl Syntax {
|
||||
/// Class definitions of `text` with their base names, in source order.
|
||||
/// Only Python is supported; every other language returns an empty vector.
|
||||
pub fn class_bases(&self, lang: &LanguageId, text: &str) -> Vec<ClassBases> {
|
||||
self.analyze(lang, text, |g, tree, text| (g.spec.name == "python").then(|| extract(tree, text)))
|
||||
.unwrap_or_default()
|
||||
}
|
||||
}
|
||||
|
||||
fn extract(tree: &Tree, text: &str) -> Vec<ClassBases> {
|
||||
let idx = LineIndex::new(text);
|
||||
let mut out = Vec::new();
|
||||
let mut cursor = tree.walk();
|
||||
visit(&mut cursor, &mut |node| {
|
||||
if node.kind() != "class_definition" {
|
||||
return;
|
||||
}
|
||||
let Some(name_node) = node.child_by_field_name("name") else { return };
|
||||
let name = text.get(name_node.byte_range()).unwrap_or_default().to_owned();
|
||||
if name.is_empty() {
|
||||
return;
|
||||
}
|
||||
// Selection-range start of the class: its name identifier (`class Foo`
|
||||
// -> `Foo`), matching `outline.rs` and the gutter marker candidates.
|
||||
let range = idx.range(name_node.start_byte(), name_node.end_byte());
|
||||
let bases = node.child_by_field_name("superclasses").map(|args| base_names(args, text)).unwrap_or_default();
|
||||
out.push(ClassBases { name, line: range.start.line, col: range.start.col, bases });
|
||||
});
|
||||
out
|
||||
}
|
||||
|
||||
/// Depth-first pre-order traversal of all descendants of `cursor`'s node.
|
||||
fn visit(cursor: &mut tree_sitter::TreeCursor, f: &mut impl FnMut(Node)) {
|
||||
loop {
|
||||
f(cursor.node());
|
||||
if cursor.goto_first_child() {
|
||||
continue;
|
||||
}
|
||||
loop {
|
||||
if cursor.goto_next_sibling() {
|
||||
break;
|
||||
}
|
||||
if !cursor.goto_parent() {
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn base_names(args: Node, text: &str) -> Vec<String> {
|
||||
let mut out = Vec::new();
|
||||
let mut cursor = args.walk();
|
||||
for arg in args.named_children(&mut cursor) {
|
||||
if let Some(name) = base_name(arg, text) {
|
||||
out.push(name);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Bare name of one base expression, or `None` for keyword arguments and
|
||||
/// expressions that are not a (possibly qualified) name.
|
||||
fn base_name(arg: Node, text: &str) -> Option<String> {
|
||||
match arg.kind() {
|
||||
"identifier" => text.get(arg.byte_range()).map(str::to_owned),
|
||||
// `pkg.mod.Base` -> `Base`.
|
||||
"attribute" => {
|
||||
let attr = arg.child_by_field_name("attribute")?;
|
||||
(attr.kind() == "identifier").then(|| text.get(attr.byte_range()).unwrap_or_default().to_owned())
|
||||
}
|
||||
// `Generic[T]` -> `Generic`.
|
||||
"subscript" => {
|
||||
let value = arg.child_by_field_name("value")?;
|
||||
(value.kind() == "identifier").then(|| text.get(value.byte_range()).unwrap_or_default().to_owned())
|
||||
}
|
||||
// `metaclass=Meta`, `**kwargs`, `*bases` and other expressions.
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn bases(src: &str) -> Vec<ClassBases> {
|
||||
static S: std::sync::OnceLock<Syntax> = std::sync::OnceLock::new();
|
||||
S.get_or_init(Syntax::new).class_bases(&LanguageId::new("python"), src)
|
||||
}
|
||||
|
||||
fn shape(src: &str) -> Vec<(String, u32, u32, Vec<String>)> {
|
||||
bases(src).into_iter().map(|c| (c.name, c.line, c.col, c.bases)).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_bases() {
|
||||
assert_eq!(shape("class A: pass"), vec![("A".to_owned(), 0, 6, vec![])]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multiple_bases_skips_metaclass() {
|
||||
assert_eq!(
|
||||
shape("class B(A, C, metaclass=Meta): pass"),
|
||||
vec![("B".to_owned(), 0, 6, vec!["A".to_owned(), "C".to_owned()])]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn qualified_name_takes_last_segment() {
|
||||
assert_eq!(shape("class D(pkg.mod.Base): pass"), vec![("D".to_owned(), 0, 6, vec!["Base".to_owned()])]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn subscript_takes_value() {
|
||||
assert_eq!(shape("class E(Generic[T]): pass"), vec![("E".to_owned(), 0, 6, vec!["Generic".to_owned()])]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multiline_bases() {
|
||||
let src = "class F(\n A,\n B,\n):\n pass";
|
||||
assert_eq!(shape(src), vec![("F".to_owned(), 0, 6, vec!["A".to_owned(), "B".to_owned()])]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nested_classes() {
|
||||
let src = "class Outer:\n class Inner(Y):\n pass";
|
||||
assert_eq!(
|
||||
shape(src),
|
||||
vec![("Outer".to_owned(), 0, 6, vec![]), ("Inner".to_owned(), 1, 10, vec!["Y".to_owned()]),]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn comments_and_strings_are_ignored() {
|
||||
let src = r#"
|
||||
# class Fake(Nope): pass
|
||||
doc = """
|
||||
class InDocstring(X): pass
|
||||
"""
|
||||
s = 'class InString(Y): pass'
|
||||
class Real(Base): pass
|
||||
"#;
|
||||
assert_eq!(shape(src), vec![("Real".to_owned(), 6, 6, vec!["Base".to_owned()])]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn other_languages_are_empty() {
|
||||
static S: std::sync::OnceLock<Syntax> = std::sync::OnceLock::new();
|
||||
let s = S.get_or_init(Syntax::new);
|
||||
assert!(s.class_bases(&LanguageId::new("rust"), "struct A;").is_empty());
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ mod external;
|
||||
mod folding;
|
||||
mod grammars;
|
||||
mod highlight;
|
||||
mod inheritance;
|
||||
pub(crate) mod lines;
|
||||
mod literals;
|
||||
mod outline;
|
||||
@@ -22,6 +23,7 @@ pub use external::{ExternalFold, ExternalSymbol, ExternalSyntax, ExternalToken,
|
||||
use grammars::{SPECS, Spec};
|
||||
pub use highlight::{SyntaxHighlightKind, SyntaxHighlights};
|
||||
use ide_core::{LanguageId, OutlineItem, Pos, Range};
|
||||
pub use inheritance::ClassBases;
|
||||
pub use lines::{shift_pos, shift_range};
|
||||
pub use literals::{Literal, LiteralKind, SpellingKind, SpellingRegion};
|
||||
pub use outline::merge_visibility;
|
||||
|
||||
@@ -218,6 +218,37 @@ impl ClientHandler {
|
||||
true
|
||||
}
|
||||
|
||||
/// Ends every running task of a tsserver that died at its heap limit: the process
|
||||
/// behind them is gone, so no `end` will ever arrive. Without this the tasks sit
|
||||
/// as zombies until [`PROGRESS_STALE`] and the server stays "loading", holding
|
||||
/// empty answers back for [`LOADING_WAIT`] — while vtsls already restarts
|
||||
/// tsserver, which re-reports its own loading.
|
||||
fn tsserver_died(&self, mgr: &LspManager) {
|
||||
let events: Vec<IdeEvent> = {
|
||||
let mut progress = self.server.progress.lock();
|
||||
progress
|
||||
.drain()
|
||||
.map(|(_, p)| IdeEvent::Task {
|
||||
id: p.task,
|
||||
title: p.title,
|
||||
message: p.message,
|
||||
fraction: Some(1.0),
|
||||
done: true,
|
||||
cancellable: false,
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
let ended = !events.is_empty();
|
||||
for event in events {
|
||||
self.server.emit(mgr.sink(), event);
|
||||
}
|
||||
if ended {
|
||||
self.server.refresh_busy_status(mgr.sink());
|
||||
self.server.progress_ended();
|
||||
}
|
||||
self.server.set_loading(mgr.sink(), false);
|
||||
}
|
||||
|
||||
fn publish_diagnostics(&self, mgr: &LspManager, params: Value) {
|
||||
// Problems come only from servers used for `diagnostics`.
|
||||
if self.server.running_gen(self.generation).is_some_and(|run| !run.caps.read().restriction.diagnostics()) {
|
||||
@@ -410,6 +441,10 @@ impl Handler for ClientHandler {
|
||||
let text = params.get("message").and_then(Value::as_str).unwrap_or_default();
|
||||
if self.server.config.tsserver_project && tsserver_ran_out_of_memory(text) {
|
||||
tracing::warn!("[{}] {text}", self.server.name);
|
||||
// The tasks of the dead tsserver never see their `end`: end them now
|
||||
// instead of letting them sit as zombies until the stale window, and
|
||||
// stop holding queries back — the restarted instance re-reports.
|
||||
self.tsserver_died(&mgr);
|
||||
if mgr.first_message(&self.server.name, TSSERVER_OUT_OF_MEMORY) {
|
||||
mgr.sink().emit(IdeEvent::Notify {
|
||||
level: NotifyLevel::Error,
|
||||
@@ -567,7 +602,12 @@ fn merge(base: &mut Value, layer: Value) {
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::tsserver_ran_out_of_memory;
|
||||
use super::*;
|
||||
use crate::server::Progress;
|
||||
use ide_core::event::SharedSink;
|
||||
use ide_core::{LanguageId, TaskId};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[test]
|
||||
fn tsserver_aborts_are_out_of_memory() {
|
||||
@@ -579,4 +619,85 @@ mod tests {
|
||||
assert!(!tsserver_ran_out_of_memory("Code: 134."));
|
||||
assert!(tsserver_ran_out_of_memory("[error] TSServer exited. Code: 134. Signal: null\n"));
|
||||
}
|
||||
|
||||
/// A vtsls server with two running tasks, one marked stalled, and `loading` set:
|
||||
/// the state a tsserver OOM leaves behind.
|
||||
fn loaded_vtsls(events: Arc<Mutex<Vec<IdeEvent>>>) -> (Arc<LspManager>, Arc<crate::server::Server>) {
|
||||
let sink: SharedSink = {
|
||||
let events = events.clone();
|
||||
Arc::new(move |event: IdeEvent| events.lock().unwrap().push(event))
|
||||
};
|
||||
let languages = Arc::new(ide_language::LanguageRegistry::from_configs(vec![]));
|
||||
let mgr = LspManager::new(PathBuf::from("/p"), languages, sink);
|
||||
let cfg: ide_language::LanguageServerConfig =
|
||||
toml::from_str("name = \"vtsls\"\ncommand = \"vtsls\"\ntsserver-project = true").unwrap();
|
||||
let server = Arc::new(crate::server::Server::new(cfg, PathBuf::from("/p"), LanguageId::new("vue")));
|
||||
server.progress.lock().insert(
|
||||
"loading".into(),
|
||||
Progress {
|
||||
task: TaskId::next(),
|
||||
title: "Initializing 'tsconfig.json'".into(),
|
||||
message: None,
|
||||
cancel: None,
|
||||
last: tokio::time::Instant::now(),
|
||||
fraction: Some(0.1),
|
||||
stalled: false,
|
||||
},
|
||||
);
|
||||
server.progress.lock().insert(
|
||||
"stuck".into(),
|
||||
Progress {
|
||||
task: TaskId::next(),
|
||||
title: "Analyzing 'index.vue' and its dependencies".into(),
|
||||
message: None,
|
||||
cancel: None,
|
||||
last: tokio::time::Instant::now(),
|
||||
fraction: Some(0.2),
|
||||
stalled: true,
|
||||
},
|
||||
);
|
||||
let quiet: SharedSink = Arc::new(|_: IdeEvent| {});
|
||||
server.set_loading(&quiet, true);
|
||||
(mgr, server)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tsserver_dying_ends_every_task_and_the_loading_state() {
|
||||
let events = Arc::new(Mutex::new(Vec::new()));
|
||||
let (mgr, server) = loaded_vtsls(events.clone());
|
||||
let handler = ClientHandler::new(Arc::downgrade(&mgr), server.clone(), 0);
|
||||
|
||||
handler.tsserver_died(&mgr);
|
||||
|
||||
// Both tasks ended (done: true), including the stalled one; none left running.
|
||||
let tasks: Vec<_> = events
|
||||
.lock()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter_map(|e| match e {
|
||||
IdeEvent::Task { title, done, .. } => Some((title.clone(), *done)),
|
||||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(tasks.len(), 2, "{tasks:?}");
|
||||
assert!(tasks.iter().all(|(_, done)| *done), "{tasks:?}");
|
||||
assert!(server.progress.lock().is_empty());
|
||||
assert!(!server.is_loading());
|
||||
assert!(!server.is_busy());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tsserver_dying_without_tasks_only_clears_loading() {
|
||||
let events = Arc::new(Mutex::new(Vec::new()));
|
||||
let (mgr, server) = loaded_vtsls(events.clone());
|
||||
server.progress.lock().clear();
|
||||
let handler = ClientHandler::new(Arc::downgrade(&mgr), server.clone(), 0);
|
||||
|
||||
handler.tsserver_died(&mgr);
|
||||
|
||||
assert!(events.lock().unwrap().is_empty());
|
||||
assert!(!server.is_loading());
|
||||
// A status event may be published, but no task events.
|
||||
assert!(server.progress.lock().is_empty());
|
||||
}
|
||||
}
|
||||
@@ -482,7 +482,8 @@ mod tests {
|
||||
std::fs::create_dir_all(output.join("test001")).unwrap();
|
||||
// As jdtls writes it: the class's source file name in the path, the library's
|
||||
// classpath attributes after the folder.
|
||||
let escaped = output.to_string_lossy().replace('/', "\\/");
|
||||
// JDT writes the folder as a portable path, `/` on Windows too, each `/` escaped.
|
||||
let escaped = output.to_string_lossy().replace('\\', "/").replace('/', "\\/");
|
||||
let handle = format!("=demo/{escaped}=/ai.openidea.kotlin=/true=/<test001(Db.class");
|
||||
let uri = format!("jdt://contents/main-0/test001/Db.kt?{}", utf8_percent_encode(&handle, PATH));
|
||||
assert_eq!(folder_class(&uri), Some(("test001".into(), "Db".into())));
|
||||
|
||||
@@ -40,22 +40,37 @@ const LOOSE: &AsciiSet = &CONTROLS
|
||||
.add(b'|')
|
||||
.add(b'}');
|
||||
|
||||
/// Percent-encodes the characters URIs don't allow in every `file:` URI in `value`, string
|
||||
/// values and object keys alike (a workspace edit's `changes`). Eclipse's jdtls sends a path
|
||||
/// as is (`file:///H:/项目/…`), which `lsp_types::Uri` rejects, failing the whole response.
|
||||
/// Fields of LSP messages whose value is a URI.
|
||||
const URI_FIELDS: [&str; 9] =
|
||||
["uri", "targetUri", "newUri", "oldUri", "baseUri", "scopeUri", "rootUri", "target", "href"];
|
||||
|
||||
/// Percent-encodes the characters URIs don't allow in the `file:` URIs of `value`: the URI
|
||||
/// fields and the keys of a workspace edit's `changes`. Eclipse's jdtls sends a path as is
|
||||
/// (`file:///H:/项目/…`), which `lsp_types::Uri` rejects, failing the whole response. Other
|
||||
/// strings stay as sent: an edit's `newText` may well start with `file:` (YAML, `.env`).
|
||||
pub fn encode_loose_file_uris(value: &mut Value) {
|
||||
match value {
|
||||
Value::String(s) => {
|
||||
if let Some(encoded) = encode_loose(s) {
|
||||
*s = encoded;
|
||||
}
|
||||
}
|
||||
Value::Array(items) => items.iter_mut().for_each(encode_loose_file_uris),
|
||||
Value::Object(map) => {
|
||||
if map.keys().any(|k| encode_loose(k).is_some()) {
|
||||
*map = std::mem::take(map).into_iter().map(|(k, v)| (encode_loose(&k).unwrap_or(k), v)).collect();
|
||||
for (key, value) in map.iter_mut() {
|
||||
match value {
|
||||
Value::String(s) if URI_FIELDS.contains(&key.as_str()) => {
|
||||
if let Some(encoded) = encode_loose(s) {
|
||||
*s = encoded;
|
||||
}
|
||||
}
|
||||
Value::Object(changes) if key == "changes" => {
|
||||
if changes.keys().any(|k| encode_loose(k).is_some()) {
|
||||
*changes = std::mem::take(changes)
|
||||
.into_iter()
|
||||
.map(|(k, v)| (encode_loose(&k).unwrap_or(k), v))
|
||||
.collect();
|
||||
}
|
||||
changes.values_mut().for_each(encode_loose_file_uris);
|
||||
}
|
||||
_ => encode_loose_file_uris(value),
|
||||
}
|
||||
}
|
||||
map.values_mut().for_each(encode_loose_file_uris);
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
@@ -161,9 +176,28 @@ mod tests {
|
||||
assert!(v["edit"]["changes"].get("file:///tmp/%E7%9B%AE%E5%BD%95/b.java").is_some());
|
||||
assert_eq!(v["text"], "项目", "only file: URIs change");
|
||||
// Already encoded: untouched.
|
||||
let mut encoded = serde_json::json!("file:///a%20b/%E9%A1%B9.java");
|
||||
let mut encoded = serde_json::json!({ "uri": "file:///a%20b/%E9%A1%B9.java" });
|
||||
encode_loose_file_uris(&mut encoded);
|
||||
assert_eq!(encoded, "file:///a%20b/%E9%A1%B9.java");
|
||||
assert_eq!(encoded["uri"], "file:///a%20b/%E9%A1%B9.java");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_that_starts_with_file_is_not_a_uri() {
|
||||
// A YAML formatting edit and a completion whose text begins with `file:`.
|
||||
let mut v = serde_json::json!({ "result": [
|
||||
{ "range": { "start": { "line": 0, "character": 0 }, "end": { "line": 1, "character": 0 } }, "newText": "file: a b\n - c\n" },
|
||||
{ "label": "file:", "insertText": "file: ${1:path}", "documentation": "file: the input" }
|
||||
], "edit": { "changes": { "file:///tmp/a b.yaml": [{ "newText": "file: x y" }] } } });
|
||||
encode_loose_file_uris(&mut v);
|
||||
assert_eq!(v["result"][0]["newText"], "file: a b\n - c\n");
|
||||
assert_eq!(v["result"][1]["insertText"], "file: ${1:path}");
|
||||
assert_eq!(v["result"][1]["documentation"], "file: the input");
|
||||
let changes = &v["edit"]["changes"]["file:///tmp/a%20b.yaml"];
|
||||
assert_eq!(changes[0]["newText"], "file: x y");
|
||||
// A diagnostic's help link is a URI too.
|
||||
let mut d = serde_json::json!({ "codeDescription": { "href": "file:///tmp/help files/rule.html" } });
|
||||
encode_loose_file_uris(&mut d);
|
||||
assert_eq!(d["codeDescription"]["href"], "file:///tmp/help%20files/rule.html");
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::io;
|
||||
use std::net::{IpAddr, Ipv4Addr};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// The discovery file, relative to the app data directory.
|
||||
@@ -39,6 +40,9 @@ pub fn kept_token(path: &Path) -> io::Result<String> {
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct Discovery {
|
||||
/// Where a client on this computer connects (the file of an IDE without this field: `127.0.0.1`).
|
||||
#[serde(default = "loopback")]
|
||||
pub address: IpAddr,
|
||||
pub port: u16,
|
||||
pub token: String,
|
||||
/// The IDE's process.
|
||||
@@ -47,6 +51,10 @@ pub struct Discovery {
|
||||
pub version: String,
|
||||
}
|
||||
|
||||
fn loopback() -> IpAddr {
|
||||
IpAddr::V4(Ipv4Addr::LOCALHOST)
|
||||
}
|
||||
|
||||
impl Discovery {
|
||||
pub fn read(path: &Path) -> io::Result<Self> {
|
||||
let bytes = std::fs::read(path)?;
|
||||
@@ -103,7 +111,13 @@ mod tests {
|
||||
fn round_trips_privately_and_only_its_owner_removes_it() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let path = discovery_path(tmp.path());
|
||||
let ours = Discovery { port: 1234, token: "t".into(), pid: std::process::id(), version: "0.1.0".into() };
|
||||
let ours = Discovery {
|
||||
address: "100.92.72.95".parse().unwrap(),
|
||||
port: 1234,
|
||||
token: "t".into(),
|
||||
pid: std::process::id(),
|
||||
version: "0.1.0".into(),
|
||||
};
|
||||
ours.write(&path).unwrap();
|
||||
assert_eq!(Discovery::read(&path).unwrap(), ours);
|
||||
#[cfg(unix)]
|
||||
@@ -118,6 +132,10 @@ mod tests {
|
||||
ours.write(&path).unwrap();
|
||||
Discovery::remove_if_ours(&path);
|
||||
assert!(!path.exists());
|
||||
|
||||
// The file of an IDE that listened on 127.0.0.1 only has no address.
|
||||
std::fs::write(&path, r#"{"port":1,"token":"t","pid":1,"version":"0.1.0"}"#).unwrap();
|
||||
assert_eq!(Discovery::read(&path).unwrap().address, loopback());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -2,14 +2,15 @@
|
||||
//! the IDE's built-in tools.
|
||||
//!
|
||||
//! - **Transport:** MCP Streamable HTTP (2025-06-18; 2025-03-26 and 2024-11-05
|
||||
//! clients too) on `127.0.0.1:<random port>`, endpoint `/mcp`. Responses are
|
||||
//! clients too) on `127.0.0.1:<random port>` (or another address of
|
||||
//! [`ServerOptions`]), endpoint `/mcp`. Responses are
|
||||
//! plain JSON (no SSE streams: `GET` is 405). Every request carries
|
||||
//! `Authorization: Bearer <token>` (random, per run); requests from another
|
||||
//! `Origin` or `Host` are refused (DNS rebinding).
|
||||
//! - **Projects:** the `X-OpenIDEA-Project` header or the `?project=` query
|
||||
//! names a folder; the server picks the open project containing it.
|
||||
//! - **Discovery:** `<app-data>/mcp/server.json` (`port`, `token`, `pid`,
|
||||
//! `version`; readable only by the user) while the server runs.
|
||||
//! - **Discovery:** `<app-data>/mcp/server.json` (`address`, `port`, `token`,
|
||||
//! `pid`, `version`; readable only by the user) while the server runs.
|
||||
//! - **Stdio:** `<app exe> --mcp-stdio [--project <path>]` reads the discovery
|
||||
//! file and relays newline-delimited JSON-RPC to the server ([`stdio`]), so
|
||||
//! persisted client configurations hold no token.
|
||||
@@ -23,7 +24,7 @@ mod server;
|
||||
pub mod stdio;
|
||||
|
||||
pub use discovery::{DISCOVERY_FILE, Discovery, TOKEN_FILE, discovery_path, kept_token, token_path};
|
||||
pub use server::{McpServer, ServerOptions};
|
||||
pub use server::{McpServer, ServerOptions, local_address};
|
||||
pub use tokio_util::sync::CancellationToken;
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
@@ -85,9 +86,15 @@ pub trait ToolRegistry: Send + Sync + 'static {
|
||||
) -> Result<String, ToolCallError>;
|
||||
}
|
||||
|
||||
/// The URL of the server at `port` for `project`.
|
||||
/// The URL of the server at `127.0.0.1:<port>` for `project`.
|
||||
pub fn endpoint_url(port: u16, project: &Path) -> String {
|
||||
format!("http://127.0.0.1:{port}/mcp?project={}", encode_query(&project.to_string_lossy()))
|
||||
endpoint_url_at(std::net::Ipv4Addr::LOCALHOST.into(), port, project)
|
||||
}
|
||||
|
||||
/// The URL of the server at `<address>:<port>` for `project`.
|
||||
pub fn endpoint_url_at(address: std::net::IpAddr, port: u16, project: &Path) -> String {
|
||||
let at = std::net::SocketAddr::new(address, port);
|
||||
format!("http://{at}/mcp?project={}", encode_query(&project.to_string_lossy()))
|
||||
}
|
||||
|
||||
/// The stdio bridge's command line for `project`: `exe --mcp-stdio --project <project>`.
|
||||
@@ -133,6 +140,9 @@ mod tests {
|
||||
fn urls_and_commands_name_the_project() {
|
||||
let url = endpoint_url(4321, Path::new("/home/me/my app/项目"));
|
||||
assert_eq!(url, "http://127.0.0.1:4321/mcp?project=/home/me/my%20app/%E9%A1%B9%E7%9B%AE");
|
||||
let ip = |s: &str| s.parse::<std::net::IpAddr>().unwrap();
|
||||
assert_eq!(endpoint_url_at(ip("100.92.72.95"), 80, Path::new("/p")), "http://100.92.72.95:80/mcp?project=/p");
|
||||
assert_eq!(endpoint_url_at(ip("fd7a::1"), 80, Path::new("/p")), "http://[fd7a::1]:80/mcp?project=/p");
|
||||
assert_eq!(
|
||||
stdio_command(Path::new("/Applications/OpenIDEA.app/Contents/MacOS/openidea"), Path::new("/p")),
|
||||
["/Applications/OpenIDEA.app/Contents/MacOS/openidea", "--mcp-stdio", "--project", "/p"]
|
||||
|
||||
@@ -9,6 +9,7 @@ use hyper::header::{self, HeaderMap, HeaderValue};
|
||||
use hyper::{Method, Request, Response, StatusCode};
|
||||
use serde_json::{Value, json};
|
||||
use std::convert::Infallible;
|
||||
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::pin::Pin;
|
||||
use std::sync::Arc;
|
||||
@@ -40,6 +41,8 @@ type Body = Either<Full<Bytes>, EventStream>;
|
||||
struct Shared {
|
||||
registry: Arc<dyn ToolRegistry>,
|
||||
token: String,
|
||||
/// The address listened on (`0.0.0.0` / `::`: every address).
|
||||
address: IpAddr,
|
||||
port: u16,
|
||||
requests: protocol::Requests,
|
||||
cancel: CancellationToken,
|
||||
@@ -48,6 +51,7 @@ struct Shared {
|
||||
/// The running server. Stopping it (or dropping it) closes the port and
|
||||
/// removes its discovery file.
|
||||
pub struct McpServer {
|
||||
address: IpAddr,
|
||||
port: u16,
|
||||
/// Why the port asked for in [`ServerOptions::port`] was not used.
|
||||
port_unavailable: Option<String>,
|
||||
@@ -60,7 +64,10 @@ pub struct McpServer {
|
||||
/// How [`McpServer::start_with`] listens.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct ServerOptions {
|
||||
/// Listen on `127.0.0.1:<port>`. When it can't be bound (in use, or reserved by the
|
||||
/// The address to listen on; `None`: `127.0.0.1`, this computer only. Another address of
|
||||
/// this computer, or `0.0.0.0` / `::` for every address, lets other computers connect.
|
||||
pub address: Option<IpAddr>,
|
||||
/// Listen on `<address>:<port>`. When it can't be bound (in use, or reserved by the
|
||||
/// system), a free port is taken instead and [`McpServer::port_unavailable`] says why.
|
||||
/// `None`: a free port.
|
||||
pub port: Option<u16>,
|
||||
@@ -86,15 +93,20 @@ impl McpServer {
|
||||
runtime: &tokio::runtime::Handle,
|
||||
options: ServerOptions,
|
||||
) -> std::io::Result<Self> {
|
||||
let address = options.address.unwrap_or(IpAddr::V4(Ipv4Addr::LOCALHOST));
|
||||
let bind = |port: u16| {
|
||||
std::net::TcpListener::bind(SocketAddr::new(address, port))
|
||||
.map_err(|e| std::io::Error::new(e.kind(), format!("{}: {e}", SocketAddr::new(address, port))))
|
||||
};
|
||||
let (listener, port_unavailable) = match options.port {
|
||||
Some(port) => match std::net::TcpListener::bind(("127.0.0.1", port)) {
|
||||
Some(port) => match bind(port) {
|
||||
Ok(listener) => (listener, None),
|
||||
Err(e) => {
|
||||
tracing::warn!("MCP server: port {port} unavailable ({e}); taking a free one");
|
||||
(std::net::TcpListener::bind(("127.0.0.1", 0))?, Some(format!("port {port}: {e}")))
|
||||
(bind(0)?, Some(format!("port {port}: {e}")))
|
||||
}
|
||||
},
|
||||
None => (std::net::TcpListener::bind(("127.0.0.1", 0))?, None),
|
||||
None => (bind(0)?, None),
|
||||
};
|
||||
listener.set_nonblocking(true)?;
|
||||
let port = listener.local_addr()?.port();
|
||||
@@ -108,6 +120,7 @@ impl McpServer {
|
||||
};
|
||||
if let Some(path) = &discovery {
|
||||
let info = Discovery {
|
||||
address: local_address(address),
|
||||
port,
|
||||
token: token.clone(),
|
||||
pid: std::process::id(),
|
||||
@@ -120,13 +133,19 @@ impl McpServer {
|
||||
let shared = Arc::new(Shared {
|
||||
registry,
|
||||
token: token.clone(),
|
||||
address,
|
||||
port,
|
||||
requests: Default::default(),
|
||||
cancel: cancel.clone(),
|
||||
});
|
||||
runtime.spawn(accept(listener, shared, stopped));
|
||||
tracing::info!("MCP server on 127.0.0.1:{port}");
|
||||
Ok(Self { port, port_unavailable, token, discovery, shutdown, cancel })
|
||||
tracing::info!("MCP server on {}", SocketAddr::new(address, port));
|
||||
Ok(Self { address, port, port_unavailable, token, discovery, shutdown, cancel })
|
||||
}
|
||||
|
||||
/// The address listened on (`0.0.0.0` / `::`: every address).
|
||||
pub fn address(&self) -> IpAddr {
|
||||
self.address
|
||||
}
|
||||
|
||||
pub fn port(&self) -> u16 {
|
||||
@@ -143,14 +162,24 @@ impl McpServer {
|
||||
&self.token
|
||||
}
|
||||
|
||||
/// The endpoint URL for `project`.
|
||||
/// The endpoint URL for `project`, at the address a client on this computer connects to.
|
||||
pub fn url(&self, project: &Path) -> String {
|
||||
crate::endpoint_url(self.port, project)
|
||||
crate::endpoint_url_at(local_address(self.address), self.port, project)
|
||||
}
|
||||
|
||||
pub fn stop(self) {}
|
||||
}
|
||||
|
||||
/// The address a client on this computer connects to for a server listening on `address`:
|
||||
/// loopback for every address (`0.0.0.0` / `::`), else `address` itself.
|
||||
pub fn local_address(address: IpAddr) -> IpAddr {
|
||||
match address {
|
||||
IpAddr::V4(a) if a.is_unspecified() => IpAddr::V4(Ipv4Addr::LOCALHOST),
|
||||
IpAddr::V6(a) if a.is_unspecified() => IpAddr::V6(Ipv6Addr::LOCALHOST),
|
||||
a => a,
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for McpServer {
|
||||
fn drop(&mut self) {
|
||||
self.cancel.cancel();
|
||||
@@ -212,7 +241,7 @@ async fn respond(req: Request<Incoming>, shared: &Arc<Shared>, scope: String) ->
|
||||
if req.uri().path() != "/mcp" {
|
||||
return plain(StatusCode::NOT_FOUND, "not found: the endpoint is /mcp");
|
||||
}
|
||||
if !local_request(req.headers(), shared.port) {
|
||||
if !local_request(req.headers(), shared.address, shared.port) {
|
||||
return plain(StatusCode::FORBIDDEN, "requests from web pages are refused");
|
||||
}
|
||||
if !authorized(req.headers(), &shared.token) {
|
||||
@@ -343,11 +372,16 @@ impl hyper::body::Body for EventStream {
|
||||
}
|
||||
}
|
||||
|
||||
/// No `Origin` other than this server's own, and a loopback `Host`: a web
|
||||
/// page (DNS rebinding) can't reach the tools even before the token check.
|
||||
fn local_request(headers: &HeaderMap, port: u16) -> bool {
|
||||
/// No `Origin` other than this server's own, and a `Host` naming this server by IP address
|
||||
/// (loopback, the address listened on, or any when listening on every address) or as
|
||||
/// `localhost`: never another host name, so a web page (DNS rebinding) can't reach the tools
|
||||
/// even before the token check.
|
||||
fn local_request(headers: &HeaderMap, address: IpAddr, port: u16) -> bool {
|
||||
let ours = |authority: &str| {
|
||||
["127.0.0.1", "localhost", "[::1]"].iter().any(|host| authority.eq_ignore_ascii_case(&format!("{host}:{port}")))
|
||||
authority.eq_ignore_ascii_case(&format!("localhost:{port}"))
|
||||
|| authority.parse::<SocketAddr>().is_ok_and(|a| {
|
||||
a.port() == port && (a.ip().is_loopback() || a.ip() == address || address.is_unspecified())
|
||||
})
|
||||
};
|
||||
let host_ok = headers.get(header::HOST).is_none_or(|h| h.to_str().is_ok_and(ours));
|
||||
let origin_ok = headers.get(header::ORIGIN).is_none_or(|o| {
|
||||
@@ -534,7 +568,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_loopback_hosts_and_origins_of_this_server_pass() {
|
||||
fn only_hosts_and_origins_of_this_server_pass() {
|
||||
let headers = |pairs: &[(header::HeaderName, &str)]| {
|
||||
let mut h = HeaderMap::new();
|
||||
for (k, v) in pairs {
|
||||
@@ -542,16 +576,43 @@ mod tests {
|
||||
}
|
||||
h
|
||||
};
|
||||
assert!(local_request(&headers(&[]), 80));
|
||||
assert!(local_request(&headers(&[(header::HOST, "127.0.0.1:80")]), 80));
|
||||
let loopback = IpAddr::V4(Ipv4Addr::LOCALHOST);
|
||||
let host = |h: &str, address: IpAddr| local_request(&headers(&[(header::HOST, h)]), address, 80);
|
||||
assert!(local_request(&headers(&[]), loopback, 80));
|
||||
assert!(host("127.0.0.1:80", loopback));
|
||||
assert!(host("[::1]:80", loopback));
|
||||
assert!(local_request(
|
||||
&headers(&[(header::HOST, "localhost:80"), (header::ORIGIN, "http://localhost:80")]),
|
||||
loopback,
|
||||
80
|
||||
));
|
||||
assert!(!local_request(&headers(&[(header::HOST, "evil.example:80")]), 80));
|
||||
assert!(!local_request(&headers(&[(header::HOST, "127.0.0.1:81")]), 80));
|
||||
assert!(!local_request(&headers(&[(header::ORIGIN, "https://evil.example")]), 80));
|
||||
assert!(!local_request(&headers(&[(header::ORIGIN, "null")]), 80));
|
||||
assert!(!host("evil.example:80", loopback));
|
||||
assert!(!host("127.0.0.1:81", loopback));
|
||||
assert!(!host("100.92.72.95:80", loopback), "another address of a loopback server");
|
||||
assert!(!local_request(&headers(&[(header::ORIGIN, "https://evil.example")]), loopback, 80));
|
||||
assert!(!local_request(&headers(&[(header::ORIGIN, "null")]), loopback, 80));
|
||||
|
||||
// Listening on one address: that address too, still never a host name.
|
||||
let lan: IpAddr = "100.92.72.95".parse().unwrap();
|
||||
assert!(host("100.92.72.95:80", lan) && host("127.0.0.1:80", lan));
|
||||
assert!(!host("100.92.72.96:80", lan) && !host("evil.example:80", lan));
|
||||
// Listening on every address: any IP address, still never a host name.
|
||||
let every: IpAddr = "0.0.0.0".parse().unwrap();
|
||||
assert!(host("192.168.1.7:80", every) && host("[fe80::1]:80", every));
|
||||
assert!(!host("evil.example:80", every) && !host("192.168.1.7:81", every));
|
||||
assert!(local_request(
|
||||
&headers(&[(header::HOST, "192.168.1.7:80"), (header::ORIGIN, "http://192.168.1.7:80")]),
|
||||
every,
|
||||
80
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clients_on_this_computer_connect_to_loopback_for_every_address() {
|
||||
let ip = |s: &str| s.parse::<IpAddr>().unwrap();
|
||||
assert_eq!(local_address(ip("0.0.0.0")), ip("127.0.0.1"));
|
||||
assert_eq!(local_address(ip("::")), ip("::1"));
|
||||
assert_eq!(local_address(ip("100.92.72.95")), ip("100.92.72.95"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -148,13 +148,14 @@ async fn post(
|
||||
)
|
||||
};
|
||||
let info = Discovery::read(discovery).map_err(|_| not_running())?;
|
||||
let stream = tokio::net::TcpStream::connect(("127.0.0.1", info.port)).await.map_err(|_| not_running())?;
|
||||
let at = std::net::SocketAddr::new(info.address, info.port);
|
||||
let stream = tokio::net::TcpStream::connect(at).await.map_err(|_| not_running())?;
|
||||
let (mut sender, conn) = hyper::client::conn::http1::handshake(hyper_util::rt::TokioIo::new(stream))
|
||||
.await
|
||||
.map_err(|e| format!("cannot reach OpenIDEA: {e}"))?;
|
||||
tokio::spawn(conn);
|
||||
let request = Request::post(format!("/mcp?project={}", crate::encode_query(&project.to_string_lossy())))
|
||||
.header(header::HOST, format!("127.0.0.1:{}", info.port))
|
||||
.header(header::HOST, at.to_string())
|
||||
.header(header::AUTHORIZATION, format!("Bearer {}", info.token))
|
||||
.header("mcp-session-id", scope)
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
|
||||
@@ -373,7 +373,7 @@ async fn a_fixed_port_and_token_are_used_and_a_taken_port_falls_back_to_a_free_o
|
||||
s.server.take().unwrap().stop();
|
||||
let free = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
|
||||
let token = "ab".repeat(32);
|
||||
let options = ServerOptions { port: Some(free), token: Some(token.clone()) };
|
||||
let options = ServerOptions { port: Some(free), token: Some(token.clone()), ..Default::default() };
|
||||
let registry = Arc::new(Registry { project: s.project.clone() });
|
||||
let runtime = tokio::runtime::Handle::current();
|
||||
s.server =
|
||||
@@ -393,3 +393,34 @@ async fn a_fixed_port_and_token_are_used_and_a_taken_port_falls_back_to_a_free_o
|
||||
second.port_unavailable()
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
async fn another_listen_address_is_used_and_one_not_on_this_computer_fails() {
|
||||
let mut s = setup();
|
||||
s.server.take().unwrap().stop();
|
||||
let registry = Arc::new(Registry { project: s.project.clone() });
|
||||
let runtime = tokio::runtime::Handle::current();
|
||||
|
||||
// Every address: clients here use loopback; other computers name the server by an address of it.
|
||||
let every = ServerOptions { address: Some("0.0.0.0".parse().unwrap()), ..Default::default() };
|
||||
s.server = Some(McpServer::start_with(registry.clone(), Some(s.discovery.clone()), &runtime, every).unwrap());
|
||||
let port = s.server().port();
|
||||
assert_eq!(s.server().address().to_string(), "0.0.0.0");
|
||||
assert!(s.server().url(&s.project).starts_with(&format!("http://127.0.0.1:{port}/mcp?")));
|
||||
assert_eq!(Discovery::read(&s.discovery).unwrap().address.to_string(), "127.0.0.1");
|
||||
let ping = json!({ "jsonrpc": "2.0", "id": 1, "method": "ping" }).to_string();
|
||||
let request = |host: String| {
|
||||
format!(
|
||||
"POST /mcp HTTP/1.1\r\nHost: {host}\r\n{}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{ping}",
|
||||
bearer(&s),
|
||||
ping.len()
|
||||
)
|
||||
};
|
||||
assert_eq!(http(port, &request(format!("192.0.2.7:{port}"))).await.0, 200);
|
||||
assert_eq!(http(port, &request(format!("evil.example:{port}"))).await.0, 403, "never a host name");
|
||||
|
||||
// Not an address of this computer: the server does not start, and says which address.
|
||||
let elsewhere = ServerOptions { address: Some("192.0.2.1".parse().unwrap()), ..Default::default() };
|
||||
let error = McpServer::start_with(registry, None, &runtime, elsewhere).err().expect("no such address here");
|
||||
assert!(error.to_string().contains("192.0.2.1"), "{error}");
|
||||
}
|
||||
@@ -209,10 +209,11 @@ impl Deployment<'_> {
|
||||
let start = out.len();
|
||||
// One `find` over SSH instead of a listing per folder (thousands of round trips on a
|
||||
// big project); the same entries the walk below would keep.
|
||||
if let Some(entries) = self.fs.tree(dir, self.exclude.prunable_names()).await {
|
||||
if self.cancel.is_cancelled() {
|
||||
return Err(IdeError::cancelled());
|
||||
}
|
||||
let tree = tokio::select! {
|
||||
tree = self.fs.tree(dir, self.exclude.prunable_names()) => tree,
|
||||
() = self.cancel.cancelled() => return Err(IdeError::cancelled()),
|
||||
};
|
||||
if let Some(entries) = tree {
|
||||
let base = dir.trim_end_matches('/');
|
||||
out.extend(entries.into_iter().filter(|e| {
|
||||
let rel = e.path.strip_prefix(base).unwrap_or(&e.path);
|
||||
|
||||
@@ -403,7 +403,7 @@ async fn download_into_copies_only_the_selected_items() {
|
||||
assert!(std::fs::read_dir(&elsewhere).unwrap().next().is_none(), "nothing written through the links");
|
||||
}
|
||||
|
||||
let missing = [server.join("gone").to_str().unwrap().to_owned()];
|
||||
let missing = [server.join("gone").to_str().unwrap().replace('\\', "/")];
|
||||
let report = service
|
||||
.download_into(&FsConnection::Local, &missing, &project, &CancellationToken::new(), &mut |_, _, _| {})
|
||||
.await
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
pub(crate) mod cmake;
|
||||
mod dotnet;
|
||||
mod zig;
|
||||
|
||||
use crate::RunConfig;
|
||||
use ide_core::project_roots::expand_members;
|
||||
@@ -72,6 +73,7 @@ pub(crate) fn detect_with_profiles(
|
||||
}
|
||||
}
|
||||
}
|
||||
detect_projects(root, &["build.zig"], zig::detect, &mut out);
|
||||
dotnet::detect(root, &mut out);
|
||||
detect_projects(root, &["Makefile", "makefile", "GNUmakefile"], detect_make, &mut out);
|
||||
detect_projects(root, &["justfile", "Justfile", ".justfile"], detect_just, &mut out);
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
//! Zig projects (`build.zig`): the build's steps (`zig build <step>`) and, for each executable it
|
||||
//! declares, the installed binary (`zig-out/bin/<name>`), built first and debuggable with CodeLLDB.
|
||||
//! Read from the build script's source: detecting configurations never runs the project's build code.
|
||||
|
||||
use super::{ConfigExt, config, read};
|
||||
use crate::{BeforeLaunchTask, RunConfig};
|
||||
use std::path::Path;
|
||||
|
||||
pub(super) const BUILD_ID: &str = "zig:build";
|
||||
|
||||
pub(super) fn detect(dir: &Path, out: &mut Vec<RunConfig>) {
|
||||
let Some(src) = read(&dir.join("build.zig")) else { return };
|
||||
let src = code_only(&src);
|
||||
out.push(config(BUILD_ID, "zig build", "zig", "zig", &["build"]));
|
||||
for step in steps(&src) {
|
||||
// `install` is what plain `zig build` runs.
|
||||
if step == "install" || step == "uninstall" {
|
||||
continue;
|
||||
}
|
||||
let mut c = config(format!("zig:{step}"), format!("zig build {step}"), "zig", "zig", &["build", &step]);
|
||||
if step == "test" || step.starts_with("test-") || step.starts_with("test_") {
|
||||
c = c.test();
|
||||
}
|
||||
out.push(c);
|
||||
}
|
||||
for exe in executables(&src) {
|
||||
let binary = if cfg!(windows) {
|
||||
format!("zig-out\\bin\\{}.exe", exe.name)
|
||||
} else {
|
||||
format!("./zig-out/bin/{}", exe.name)
|
||||
};
|
||||
let mut c = config(format!("zig:exe:{}", exe.name), exe.name.clone(), "zig-application", &binary, &[])
|
||||
.debug("codelldb");
|
||||
c.extras.before_launch.push(BeforeLaunchTask { kind: "run".into(), target: BUILD_ID.into() });
|
||||
if let Some(source) = exe.source {
|
||||
c.extras.params.insert("source".into(), source);
|
||||
}
|
||||
out.push(c);
|
||||
}
|
||||
}
|
||||
|
||||
/// `src` with comments blanked out (Zig has only `//` comments) and string contents kept.
|
||||
fn code_only(src: &str) -> String {
|
||||
let mut out = String::with_capacity(src.len());
|
||||
for line in src.lines() {
|
||||
let mut in_string = false;
|
||||
let mut escaped = false;
|
||||
let mut end = line.len();
|
||||
let bytes = line.as_bytes();
|
||||
for (i, &b) in bytes.iter().enumerate() {
|
||||
if in_string {
|
||||
match b {
|
||||
_ if escaped => escaped = false,
|
||||
b'\\' => escaped = true,
|
||||
b'"' => in_string = false,
|
||||
_ => {}
|
||||
}
|
||||
} else if b == b'"' {
|
||||
in_string = true;
|
||||
} else if b == b'/' && bytes.get(i + 1) == Some(&b'/') {
|
||||
end = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
out.push_str(&line[..end]);
|
||||
out.push('\n');
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The string literal starting at `s` (after optional whitespace), unescaped only for `\"`.
|
||||
fn string_at(s: &str) -> Option<String> {
|
||||
let rest = s.trim_start().strip_prefix('"')?;
|
||||
let mut value = String::new();
|
||||
let mut chars = rest.chars();
|
||||
while let Some(c) = chars.next() {
|
||||
match c {
|
||||
'"' => return Some(value),
|
||||
'\\' => value.push(chars.next()?),
|
||||
c => value.push(c),
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Step names from `<build>.step("name", "description")`, in order and without duplicates.
|
||||
fn steps(src: &str) -> Vec<String> {
|
||||
let mut out: Vec<String> = Vec::new();
|
||||
for (i, _) in src.match_indices(".step(") {
|
||||
let preceded = src[..i].chars().next_back().is_some_and(|c| c.is_alphanumeric() || c == '_');
|
||||
if let Some(name) = string_at(&src[i + ".step(".len()..])
|
||||
&& preceded
|
||||
&& !name.is_empty()
|
||||
&& !out.contains(&name)
|
||||
{
|
||||
out.push(name);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
struct Executable {
|
||||
name: String,
|
||||
/// Root source file, relative to the build script's folder.
|
||||
source: Option<String>,
|
||||
}
|
||||
|
||||
/// `addExecutable(.{ .name = "…", … .root_source_file = <b>.path("…") … })`: the name at the
|
||||
/// options' top level (an `imports` entry has its own `.name`) and the first root source file.
|
||||
fn executables(src: &str) -> Vec<Executable> {
|
||||
let mut out: Vec<Executable> = Vec::new();
|
||||
for (start, _) in src.match_indices("addExecutable(") {
|
||||
let body = &src[start + "addExecutable(".len()..];
|
||||
let (mut depth, mut in_string, mut escaped) = (1usize, false, false);
|
||||
let mut name = None;
|
||||
let mut source = None;
|
||||
for (i, c) in body.char_indices() {
|
||||
if in_string {
|
||||
match c {
|
||||
_ if escaped => escaped = false,
|
||||
'\\' => escaped = true,
|
||||
'"' => in_string = false,
|
||||
_ => {}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
match c {
|
||||
'"' => in_string = true,
|
||||
'(' | '{' | '[' => depth += 1,
|
||||
')' | '}' | ']' => {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
'.' => {
|
||||
let field = &body[i + 1..];
|
||||
let value =
|
||||
|key: &str| field.strip_prefix(key)?.trim_start().strip_prefix('=').map(str::trim_start);
|
||||
if depth == 2 && name.is_none() {
|
||||
name = value("name").and_then(string_at);
|
||||
}
|
||||
if source.is_none()
|
||||
&& let Some(v) = value("root_source_file")
|
||||
&& let Some(call) = v.find(".path(")
|
||||
&& v[..call].chars().all(|c| c.is_alphanumeric() || c == '_')
|
||||
{
|
||||
source = string_at(&v[call + ".path(".len()..]);
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if let Some(name) = name
|
||||
&& !name.is_empty()
|
||||
&& !out.iter().any(|e| e.name == name)
|
||||
{
|
||||
out.push(Executable { name, source });
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const INIT: &str = r#"const std = @import("std");
|
||||
|
||||
pub fn build(b: *std.Build) void {
|
||||
const target = b.standardTargetOptions(.{});
|
||||
const mod = b.addModule("app", .{
|
||||
.root_source_file = b.path("src/root.zig"),
|
||||
.target = target,
|
||||
});
|
||||
// const old = b.addExecutable(.{ .name = "commented-out" });
|
||||
const exe = b.addExecutable(.{
|
||||
.name = "app",
|
||||
.root_module = b.createModule(.{
|
||||
.root_source_file = b.path("src/main.zig"),
|
||||
.target = target,
|
||||
.imports = &.{
|
||||
.{ .name = "app", .module = mod },
|
||||
},
|
||||
}),
|
||||
});
|
||||
b.installArtifact(exe);
|
||||
const tool = b.addExecutable(.{ .name = "gen-docs", .root_module = b.createModule(.{
|
||||
.root_source_file = b.path("tools/gen_docs.zig"),
|
||||
}) });
|
||||
_ = tool;
|
||||
const run_step = b.step("run", "Run the app");
|
||||
const test_step = b.step("test", "Run tests");
|
||||
const docs = b.step("docs", "Generate docs // not a comment");
|
||||
_ = .{ run_step, test_step, docs };
|
||||
}
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn steps_and_executables_come_from_the_build_script() {
|
||||
let src = code_only(INIT);
|
||||
assert_eq!(steps(&src), ["run", "test", "docs"]);
|
||||
let exes = executables(&src);
|
||||
assert_eq!(
|
||||
exes.iter().map(|e| (e.name.as_str(), e.source.as_deref())).collect::<Vec<_>>(),
|
||||
[("app", Some("src/main.zig")), ("gen-docs", Some("tools/gen_docs.zig"))]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zig_project_gets_build_steps_and_debuggable_executables() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
std::fs::write(dir.path().join("build.zig"), INIT).unwrap();
|
||||
let mut out = Vec::new();
|
||||
detect(dir.path(), &mut out);
|
||||
let ids: Vec<_> = out.iter().map(|c| c.id.as_str()).collect();
|
||||
assert_eq!(ids, ["zig:build", "zig:run", "zig:test", "zig:docs", "zig:exe:app", "zig:exe:gen-docs"]);
|
||||
let test = out.iter().find(|c| c.id == "zig:test").unwrap();
|
||||
assert!(test.is_test);
|
||||
assert_eq!(
|
||||
(test.command.as_str(), test.args.as_slice()),
|
||||
("zig", ["build".to_owned(), "test".to_owned()].as_slice())
|
||||
);
|
||||
let app = out.iter().find(|c| c.id == "zig:exe:app").unwrap();
|
||||
assert_eq!(app.kind, "zig-application");
|
||||
assert!(app.command.ends_with(if cfg!(windows) { "app.exe" } else { "zig-out/bin/app" }));
|
||||
assert_eq!(app.debug_adapter.as_deref(), Some("codelldb"));
|
||||
assert_eq!(app.extras.before_launch[0].kind, "run");
|
||||
assert_eq!(app.extras.before_launch[0].target, BUILD_ID);
|
||||
assert_eq!(app.extras.params.get("source").map(String::as_str), Some("src/main.zig"));
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,8 @@
|
||||
//! `.vscode/launch.json`: JSON with comments and trailing commas, `configurations` and
|
||||
//! `compounds`. Python (`debugpy`, the older `python`) and Node.js (`node`, `pwa-node`,
|
||||
//! `node-terminal`) launches become run configurations of the matching kind, their attach
|
||||
//! requests Python Remote Debug / Attach to Node.js configurations. Variables with a fixed
|
||||
//! requests Python Remote Debug / Attach to Node.js configurations. A browser launch
|
||||
//! (`chrome`, `msedge` and their `pwa-` names) becomes a JavaScript Debug configuration. Variables with a fixed
|
||||
//! meaning (`${workspaceFolder}`, `${userHome}`, `${env:NAME}`…) are replaced when importing;
|
||||
//! an entry using one that depends on the moment VS Code launches it (`${file}`,
|
||||
//! `${input:…}`, `${command:…}`) is skipped.
|
||||
@@ -46,6 +47,7 @@ const PYTHON_ATTACH: &[&str] = &["connect", "host", "port", "pathMappings"];
|
||||
const NODE_LAUNCH: &[&str] = &["program", "args", "runtimeExecutable", "runtimeArgs"];
|
||||
const NODE_ATTACH: &[&str] = &["address", "port", "localRoot", "remoteRoot"];
|
||||
const NODE_TERMINAL: &[&str] = &["command"];
|
||||
const BROWSER_LAUNCH: &[&str] = &["url", "webRoot"];
|
||||
const COMPOUND: &[&str] = &["configurations"];
|
||||
|
||||
const PACKAGE_MANAGERS: &[&str] = &["npm", "pnpm", "yarn", "bun"];
|
||||
@@ -221,7 +223,14 @@ impl Cx<'_> {
|
||||
("node" | "pwa-node", "launch") => (self.node_launch(name, obj)?, NODE_LAUNCH),
|
||||
("node" | "pwa-node", "attach") => (self.node_attach(name, obj)?, NODE_ATTACH),
|
||||
("node-terminal", "launch") => (self.node_terminal(name, obj)?, NODE_TERMINAL),
|
||||
("debugpy" | "python" | "node" | "pwa-node" | "node-terminal", other) => {
|
||||
("chrome" | "pwa-chrome" | "msedge" | "pwa-msedge", "launch") => {
|
||||
(self.browser_launch(name, obj, ty)?, BROWSER_LAUNCH)
|
||||
}
|
||||
(
|
||||
"debugpy" | "python" | "node" | "pwa-node" | "node-terminal" | "chrome" | "pwa-chrome" | "msedge"
|
||||
| "pwa-msedge",
|
||||
other,
|
||||
) => {
|
||||
return Err(SkipReason::UnsupportedRequest { request: other.to_owned() });
|
||||
}
|
||||
(other, _) => return Err(SkipReason::UnsupportedType { type_: other.to_owned() }),
|
||||
@@ -340,6 +349,23 @@ impl Cx<'_> {
|
||||
Ok(c)
|
||||
}
|
||||
|
||||
/// A page opened in Chrome or Edge with the debugger attached: a JavaScript Debug configuration.
|
||||
fn browser_launch(&self, name: &str, obj: &Map<String, Value>, ty: &str) -> Res<RunConfig> {
|
||||
let Some(url) = self.string(obj, "url")? else {
|
||||
return Err(SkipReason::NothingToRun);
|
||||
};
|
||||
let browser = if ty.ends_with("msedge") { "edge" } else { "chrome" };
|
||||
let mut c = config(name, "js-browser", String::new(), Vec::new(), Some("js-debug"));
|
||||
c.extras.params = BTreeMap::from([("url".into(), url), ("browser".into(), browser.into())]);
|
||||
if let Some(web_root) = self.string(obj, "webRoot")? {
|
||||
let web_root = self.path_from(web_root, self.root);
|
||||
if web_root != "." {
|
||||
c.extras.params.insert("webRoot".into(), web_root);
|
||||
}
|
||||
}
|
||||
Ok(c)
|
||||
}
|
||||
|
||||
/// `localRoot` (relative to the project root, empty for the root itself) and `remoteRoot`.
|
||||
fn roots(&self, obj: &Map<String, Value>, c: &mut RunConfig) -> Res<()> {
|
||||
if let Some(local) = self.string(obj, "localRoot")? {
|
||||
|
||||
@@ -229,7 +229,12 @@ fn nextjs_recipes() {
|
||||
assert_eq!(server.args, s(&["run", "dev"]));
|
||||
assert_eq!(server.debug_adapter.as_deref(), Some("js-debug"));
|
||||
|
||||
assert_eq!(reason(&r, "Next.js: debug client-side"), &SkipReason::UnsupportedType { type_: "chrome".into() });
|
||||
let client = get(&r, "Next.js: debug client-side");
|
||||
assert_eq!((client.kind.as_str(), client.command.as_str()), ("js-browser", ""));
|
||||
assert_eq!(client.debug_adapter.as_deref(), Some("js-debug"));
|
||||
assert_eq!(client.extras.params["url"], "http://localhost:3000");
|
||||
assert_eq!(client.extras.params["browser"], "chrome");
|
||||
assert!(ignored(&r, "Next.js: debug client-side").is_empty());
|
||||
|
||||
let full = get(&r, "Next.js: debug full stack");
|
||||
assert_eq!((full.kind.as_str(), full.command.as_str()), ("node", "node"));
|
||||
@@ -403,6 +408,37 @@ fn attach_requests_become_remote_debug_configurations() {
|
||||
assert_eq!(reason(&r, "Odd"), &SkipReason::UnsupportedRequest { request: "test".into() });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn browser_launches_become_javascript_debug_configurations() {
|
||||
let (_dir, r) = run(r#"{
|
||||
"configurations": [
|
||||
{ "name": "Edge", "type": "pwa-msedge", "request": "launch", "url": "http://localhost:5173/app",
|
||||
"webRoot": "${workspaceFolder}/web", "sourceMaps": true, "userDataDir": false },
|
||||
{ "name": "Root", "type": "chrome", "request": "launch", "url": "http://localhost:8080",
|
||||
"webRoot": "${workspaceFolder}" },
|
||||
{ "name": "Page file", "type": "chrome", "request": "launch", "file": "${workspaceFolder}/index.html" },
|
||||
{ "name": "Running browser", "type": "chrome", "request": "attach", "port": 9222 }
|
||||
]
|
||||
}"#);
|
||||
let edge = get(&r, "Edge");
|
||||
assert_eq!((edge.kind.as_str(), edge.debug_adapter.as_deref()), ("js-browser", Some("js-debug")));
|
||||
assert_eq!(
|
||||
edge.extras.params,
|
||||
BTreeMap::from([
|
||||
("browser".into(), "edge".into()),
|
||||
("url".into(), "http://localhost:5173/app".into()),
|
||||
("webRoot".into(), "web".into())
|
||||
])
|
||||
);
|
||||
assert_eq!(ignored(&r, "Edge"), ["userDataDir"]);
|
||||
|
||||
// The project root is the default web root.
|
||||
assert!(!get(&r, "Root").extras.params.contains_key("webRoot"));
|
||||
|
||||
assert_eq!(reason(&r, "Page file"), &SkipReason::NothingToRun);
|
||||
assert_eq!(reason(&r, "Running browser"), &SkipReason::UnsupportedRequest { request: "attach".into() });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_block_for_this_os_overrides_the_entry() {
|
||||
let os = if cfg!(windows) {
|
||||
|
||||
@@ -1168,6 +1168,22 @@ pub fn configure_server(cfg: &mut LanguageServerConfig, root: &Path, server_root
|
||||
let plugin = vue::tsserver_plugin(&vue);
|
||||
merge_init_options(cfg, json!({ "vtsls": { "tsserver": { "globalPlugins": [plugin] } } }));
|
||||
cfg.tsserver_project_extras.extensions = vec![vue::EXTENSION.into()];
|
||||
// Hybrid mode: one program holds every .vue's virtual TypeScript, so the
|
||||
// heap must clear it (vtsls' default 3072 dies in a GC death spiral), and
|
||||
// the syntax tsserver duplicates the plugin and its file watching.
|
||||
merge_init_options(
|
||||
cfg,
|
||||
json!({ "typescript": { "tsserver": { "useSyntaxServer": "never", "maxTsServerMemory": 8192 } } }),
|
||||
);
|
||||
// A configured project resolves its own types; the typings installer only
|
||||
// serves inferred projects and keeps scanning npm in the background.
|
||||
if server_root.ancestors().any(|d| d.join("tsconfig.json").is_file() || d.join("jsconfig.json").is_file()) {
|
||||
merge_init_options(cfg, json!({ "typescript": { "disableAutomaticTypeAcquisition": true } }));
|
||||
}
|
||||
}
|
||||
let memory = s.languages.typescript.tsserver_max_memory;
|
||||
if name == "vtsls" && memory > 0 {
|
||||
merge_init_options(cfg, json!({ "typescript": { "tsserver": { "maxTsServerMemory": memory } } }));
|
||||
}
|
||||
if name == "vtsls"
|
||||
&& cfg.tsserver_project
|
||||
@@ -1814,10 +1830,17 @@ pub fn without_derived_in(config: &RunConfig, root: &Path, fallback: Option<&Pat
|
||||
if path.is_absolute()
|
||||
&& let Some(name) = path.file_name().map(|n| n.to_string_lossy().into_owned())
|
||||
{
|
||||
let mut probe = out.clone();
|
||||
probe.command = name.clone();
|
||||
if applied(&probe).command == config.command {
|
||||
out.command = name;
|
||||
// Generated configurations name the bare program (`java`), also on Windows where the
|
||||
// derived path ends in `java.exe`.
|
||||
let bare =
|
||||
name.len().checked_sub(4).filter(|&at| name[at..].eq_ignore_ascii_case(".exe")).map(|at| &name[..at]);
|
||||
for candidate in bare.into_iter().chain([name.as_str()]) {
|
||||
let mut probe = out.clone();
|
||||
probe.command = candidate.to_owned();
|
||||
if applied(&probe).command == config.command {
|
||||
out.command = candidate.to_owned();
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
@@ -2104,6 +2127,74 @@ mod gradle_runtime_tests {
|
||||
mod maven_linkage_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn hybrid_vtsls_gets_one_server_a_full_heap_and_no_type_acquisition() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = tmp.path();
|
||||
// The Vue package the hybrid block finds through the command's package dir.
|
||||
let bin = root.join("tools/node_modules/.bin/vue-language-server");
|
||||
let pkg = root.join("tools/node_modules/@vue/language-server/package.json");
|
||||
std::fs::create_dir_all(bin.parent().unwrap()).unwrap();
|
||||
std::fs::create_dir_all(pkg.parent().unwrap()).unwrap();
|
||||
std::fs::write(&bin, "#!/bin/sh\n").unwrap();
|
||||
std::fs::write(&pkg, "{}").unwrap();
|
||||
let mut s = Settings::default();
|
||||
s.tools
|
||||
.insert(vue::SERVER.into(), ide_core::settings::ToolSettings { path: bin.to_string_lossy().into_owned() });
|
||||
std::fs::write(root.join("tsconfig.json"), "{}").unwrap();
|
||||
let mut cfg = custom_server("vtsls", &LanguageServerSettings::default());
|
||||
configure_server(&mut cfg, root, root, &s);
|
||||
let options = serde_json::to_value(cfg.initialization_options).unwrap();
|
||||
assert_eq!(options.pointer("/typescript/tsserver/useSyntaxServer"), Some(&json!("never")));
|
||||
assert_eq!(options.pointer("/typescript/tsserver/maxTsServerMemory"), Some(&json!(8192)));
|
||||
assert_eq!(options.pointer("/typescript/disableAutomaticTypeAcquisition"), Some(&json!(true)));
|
||||
// A user-set limit wins over the automatic one.
|
||||
s.languages.typescript.tsserver_max_memory = 6144;
|
||||
let mut cfg = custom_server("vtsls", &LanguageServerSettings::default());
|
||||
configure_server(&mut cfg, root, root, &s);
|
||||
let options = serde_json::to_value(cfg.initialization_options).unwrap();
|
||||
assert_eq!(options.pointer("/typescript/tsserver/maxTsServerMemory"), Some(&json!(6144)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hybrid_without_a_config_keeps_type_acquisition_and_plain_vtsls_stays_default() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let root = tmp.path();
|
||||
let bin = root.join("tools/node_modules/.bin/vue-language-server");
|
||||
let pkg = root.join("tools/node_modules/@vue/language-server/package.json");
|
||||
std::fs::create_dir_all(bin.parent().unwrap()).unwrap();
|
||||
std::fs::create_dir_all(pkg.parent().unwrap()).unwrap();
|
||||
std::fs::write(&bin, "#!/bin/sh\n").unwrap();
|
||||
std::fs::write(&pkg, "{}").unwrap();
|
||||
let mut s = Settings::default();
|
||||
s.tools
|
||||
.insert(vue::SERVER.into(), ide_core::settings::ToolSettings { path: bin.to_string_lossy().into_owned() });
|
||||
// No tsconfig/jsconfig anywhere: inferred projects still want the typings installer.
|
||||
let mut cfg = custom_server("vtsls", &LanguageServerSettings::default());
|
||||
configure_server(&mut cfg, root, root, &s);
|
||||
let options = serde_json::to_value(cfg.initialization_options).unwrap();
|
||||
assert!(options.pointer("/typescript/disableAutomaticTypeAcquisition").is_none());
|
||||
assert_eq!(options.pointer("/typescript/tsserver/maxTsServerMemory"), Some(&json!(8192)));
|
||||
// Without the Vue package there is no hybrid mode: nothing is injected. The
|
||||
// command points at a bare binary, so the package lookup finds nothing
|
||||
// (a bare name would resolve on the host's PATH and break the test).
|
||||
std::fs::write(root.join("bare-vue-language-server"), "#!/bin/sh\n").unwrap();
|
||||
s.tools
|
||||
.insert(vue::SERVER.into(), ide_core::settings::ToolSettings { path: "bare-vue-language-server".into() });
|
||||
let mut cfg = custom_server("vtsls", &LanguageServerSettings::default());
|
||||
configure_server(&mut cfg, root, root, &s);
|
||||
let options = serde_json::to_value(cfg.initialization_options).unwrap();
|
||||
assert!(options.pointer("/typescript/tsserver/useSyntaxServer").is_none());
|
||||
assert!(options.pointer("/typescript/tsserver/maxTsServerMemory").is_none());
|
||||
// A user-set limit applies to plain TypeScript projects too.
|
||||
s.languages.typescript.tsserver_max_memory = 12288;
|
||||
let mut cfg = custom_server("vtsls", &LanguageServerSettings::default());
|
||||
configure_server(&mut cfg, root, root, &s);
|
||||
let options = serde_json::to_value(cfg.initialization_options).unwrap();
|
||||
assert_eq!(options.pointer("/typescript/tsserver/maxTsServerMemory"), Some(&json!(12288)));
|
||||
assert!(options.pointer("/typescript/tsserver/useSyntaxServer").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn default_maven_repository_reads_the_user_settings() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
|
||||
@@ -35,7 +35,7 @@ pub(crate) fn configure(cfg: &mut LanguageServerConfig, root: &Path, server_root
|
||||
});
|
||||
if node.is_none() || find_command("node", &search, server_root).is_none() {
|
||||
cfg.launch_error = Some(
|
||||
"Astro Language Server needs Node.js. Install it or select its executable in Settings › Languages & Frameworks › Node.js."
|
||||
"Astro Language Server needs Node.js. Install it or select its executable in Settings › Languages & Frameworks › JavaScript Runtime."
|
||||
.into(),
|
||||
);
|
||||
return;
|
||||
|
||||
@@ -332,6 +332,12 @@ fn cached_cancellable(
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// What `ecosystem` last found under `root` with the same settings, from before its manifests changed.
|
||||
fn last_found(root: &Path, ecosystem: &'static str, key: &str) -> Option<Vec<LibraryGroup>> {
|
||||
let cache = CACHE.lock().unwrap_or_else(|e| e.into_inner());
|
||||
cache.get(&(root.to_path_buf(), ecosystem)).filter(|c| c.key == key).map(|c| c.groups.clone())
|
||||
}
|
||||
|
||||
fn cached_optional(
|
||||
root: &Path,
|
||||
ecosystem: &'static str,
|
||||
@@ -807,6 +813,9 @@ fn rust_groups(root: &Path, s: &Settings, locked: bool, cancel: &CancellationTok
|
||||
let roots = parse_cargo_metadata(&json)?;
|
||||
Some(group("Cargo dependencies", "cargo", named_roots(roots)).into_iter().collect())
|
||||
});
|
||||
// An edited manifest that names crates not downloaded yet fails offline until the next
|
||||
// resolution writes Cargo.lock: the last graph stays meanwhile, as a failed sync keeps the model.
|
||||
let graph = graph.or_else(|| last_found(root, "cargo", &r.toolchain));
|
||||
out.extend(graph.unwrap_or_else(|| {
|
||||
let home = ide_core::process::host_var_os("CARGO_HOME")
|
||||
.map(PathBuf::from)
|
||||
@@ -1107,6 +1116,20 @@ mod go_cache_tests {
|
||||
assert_eq!(groups[0].kind, "cargo");
|
||||
assert_eq!(groups[0].roots[0].path, source);
|
||||
assert_eq!(std::fs::read_to_string(calls).unwrap().lines().count(), 1);
|
||||
|
||||
// The manifest now names a crate that isn't downloaded: the offline graph fails, the last one stays.
|
||||
std::fs::write(
|
||||
bin.join("cargo"),
|
||||
format!(
|
||||
"#!/bin/sh\nif [ \"$1\" = locate-project ]; then echo '{}'; exit; fi\nexit 101\n",
|
||||
root.join("Cargo.toml").display()
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
std::fs::write(root.join("Cargo.toml"), "[workspace]\nmembers=['members/*']\n# edited\n").unwrap();
|
||||
let groups = project_libraries(root, &settings, &[]);
|
||||
assert_eq!(groups.len(), 1);
|
||||
assert_eq!(groups[0].roots[0].path, source);
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
|
||||
@@ -194,14 +194,23 @@ impl Session {
|
||||
}
|
||||
channel.eof().await.map_err(|e| self.failed(e))?;
|
||||
let mut out = ExecOutput::default();
|
||||
while let Some(msg) = channel.wait().await {
|
||||
loop {
|
||||
// After the exit status, output still in flight is read until the channel goes quiet: a
|
||||
// background child that inherited stdout keeps the channel open past the exit (as in `spawn`).
|
||||
let msg = match out.code {
|
||||
None => channel.wait().await,
|
||||
Some(_) => tokio::time::timeout(EXIT_DRAIN, channel.wait()).await.ok().flatten(),
|
||||
};
|
||||
match msg {
|
||||
ChannelMsg::Data { data } => out.stdout.extend_from_slice(&data),
|
||||
ChannelMsg::ExtendedData { data, ext: 1 } => out.stderr.extend_from_slice(&data),
|
||||
ChannelMsg::ExitStatus { exit_status } => out.code = Some(exit_status),
|
||||
_ => {}
|
||||
Some(ChannelMsg::Data { data }) => out.stdout.extend_from_slice(&data),
|
||||
Some(ChannelMsg::ExtendedData { data, ext: 1 }) => out.stderr.extend_from_slice(&data),
|
||||
Some(ChannelMsg::ExitStatus { exit_status }) => out.code = Some(exit_status),
|
||||
Some(_) => {}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
// Closed here, the channel doesn't stay open on the server for that child.
|
||||
let _ = channel.close().await;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
//! An in-process SSH server for tests (feature `testing`): password `dev` /
|
||||
//! `secret`, commands answered by a closure, SFTP served from a local
|
||||
//! folder (remote `/x` is `<root>/x`), and a shell that echoes its input
|
||||
//! (`exit` ends it with status 0).
|
||||
//! (`exit` ends it with status 0). `direct-tcpip` channels are forwarded to local TCP
|
||||
//! ports, and refused when nothing listens there.
|
||||
|
||||
use russh::server::{self, Auth, ChannelOpenHandle, Msg, Server as _, Session};
|
||||
use russh::{Channel, ChannelId};
|
||||
@@ -71,6 +72,27 @@ impl server::Handler for Conn {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn channel_open_direct_tcpip(
|
||||
&mut self,
|
||||
channel: Channel<Msg>,
|
||||
host: &str,
|
||||
port: u32,
|
||||
_originator_address: &str,
|
||||
_originator_port: u32,
|
||||
reply: ChannelOpenHandle,
|
||||
_session: &mut Session,
|
||||
) -> Result<(), Self::Error> {
|
||||
let Ok(mut upstream) = tokio::net::TcpStream::connect((host, port as u16)).await else {
|
||||
return Ok(());
|
||||
};
|
||||
reply.accept().await;
|
||||
tokio::spawn(async move {
|
||||
let mut stream = channel.into_stream();
|
||||
let _ = tokio::io::copy_bidirectional(&mut stream, &mut upstream).await;
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn exec_request(
|
||||
&mut self,
|
||||
channel: ChannelId,
|
||||
|
||||
@@ -321,6 +321,20 @@ async fn a_process_that_exits_finishes_even_while_its_channel_stays_open() {
|
||||
assert_eq!((code, seen.lock().as_str()), (Some(3), "ran: linger"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_command_that_exits_returns_even_while_its_channel_stays_open() {
|
||||
let env = env().await;
|
||||
let session = env.connect_trusted(&env.target(SshAuth::Password), Some("secret")).await.ok().unwrap();
|
||||
let out = tokio::time::timeout(std::time::Duration::from_secs(10), session.exec("linger", None))
|
||||
.await
|
||||
.expect("the exit status must end the command")
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
(out.code, out.stdout.as_slice(), out.stderr.as_slice()),
|
||||
(Some(3), b"ran: linger".as_slice(), b"oops".as_slice())
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn upload_reports_a_command_that_stops_reading() {
|
||||
let env = env().await;
|
||||
|
||||
@@ -41,6 +41,14 @@ pub trait Backend: Send + Sync {
|
||||
/// as `unstaged`.
|
||||
fn status(&self) -> IdeResult<Vec<FileStatus>>;
|
||||
|
||||
/// Working-area changes for `paths` only (absolute; those outside the root
|
||||
/// are ignored), which is what a watcher batch needs. The default is the
|
||||
/// full status: always correct, just not proportional to the batch.
|
||||
fn status_paths(&self, paths: &[PathBuf]) -> IdeResult<Vec<FileStatus>> {
|
||||
let _ = paths;
|
||||
self.status()
|
||||
}
|
||||
|
||||
/// Text of `path` at `rev`; `None` when absent there.
|
||||
fn file_at(&self, path: &Path, rev: Option<&str>, encoding: TextEncoding) -> IdeResult<Option<String>>;
|
||||
|
||||
@@ -189,6 +197,10 @@ impl Backend for Repo {
|
||||
Repo::status(self)
|
||||
}
|
||||
|
||||
fn status_paths(&self, paths: &[PathBuf]) -> IdeResult<Vec<FileStatus>> {
|
||||
Repo::status_paths(self, paths)
|
||||
}
|
||||
|
||||
fn file_at(&self, path: &Path, rev: Option<&str>, encoding: TextEncoding) -> IdeResult<Option<String>> {
|
||||
Repo::file_at(self, path, rev, encoding)
|
||||
}
|
||||
|
||||
@@ -764,6 +764,24 @@ impl Repo {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
status::status(self)
|
||||
}
|
||||
/// Status of `paths` only (absolute; those outside the repository are ignored), for the
|
||||
/// dirty-scope refresh after a watcher batch. An empty list is the full status.
|
||||
pub fn status_paths(&self, paths: &[PathBuf]) -> IdeResult<Vec<FileStatus>> {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
// Restricted Mode starts no program of the repository, and the scoped read is `git status`
|
||||
// with a pathspec: answer with the full gitoxide status instead.
|
||||
if !self.execution_allowed.load(std::sync::atomic::Ordering::Acquire) {
|
||||
return status::status(self);
|
||||
}
|
||||
if paths.is_empty() {
|
||||
return status::status_paths(self, &[]);
|
||||
}
|
||||
let rel: Vec<String> = paths.iter().filter_map(|p| self.given_rel_path(p).ok()).collect();
|
||||
if rel.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
status::status_paths(self, &rel)
|
||||
}
|
||||
pub fn branch_info(&self) -> IdeResult<BranchInfo> {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
refs::branch_info(self)
|
||||
@@ -866,6 +884,12 @@ impl Repo {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
write::blame(self, path, current_text, revision, encoding)
|
||||
}
|
||||
/// Adds `commit` to the list `blame.ignoreRevsFile` names, so annotate skips it; with none
|
||||
/// configured, to `.git-blame-ignore-revs` at the worktree root, which the local config then names.
|
||||
pub fn ignore_blame_revision(&self, commit: &str) -> IdeResult<()> {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
write::ignore_blame_revision(self, commit)
|
||||
}
|
||||
/// History for Selection: commits that changed `lines` (0-based, inclusive, of the
|
||||
/// current text: `current_text`, or the file on disk), mapped to HEAD's lines first.
|
||||
pub fn block_history(
|
||||
@@ -1244,10 +1268,17 @@ impl Repo {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
write::rebase(self, onto)
|
||||
}
|
||||
/// `paths` (absolute) stashes only those files; empty stashes the whole working tree.
|
||||
/// `keep_index`: changes already staged stay in the index (and the working tree).
|
||||
pub fn stash_push(&self, message: Option<&str>, include_untracked: bool, keep_index: bool) -> IdeResult<()> {
|
||||
pub fn stash_push(
|
||||
&self,
|
||||
message: Option<&str>,
|
||||
include_untracked: bool,
|
||||
keep_index: bool,
|
||||
paths: &[PathBuf],
|
||||
) -> IdeResult<()> {
|
||||
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
|
||||
write::stash_push(self, message, include_untracked, keep_index)
|
||||
write::stash_push(self, message, include_untracked, keep_index, paths)
|
||||
}
|
||||
/// `reinstate_index`: also restore what was staged when the stash was made.
|
||||
pub fn stash_apply(&self, index: u32, pop: bool, reinstate_index: bool) -> IdeResult<()> {
|
||||
|
||||
@@ -148,7 +148,7 @@ impl Repo {
|
||||
}
|
||||
|
||||
/// `path` relative to the worktree, spelled as given.
|
||||
fn given_rel_path(&self, path: &Path) -> IdeResult<String> {
|
||||
pub(crate) fn given_rel_path(&self, path: &Path) -> IdeResult<String> {
|
||||
let rel = if path.is_absolute() {
|
||||
self.strip_root(path)
|
||||
.ok_or_else(|| IdeError::invalid(format!("{} is outside the repository", path.display())))?
|
||||
|
||||
@@ -146,17 +146,34 @@ impl Repos {
|
||||
/// The reads go through [`Backend`], the Changes-view surface — repositories of a
|
||||
/// future second backend slot in here once `Repos` learns to hold them.
|
||||
pub fn status(&self) -> IdeResult<Vec<FileStatus>> {
|
||||
let groups: Vec<(Arc<Repo>, Vec<PathBuf>)> = self.repos.iter().map(|repo| (repo.clone(), Vec::new())).collect();
|
||||
self.read_status(groups, false)
|
||||
}
|
||||
|
||||
/// [`status`](Self::status) for `paths` only: each path goes to its innermost repository,
|
||||
/// so a repository without one is not read at all and a nested repository keeps its own
|
||||
/// files (the enclosing one's "untracked folder" entry for them is dropped).
|
||||
pub fn status_paths(&self, paths: &[PathBuf]) -> IdeResult<Vec<FileStatus>> {
|
||||
self.read_status(self.group(paths), true)
|
||||
}
|
||||
|
||||
/// Reads one status per repository: the whole working tree, or `scoped` the paths it was
|
||||
/// given. A nested repository reports its own files, so the enclosing one's "untracked
|
||||
/// folder" entry for it is dropped. A repository that cannot be read is left out; fails
|
||||
/// only when all fail.
|
||||
fn read_status(&self, groups: Vec<(Arc<Repo>, Vec<PathBuf>)>, scoped: bool) -> IdeResult<Vec<FileStatus>> {
|
||||
let mut out = Vec::new();
|
||||
let mut first_error = None;
|
||||
let mut read = 0;
|
||||
for repo in &self.repos {
|
||||
for (repo, mine) in groups {
|
||||
let nested: Vec<&Path> = self
|
||||
.repos
|
||||
.iter()
|
||||
.map(|r| r.root())
|
||||
.filter(|r| *r != repo.root() && r.starts_with(repo.root()))
|
||||
.collect();
|
||||
let status = match (repo.as_ref() as &dyn Backend).status() {
|
||||
let backend = repo.as_ref() as &dyn Backend;
|
||||
let status = match if scoped { backend.status_paths(&mine) } else { backend.status() } {
|
||||
Ok(status) => {
|
||||
read += 1;
|
||||
status
|
||||
|
||||
@@ -24,9 +24,38 @@ pub(crate) fn status(r: &Repo) -> IdeResult<Vec<FileStatus>> {
|
||||
tracing::debug!("status via gitoxide failed, asking git: {}", e.message);
|
||||
cli_status(r, &ignored)
|
||||
})?;
|
||||
Ok(collect(r, &ignored, map, conflicts))
|
||||
}
|
||||
|
||||
/// Status of `paths` only (repository-relative, `/`-separated): the dirty-scope refresh.
|
||||
///
|
||||
/// A watcher batch touches a few files, and the full status walks the whole tree; git's own
|
||||
/// pathspec filtering keeps that batch proportional to what changed. An empty list is the
|
||||
/// full status, so a caller that lost the paths still gets a correct answer.
|
||||
pub(crate) fn status_paths(r: &Repo, paths: &[String]) -> IdeResult<Vec<FileStatus>> {
|
||||
// Pathspecs always take `/`, and a path that *is* the repository root names everything: both
|
||||
// matter, because `Repo::status_paths` hands over paths as the platform spells them.
|
||||
let patterns: Vec<String> =
|
||||
paths.iter().map(|path| path.replace('\\', "/")).filter(|path| !path.is_empty()).collect();
|
||||
if patterns.is_empty() {
|
||||
return status(r);
|
||||
}
|
||||
let ignored = r.ignored.read().unwrap_or_else(|p| p.into_inner()).clone();
|
||||
// `git status` with the pathspec, for the same reason the full read falls back to it: gitoxide's
|
||||
// in-process pathspec matched nothing for a plain relative path here, and a silent empty
|
||||
// answer is worse than the ~40 ms a `git` process costs on Windows.
|
||||
let (map, conflicts) = cli_status_paths(r, &ignored, &patterns)?;
|
||||
Ok(collect(r, &ignored, map, conflicts))
|
||||
}
|
||||
|
||||
fn collect(
|
||||
r: &Repo,
|
||||
ignored: &IgnoredNames,
|
||||
map: BTreeMap<String, Entry>,
|
||||
conflicts: HashSet<String>,
|
||||
) -> Vec<FileStatus> {
|
||||
let mut disk = r.gix().is_ok_and(|repo| crate::repo::ignores_case(&repo)).then(DiskSpelling::default);
|
||||
Ok(map
|
||||
.into_iter()
|
||||
map.into_iter()
|
||||
.filter(|(_, e)| e.staged.is_some() || e.unstaged.is_some())
|
||||
// Ignored names mean "the IDE treats these as nonexistent": hide their tracked
|
||||
// changes too, not just untracked hits above. Conflicts stay visible so merges
|
||||
@@ -51,7 +80,7 @@ pub(crate) fn status(r: &Repo) -> IdeResult<Vec<FileStatus>> {
|
||||
renamed_from: e.renamed_from,
|
||||
}
|
||||
})
|
||||
.collect())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Spells repository paths the way a case-insensitive disk does, so that the tree, tabs and
|
||||
@@ -216,6 +245,21 @@ fn cli_status(r: &Repo, ignored: &IgnoredNames) -> IdeResult<Entries> {
|
||||
Ok(parse_porcelain_v2(&String::from_utf8_lossy(&out.stdout), ignored))
|
||||
}
|
||||
|
||||
/// `git status` for `paths` only, parsed and filtered like the full one. `--literal-pathspecs`
|
||||
/// keeps a `*` or `[` in a file name from being read as a pattern.
|
||||
fn cli_status_paths(r: &Repo, ignored: &IgnoredNames, paths: &[String]) -> IdeResult<Entries> {
|
||||
let overrides = crate::cli::filter_overrides(r.root(), &ide_core::CancellationToken::new())?;
|
||||
let mut args: Vec<&str> = overrides.iter().map(String::as_str).collect();
|
||||
args.extend(["--literal-pathspecs", "status", "--porcelain=v2", "-z", "--untracked-files=all"]);
|
||||
if !crate::execution_allowed() {
|
||||
args.push("--ignore-submodules=dirty");
|
||||
}
|
||||
args.push("--");
|
||||
args.extend(paths.iter().map(String::as_str));
|
||||
let out = crate::cli::run(r.root(), &args, None)?;
|
||||
Ok(parse_porcelain_v2(&String::from_utf8_lossy(&out.stdout), ignored))
|
||||
}
|
||||
|
||||
fn parse_porcelain_v2(out: &str, ignored: &IgnoredNames) -> Entries {
|
||||
let kind = |c: u8| match c {
|
||||
b'M' | b'T' => Some(ChangeKind::Modified),
|
||||
|
||||
@@ -43,6 +43,70 @@ fn run_paths(r: &Repo, args: &[&str], paths: &[String]) -> IdeResult<()> {
|
||||
cli::run_paths(r.root(), &base, paths).map(|_| ())
|
||||
}
|
||||
|
||||
/// The annotate ignore list at the worktree root: git's conventional `blame.ignoreRevsFile`.
|
||||
pub const BLAME_IGNORE_REVS_FILE: &str = ".git-blame-ignore-revs";
|
||||
|
||||
/// The file `blame.ignoreRevsFile` names, resolved from the worktree root as git does (it may not
|
||||
/// exist). The value comes from the gitoxide config snapshot, which Restricted Mode still parses,
|
||||
/// so resolving it starts no program.
|
||||
fn configured_ignore_revs(r: &Repo) -> Option<PathBuf> {
|
||||
let repo = r.gix().ok()?;
|
||||
let configured = repo.config_snapshot().string("blame.ignoreRevsFile")?;
|
||||
let configured = configured.to_str_lossy();
|
||||
let configured = configured.trim();
|
||||
if configured.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let path = Path::new(configured);
|
||||
Some(if path.is_absolute() { path.to_path_buf() } else { r.root().join(path) })
|
||||
}
|
||||
|
||||
/// Arguments for the file `blame.ignoreRevsFile` names, which git does not accept as an
|
||||
/// absent file: with a value configured but missing from disk, `--no-ignore-revs-file`
|
||||
/// clears it so annotate still works.
|
||||
fn ignore_revs_args(r: &Repo) -> Vec<String> {
|
||||
match configured_ignore_revs(r) {
|
||||
Some(path) if path.is_file() => vec!["--ignore-revs-file".to_string(), path.to_string_lossy().into_owned()],
|
||||
Some(_) => vec!["--no-ignore-revs-file".to_string()],
|
||||
None => Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// `existing` (an ignore list's text) with `commit` appended: lines keep their text and order, a
|
||||
/// listed commit isn't repeated, and the result ends with a newline in the list's line ending.
|
||||
fn append_revision(existing: &str, commit: &str) -> String {
|
||||
let revision = commit.trim();
|
||||
let newline = if existing.contains("\r\n") { "\r\n" } else { "\n" };
|
||||
let mut lines: Vec<&str> = existing.lines().collect();
|
||||
if !revision.is_empty() && !lines.iter().any(|line| line.trim() == revision) {
|
||||
lines.push(revision);
|
||||
}
|
||||
if lines.is_empty() { String::new() } else { lines.join(newline) + newline }
|
||||
}
|
||||
|
||||
/// Adds `commit` to the list `blame.ignoreRevsFile` names; with none configured, to
|
||||
/// `.git-blame-ignore-revs` at the worktree root, which the local config then names.
|
||||
pub(crate) fn ignore_blame_revision(r: &Repo, commit: &str) -> IdeResult<()> {
|
||||
let configured = configured_ignore_revs(r);
|
||||
let path = configured.clone().unwrap_or_else(|| r.root().join(BLAME_IGNORE_REVS_FILE));
|
||||
let existing = match std::fs::read_to_string(&path) {
|
||||
Ok(text) => text,
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => String::new(),
|
||||
Err(e) => return Err(e.into()),
|
||||
};
|
||||
let updated = append_revision(&existing, commit);
|
||||
if updated != existing {
|
||||
if let Some(dir) = path.parent() {
|
||||
std::fs::create_dir_all(dir)?;
|
||||
}
|
||||
std::fs::write(&path, updated)?;
|
||||
}
|
||||
if configured.is_none() {
|
||||
cli::run(r.root(), &["config", "--local", "blame.ignoreRevsFile", BLAME_IGNORE_REVS_FILE], None)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn blame(
|
||||
r: &Repo,
|
||||
path: &Path,
|
||||
@@ -61,8 +125,9 @@ pub(crate) fn blame(
|
||||
Err(error) => return Err(error.into()),
|
||||
};
|
||||
let text = current_text.map(str::to_owned).unwrap_or_else(|| decode(&disk, encoding).into_owned());
|
||||
let ignore = ignore_revs_args(r);
|
||||
if encoding.has_ascii_conversion() {
|
||||
return converted_blame(r, path, &rel, &text, revision, encoding);
|
||||
return converted_blame(r, path, &rel, &text, revision, encoding, &ignore);
|
||||
}
|
||||
let contents = match current_text {
|
||||
Some(_) if crate::diff::normalize_eol(&text) == crate::diff::normalize_eol(&decode(&disk, encoding)) => disk,
|
||||
@@ -94,23 +159,32 @@ pub(crate) fn blame(
|
||||
// older releases reject the pair (`fatal: cannot use --contents with final
|
||||
// commit object name`) and take the same plain path — the edited lines are
|
||||
// marked uncommitted below either way (issue #755).
|
||||
let contents_args = ["--literal-pathspecs", "blame", "--porcelain", "--contents", "-", revision, "--", &rel];
|
||||
let plain_args: &[&str] = if crate::execution_allowed() {
|
||||
&["--literal-pathspecs", "blame", "--porcelain", revision, "--", &rel]
|
||||
} else {
|
||||
&["--literal-pathspecs", "blame", "--no-textconv", "--porcelain", revision, "--", &rel]
|
||||
let contents_args = {
|
||||
let mut args = vec!["--literal-pathspecs", "blame", "--porcelain"];
|
||||
args.extend(ignore.iter().map(String::as_str));
|
||||
args.extend(["--contents", "-", revision, "--", &rel]);
|
||||
args
|
||||
};
|
||||
let plain_args: Vec<&str> = {
|
||||
let mut args = vec!["--literal-pathspecs", "blame"];
|
||||
if !crate::execution_allowed() {
|
||||
args.push("--no-textconv");
|
||||
}
|
||||
args.extend(ignore.iter().map(String::as_str));
|
||||
args.extend(["--porcelain", revision, "--", &rel]);
|
||||
args
|
||||
};
|
||||
let (out, contents_blame) = if crate::execution_allowed() {
|
||||
let out = cli::run_raw(r.root(), &contents_args, Some(&contents))?;
|
||||
if out.code == Some(0) {
|
||||
(out, true)
|
||||
} else if String::from_utf8_lossy(&out.stderr).contains("cannot use --contents") {
|
||||
(run(r.root(), plain_args, None)?, false)
|
||||
(run(r.root(), &plain_args, None)?, false)
|
||||
} else {
|
||||
return Err(cli::failure(&contents_args, &out.stderr, &out.stdout, out.code));
|
||||
}
|
||||
} else {
|
||||
(run(r.root(), plain_args, None)?, false)
|
||||
(run(r.root(), &plain_args, None)?, false)
|
||||
};
|
||||
let mut lines = crate::blame::parse_porcelain(&out.stdout_str());
|
||||
for line in &mut lines {
|
||||
@@ -142,16 +216,21 @@ fn converted_blame(
|
||||
text: &str,
|
||||
revision: &str,
|
||||
encoding: TextEncoding,
|
||||
ignore: &[String],
|
||||
) -> IdeResult<Vec<BlameLine>> {
|
||||
let current = crate::diff::normalize_eol(text);
|
||||
let mut result = uncommitted_lines(current.split_inclusive('\n').count());
|
||||
let Some(base) = r.file_bytes_at(path, Some(revision))? else { return Ok(result) };
|
||||
let args: &[&str] = if crate::execution_allowed() {
|
||||
&["--literal-pathspecs", "blame", "--porcelain", revision, "--", rel]
|
||||
} else {
|
||||
&["--literal-pathspecs", "blame", "--no-textconv", "--porcelain", revision, "--", rel]
|
||||
let args: Vec<&str> = {
|
||||
let mut args = vec!["--literal-pathspecs", "blame"];
|
||||
if !crate::execution_allowed() {
|
||||
args.push("--no-textconv");
|
||||
}
|
||||
args.extend(ignore.iter().map(String::as_str));
|
||||
args.extend(["--porcelain", revision, "--", rel]);
|
||||
args
|
||||
};
|
||||
let out = run(r.root(), args, None)?;
|
||||
let out = run(r.root(), &args, None)?;
|
||||
let physical = crate::blame::parse_porcelain(&out.stdout_str());
|
||||
let raw = crate::text_encoding::decode_raw(&base, encoding);
|
||||
let raw = crate::diff::normalize_eol(&raw);
|
||||
@@ -1049,8 +1128,18 @@ pub(crate) fn rebase(r: &Repo, onto: &str) -> IdeResult<()> {
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn stash_push(r: &Repo, message: Option<&str>, include_untracked: bool, keep_index: bool) -> IdeResult<()> {
|
||||
let mut args = vec!["stash", "push", "-q"];
|
||||
pub(crate) fn stash_push(
|
||||
r: &Repo,
|
||||
message: Option<&str>,
|
||||
include_untracked: bool,
|
||||
keep_index: bool,
|
||||
paths: &[PathBuf],
|
||||
) -> IdeResult<()> {
|
||||
let rel = r.rel_paths(paths)?;
|
||||
// Literal pathspecs only for the chosen files: git's whole-tree stash passes itself `:/`,
|
||||
// which a literal pathspec turns into a file of that name.
|
||||
let mut args = if rel.is_empty() { vec![] } else { vec!["--literal-pathspecs"] };
|
||||
args.extend(["stash", "push", "-q"]);
|
||||
if include_untracked {
|
||||
args.push("--include-untracked");
|
||||
}
|
||||
@@ -1060,6 +1149,10 @@ pub(crate) fn stash_push(r: &Repo, message: Option<&str>, include_untracked: boo
|
||||
if let Some(m) = message.filter(|m| !m.trim().is_empty()) {
|
||||
args.extend(["-m", m]);
|
||||
}
|
||||
if !rel.is_empty() {
|
||||
args.push("--");
|
||||
args.extend(rel.iter().map(String::as_str));
|
||||
}
|
||||
run(r.root(), &args, None).map(|_| ())
|
||||
}
|
||||
|
||||
@@ -1383,3 +1476,19 @@ fn decode_diff(bytes: &[u8], encoding: TextEncoding) -> String {
|
||||
}
|
||||
text
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::append_revision;
|
||||
|
||||
#[test]
|
||||
fn a_revision_is_appended_once_keeping_the_list() {
|
||||
let (a, b) = ("a".repeat(40), "b".repeat(40));
|
||||
assert_eq!(append_revision("", &b), format!("{b}\n"));
|
||||
assert_eq!(append_revision(&format!("{a}\n# comment\n"), &b), format!("{a}\n# comment\n{b}\n"));
|
||||
assert_eq!(append_revision(&a, &b), format!("{a}\n{b}\n"));
|
||||
let listed = format!("# reformat\n {b} \n{a}\n");
|
||||
assert_eq!(append_revision(&listed, &b), listed);
|
||||
assert_eq!(append_revision(&format!("{a}\r\n"), &b), format!("{a}\r\n{b}\r\n"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
//! Annotate skips the commits listed in `blame.ignoreRevsFile`.
|
||||
|
||||
use ide_vcs::Repo;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
struct Fixture {
|
||||
_tmp: tempfile::TempDir,
|
||||
dir: PathBuf,
|
||||
}
|
||||
|
||||
fn git_in(dir: &Path, args: &[&str]) -> String {
|
||||
let out = Command::new("git")
|
||||
.current_dir(dir)
|
||||
.args(["-c", "init.defaultBranch=main", "-c", "core.quotepath=off"])
|
||||
.args(args)
|
||||
.env("GIT_TERMINAL_PROMPT", "0")
|
||||
.output()
|
||||
.expect("git runs");
|
||||
assert!(out.status.success(), "git {args:?} failed: {}", String::from_utf8_lossy(&out.stderr));
|
||||
String::from_utf8_lossy(&out.stdout).trim().to_string()
|
||||
}
|
||||
|
||||
impl Fixture {
|
||||
fn new() -> Self {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let dir = tmp.path().join("work");
|
||||
std::fs::create_dir(&dir).unwrap();
|
||||
git_in(&dir, &["init", "-q"]);
|
||||
for (k, v) in [
|
||||
("user.name", "Test User"),
|
||||
("user.email", "[email protected]"),
|
||||
("commit.gpgsign", "false"),
|
||||
("core.autocrlf", "false"),
|
||||
] {
|
||||
git_in(&dir, &["config", k, v]);
|
||||
}
|
||||
Fixture { _tmp: tmp, dir }
|
||||
}
|
||||
fn git(&self, args: &[&str]) -> String {
|
||||
git_in(&self.dir, args)
|
||||
}
|
||||
fn write(&self, rel: &str, content: &str) {
|
||||
std::fs::write(self.dir.join(rel), content).unwrap();
|
||||
}
|
||||
fn commit_all(&self, msg: &str) -> String {
|
||||
self.git(&["add", "-A"]);
|
||||
self.git(&["commit", "-q", "-m", msg]);
|
||||
self.git(&["rev-parse", "HEAD"])
|
||||
}
|
||||
fn path(&self, rel: &str) -> PathBuf {
|
||||
self.dir.join(rel)
|
||||
}
|
||||
/// The repository of a trusted project.
|
||||
fn repo(&self) -> Repo {
|
||||
let repo = Repo::discover(&self.dir).unwrap().expect("repository found");
|
||||
repo.set_execution_allowed(true);
|
||||
repo
|
||||
}
|
||||
/// The repository of a project in Restricted Mode.
|
||||
fn restricted(&self) -> Repo {
|
||||
Repo::discover(&self.dir).unwrap().expect("repository found")
|
||||
}
|
||||
}
|
||||
|
||||
/// A line added by `first` and reformatted by `second`; annotate attributes it to
|
||||
/// `second` until `second` is ignored, then back to `first`.
|
||||
fn reformatted_file() -> (Fixture, String, String) {
|
||||
let fx = Fixture::new();
|
||||
fx.write("file.txt", "first\nsecond\n");
|
||||
let first = fx.commit_all("add file");
|
||||
fx.write("file.txt", "first\n second\n");
|
||||
let second = fx.commit_all("reformat");
|
||||
(fx, first, second)
|
||||
}
|
||||
|
||||
fn commits(repo: &Repo, path: &Path) -> Vec<String> {
|
||||
repo.blame(path, None, None, encoding_rs::UTF_8).unwrap().into_iter().map(|line| line.commit).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_listed_commit_gives_its_lines_back_to_the_earlier_one() {
|
||||
let (fx, first, second) = reformatted_file();
|
||||
let file = fx.path("file.txt");
|
||||
assert_eq!(commits(&fx.repo(), &file), vec![first.clone(), second.clone()]);
|
||||
|
||||
fx.write(".git-blame-ignore-revs", &format!("{second}\n"));
|
||||
fx.git(&["config", "blame.ignoreRevsFile", ".git-blame-ignore-revs"]);
|
||||
let ignored = vec![first.clone(), first.clone()];
|
||||
assert_eq!(commits(&fx.repo(), &file), ignored);
|
||||
// Restricted Mode reads the same configuration and skips the same commit.
|
||||
assert_eq!(commits(&fx.restricted(), &file), ignored);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_configured_file_that_does_not_exist_is_skipped() {
|
||||
let (fx, _, second) = reformatted_file();
|
||||
fx.git(&["config", "blame.ignoreRevsFile", ".git-blame-ignore-revs"]);
|
||||
let file = fx.path("file.txt");
|
||||
// The missing list is passed to no blame, so annotate still works.
|
||||
assert_eq!(commits(&fx.repo(), &file)[1], second);
|
||||
assert_eq!(commits(&fx.restricted(), &file)[1], second);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ignoring_a_commit_starts_the_list_and_configures_it_once() {
|
||||
let (fx, first, second) = reformatted_file();
|
||||
fx.repo().ignore_blame_revision(&second).unwrap();
|
||||
fx.repo().ignore_blame_revision(&second).unwrap();
|
||||
assert_eq!(std::fs::read_to_string(fx.path(".git-blame-ignore-revs")).unwrap(), format!("{second}\n"));
|
||||
assert_eq!(fx.git(&["config", "--local", "--get", "blame.ignoreRevsFile"]), ".git-blame-ignore-revs");
|
||||
let repo = fx.repo();
|
||||
assert_eq!(commits(&repo, &fx.path("file.txt")), vec![first.clone(), first.clone()]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ignoring_a_commit_adds_it_to_the_configured_list() {
|
||||
let (fx, first, second) = reformatted_file();
|
||||
fx.write("custom-revs", &format!("# formatting\r\n{first}\r\n"));
|
||||
fx.git(&["config", "--local", "blame.ignoreRevsFile", "custom-revs"]);
|
||||
fx.repo().ignore_blame_revision(&second).unwrap();
|
||||
// Lines and line endings kept; the configured file is the one annotate reads.
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(fx.path("custom-revs")).unwrap(),
|
||||
format!("# formatting\r\n{first}\r\n{second}\r\n")
|
||||
);
|
||||
assert!(!fx.path(".git-blame-ignore-revs").exists());
|
||||
assert_eq!(fx.git(&["config", "--local", "--get", "blame.ignoreRevsFile"]), "custom-revs");
|
||||
let blamed = commits(&fx.repo(), &fx.path("file.txt"));
|
||||
assert_ne!(blamed[1], second);
|
||||
}
|
||||
@@ -769,11 +769,11 @@ fn stash_push_list_apply_pop() {
|
||||
let repo = fx.repo();
|
||||
|
||||
fx.write("s.txt", "change 1\n");
|
||||
repo.stash_push(Some("first stash"), false, false).unwrap();
|
||||
repo.stash_push(Some("first stash"), false, false, &[]).unwrap();
|
||||
assert_eq!(fx.read("s.txt"), "base\n");
|
||||
fx.write("s.txt", "change 2\n");
|
||||
fx.write("u.txt", "untracked\n");
|
||||
repo.stash_push(None, true, false).unwrap();
|
||||
repo.stash_push(None, true, false, &[]).unwrap();
|
||||
assert!(!fx.path("u.txt").exists());
|
||||
|
||||
let stashes = repo.stashes().unwrap();
|
||||
@@ -808,7 +808,7 @@ fn stash_keep_index_reinstate_index_and_branch() {
|
||||
fx.write("a.txt", "a staged\n");
|
||||
fx.git(&["add", "a.txt"]);
|
||||
fx.write("b.txt", "b unstaged\n");
|
||||
repo.stash_push(Some("keep"), false, true).unwrap();
|
||||
repo.stash_push(Some("keep"), false, true, &[]).unwrap();
|
||||
// Keep index: the staged change stays, the unstaged one is stashed.
|
||||
assert_eq!(fx.read("a.txt"), "a staged\n");
|
||||
assert_eq!(fx.read("b.txt"), "b\n");
|
||||
@@ -2065,3 +2065,48 @@ fn a_case_only_directory_rename_keeps_status_and_head_content() {
|
||||
assert_eq!(status_of(&st, "src/financial/table/index.vue").staged, Some(ChangeKind::Modified));
|
||||
assert_eq!(fx.git(&["diff", "--cached", "--name-only"]), "src/financial/table/index.vue");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn status_paths_reads_only_the_named_paths() {
|
||||
// The dirty-scope refresh: a watcher batch names a few paths, and git's pathspec
|
||||
// filtering keeps the answer proportional to the batch instead of the whole tree.
|
||||
let fx = Fixture::new();
|
||||
fx.write("a.txt", "a\n");
|
||||
fx.write("b.txt", "a\n");
|
||||
fx.write("dir/c.txt", "a\n");
|
||||
fx.commit_all("base");
|
||||
fx.write("a.txt", "b\n");
|
||||
fx.write("b.txt", "b\n");
|
||||
fx.write("dir/c.txt", "b\n");
|
||||
fx.write("new.txt", "n\n");
|
||||
let repo = fx.repo();
|
||||
repo.set_ignored_names(ide_core::settings::Settings::default().ignored_names());
|
||||
|
||||
let one = repo.status_paths(&[fx.path("a.txt")]).unwrap();
|
||||
assert_eq!(one.len(), 1, "{one:#?}");
|
||||
assert_eq!(one[0].rel_path, "a.txt");
|
||||
assert_eq!(one[0].unstaged, Some(ChangeKind::Modified));
|
||||
|
||||
// A directory stands for everything below it, as the watcher reports a removed one.
|
||||
let dir = repo.status_paths(&[fx.path("dir")]).unwrap();
|
||||
assert_eq!(dir.iter().map(|s| s.rel_path.as_str()).collect::<Vec<_>>(), ["dir/c.txt"]);
|
||||
|
||||
// A nested file: the path a watcher hands over is spelled by the platform, and the pathspec
|
||||
// needs `/` whatever the platform's separator is.
|
||||
let nested = repo.status_paths(&[fx.path("dir").join("c.txt")]).unwrap();
|
||||
assert_eq!(nested.iter().map(|s| s.rel_path.as_str()).collect::<Vec<_>>(), ["dir/c.txt"]);
|
||||
|
||||
// The repository root itself names everything, not a path called "".
|
||||
assert_eq!(repo.status_paths(&[fx.path("")]).unwrap().len(), 4);
|
||||
|
||||
// A new file the batch names is reported as untracked, not as a directory entry.
|
||||
let new = repo.status_paths(&[fx.path("new.txt")]).unwrap();
|
||||
assert_eq!((new[0].rel_path.as_str(), new[0].unstaged), ("new.txt", Some(ChangeKind::Untracked)));
|
||||
|
||||
// Paths outside the repository are ignored rather than an error; nothing named is nothing read.
|
||||
let outside = if cfg!(windows) { PathBuf::from("C:/elsewhere/x.txt") } else { PathBuf::from("/elsewhere/x.txt") };
|
||||
assert!(repo.status_paths(&[outside]).unwrap().is_empty());
|
||||
|
||||
// An empty list is the full status, so a caller that lost the paths stays correct.
|
||||
assert_eq!(repo.status_paths(&[]).unwrap().len(), 4);
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
//! Stashing only the selected files.
|
||||
|
||||
use ide_core::error::ErrorKind;
|
||||
use ide_vcs::Repo;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
struct Fixture {
|
||||
_tmp: tempfile::TempDir,
|
||||
dir: PathBuf,
|
||||
}
|
||||
|
||||
fn git_in(dir: &Path, args: &[&str]) -> String {
|
||||
let out = Command::new("git")
|
||||
.current_dir(dir)
|
||||
.args(["-c", "init.defaultBranch=main", "-c", "core.quotepath=off"])
|
||||
.args(args)
|
||||
.env("GIT_TERMINAL_PROMPT", "0")
|
||||
.output()
|
||||
.expect("git runs");
|
||||
assert!(out.status.success(), "git {args:?} failed: {}", String::from_utf8_lossy(&out.stderr));
|
||||
String::from_utf8_lossy(&out.stdout).trim().to_string()
|
||||
}
|
||||
|
||||
fn configure(dir: &Path) {
|
||||
for (k, v) in [
|
||||
("user.name", "Test User"),
|
||||
("user.email", "[email protected]"),
|
||||
("commit.gpgsign", "false"),
|
||||
("core.autocrlf", "false"),
|
||||
] {
|
||||
git_in(dir, &["config", k, v]);
|
||||
}
|
||||
}
|
||||
|
||||
impl Fixture {
|
||||
fn new() -> Self {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
let dir = tmp.path().join("work");
|
||||
std::fs::create_dir(&dir).unwrap();
|
||||
git_in(&dir, &["init", "-q"]);
|
||||
configure(&dir);
|
||||
Fixture { _tmp: tmp, dir }
|
||||
}
|
||||
fn git(&self, args: &[&str]) -> String {
|
||||
git_in(&self.dir, args)
|
||||
}
|
||||
fn write(&self, rel: &str, content: &str) {
|
||||
std::fs::write(self.dir.join(rel), content).unwrap();
|
||||
}
|
||||
fn read(&self, rel: &str) -> String {
|
||||
std::fs::read_to_string(self.dir.join(rel)).unwrap()
|
||||
}
|
||||
fn commit_all(&self, msg: &str) {
|
||||
self.git(&["add", "-A"]);
|
||||
self.git(&["commit", "-q", "-m", msg]);
|
||||
}
|
||||
fn repo(&self) -> Repo {
|
||||
let repo = Repo::discover(&self.dir).unwrap().expect("repository found");
|
||||
repo.set_execution_allowed(true);
|
||||
repo
|
||||
}
|
||||
fn path(&self, rel: &str) -> PathBuf {
|
||||
self.dir.join(rel)
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stash_push_stashes_only_the_given_paths() {
|
||||
let fx = Fixture::new();
|
||||
fx.write("a.txt", "a\n");
|
||||
fx.write("b.txt", "b\n");
|
||||
fx.commit_all("base");
|
||||
let repo = fx.repo();
|
||||
|
||||
fx.write("a.txt", "a changed\n");
|
||||
fx.write("b.txt", "b changed\n");
|
||||
repo.stash_push(Some("only a"), false, false, &[fx.path("a.txt")]).unwrap();
|
||||
assert_eq!(fx.read("a.txt"), "a\n");
|
||||
assert_eq!(fx.read("b.txt"), "b changed\n");
|
||||
assert_eq!(repo.stashes().unwrap().len(), 1);
|
||||
|
||||
repo.stash_apply(0, true, false).unwrap();
|
||||
assert_eq!(fx.read("a.txt"), "a changed\n");
|
||||
assert_eq!(fx.read("b.txt"), "b changed\n");
|
||||
assert!(repo.stashes().unwrap().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stash_push_stashes_only_the_given_untracked_paths() {
|
||||
let fx = Fixture::new();
|
||||
fx.write("a.txt", "a\n");
|
||||
fx.commit_all("base");
|
||||
let repo = fx.repo();
|
||||
|
||||
fx.write("new.txt", "new\n");
|
||||
fx.write("other.txt", "other\n");
|
||||
repo.stash_push(None, true, false, &[fx.path("new.txt")]).unwrap();
|
||||
assert!(!fx.path("new.txt").exists());
|
||||
assert!(fx.path("other.txt").exists());
|
||||
assert_eq!(repo.stashes().unwrap().len(), 1);
|
||||
|
||||
repo.stash_apply(0, true, false).unwrap();
|
||||
assert_eq!(fx.read("new.txt"), "new\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stash_push_rejects_paths_outside_the_repository() {
|
||||
let fx = Fixture::new();
|
||||
fx.write("a.txt", "a\n");
|
||||
fx.commit_all("base");
|
||||
let repo = fx.repo();
|
||||
fx.write("a.txt", "a changed\n");
|
||||
|
||||
let outside = tempfile::tempdir().unwrap();
|
||||
std::fs::write(outside.path().join("outside.txt"), "outside\n").unwrap();
|
||||
let error = repo.stash_push(None, false, false, &[outside.path().join("outside.txt")]).unwrap_err();
|
||||
assert_eq!(error.kind, ErrorKind::InvalidInput);
|
||||
assert!(error.message.contains("outside the repository"), "{error:?}");
|
||||
|
||||
// Nothing was stashed and the change is still in the working tree.
|
||||
assert!(repo.stashes().unwrap().is_empty());
|
||||
assert_eq!(fx.read("a.txt"), "a changed\n");
|
||||
}
|
||||
@@ -50,7 +50,12 @@ Agents are unavailable in a project you have not marked as trusted.
|
||||
- **Local checkout**: the agent edits your files directly.
|
||||
</Step>
|
||||
<Step title="Pick the agent and its settings">
|
||||
Choose the agent, and its model, mode and other options when it offers them. Models and modes appear after the agent's first task.
|
||||
Choose the agent, and its model, mode and other options when it offers them. Cached models appear immediately; the selected agent's catalog refreshes in the background when it is more than 15 minutes old, without sending a prompt or attaching project MCP servers.
|
||||
|
||||
After changing the agent CLI or account, open the model picker and choose **Refresh models**. Refreshes share one lookup and have a 30-second cooldown after completion, including failures. Cached choices remain usable while refreshing or after a failure.
|
||||
|
||||
Available models depend on the installed agent and account. If a selected model is removed, choose another before starting. Refreshing does not change running tasks.
|
||||
|
||||
</Step>
|
||||
<Step title="Send">
|
||||
The task opens as an editor tab and starts.
|
||||
@@ -67,6 +72,12 @@ The task tab shows the conversation: your messages, the agent's replies, its rea
|
||||
|
||||
On your own messages you can **Edit** (the messages after it are replaced; files already changed stay as they are) or **Retract Message**. On replies you can **Copy Markdown**, **Quote Reply**, **Regenerate Response** and **Branch Conversation from Here**, which starts a new task from that point.
|
||||
|
||||
**Focus Agent Input** defaults to `⌘L` on macOS and `Ctrl+L` on Windows / Linux in every built-in keymap. When focus is inside your writable task conversation, in either an editor tab or the tool window, it returns focus to the message input without changing your draft, caret position or selection—even when pressed repeatedly. It does not send a message or jump through message history.
|
||||
|
||||
The shortcut does not apply to archived tasks, subagent conversations or the New Task page, and does not take focus from open popups or dialogs. Outside the task conversation, the keys keep their usual editor or terminal behavior. To change or remove the binding, open **Settings › Keymap › Agents › Focus Agent Input**.
|
||||
|
||||
Right-click a link and choose **Copy Link Address** to copy its URL; **Copy** copies the selected text.
|
||||
|
||||
### Permissions
|
||||
|
||||
When an agent wants to run a command or call a tool that needs approval, the task shows the request and is listed as **Needs Input**. Choose one of the options the agent offers, such as **Allow Once**, **Allow for This Session**, **Always Allow**, **Decline**, **Always Decline** or **Decline and Stop**. Many agents also have a mode picker that approves edits or everything automatically; use it with care.
|
||||
|
||||
@@ -107,6 +107,8 @@ To format on every save, enable **Reformat code** (and optionally **Optimize imp
|
||||
|
||||
Tab placement, the tab limit and what happens when it is reached, the preview tab, sorting, and which tab is activated after one closes are set in **Settings › Editor › General › Editor Tabs**.
|
||||
|
||||
With tabs placed left or right, the tab column is as wide as its longest name, up to a limit. Drag the column's inner edge (or focus it and press `←` / `→`) to change the limit; it applies to every editor group and is remembered.
|
||||
|
||||
## Split editors
|
||||
|
||||
| Action | macOS | Windows / Linux |
|
||||
|
||||
Loaded 100 of 457 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Заблокувати користувача