Порівняти коміти

...
Автор SHA1 Повідомлення Дата
熊猫同学 646fb1e876 fix(ide-workspace): declare GetProcessTimes consistently
Two modules declared the same kernel32 function with different
out-pointer types (FileTime vs packed u64), which clippy rejects
under -D warnings. Both now read the packed u64.
2026-11-15 08:23:23 +08:00
l0ng-ai e26e1d1d7d fix(themes): add the shell command token to the theme template and docs (#3464)
#3458 added --syn-shell-command without setting it in the Harbor Night
scheme (the template new themes start from) or regenerating plugins/themes.md,
so scripts/theme/theme.test.ts failed. Imported IntelliJ schemes map
BASH.EXTERNAL_COMMAND to it.
2026-11-15 08:23:20 +08:00
l0ng-ai 0af3bd00e7 fix(editor): highlight shell commands, function names and local from the grammar (#3458)
The legacy shell stream mode works from fixed word lists: local, declare and
return were plain text, and so were function definitions and any command
outside its ~70 builtins. The bundled tree-sitter-bash grammar now draws its
highlights over the stream mode, as Kotlin's does: declaration keywords,
function definition names, command names and variables. Command names get
their own colour (--syn-shell-command, Color Scheme › Shell Script ›
Command), blue in light and orange in dark schemes, falling back to the
function-call colour in colour themes.
2026-11-15 05:10:11 +08:00
SunZhiQiang 951e81e065 feat(editor): untitled buffers and Save As with a file type (#3427)
File › New Untitled File opens an editable untitled buffer (Untitled-N), also from the welcome page (New File, plus Open File beside Open), the LightEdit empty-editor hints, and Meta+N in the VS Code keymap. Saving asks for a path in a Save As dialog whose type dropdown lists every registered language; the backend re-keys the buffer under the operations lock, keeps its undo history and re-detects the language. Untitled buffers belong to the window and follow it into and out of projects; closing a dirty one asks to save it. A LightEdit window with only untitled buffers offers Open Project via the folder picker.

Closes #3426

Co-authored-by: panda-student <[email protected]>
2026-11-15 04:32:12 +08:00
l0ng-ai 023c0a6ee6 fix(debugger): silence the unix-only runInTerminal pid on Windows (#3456)
ProgramTerminal's pid and process_id are read only by the unix process-group
cleanup, so Windows clippy failed with dead-code warnings.
2026-11-15 04:11:35 +08:00
rohitkumardandaтаrohitdanda c5481e8fe4 fix(lsp): language servers that give up, start slowly or send bad data (#3449)
Several causes behind servers that ended up absent or stuck:
- Fallback servers now take over when the preferred one gives up (missing, not launchable, crashed for good, expired), not only when it is missing, and hand back after a manual restart.
- Crash-budget uptime counts from ready, not from launch; the crash message carries why initialize failed (timeout or the server's error).
- Restart works while a server is still starting and no longer holds the reconfigure lock while it waits; a server dropped during its crash backoff no longer comes back.
- Progress of a still-starting server is watched.
- One malformed diagnostic no longer drops the file's other problems; DocumentLink.target URIs are escaped.
- Semantic tokens, inlay hints and code lens are asked again on ContentModified (staleRequestSupport).
- rustup toolchain lookup uses the platform executable suffix and USERPROFILE, and probes in the project root with a timeout.
- Server output lines are capped at 8 KiB.
- Go to Declaration from Java to a Kotlin class declared in a differently named file finds the .kt source.
- The kotlin-lsp install hint names the Language Servers settings page.

Co-authored-by: rohitdanda <[email protected]>
2026-11-15 04:10:44 +08:00
Deng Liuтаdengliu dfacb45575 fix(agents): sanitize unsupported ambient GitHub tokens for Copilot CLI (#3443)
Copilot CLI rejects a classic PAT (ghp_…) in GITHUB_TOKEN / GH_TOKEN / COPILOT_GITHUB_TOKEN outright instead of falling back to its stored login, and a stale or narrow-scope token from the login shell made model discovery fail with "Not authenticated". Classic PATs inherited from the shell are now cleared for copilot; when discovery still fails with an authentication error and such a token is inherited, it retries once without the ambient tokens and keeps doing so for later launches. Tokens set in the agent's own environment settings are kept.

Co-authored-by: dengliu <[email protected]>
2026-11-15 04:07:51 +08:00
Mr0Bread 3363bb8a46 feat(agents): resize the task Changes pane by dragging its edge (#3451)
The Changes pane beside a task's conversation was fixed at 280 px. Its left edge is now a separator that drags (or moves with ←/→ when focused); the width applies to every task, is remembered across restarts, and always leaves the conversation 240 px (half of a narrower session). In the tool window the Changes view fills the window as before.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-15 04:05:17 +08:00
Mr0Bread 39f34ec22f fix(agents): clicking the bottom of the message box puts the caret in it (#3450)
The composer's 44 px minimum height sat on CodeMirror's scroller, while the text inside stopped at 32 px (its min-height: 100% doesn't resolve against a min-height parent), so a click in the strip below focused the bare scroller and showed no caret. The minimum height now sits on .cm-content.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-15 04:05:07 +08:00
Taylor BurkeтаTaylorBurke 34ae8acbf8 fix(window): lower the minimum window width to 400 px and let the UI fit it (#3440)
Tiling compositors ignore a client's minimum size, so a tile narrower than 800 px laid the webview out at 800 px and scaled it down. The minimum is now 400 px on every platform. At that width the welcome page header and action tiles wrap, the title bar's run and VCS widgets ellipsize instead of covering the toolbar, dialog footers wrap, and below 560 px the left and right tool windows collapse (their stripe buttons stay, the saved layout is unchanged, a wider window brings them back).

Closes #3046

Co-authored-by: TaylorBurke <[email protected]>
2026-11-15 04:02:55 +08:00
l0ng-ai 6bafcb2b5c fix(i18n): translate the memory popup's PID tooltip in Chinese (#3453)
The PID tooltips from #3437 kept the English text in zh.json, which the i18n catalog test rejects; they now read 进程号 {pid}.
2026-11-15 03:58:02 +08:00
l0ng-ai a6ed65e6d1 fix(editor): run icons, inheritance markers and the doc toggle share one gutter column (#3452)
Each kind of gutter icon had its own column, so a file with doc comments kept an empty column for the hover-only rendered-view toggle beside the inheritance markers, and run icons took a third. A shared icon area (iconAreaSource facet in core, exported from @/editor/api) now holds run icons, implements/overrides markers and the doc toggle in one column, as wide as the most icons on any line; empty areas take no width, sources can reserve an icon's width and handle their own presses. Hiding run icons drops them from the area.
2026-11-15 03:56:38 +08:00
l0ng-ai 009e61d27b fix(editor): clicking the author hint again closes the blame annotations (#3448)
The Code Vision author hint ran Annotate with Git Blame only while the annotations were hidden, so a second click left them open. It now runs vcs.annotate on every click, toggling the annotations like the action itself.
2026-11-15 03:46:11 +08:00
l0ng-ai a89c0dd229 feat(ui): small raster data-URI images as icons (#3446)
Icon draws a raster data: URI (PNG, JPEG, GIF or WebP, base64, at most 64 KiB) as an image of the icon's size, so plugin view items and buttons can show avatars and status badges the plugin fetched with its net grant; the UI never loads URLs for icons, and older hosts show the file icon. Documented in the views guide.
2026-11-15 03:33:29 +08:00
l0ng-ai ea644dcdb0 feat(plugins): diffs of up to 8 MiB per side from open-diff (#3445)
views.open-diff and content tab reply diffs accept 8 MiB per side (was 1 MiB). A larger side fails with invalid("diff too large: the <side> side is over 8 MiB") from the host call and "Diff too large" for a web view's open-diff message, so plugins can tell the size limit from a malformed request. WIT docs updated and plugin components rebuilt.
2026-11-15 03:32:29 +08:00
l0ng-ai 519c8ad8fc feat(plugins): Markdown preview for Markdown content tabs (#3442)
A Markdown content tab (language markdown, or a .md file name) gets the Markdown editor's layouts (Editor Only / Editor and Preview / Preview Only) and opens on the preview when read-only. The preview uses the editor's sanitized Markdown pipeline, highlights fenced code, follows reloaded content and opens web links in the browser. No API change; documented in the plugin README and views guide.
2026-11-15 03:30:52 +08:00
l0ng-ai 27fdbe71e5 fix(ui): leave language status widgets out of projects without the language (#3439)
The status bar context menu listed PHP Language Level and Python Interpreter in every project, although neither widget shows without PHP or Python files. Status widgets can now be marked unavailable for the open project (setStatusWidgetAvailable): neither rendered nor listed in the menus. PHP and Python track availability from activation (so hidden widgets are judged too) and re-check on project, index, file-set and settings changes.
2026-11-15 03:01:18 +08:00
l0ng-ai d5cd5e7a55 feat(perf): CPU, PID and End Process in the memory popup (#3437)
The Memory & Diagnostics popup now shows each row's CPU use since the previous measurement (percent of one core; proc_pid_rusage / /proc/<pid>/stat / GetProcessTimes via ide-workspace CpuSampler), the PID in the row tooltip, and an End Process button for language server, run and terminal rows. end_process_tree sends SIGTERM, then SIGKILL after a grace period (TerminateProcess on Windows), and refuses pids that don't descend from the backend; owners see the exit as usual (language servers restart, terminals close).
2026-11-15 02:55:37 +08:00
l0ng-ai c32a68c0b8 feat(ui): Project Structure toolbars and the type hierarchy root's package (#3436)
Project Structure: Modules gets an icon toolbar (+ menu with Add Module Overrides / Import Module from Existing Sources, -); Libraries gets + (JAR or Directory, Classes Folder), - and Copy. Type Hierarchy: the root type shows the server's detail (Java: its package) beside its name, through the additive lsp_prepare_type_hierarchy command.

Part of #3409
2026-11-15 02:29:04 +08:00
pravdomil 59cc84521f fix(project-tree): Autoscroll to Source no longer opens files in external apps (#3429)
Autoscroll to Source used the same EditorService.openFile as an explicit open, so files matching Files opened in associated applications (PDF, Office documents) were launched in the system app whenever the selection touched them. OpenOptions.external = false now leaves such files unopened; ExternalFileOpenHandler gains matches() so the editor can tell without launching anything. Explicit opens (double-click, Enter, Jump to Source) are unchanged.

Co-authored-by: pravdomil <[email protected]>
2026-11-15 02:15:18 +08:00
l0ng-ai 4faa37aaea fix(vcs): Pull, Merge and Rebase start on the repository of the context (#3435)
The context menus' Git > Pull... ran Update Project (every repository) while the rest of the submenu acts on the selection. It now opens the Pull dialog. Pull, Merge and Rebase start on the repository of the action's files, else the focused Project view selection, else the active editor's file, else the selected repository; Update Project still updates every repository.

Closes #3404
2026-11-15 02:13:34 +08:00
l0ng-ai aaf25d9f0d fix(vcs): keep the Project view branch suffix of workspace members current (#3434)
Only the project folder's watcher emitted VcsChanged on .git changes, so members outside the workspace folder kept a stale branch suffix after a checkout, fetch, pull or push. Member watchers now emit it too (same own-index-write filter), and the suffix is read again when a repository's HEAD moves or a fetch / pull / push finishes.

Closes #3405
2026-11-15 02:13:23 +08:00
l0ng-ai 913ad6616c feat(vcs): View Commits after Update Project opens an Update Info tab (#3432)
View Commits opened a modal dialog with one repository's commits. It now opens a closable Update Info tab in the Git tool window: the Log of every updated repository, each limited to the range the update brought in (LogFilters.repoRanges, sent per repository as the query's ranges), with the Log's filters, details and changed files; each update opens a tab of its own.

Closes #3403
2026-11-15 02:00:12 +08:00
l0ng-ai b80f0f9e39 fix(ui): collapsing a tree node also collapses everything expanded below it (#3430)
Collapsing a node kept its expanded descendants in the expanded set, so expanding it again brought the whole subtree back. The shared Tree now drops the node's loaded descendants when it collapses (chevron, Left, Enter, single-click expand), so every tree (Project view, Structure, Git, Debugger variables, ...) shows only direct children on the next expand; trees that persist their expanded set save the result.

Closes #3402
2026-11-15 01:48:20 +08:00
l0ng-ai 460027fd4b feat(layout): Restore Current Layout and Save Changes in Current Layout (#3421)
Restore Current Layout applies the active saved tool window layout again, or the default layout when none is active; Shift+F12 in the IntelliJ keymaps now runs it. Save Changes in Current Layout stores the current windows into the active layout. Both are in Window > Tool Window Layouts.

Closes #3399
2026-11-15 01:28:16 +08:00
Muuтаmuutot 33c5627bdb feat(analysis): Python inheritance inspections from the IDE graph (#3419)
Four Python inspections under a new Python category in Settings › Inspections, computed from the IDE's inheritance graph with facts the backend extracts from the syntax tree (doc_python_facts): circular inheritance (error), @override on a method no base declares (warning), abstract method not implemented (warning), redundant override (weak warning). A rule stays silent while a base on the lookup path is outside the project; object, ABC, Generic and Protocol count as known. A class that is itself abstract (own abstract methods, or ABC / Protocol as a base) isn't asked to implement inherited abstract methods.

Co-authored-by: muutot <[email protected]>
2026-11-15 01:20:26 +08:00
akshathayadav378-boopтаakshathayadav378-boop 3a22a2d0f0 fix(vcs): show cached inline blame without the 300 ms debounce (#3410)
A caret move in a document whose blame is already loaded shows the new line's inline blame at once (BlameCache keeps the resolved lines and peek(doc) returns them for the current document version); edits and first loads keep the 300 ms debounce, so typing starts at most one git blame per pause. This replaces the equivalent change merged in #3418.

Co-authored-by: akshathayadav378-boop <[email protected]>
2026-11-15 01:11:43 +08:00
l0ng-ai c169b8ab3c fix(vcs): show the current line's inline blame at once when the file is blamed (#3418)
A caret move within a document whose blame is loaded (or loading) now shows the new line's inline blame at once; edits and a document not blamed yet still wait 300 ms, so typing doesn't run git per keystroke.
2026-11-15 01:10:23 +08:00
l0ng-ai 0dbce5d09e feat(vcs): show the number of changed files in the Project view's branch suffix (#3416)
With "Show branch and incoming/outgoing commits in the Project view" on, a repository root with local changes also shows how many files changed (staged, unstaged or untracked; ignored aside), with a tooltip; a clean repository shows nothing extra. The count follows the working tree status, re-rendered from the branches already read without asking git again.
2026-11-15 01:08:47 +08:00
l0ng-ai 7fe6f5f58d feat(run): run icons on the profiles of launchSettings.json (#3415)
A .NET project's Properties/launchSettings.json shows a run icon on each Project profile; Run / Debug launch the project's configuration for that profile, or a temporary copy for a profile added since detection ran. The package.json scripts gutter is generalised into entryGutter for this.
2026-11-15 01:04:59 +08:00
l0ng-ai abf904553d feat(build): build .NET solutions and projects in the Build tool window (#3414)
Build Project / Build Module / Rebuild / Build Artifacts build a .NET solution or project (dotnet build, --no-incremental, dotnet publish), and MSBuild's canonical file(line,col): error CODE lines become Build messages linking to the source. A .NET run configuration's "Build .NET project" before-launch task builds its project in the Build tool window instead of a Run tab.
2026-11-15 01:00:28 +08:00
l0ng-ai d425388f1a fix(run): read .NET launch profiles from a launchSettings.json with a byte order mark (#3412)
Visual Studio saves launchSettings.json with a UTF-8 byte order mark, which serde_json rejects, so a project's launch profiles came out empty and a configuration naming one failed with "launch profile is missing". jsonc::strip drops a leading byte order mark, which also covers devcontainer.json, the .vscode/launch.json import, tsconfig.json and run detection.
2026-11-15 00:54:37 +08:00
Akbar Husain de123e2a21 perf(index): stream the cold scan instead of chunk joins (#3407)
The initial scan parsed files in 512-file chunks (rayon join, then insert and redb commit), so every join idled the pool on its slowest file and insert/persist serialized between chunks. The scan now parses the whole todo list in one rayon pass, serializes cache records inside the parse, and streams them over a bounded channel to the index thread, which inserts and persists in batches of 2048 alongside the parse. Records stay byte-identical; the scan's log line carries walk / classify / extract / insert / persist / durable laps.

Co-authored-by: akbar1214 <[email protected]>
2026-11-15 00:48:27 +08:00
Muuтаmuutot d19dbec56a feat(editor): Python inheritance in navigation from one shared graph (#3397)
Python's servers answer no type hierarchy, so the navigation built on it did nothing for Python. The inheritance graph moves out of the gutter into src/editor/navigation/inheritanceGraph.ts (one fetch per project, the name map and base resolver), and Type Hierarchy, Structure's Show Inherited, Super Method and Go to Implementation use it for Python, merged with whatever the server answers. The gutter keeps its per-file import reading.

The subclass map is built when a subtype lookup first needs it; the files whose imports decide a base are read in one batch, bases resolve in parallel, and an import whose name is no class of the project is not looked up.

Co-authored-by: muutot <[email protected]>
2026-11-15 00:41:25 +08:00
l0ng-ai 1a204b5f4b fix(lsp): show TypeScript 7 as the server when its own tsc --lsp runs (#3408)
A project on TypeScript 7 runs the compiler's native tsc --lsp in place of vtsls / typescript-language-server, but the status bar and the server popup still named the configured server. The settings hook gives such a server a display name, carried by ServerInfo and the languageServer event as an optional label; the server's name stays its identity.
2026-11-15 00:25:51 +08:00
l0ng-ai 3ec7daf845 fix(profiler): name js-debug as the recorder of a page's CPU profile (#3406)
A page's CPU profile is read like a Node.js run's .cpuprofile files, so its snapshot header said "node --cpu-prof". profiler.showCpuProfiles takes an optional profiler name; the debugger passes js-debug.
2026-11-15 00:16:52 +08:00
ntbang0901 112ac05052 feat(debugger): replace the responses a debugged page receives (#3329)
A page session can answer requests itself: rules (URL part → status, content type, body) are kept per project, edited from the Network tab (Overrides, Override Response… on a request), and applied through js-debug's CDP proxy with Fetch.enable / Fetch.fulfillRequest. Interception is switched off before the connection is dropped, unknown statuses get a reason phrase, cross-origin pages get CORS headers and preflight answers, and rules present at page start wait for the launch response.

The Network list marks a request overridden only when it was sent after the rule that answers it was saved or enabled.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-15 00:15:09 +08:00
ntbang0901 10b7340e2a feat(debugger): CPU profile of a debugged page (#3320)
The More menu of a page session has Start / Stop CPU Profile (js-debug's startProfile / stopProfile); the recording opens in the Profiler tool window as a snapshot named after the session. The .cpuprofile reader prefers js-debug's $vscode.locations, so a frame opens the project's source at its line instead of the URL the browser ran.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-15 00:01:16 +08:00
Andrea Basileтаandreabasile 6fa5b368e6 feat(python): a Docker or Compose run target as the project interpreter (#3364)
A Docker or Compose run target can be the project's Python interpreter (languages.python.interpreter = "target:<id>"). The IDE runs the target's Python once in a new container (entrypoint and service dependencies skipped), copies its sys.path folders outside the standard library and the project with docker cp, once per image, and gives them to basedpyright / Pyright as extraPaths with an empty local virtualenv as interpreter. Packages show read-only with a refresh that copies again; Python run configurations without a target of their own run on that target; the copy shows under External Libraries. Snapshots require a trusted project.

Co-authored-by: andreabasile <[email protected]>
2026-11-14 23:53:40 +08:00
Mr0Bread dd40273952 test(database): cover result submission lifecycle interleavings (#3346)
Covers paging staying disabled while a remounted result applies edits, and an apply finishing after its result tab closed without refreshing another tab. (The refresh-failure case is already covered on main.)

Co-authored-by: Mr0Bread <[email protected]>
2026-11-14 23:43:23 +08:00
l0ng-ai 21bc81c6da fix(ui): remember splitter positions in the Git and other tool windows (#3401)
None of the Git views gave SplitGroup a storageKey, so their splitters reset whenever the view was recreated. They, the Commit dialog and the Todo, Find, Problems, Usages, Build, HTTP Client and Dependency Analyzer splits now store their sizes. A view whose preview comes and goes keys sizes per panel set, and SplitGroup applies that set's sizes when the key changes, without remounting the view.
2026-11-14 23:42:02 +08:00
l0ng-ai 1dc1e3cca1 feat(debugger): Focus On Breakpoint toggle for the Threads & Variables tab (#3400)
A breakpoint hit always brought Threads & Variables to the front. Its tab's context menu now has Focus On Breakpoint (on by default, client setting debugger.variablesOnBreakpoint); turned off, a hit keeps the current tab, e.g. Console. Finishing a step already kept it.
2026-11-14 23:34:17 +08:00
l0ng-ai 4a82c22fdc fix(plugins): keep document indexer markers on their lines while typing (#3398)
Every edit emptied the indexed-marker gutter until the host re-indexed the file; the column has no fixed width, so the text shifted left and back on each keystroke. The markers are now mapped through the change (a deleted line drops its own), as line markers and bookmarks already do.
2026-11-14 23:27:33 +08:00
Mr0Bread 010e7ba6b4 feat(agents): add configurable task input focus shortcut (#3382)
Focus Agent Input (⌘L on macOS, Ctrl+L on Windows / Linux, in every built-in keymap) returns focus to the current writable task's message input from its conversation, in an editor tab or the tool window, keeping the draft, caret and selection. It applies only while focus is inside an agent task, so the keys keep their editor and terminal meaning elsewhere; the binding is under Settings › Keymap › Agents.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-14 23:24:59 +08:00
Mr0Bread 9984e40b83 fix(agents): refresh available models before starting tasks (#3366)
New Task showed no models until an agent's first task. It now shows the cached catalog at once and discovers the agent's models without starting a task (no prompt, no project MCP servers): one shared lookup per provider and project, fresh for 15 minutes, with a Refresh models action and a 30-second cooldown. Catalogs persist per effective launch configuration (a digest, never the arguments or credentials); a selected model the latest catalog dropped blocks Start until another is chosen. Discovery requires a trusted project.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-14 23:22:14 +08:00
Muuтаmuutot 36a6e6359d feat(editor): resolve an ambiguous Python base name through the file's imports (#3384)
A base name several modules declare was skipped, so a project with many Config classes got no marker for them. The subclass file's from-imports (aliases included) now pick the declaration; with no import naming it the base is still ignored. Imports are read only when a base may name a class of the open file, so a marker refresh does not read every subclass file of the project.

Co-authored-by: muutot <+[email protected]>
2026-11-14 23:14:53 +08:00
Muuтаmuutot 4f64326f89 fix(editor): keep close gestures when the close button is hidden (#3354)
EditorArea passed closable: showCloseButton, and closable gates every close gesture in Tabs, so Close button position: None made editor tabs unclosable by middle click, Shift+click and Delete. Tabs gets a visual-only showCloseButton that hides the button while the tab stays closable.

Co-authored-by: muutot <[email protected]>
2026-11-14 23:12:14 +08:00
akshathayadav378-boopтаakshathayadav378-boop b5c2d0dc5e fix(settings): let Project View branch decorations be bold and italic (#3387)
The six Project View branch attributes never set fontStyle, so Settings disabled Bold and Italic for them although the Project view's branch suffix already reads --git-<attr>-bold/-italic/-effect. Setting it enables the checkboxes and makes the overrides emit those variables.

Co-authored-by: akshathayadav378-boop <[email protected]>
2026-11-14 23:10:13 +08:00
l0ng-ai c9dca15972 fix(debugger): let a Node.js program read input while it is debugged (#3395)
js-debug launched the program with the internal console, so it had no standard input. Run it in the debugger's pseudo-terminal instead, and reap a runInTerminal program as soon as it exits: an unreaped zombie looked alive to js-debug's processId polling and kept the session open.
2026-11-14 23:08:36 +08:00
l0ng-ai 41b581eafd fix(github): recognise remotes cloned through ssh.github.com (#3391)
A clone over GitHub's SSH-over-HTTPS host (ssh://[email protected]:443/o/r.git or [email protected]:o/r.git) wasn't recognised as a GitHub repository. parseGitHubRemote now accepts ssh.github.com for SSH remotes; https://ssh.github.com stays rejected. Sibling of #3379.
2026-11-14 22:57:59 +08:00
akshathayadav378-boopтаakshathayadav378-boop 13e18af777 fix(vcs): open Bitbucket SSH clones on bitbucket.org (#3379)
For a Bitbucket Cloud repository cloned over SSH ([email protected]:team/repo.git), Open In used ssh.bitbucket.org as the web host and GitLab routes. remoteWebUrl now maps the services' SSH front hosts (ssh.bitbucket.org, ssh.github.com, altssh.gitlab.com) to their web hosts before building the URL.

Co-authored-by: akshathayadav378-boop <[email protected]>
2026-11-14 22:54:44 +08:00
charlie-morrisonтаcharlie-morrison 488e819b46 fix(vcs): keep the merge editor responsive on long files (#3381)
Opening a conflicted package-lock.json in the merge editor froze the app. The Your/Their panes diffed each side against the base with @codemirror/merge's default config, which gives up past ~16 KB of changed span and returns one chunk from the first change to the last, so each pane rendered the whole base as one deletion widget; syncMergeScroll ran an unbounded character diff on first scroll. Both now use the IDEA line comparison the diff viewer uses (ideaDiffConfig).

Co-authored-by: charlie-morrison <[email protected]>
2026-11-14 22:53:04 +08:00
l0ng-ai ad8df8fe00 feat(database): open several query consoles per data source (#3386)
New › Query Console always reopened the single console of a data source. It now creates another console each time (console, console_1, console_2…, as in DataGrip); Jump to Query Console opens the only console directly or lists them with New Query Console; Tools › Query Console keeps opening the default one. Backend: db_consoles(id, create).
2026-11-14 22:50:51 +08:00
l0ng-ai 19d6eab21a feat(http-client): find in the response body and open the files .http lines name (#3383)
The response pane had no find: ⌘F (or a Find button next to Show Raw) now opens the console find bar with matches marked in the body. ⌘-click / ⌘B on the file a .http line names (< body, > handler, >> redirect, <> saved response) opens it; saved responses resolve against .idea/httpRequests/ (where IntelliJ IDEA writes them), then the request file's folder.
2026-11-14 22:32:50 +08:00
Muuтаmuutot 5c0befeff0 feat(lsp): compute Python inheritance gutter markers from the IDE index (#3359)
Python's overrides / is-overridden gutter markers never appeared: basedpyright and pyrefly declare no typeHierarchyProvider, and a single-root server cannot see classes in attached projects. The IDE now extracts every class X(Base) with tree-sitter across the project (content roots included) and nav_inheritance exposes them; the editor merges this graph with the server's answer. A base name is trusted only when it names one class (unique in the project or declared in the same file). Class bases are cached per file and parsed again only when the file changes.

Co-authored-by: muutot <[email protected]>
2026-11-14 22:24:48 +08:00
Akbar Husain 07bd8ded5e feat(run): report Zig test runs in the test runner (#3355)
Zig test blocks had run markers but their runs never reached the test runner: no parser read the runners' output. A parser now reads zig test's per-test progress lines, error-return traces and summary, and zig build test's failure blocks and --summary all counts (added to those runs). Results are rooted at the module stem so gutter icons turn green/red per test; Rerun Failed and Run Selected build --test-filter configs for standalone zig test runs. test names are unescaped in run targets so the marker label, --test-filter and the runner's names agree.

Co-authored-by: akbar1214 <[email protected]>
2026-11-14 22:16:35 +08:00
Mr0Bread 4ab85d5f49 feat(editor): resize the vertical editor tab column by dragging its edge (#3365)
With tabs placed left or right, the column could not be resized and was never narrower than 140 px. It now fits its longest name up to a limit (260 px by default); dragging its inner edge, or ←/→ on the focused edge, changes the limit for every editor group and it is remembered. The editor keeps at least 60% of each group.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-14 22:15:22 +08:00
Mansur Akhmadiтаmansur-akhmadi 3c91adfae2 feat(vcs): choose the repository in the Rebase dialog when the project has several (#3351)
Git > Rebase… always rebased the repository selected in the branch widget. With two or more repositories the dialog now has the repository combo: picking one reloads its branches and the label, clears the chosen upstream and --onto base, and Rebase runs in that repository.

Co-authored-by: mansur-akhmadi <[email protected]>
2026-11-14 22:12:34 +08:00
Mansur Akhmadiтаmansur-akhmadi 5cfa6614a1 feat(vcs): choose the repository in the Merge dialog when the project has several (#3332)
Git > Merge… always merged into the repository selected in the branch widget. With two or more repositories the dialog now has the repository combo (as in Pull): picking one reloads its branches and title, clears the chosen branch, and Merge runs in that repository.

Co-authored-by: mansur-akhmadi <[email protected]>
2026-11-14 22:12:04 +08:00
Shohruh Ismoilovтаshohruh-genius e875d97f7c fix(project): show relevant file templates in New menus (#3342)
The New menu offered every language template in every project. It now uses the detected type of the project owning the selected folder for language-specific templates (Rust, Python, Go, PHP, Java/Kotlin); generic and web templates (JavaScript, TypeScript) stay in every project, as in IntelliJ IDEA Ultimate, and From Template still lists the whole catalog. Tomcat Context Descriptor is offered only when the project has a Tomcat run configuration.

Co-authored-by: shohruh-genius <[email protected]>
2026-11-14 22:07:27 +08:00
Mr0Bread 31016ee684 fix(agents): add Copy Link Address to the context menu of links in readable text (#3371)
Right-clicking a link in Agent Task chat (and other readable panes using the shared Copy menu) only offered Copy, which copied the selected label. Links now also get Copy Link Address, as in browsers; Copy keeps copying the selection, and in-page anchors get no link item.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-14 22:05:29 +08:00
l0ng-ai 6a93f1cce6 fix(debugger): show a JVM session's Memory tab from the start; pass lldb-dap's environment for cargo debugging (#3380)
The Memory tab appeared only once the debugger reported its capabilities, so a Java or Kotlin session's tab bar grew a tab after the program started; it is now shown from the start with a placeholder until the program runs. A cargo configuration debugged with lldb-dap passed CodeLLDB's sourceLanguages and an environment object; lldb-dap now gets KEY=VALUE strings like other native programs.
2026-11-14 22:01:34 +08:00
l0ng-ai 2b4e3c79f8 fix(debugger): let a Python program read input while it is debugged (#3374)
debugpy launched the program with console "internalConsole", which gives it no standard input, so input() raised EOFError. It now runs with console "integratedTerminal": debugpy starts it through runInTerminal on the pseudo-terminal the debugger already provides, and what is typed in the debug console reaches it. Output comes from that terminal, so redirectOutput is dropped (it doubled every line). Node under js-debug still lacks input; its terminal launch needs its own fix.
2026-11-14 21:34:21 +08:00
452 змінених файлів з 16976 додано та 1202 видалено

No files matched your search

Створено
+3
Переглянути файл
@@ -4146,6 +4146,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
name = "ide-agent"
version = "0.1.45"
dependencies = [
"blake3",
"dunce",
"encoding_rs",
"ide-core",
@@ -4268,6 +4269,7 @@ dependencies = [
name = "ide-dap"
version = "0.1.45"
dependencies = [
"base64 0.22.1",
"futures",
"ide-core",
"ide-heap",
@@ -4286,6 +4288,7 @@ dependencies = [
"specta",
"tempfile",
"tokio",
"tokio-tungstenite",
"tracing",
"tracing-subscriber",
"windows-sys 0.61.2",
+4
Переглянути файл
@@ -7,6 +7,7 @@ license.workspace = true
[dependencies]
encoding_rs.workspace = true
blake3.workspace = true
ide-vfs.workspace = true
ide-core.workspace = true
ide-mcp.workspace = true
@@ -24,3 +25,6 @@ dunce.workspace = true
uuid.workspace = true
tempfile.workspace = true
which = "8"
[dev-dependencies]
tokio = { workspace = true, features = ["test-util"] }
+55 -28
Переглянути файл
@@ -3,6 +3,7 @@
//! `initialize` alone).
use super::{Conn, Dialect};
use crate::catalogs::Discovered;
use crate::config;
use crate::model::AgentConfigOption;
use crate::provider::ProviderSpec;
@@ -15,22 +16,41 @@ use std::time::Duration;
const DISCOVER_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(3) } else { Duration::from_secs(120) };
/// Opens `session/new`, reads `configOptions`, and records per-model `thought_level` choices.
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(), String> {
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<Discovered, String> {
let prefer_stored = crate::provider::prefer_stored_copilot_auth();
match run_once(shared, spec, prefer_stored).await {
Err(err)
if !prefer_stored
&& crate::provider::is_copilot_provider(&spec.id)
&& crate::provider::has_unconfigured_copilot_ambient_token(&spec.env)
&& super::auth_required(0, &err) =>
{
let retry = run_once(shared, spec, true).await;
if retry.is_ok() {
crate::provider::mark_prefer_stored_copilot_auth();
}
retry
}
other => other,
}
}
async fn run_once(shared: &Arc<Shared>, spec: &ProviderSpec, strip_ambient: bool) -> Result<Discovered, String> {
let root = shared.paths.project_root.clone();
let launch = spec.clone();
let launch = tokio::task::spawn_blocking({
let root = root.clone();
move || launch.resolve_for_launch(&root)
move || launch.resolve_for_launch_with_copilot_env(&root, strip_ambient)
})
.await
.map_err(|e| e.to_string())?;
let cwd = dunce::canonicalize(&root).unwrap_or(root);
let stderr: Arc<Mutex<Vec<String>>> = Arc::default();
let lines = stderr.clone();
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(move |l| lines.lock().push(l));
let client = Dialect::wrap(Arc::new(ide_jsonrpc::NoopHandler));
let updates = Arc::new(DiscoveryUpdates { provider: spec.id.clone(), commands: Mutex::new(None) });
let client = Dialect::wrap(updates.clone());
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(|_| {});
let peer = (shared.connector)(&launch, &cwd, client.clone(), on_stderr)
.map_err(|e| format!("Couldn't start {} (`{}`): {e}", launch.name, launch.command))?;
let _stop = crate::catalogs::KillPeer(peer.clone());
let conn = Conn::new(peer.clone(), client);
let work = async {
conn.initialize(&client_info()).await.map_err(|e| e.to_string())?;
@@ -51,34 +71,41 @@ pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(),
(Some(model), Some(effort)) => model.current.clone().map(|m| vec![(m, effort)]).unwrap_or_default(),
_ => Vec::new(),
};
let option_ids: Vec<_> = options.iter().map(|o| o.id.as_str()).collect();
tracing::info!(
"agent {}: settings discovery got {} config option(s) {:?}, {} model effort list(s)",
launch.id,
options.len(),
option_ids,
levels.len()
);
shared.remember_discovered(&launch.id, &options, &[], &levels);
Ok::<(), String>(())
let commands = updates
.commands
.lock()
.clone()
.or_else(|| new.get("availableCommands").map(|_| config::commands_of(&new, &launch.id)));
Ok::<_, String>(Discovered { config: options, commands, levels: levels.into_iter().collect() })
};
let result = tokio::time::timeout(DISCOVER_TIMEOUT, work).await;
peer.kill();
match result {
Ok(Ok(())) => Ok(()),
Ok(Err(e)) => Err(match stderr.lock().last() {
Some(line) => format!("{e}: {line}"),
None => e,
}),
Err(_) => {
let tail = stderr.lock().join("\n");
if tail.is_empty() {
Err("the agent did not answer in time".into())
} else {
Err(format!("the agent did not answer in time:\n{tail}"))
}
Ok(result) => result,
Err(_) => Err("the agent did not answer in time".into()),
}
}
#[derive(Default)]
struct DiscoveryUpdates {
provider: String,
commands: Mutex<Option<Vec<crate::AgentCommand>>>,
}
impl ide_jsonrpc::Handler for DiscoveryUpdates {
fn on_notification(&self, method: &str, params: Value) {
if method == "session/update"
&& let Some(update) = params.get("update")
&& update["sessionUpdate"] == "available_commands_update"
{
// Commands are protocol data, not stderr (which may contain credentials).
*self.commands.lock() = Some(config::commands_of(update, &self.provider));
}
}
fn on_request(&self, method: &str, _: Value) -> Result<Value, (i64, String)> {
Err((ide_jsonrpc::codes::METHOD_NOT_FOUND, format!("unsupported discovery request {method}")))
}
}
fn is_thought_level(o: &AgentConfigOption) -> bool {
+1
Переглянути файл
@@ -108,6 +108,7 @@ fn manager(dir: &Path, agent: Arc<FakeV2Agent>) -> AgentManager {
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
catalogs: Arc::default(),
},
tokio::runtime::Handle::current(),
connector,
+1
Переглянути файл
@@ -155,6 +155,7 @@ async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent()
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
catalogs: Arc::default(),
},
tokio::runtime::Handle::current(),
connector,
+427
Переглянути файл
@@ -0,0 +1,427 @@
//! Model availability for new tasks. Sessions own selections; discovery owns catalogs.
use crate::{
AgentCommand, AgentConfigOption, AgentEvent, AgentModelCatalog, Known, Shared, acp, config, native, provider,
};
use ide_core::{IdeError, IdeResult};
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, HashMap};
use std::path::PathBuf;
use std::sync::{Arc, Weak};
use std::time::Duration;
use tokio::sync::watch;
use tokio::time::Instant;
const FRESH_FOR: Duration = Duration::from_secs(15 * 60);
const COOLDOWN: Duration = Duration::from_secs(30);
type Store = Arc<Mutex<HashMap<String, Known>>>;
type Key = (PathBuf, PathBuf, String);
/// Backend-wide single-flight model lookups, scoped to project and effective launch configuration.
#[derive(Default)]
pub struct AgentCatalogs {
state: Mutex<State>,
#[cfg(test)]
clock_ms: std::sync::atomic::AtomicU64,
}
#[derive(Default)]
struct State {
stores: HashMap<PathBuf, Store>,
entries: HashMap<Key, Entry>,
subscribers: Vec<Weak<Shared>>,
generation: u64,
}
struct Entry {
// Launch credentials remain only in memory, never serialized or logged.
context: LaunchContext,
generation: u64,
completed: Option<Instant>,
successful: Option<Instant>,
retry_at_ms: Option<f64>,
error: Option<String>,
flight: Option<watch::Receiver<bool>>,
abort: Option<tokio::task::AbortHandle>,
catalog: Option<CatalogData>,
}
#[derive(Clone, Default, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct CatalogData {
pub config: Vec<AgentConfigOption>,
pub thought_levels: BTreeMap<String, Vec<AgentConfigOption>>,
pub refreshed_at_ms: Option<f64>,
}
#[derive(Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct SavedCatalog {
/// An opaque cryptographic digest, never raw launch arguments or credentials.
fingerprint: String,
#[serde(flatten)]
catalog: CatalogData,
}
#[derive(PartialEq)]
struct LaunchContext {
spec: provider::ProviderSpec,
executable: PathBuf,
modified: Option<std::time::SystemTime>,
len: Option<u64>,
}
impl LaunchContext {
fn new(spec: provider::ProviderSpec, root: &std::path::Path) -> Self {
let resolved = spec.clone().resolve_for_launch(root);
let executable = dunce::canonicalize(&resolved.command).unwrap_or_else(|_| PathBuf::from(&resolved.command));
let metadata = std::fs::metadata(&executable).ok();
Self {
spec,
executable,
modified: metadata.as_ref().and_then(|m| m.modified().ok()),
len: metadata.as_ref().map(std::fs::Metadata::len),
}
}
fn fingerprint(&self) -> String {
let mut hash = blake3::Hasher::new();
fn field(hash: &mut blake3::Hasher, value: &[u8]) {
hash.update(&(value.len() as u64).to_le_bytes());
hash.update(value);
}
field(&mut hash, b"openidea-model-catalog-v1");
field(
&mut hash,
match self.spec.driver {
provider::Driver::Acp => b"acp",
provider::Driver::Codex => b"codex",
provider::Driver::Claude => b"claude",
},
);
field(&mut hash, self.spec.command.as_bytes());
hash.update(&(self.spec.args.len() as u64).to_le_bytes());
for arg in &self.spec.args {
field(&mut hash, arg.as_bytes());
}
hash.update(&(self.spec.env.len() as u64).to_le_bytes());
for (name, value) in &self.spec.env {
field(&mut hash, name.as_bytes());
field(&mut hash, value.as_bytes());
}
field(&mut hash, self.executable.to_string_lossy().as_bytes());
if let Some(modified) = self.modified.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) {
hash.update(&modified.as_nanos().to_le_bytes());
}
if let Some(len) = self.len {
hash.update(&len.to_le_bytes());
}
hash.finalize().to_hex().to_string()
}
}
pub(crate) struct Discovered {
pub config: Vec<AgentConfigOption>,
pub commands: Option<Vec<AgentCommand>>,
pub levels: BTreeMap<String, Vec<AgentConfigOption>>,
}
pub(crate) struct KillPeer(pub ide_jsonrpc::Peer);
impl Drop for KillPeer {
fn drop(&mut self) {
self.0.kill();
}
}
impl AgentCatalogs {
fn now(&self) -> Instant {
let now = Instant::now();
#[cfg(test)]
let now = now + Duration::from_millis(self.clock_ms.load(std::sync::atomic::Ordering::Relaxed));
now
}
fn epoch_ms(&self) -> f64 {
let now = crate::now_ms();
#[cfg(test)]
let now = now + self.clock_ms.load(std::sync::atomic::Ordering::Relaxed) as f64;
now
}
pub(crate) fn store(&self, path: &std::path::Path) -> Store {
self.state
.lock()
.stores
.entry(path.to_path_buf())
.or_insert_with(|| {
Arc::new(Mutex::new(
std::fs::read(path).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default(),
))
})
.clone()
}
pub(crate) fn subscribe(&self, shared: &Arc<Shared>) {
let mut state = self.state.lock();
state.subscribers.retain(|s| s.strong_count() > 0);
state.subscribers.push(Arc::downgrade(shared));
}
fn key(shared: &Shared, provider: &str) -> Key {
(shared.paths.models_file.clone(), shared.paths.project_root.clone(), provider.to_owned())
}
fn fallback(shared: &Shared, provider: &str) -> CatalogData {
let cache = shared.known.lock();
let Some(known) = cache.get(provider) else { return CatalogData::default() };
known.catalogs.get(&shared.root_key()).map(|saved| saved.catalog.clone()).unwrap_or_else(|| CatalogData {
config: known.config.clone(),
thought_levels: known.thought_levels.clone(),
refreshed_at_ms: None,
})
}
pub(crate) fn cached(&self, shared: &Shared, provider: &str) -> CatalogData {
self.state
.lock()
.entries
.get(&Self::key(shared, provider))
.and_then(|e| e.catalog.clone())
.unwrap_or_else(|| Self::fallback(shared, provider))
}
fn snapshot(&self, shared: &Shared, provider: &str, entry: &Entry) -> AgentModelCatalog {
let catalog = entry.catalog.clone().unwrap_or_else(|| Self::fallback(shared, provider));
AgentModelCatalog {
provider: provider.to_owned(),
config: shared.config_for_new_task(catalog.config, provider),
thought_levels: catalog.thought_levels,
refreshing: entry.flight.is_some(),
refreshed_at_ms: catalog.refreshed_at_ms,
retry_at_ms: entry
.retry_at_ms
.filter(|_| entry.completed.is_some_and(|at| self.now().duration_since(at) < COOLDOWN)),
error: entry.error.clone(),
}
}
/// Start or join a lookup. No queued refresh can outlive the request that asked for it.
pub(crate) fn request(
self: &Arc<Self>,
shared: &Arc<Shared>,
spec: provider::ProviderSpec,
force: bool,
) -> (AgentModelCatalog, Option<watch::Receiver<bool>>) {
let key = Self::key(shared, &spec.id);
let context = LaunchContext::new(spec.clone(), &shared.paths.project_root);
let now = self.now();
let mut state = self.state.lock();
let changed = state.entries.get(&key).is_none_or(|e| e.context != context);
if changed {
if let Some(abort) = state.entries.get_mut(&key).and_then(|entry| entry.abort.take()) {
abort.abort();
}
state.generation += 1;
let generation = state.generation;
let saved =
shared.known.lock().get(&spec.id).and_then(|known| known.catalogs.get(&shared.root_key())).cloned();
let matching = saved.as_ref().is_some_and(|saved| saved.fingerprint == context.fingerprint());
let age = saved
.as_ref()
.filter(|_| matching)
.and_then(|saved| saved.catalog.refreshed_at_ms)
.map(|at| self.epoch_ms() - at)
.filter(|age| *age >= 0.0 && *age < FRESH_FOR.as_millis() as f64)
.map(|age| Duration::from_secs_f64(age / 1000.0));
let successful = age.and_then(|age| now.checked_sub(age));
let completed = age.filter(|age| *age < COOLDOWN).and_then(|age| now.checked_sub(age));
let retry_at_ms = completed
.and_then(|_| saved.as_ref().and_then(|s| s.catalog.refreshed_at_ms))
.map(|at| at + COOLDOWN.as_millis() as f64);
let catalog = saved.map(|saved| {
let mut catalog = saved.catalog;
if !matching {
catalog.refreshed_at_ms = None;
}
catalog
});
state.entries.insert(
key.clone(),
Entry {
context,
generation,
completed,
successful,
retry_at_ms,
catalog,
error: None,
flight: None,
abort: None,
},
);
}
let entry = state.entries.get_mut(&key).expect("catalog entry");
let cooling = entry.completed.is_some_and(|at| now.duration_since(at) < COOLDOWN);
let fresh = entry.successful.is_some_and(|at| now.duration_since(at) < FRESH_FOR);
let mut start = None;
if entry.flight.is_none() && !cooling && (force || !fresh) {
let (done, ready) = watch::channel(false);
entry.flight = Some(ready);
entry.error = None;
entry.retry_at_ms = None;
start = Some((entry.generation, done));
}
let snapshot = self.snapshot(shared, &spec.id, entry);
let flight = entry.flight.clone();
drop(state);
if let Some((generation, done)) = start {
self.broadcast(shared, &snapshot);
let coordinator = self.clone();
let shared = shared.clone();
let flight_key = key.clone();
let task = shared.runtime.clone().spawn(async move {
let result = discover(&shared, spec).await;
coordinator.complete(&shared, flight_key, generation, result);
let _ = done.send(true);
});
// A subscriber can re-enter and change configuration during the start event.
let mut state = self.state.lock();
if let Some(entry) = state.entries.get_mut(&key).filter(|entry| entry.generation == generation) {
if entry.flight.is_some() {
entry.abort = Some(task.abort_handle());
}
} else {
task.abort();
}
}
(snapshot, flight)
}
fn complete(&self, shared: &Shared, key: Key, generation: u64, result: Result<Discovered, String>) {
let current = provider::load(shared.paths.custom_agents_dir.as_deref(), shared.env_secrets())
.into_iter()
.find(|spec| spec.id == key.2)
.map(|spec| LaunchContext::new(spec, &shared.paths.project_root));
let mut state = self.state.lock();
let Some(entry) = state.entries.get_mut(&key).filter(|e| e.generation == generation) else { return };
// Configuration may have changed without another refresh request arriving.
if current.as_ref() != Some(&entry.context) {
entry.flight = None;
entry.abort = None;
entry.completed = None;
entry.retry_at_ms = None;
entry.error = None;
let snapshot = self.snapshot(shared, &key.2, entry);
state.entries.remove(&key);
drop(state);
self.broadcast(shared, &snapshot);
return;
}
let now = self.now();
entry.completed = Some(now);
entry.retry_at_ms = Some(self.epoch_ms() + COOLDOWN.as_millis() as f64);
entry.flight = None;
entry.abort = None;
let success = match result {
Ok(found) => {
let mut cache = shared.known.lock();
let known = cache.entry(key.2.clone()).or_default();
let catalog = CatalogData {
config: found.config,
thought_levels: found.levels,
refreshed_at_ms: Some(self.epoch_ms()),
};
known.config = catalog.config.clone();
known.thought_levels = catalog.thought_levels.clone();
if let Some(commands) = found.commands {
known.commands.insert(shared.root_key(), commands);
}
known.authoritative = true;
known.refreshed_at_ms = catalog.refreshed_at_ms;
known.catalogs.insert(
shared.root_key(),
SavedCatalog { fingerprint: entry.context.fingerprint(), catalog: catalog.clone() },
);
entry.catalog = Some(catalog);
shared.persist_known(&cache);
entry.successful = Some(now);
entry.error = None;
true
}
Err(error) => {
entry.error = Some(error);
false
}
};
let snapshot = self.snapshot(shared, &key.2, entry);
drop(state);
self.broadcast(shared, &snapshot);
if success {
for subscriber in self.subscribers(shared) {
subscriber.emit(AgentEvent::Providers);
}
}
}
fn subscribers(&self, shared: &Shared) -> Vec<Arc<Shared>> {
self.state
.lock()
.subscribers
.iter()
.filter_map(Weak::upgrade)
.filter(|s| {
s.paths.models_file == shared.paths.models_file && s.paths.project_root == shared.paths.project_root
})
.collect()
}
fn broadcast(&self, shared: &Shared, catalog: &AgentModelCatalog) {
for subscriber in self.subscribers(shared) {
subscriber.emit(AgentEvent::ModelCatalog { catalog: catalog.clone() });
}
}
}
impl crate::AgentManager {
/// Return cached choices immediately and schedule discovery when eligible.
pub fn refresh_models(&self, provider: &str, force: bool) -> IdeResult<AgentModelCatalog> {
// Settings and environment secrets can change while this window is open.
*self.shared.providers.write() =
provider::load(self.shared.paths.custom_agents_dir.as_deref(), self.shared.env_secrets());
let spec = self.shared.spec(provider).ok_or_else(|| IdeError::invalid(format!("unknown agent {provider}")))?;
Ok(self.shared.paths.catalogs.request(&self.shared, spec, force).0)
}
}
async fn discover(shared: &Arc<Shared>, spec: provider::ProviderSpec) -> Result<Discovered, String> {
match spec.driver {
provider::Driver::Acp => acp::discover::run(shared, &spec).await,
provider::Driver::Codex => native::codex::discover(spec, shared.paths.project_root.clone()).await,
provider::Driver::Claude => {
let found = native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await?;
Ok(Discovered {
commands: Some(config::commands_of(
&serde_json::json!({ "availableCommands": found.commands }),
&spec.id,
)),
config: config::options_of(&serde_json::json!({ "configOptions": found.options })),
levels: found
.efforts
.into_iter()
.map(|(model, option)| {
let levels = option
.map(|o| config::options_of(&serde_json::json!({ "configOptions": [o] })))
.unwrap_or_default();
(model, levels)
})
.collect(),
})
}
}
}
#[cfg(test)]
mod tests;
+501
Переглянути файл
@@ -0,0 +1,501 @@
use super::*;
use crate::{AgentManager, AgentPaths, Connector};
use ide_jsonrpc::{Framing, Peer};
use serde_json::{Value, json};
use std::path::Path;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::sync::{mpsc, oneshot};
const PROVIDER: &str = "catalog-test";
#[derive(Debug)]
struct Pending {
params: Value,
reply: oneshot::Sender<Result<Value, String>>,
}
struct Harness {
connector: Connector,
pending: mpsc::UnboundedReceiver<Pending>,
launches: Arc<AtomicUsize>,
live: Arc<AtomicUsize>,
}
struct Live(Arc<AtomicUsize>);
impl Drop for Live {
fn drop(&mut self) {
self.0.fetch_sub(1, Ordering::SeqCst);
}
}
impl Harness {
fn new() -> Self {
let (pending_tx, pending) = mpsc::unbounded_channel();
let launches = Arc::new(AtomicUsize::new(0));
let live = Arc::new(AtomicUsize::new(0));
let connector: Connector = Arc::new({
let launches = launches.clone();
let live = live.clone();
move |_, _, client, _| {
launches.fetch_add(1, Ordering::SeqCst);
live.fetch_add(1, Ordering::SeqCst);
let guard = Live(live.clone());
let pending = pending_tx.clone();
let (client_io, server_io) = tokio::io::duplex(1 << 16);
let (read, mut write) = tokio::io::split(server_io);
tokio::spawn(async move {
let _guard = guard;
let mut lines = BufReader::new(read).lines();
while let Some(line) = lines.next_line().await.unwrap() {
let request: Value = serde_json::from_str(&line).unwrap();
let id = &request["id"];
if id.is_null() {
continue;
}
let response = match request["method"].as_str().unwrap() {
"initialize" => json!({ "id": id, "jsonrpc": "2.0", "result": {
"protocolVersion": 1, "agentCapabilities": {}, "authMethods": [] } }),
"session/new" => {
let (reply, result) = oneshot::channel();
pending.send(Pending { params: request["params"].clone(), reply }).unwrap();
let result = tokio::select! {
result = result => result.unwrap(),
_ = lines.next_line() => break,
};
match result {
Ok(result) => json!({ "id": id, "jsonrpc": "2.0", "result": result }),
Err(error) => {
json!({ "id": id, "jsonrpc": "2.0", "error": { "code": -32000, "message": error } })
}
}
}
method => panic!("discovery must not call {method}"),
};
if write.write_all(format!("{response}\n").as_bytes()).await.is_err() {
break;
}
}
});
let (read, write) = tokio::io::split(client_io);
Ok(Peer::connect_framed(read, write, client, Framing::Lines))
}
});
Self { connector, pending, launches, live }
}
async fn next(&mut self) -> Pending {
tokio::time::timeout(Duration::from_secs(5), self.pending.recv()).await.unwrap().unwrap()
}
}
fn configure(root: &Path, version: &str) {
std::fs::create_dir_all(root.join("agents")).unwrap();
let executable = root.join("catalog-test-bin");
if !executable.exists() {
std::fs::write(&executable, "first executable").unwrap();
}
std::fs::write(
root.join("agents/catalog-test.toml"),
format!(
"name = \"Catalog Test\"\ncommand = {}\n[env]\nTOKEN = \"secret-{version}\"\n",
serde_json::to_string(&executable.to_string_lossy()).unwrap()
),
)
.unwrap();
}
fn make_manager(root: &Path, catalogs: Arc<AgentCatalogs>, harness: &Harness) -> AgentManager {
make_manager_at(root, &root.join("models.json"), catalogs, harness)
}
fn make_manager_at(root: &Path, models_file: &Path, catalogs: Arc<AgentCatalogs>, harness: &Harness) -> AgentManager {
AgentManager::with_connector(
AgentPaths {
project_root: root.to_path_buf(),
store_dir: root.join("store"),
worktrees_dir: root.join("worktrees"),
models_file: models_file.to_path_buf(),
catalogs,
custom_agents_dir: Some(root.join("agents")),
home_dir: Some(root.to_path_buf()),
env_secrets: None,
},
tokio::runtime::Handle::current(),
harness.connector.clone(),
)
}
fn events(manager: &AgentManager) -> mpsc::UnboundedReceiver<AgentEvent> {
let (tx, rx) = mpsc::unbounded_channel();
manager.subscribe(Arc::new(move |event| {
let _ = tx.send(event);
}));
rx
}
async fn completed(events: &mut mpsc::UnboundedReceiver<AgentEvent>) -> AgentModelCatalog {
tokio::time::timeout(Duration::from_secs(5), async {
loop {
if let AgentEvent::ModelCatalog { catalog } = events.recv().await.unwrap()
&& !catalog.refreshing
{
return catalog;
}
}
})
.await
.unwrap()
}
fn response(models: &[&str], effort: &[&str]) -> Value {
json!({ "sessionId": "discovery-only", "configOptions": [
{ "id": "model", "type": "select", "category": "model", "currentValue": models.first(),
"options": models.iter().map(|m| json!({ "value": m, "name": m })).collect::<Vec<_>>() },
{ "id": "effort", "type": "select", "category": "thought_level", "currentValue": effort.first(),
"options": effort.iter().map(|e| json!({ "value": e })).collect::<Vec<_>>() }
] })
}
fn model_values(catalog: &AgentModelCatalog) -> Vec<&str> {
catalog
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
.collect()
}
async fn stopped(harness: &Harness) {
tokio::time::timeout(Duration::from_secs(5), async {
while harness.live.load(Ordering::SeqCst) != 0 {
tokio::task::yield_now().await;
}
})
.await
.unwrap();
}
fn advance(catalogs: &AgentCatalogs, seconds: u64) {
catalogs.clock_ms.fetch_add(seconds * 1000, Ordering::Relaxed);
}
#[tokio::test]
async fn successful_refresh_replaces_models_and_efforts_and_old_sessions_cannot_revert_it() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
let mut events = events(&manager);
let old = config::options_of(&response(&["old", "retained"], &["low", "high"]));
manager.shared.remember(PROVIDER, Some(&old), None, Some("old"));
let cached = manager.refresh_models(PROVIDER, false).unwrap();
assert!(cached.refreshing);
assert_eq!(model_values(&cached), ["old", "retained"]);
assert!(cached.refreshed_at_ms.is_none(), "session reports are not fresh discovery");
let pending = harness.next().await;
assert_eq!(pending.params["mcpServers"], json!([]));
pending.reply.send(Ok(response(&["retained", "new"], &["medium"]))).unwrap();
let fresh = completed(&mut events).await;
assert_eq!(model_values(&fresh), ["retained", "new"]);
assert_eq!(fresh.thought_levels.keys().map(String::as_str).collect::<Vec<_>>(), ["new", "retained"]);
assert_eq!(fresh.thought_levels["retained"][0].values[0].value, "medium");
assert!(fresh.refreshed_at_ms.is_some() && fresh.retry_at_ms.is_some());
manager.shared.remember(PROVIDER, Some(&old), None, Some("old"));
assert_eq!(manager.refresh_models(PROVIDER, false).unwrap().config, fresh.config);
let provider = manager.providers().into_iter().find(|p| p.id == PROVIDER).unwrap();
assert_eq!(provider.config, fresh.config);
let persisted = std::fs::read_to_string(tmp.path().join("models.json")).unwrap();
assert!(!persisted.contains("secret-one") && !persisted.contains("TOKEN"));
let restored = make_manager(tmp.path(), Arc::default(), &harness);
let restored_snapshot = restored.refresh_models(PROVIDER, false).unwrap();
assert!(!restored_snapshot.refreshing, "matching persisted launch context remains fresh");
assert_eq!(restored_snapshot.refreshed_at_ms, fresh.refreshed_at_ms);
let provider = restored.providers().into_iter().find(|p| p.id == PROVIDER).unwrap();
assert_eq!(provider.config, fresh.config);
restored.shared.remember(PROVIDER, Some(&old), None, Some("old"));
assert_eq!(restored.shared.known_config(PROVIDER), fresh.config);
assert_eq!(restored.shared.known.lock()[PROVIDER].refreshed_at_ms, fresh.refreshed_at_ms);
assert!(!tmp.path().join("store").exists(), "refresh never creates tasks");
}
#[tokio::test]
async fn legacy_cache_loads_without_claiming_authoritative_freshness() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let options = config::options_of(&response(&["legacy"], &["low"]));
std::fs::write(
tmp.path().join("models.json"),
serde_json::to_vec(&json!({
(PROVIDER): { "config": options, "thoughtLevels": {}, "projectCommands": {} }
}))
.unwrap(),
)
.unwrap();
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), Arc::default(), &harness);
let mut events = events(&manager);
let cached = manager.refresh_models(PROVIDER, false).unwrap();
assert_eq!(model_values(&cached), ["legacy"]);
assert!(cached.refreshing && cached.refreshed_at_ms.is_none());
harness.next().await.reply.send(Err("Sign in required".into())).unwrap();
let failed = completed(&mut events).await;
assert_eq!(failed.config, cached.config);
assert!(failed.error.as_deref().unwrap().contains("Sign in required"));
}
#[tokio::test]
async fn concurrent_windows_commands_and_manual_requests_share_one_flight_and_cooldown() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let first = Arc::new(make_manager(tmp.path(), catalogs.clone(), &harness));
let second = make_manager(tmp.path(), catalogs.clone(), &harness);
let mut first_events = events(&first);
let mut second_events = events(&second);
assert!(first.refresh_models(PROVIDER, false).unwrap().refreshing);
let pending = harness.next().await;
for _ in 0..10 {
assert!(second.refresh_models(PROVIDER, true).unwrap().refreshing);
assert!(first.refresh_models(PROVIDER, true).unwrap().refreshing);
}
let commands = tokio::spawn({
let first = first.clone();
async move { first.commands(PROVIDER, None).await }
});
tokio::task::yield_now().await;
assert_eq!(harness.launches.load(Ordering::SeqCst), 1);
pending.reply.send(Ok(response(&["one"], &["low"]))).unwrap();
let done = completed(&mut first_events).await;
assert_eq!(completed(&mut second_events).await, done);
commands.await.unwrap().unwrap();
for force in [false, true, false, true] {
assert!(!second.refresh_models(PROVIDER, force).unwrap().refreshing);
}
advance(&catalogs, 29);
assert!(!first.refresh_models(PROVIDER, true).unwrap().refreshing);
advance(&catalogs, 1);
assert!(!first.refresh_models(PROVIDER, false).unwrap().refreshing, "automatic refresh remains fresh");
assert!(first.refresh_models(PROVIDER, true).unwrap().refreshing);
harness.next().await.reply.send(Ok(response(&["two"], &["high"]))).unwrap();
let newer = completed(&mut first_events).await;
assert_eq!(model_values(&newer), ["two"]);
assert_eq!(completed(&mut second_events).await, newer);
advance(&catalogs, 15 * 60 - 1);
assert!(!second.refresh_models(PROVIDER, false).unwrap().refreshing);
advance(&catalogs, 1);
assert!(second.refresh_models(PROVIDER, false).unwrap().refreshing);
harness.next().await.reply.send(Ok(response(&["three"], &["low"]))).unwrap();
completed(&mut first_events).await;
assert_eq!(harness.launches.load(Ordering::SeqCst), 3);
}
#[tokio::test]
async fn failure_keeps_successful_choices_and_throttles_manual_and_auto_retries() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
let mut events = events(&manager);
manager.refresh_models(PROVIDER, true).unwrap();
harness.next().await.reply.send(Ok(response(&["cached"], &["low"]))).unwrap();
let success = completed(&mut events).await;
let disk = std::fs::read(tmp.path().join("models.json")).unwrap();
advance(&catalogs, 15 * 60);
manager.refresh_models(PROVIDER, false).unwrap();
harness.next().await.reply.send(Err("Authentication expired".into())).unwrap();
let failed = completed(&mut events).await;
assert_eq!(failed.config, success.config);
assert_eq!(failed.thought_levels, success.thought_levels);
assert_eq!(failed.refreshed_at_ms, success.refreshed_at_ms);
assert!(failed.error.as_deref().unwrap().contains("Authentication expired"));
assert_eq!(std::fs::read(tmp.path().join("models.json")).unwrap(), disk);
assert!(!manager.refresh_models(PROVIDER, false).unwrap().refreshing);
assert!(!manager.refresh_models(PROVIDER, true).unwrap().refreshing);
advance(&catalogs, 29);
assert!(!manager.refresh_models(PROVIDER, true).unwrap().refreshing);
advance(&catalogs, 1);
assert!(manager.refresh_models(PROVIDER, false).unwrap().refreshing);
harness.next().await.reply.send(Ok(response(&["recovered"], &["high"]))).unwrap();
let recovered = completed(&mut events).await;
assert_eq!(model_values(&recovered), ["recovered"]);
assert!(recovered.error.is_none());
}
#[tokio::test]
async fn timeout_cleans_up_and_enters_the_same_failure_cooldown() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
let mut events = events(&manager);
manager.refresh_models(PROVIDER, false).unwrap();
let pending = harness.next().await;
tokio::time::pause();
tokio::time::advance(Duration::from_secs(4)).await;
let failed = completed(&mut events).await;
assert!(failed.error.as_deref().unwrap().contains("time"));
assert!(failed.retry_at_ms.is_some());
assert!(!manager.refresh_models(PROVIDER, true).unwrap().refreshing);
assert!(!manager.refresh_models(PROVIDER, false).unwrap().refreshing);
tokio::time::resume();
stopped(&harness).await;
assert_eq!(harness.live.load(Ordering::SeqCst), 0, "timed-out connection must close");
assert!(pending.reply.is_closed());
advance(&catalogs, 30);
assert!(manager.refresh_models(PROVIDER, false).unwrap().refreshing);
harness.next().await.reply.send(Ok(response(&["retry"], &["low"]))).unwrap();
completed(&mut events).await;
}
#[tokio::test]
async fn changed_effective_config_supersedes_old_completion_without_queueing() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let first = make_manager(tmp.path(), catalogs.clone(), &harness);
let second = make_manager(tmp.path(), catalogs.clone(), &harness);
let mut events = events(&second);
first.refresh_models(PROVIDER, false).unwrap();
let obsolete = harness.next().await;
configure(tmp.path(), "two");
assert!(second.refresh_models(PROVIDER, false).unwrap().refreshing);
let latest = harness.next().await;
latest.reply.send(Ok(response(&["latest"], &["high"]))).unwrap();
let fresh = completed(&mut events).await;
let _ = obsolete.reply.send(Ok(response(&["obsolete"], &["low"])));
stopped(&harness).await;
let shown = first.refresh_models(PROVIDER, false).unwrap();
assert_eq!(shown.config, fresh.config);
assert_eq!(shown.refreshed_at_ms, fresh.refreshed_at_ms);
assert!(!shown.refreshing);
assert_eq!(harness.launches.load(Ordering::SeqCst), 2);
assert!(!std::fs::read_to_string(tmp.path().join("models.json")).unwrap().contains("obsolete"));
// A changed configuration also invalidates successful freshness/cooldown.
configure(tmp.path(), "three");
assert!(first.refresh_models(PROVIDER, false).unwrap().refreshing);
harness.next().await.reply.send(Ok(response(&["new-config"], &["medium"]))).unwrap();
completed(&mut events).await;
}
#[tokio::test]
async fn invalid_provider_is_a_request_error_and_does_not_launch() {
let tmp = tempfile::tempdir().unwrap();
let harness = Harness::new();
let manager = make_manager(tmp.path(), Arc::default(), &harness);
assert!(manager.refresh_models("missing-provider", true).is_err());
assert_eq!(harness.launches.load(Ordering::SeqCst), 0);
}
#[tokio::test]
async fn replacing_the_executable_at_the_same_path_invalidates_freshness_and_obsolete_responses() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "unchanged-environment");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), catalogs, &harness);
let mut events = events(&manager);
manager.refresh_models(PROVIDER, false).unwrap();
let obsolete = harness.next().await;
std::fs::write(tmp.path().join("catalog-test-bin"), "replacement executable with different size").unwrap();
manager.refresh_models(PROVIDER, false).unwrap();
let latest = harness.next().await;
latest.reply.send(Ok(response(&["updated"], &["high"]))).unwrap();
let fresh = completed(&mut events).await;
let _ = obsolete.reply.send(Ok(response(&["obsolete"], &["low"])));
stopped(&harness).await;
assert_eq!(manager.refresh_models(PROVIDER, false).unwrap().config, fresh.config);
std::fs::write(tmp.path().join("catalog-test-bin"), "another executable").unwrap();
assert!(manager.refresh_models(PROVIDER, false).unwrap().refreshing);
harness.next().await.reply.send(Ok(response(&["new-install"], &["medium"]))).unwrap();
assert_eq!(model_values(&completed(&mut events).await), ["new-install"]);
assert_eq!(harness.launches.load(Ordering::SeqCst), 3);
}
#[tokio::test]
async fn project_catalogs_and_persisted_freshness_do_not_leak_between_contexts() {
let tmp = tempfile::tempdir().unwrap();
let a = tmp.path().join("project-a");
let b = tmp.path().join("project-b");
configure(&a, "account-a");
configure(&b, "account-b");
let models_file = tmp.path().join("shared-models.json");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let first = make_manager_at(&a, &models_file, catalogs.clone(), &harness);
let second = make_manager_at(&b, &models_file, catalogs, &harness);
let mut first_events = events(&first);
let mut second_events = events(&second);
first.refresh_models(PROVIDER, false).unwrap();
harness.next().await.reply.send(Ok(response(&["account-a-model"], &["low"]))).unwrap();
let a_catalog = completed(&mut first_events).await;
second.refresh_models(PROVIDER, false).unwrap();
harness.next().await.reply.send(Ok(response(&["account-b-model"], &["high"]))).unwrap();
let b_catalog = completed(&mut second_events).await;
assert_eq!(first.refresh_models(PROVIDER, false).unwrap().config, a_catalog.config);
assert_eq!(second.refresh_models(PROVIDER, false).unwrap().config, b_catalog.config);
assert_eq!(first.providers().into_iter().find(|p| p.id == PROVIDER).unwrap().config, a_catalog.config);
for (root, expected) in [(&a, &a_catalog), (&b, &b_catalog)] {
let restored = make_manager_at(root, &models_file, Arc::default(), &harness);
let cached = restored.refresh_models(PROVIDER, false).unwrap();
assert!(!cached.refreshing, "matching persisted context should retain freshness");
assert_eq!(cached.config, expected.config);
assert_eq!(cached.refreshed_at_ms, expected.refreshed_at_ms);
}
assert_eq!(harness.launches.load(Ordering::SeqCst), 2);
configure(&a, "new-account");
let changed = make_manager_at(&a, &models_file, Arc::default(), &harness);
let mut events = events(&changed);
let stale = changed.refresh_models(PROVIDER, false).unwrap();
assert!(stale.refreshing, "a different launch environment cannot reuse persisted freshness");
assert!(stale.refreshed_at_ms.is_none());
assert_eq!(stale.config, a_catalog.config, "stale cached choices remain selectable");
harness.next().await.reply.send(Ok(response(&["new-account-model"], &["medium"]))).unwrap();
completed(&mut events).await;
let persisted = std::fs::read_to_string(models_file).unwrap();
assert!(!persisted.contains("secret-") && !persisted.contains("TOKEN"));
}
#[tokio::test]
async fn configuration_changed_without_another_request_also_discards_old_discovery() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "before");
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), Arc::default(), &harness);
let mut events = events(&manager);
manager.refresh_models(PROVIDER, false).unwrap();
let obsolete = harness.next().await;
configure(tmp.path(), "after");
obsolete.reply.send(Ok(response(&["obsolete"], &["low"]))).unwrap();
let discarded = completed(&mut events).await;
assert!(model_values(&discarded).is_empty());
assert!(discarded.refreshed_at_ms.is_none());
assert!(!tmp.path().join("models.json").exists());
manager.refresh_models(PROVIDER, false).unwrap();
harness.next().await.reply.send(Ok(response(&["current"], &["high"]))).unwrap();
assert_eq!(model_values(&completed(&mut events).await), ["current"]);
}
#[tokio::test]
async fn successful_empty_catalog_removes_all_previous_models_and_efforts() {
let tmp = tempfile::tempdir().unwrap();
configure(tmp.path(), "one");
let catalogs = Arc::new(AgentCatalogs::default());
let mut harness = Harness::new();
let manager = make_manager(tmp.path(), catalogs.clone(), &harness);
let mut events = events(&manager);
manager.refresh_models(PROVIDER, true).unwrap();
harness.next().await.reply.send(Ok(response(&["removed"], &["low"]))).unwrap();
completed(&mut events).await;
advance(&catalogs, 30);
manager.refresh_models(PROVIDER, true).unwrap();
harness.next().await.reply.send(Ok(response(&[], &[]))).unwrap();
let empty = completed(&mut events).await;
assert!(model_values(&empty).is_empty() && empty.thought_levels.is_empty());
assert!(empty.refreshed_at_ms.is_some() && empty.error.is_none());
}
+63 -149
Переглянути файл
@@ -8,6 +8,7 @@
//! next prompt when the agent supports `session/load`.
mod acp;
mod catalogs;
mod config;
mod conversation;
mod external;
@@ -21,6 +22,7 @@ mod skills;
mod timeline;
mod worktree;
pub use catalogs::AgentCatalogs;
pub use external::ExternalSession;
pub use model::*;
@@ -88,6 +90,8 @@ pub struct AgentPaths {
pub worktrees_dir: PathBuf,
/// Settings each agent offered last time (shared by all projects), and its commands per project.
pub models_file: PathBuf,
/// Shared by every project/window on this backend.
pub catalogs: Arc<AgentCatalogs>,
/// User-defined agents, one `<id>.toml` each (`name`, `command`, `args`, `env`).
pub custom_agents_dir: Option<PathBuf>,
/// The user's home: its `.claude` holds their own skills and commands.
@@ -123,12 +127,10 @@ struct Shared {
sink: RwLock<Option<AgentSink>>,
security: RwLock<Option<SecurityHook>>,
new_task_approvals: RwLock<Option<NewTaskApprovalsHook>>,
known: Mutex<HashMap<String, Known>>,
known: Arc<Mutex<HashMap<String, Known>>>,
providers: RwLock<Vec<provider::ProviderSpec>>,
/// Last probe of each provider.
probes: Mutex<HashMap<String, AgentProbe>>,
/// One pre-session lookup of commands and settings per provider, including failed attempts.
command_fetches: Mutex<HashMap<String, tokio::sync::watch::Receiver<bool>>>,
/// [`IDLE_RELEASE`], in milliseconds (shorter in tests).
idle_release_ms: AtomicU64,
mcp_servers_hook: RwLock<Option<Arc<McpServersFn>>>,
@@ -145,6 +147,14 @@ struct Known {
/// By project root: skills and custom commands differ per project.
#[serde(default, rename = "projectCommands")]
commands: HashMap<String, Vec<AgentCommand>>,
/// Successful discovery, unlike a session report, owns availability.
#[serde(default)]
authoritative: bool,
#[serde(default, rename = "refreshedAtMs")]
refreshed_at_ms: Option<f64>,
/// Authoritative availability and freshness are scoped to project and launch context.
#[serde(default, rename = "projectCatalogs")]
catalogs: HashMap<String, catalogs::SavedCatalog>,
}
impl Shared {
@@ -171,7 +181,11 @@ impl Shared {
}
fn known_config(&self, provider: &str) -> Vec<AgentConfigOption> {
let mut config = self.known.lock().get(provider).map(|k| k.config.clone()).unwrap_or_default();
let config = self.paths.catalogs.cached(self, provider).config;
self.config_for_new_task(config, provider)
}
fn config_for_new_task(&self, mut config: Vec<AgentConfigOption>, provider: &str) -> Vec<AgentConfigOption> {
// Model-specific options are offered separately, never as provider-wide defaults.
config.retain(|o| o.category.as_deref() != Some("thought_level"));
config::add_client_options(provider, &mut config);
@@ -197,80 +211,6 @@ impl Shared {
cache.get(provider).and_then(|k| k.commands.get(&self.root_key())).cloned().unwrap_or_default()
}
async fn fetch_settings(self: &Arc<Self>, spec: provider::ProviderSpec) {
let mut ready = {
let mut fetches = self.command_fetches.lock();
fetches
.entry(spec.id.clone())
.or_insert_with(|| {
let (done, ready) = tokio::sync::watch::channel(false);
let shared = self.clone();
// Keep the lookup alive if a composer unmounts while it is waiting.
self.runtime.spawn(async move {
tracing::info!("agent {}: settings discovery started", spec.id);
let discovered = if spec.driver == provider::Driver::Acp {
acp::discover::run(&shared, &spec).await.map(|_| None)
} else if spec.driver == provider::Driver::Claude {
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await.map(Some)
} else {
return;
};
if let Ok(Some(found)) = discovered {
let commands =
config::commands_of(&json!({ "availableCommands": found.commands }), &spec.id);
let options = config::options_of(&json!({ "configOptions": found.options }));
let levels: Vec<_> = found
.efforts
.into_iter()
.map(|(model, o)| {
let options = o.map(|o| config::options_of(&json!({ "configOptions": [o] })));
(model, options.unwrap_or_default())
})
.collect();
let _ = tokio::task::spawn_blocking(move || {
// Notifies subscribers itself when the discovery
// changed the cached config; unchanged results
// stay silent.
shared.remember_discovered(&spec.id, &options, &commands, &levels);
})
.await;
} else if let Err(e) = discovered {
tracing::info!("agent {}: settings discovery failed: {e}", spec.id);
}
let _ = done.send(true);
});
ready
})
.clone()
};
while !*ready.borrow_and_update() {
if ready.changed().await.is_err() {
break;
}
}
}
fn has_config(&self, provider: &str) -> bool {
self.known.lock().get(provider).is_some_and(|k| !k.config.is_empty())
}
/// Whether the thinking levels of every model the agent offers are known (#1686).
fn knows_all_levels(&self, provider: &str) -> bool {
let cache = self.known.lock();
let Some(known) = cache.get(provider) else { return false };
let models: Vec<&str> = known
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
.collect();
// No advertised models yet: nothing to attach effort levels to.
if models.is_empty() {
return false;
}
models.iter().all(|id| known.thought_levels.contains_key(*id))
}
fn root_key(&self) -> String {
self.paths.project_root.to_string_lossy().into_owned()
}
@@ -283,78 +223,48 @@ impl Shared {
commands: Option<&[AgentCommand]>,
model: Option<&str>,
) {
self.remember_inner(provider, config, commands, model, &[], false);
}
/// `levels`: each model's thinking-level options, so a new task offers them for any model
/// it picks (#1686). A session's report for a model wins over these.
fn remember_discovered(
&self,
provider: &str,
config: &[AgentConfigOption],
commands: &[AgentCommand],
levels: &[(String, Vec<AgentConfigOption>)],
) {
self.remember_inner(provider, Some(config), Some(commands), model_of(config).as_deref(), levels, true);
}
fn remember_inner(
&self,
provider: &str,
config: Option<&[AgentConfigOption]>,
commands: Option<&[AgentCommand]>,
model: Option<&str>,
discovered_levels: &[(String, Vec<AgentConfigOption>)],
discovered: bool,
) {
// Keep the update and its disk write ordered; a session report wins over discovery.
let mut cache = self.known.lock();
let known = cache.entry(provider.to_owned()).or_default();
let mut changed = false;
for (model, levels) in discovered_levels {
if !known.thought_levels.contains_key(model) {
known.thought_levels.insert(model.clone(), levels.clone());
changed = true;
}
}
if let Some(c) = config.filter(|_| !discovered || known.config.is_empty()) {
// Running tasks retain their own reported settings. Once discovery has
// succeeded, a session's older list/defaults cannot change New Task choices.
if !known.authoritative
&& let Some(config) = config.filter(|c| !c.is_empty())
{
if let Some(model) = model {
let levels: Vec<_> =
c.iter().filter(|o| o.category.as_deref() == Some("thought_level")).cloned().collect();
config.iter().filter(|o| o.category.as_deref() == Some("thought_level")).cloned().collect();
if known.thought_levels.get(model) != Some(&levels) {
known.thought_levels.insert(model.to_owned(), levels);
changed = true;
}
}
if !c.is_empty() && known.config != c {
known.config = c.to_vec();
if known.config != config {
known.config = config.to_vec();
changed = true;
}
}
let root = self.root_key();
if let Some(c) = commands.filter(|c| {
!c.is_empty()
&& (!discovered || known.commands.get(&root).is_none_or(Vec::is_empty))
&& known.commands.get(&root).map(Vec::as_slice) != Some(c)
}) {
known.commands.insert(root, c.to_vec());
if let Some(commands) = commands.filter(|c| !c.is_empty())
&& known.commands.get(&self.root_key()).map(Vec::as_slice) != Some(commands)
{
known.commands.insert(self.root_key(), commands.to_vec());
changed = true;
}
if !changed {
return;
if changed {
self.persist_known(&cache);
}
if let Ok(bytes) = serde_json::to_vec_pretty(&*cache)
drop(cache);
if changed {
self.emit(AgentEvent::Providers);
}
}
fn persist_known(&self, cache: &HashMap<String, Known>) {
if let Ok(bytes) = serde_json::to_vec_pretty(cache)
&& let Err(e) = write_file(&self.paths.models_file, &bytes)
{
tracing::warn!("saving agent settings: {e}");
}
// The cached provider config actually changed (a session report, discovery,
// or a turn update): tell subscribers to re-list, so a changed model list
// reaches the composer without waiting for the next task (#1490). Unchanged
// reports stay silent, so routine session traffic emits nothing. The cache
// lock is released first; the sink never runs under it.
drop(cache);
self.emit(AgentEvent::Providers);
}
}
@@ -469,8 +379,7 @@ impl AgentManager {
}
fn with_connector(paths: AgentPaths, runtime: tokio::runtime::Handle, connector: Connector) -> Self {
let known =
std::fs::read(&paths.models_file).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default();
let known = paths.catalogs.store(&paths.models_file);
let providers = RwLock::new(provider::load(paths.custom_agents_dir.as_deref(), paths.env_secrets.as_deref()));
let shared = Arc::new(Shared {
paths,
@@ -479,13 +388,13 @@ impl AgentManager {
sink: RwLock::new(None),
security: RwLock::new(None),
new_task_approvals: RwLock::new(None),
known: Mutex::new(known),
known,
providers,
probes: Mutex::new(HashMap::new()),
command_fetches: Mutex::new(HashMap::new()),
idle_release_ms: AtomicU64::new(IDLE_RELEASE.as_millis() as u64),
mcp_servers_hook: RwLock::new(None),
});
shared.paths.catalogs.subscribe(&shared);
let mut tasks = HashMap::new();
for file in std::fs::read_dir(&shared.paths.store_dir).into_iter().flatten().flatten() {
let path = file.path();
@@ -542,9 +451,9 @@ impl AgentManager {
.filter(|p| !provider::is_unlisted(p) || in_use.contains(&p.id))
.map(|p| {
let mut info = provider::describe(p);
info.config = self.shared.known_config(&p.id);
info.thought_levels =
self.shared.known.lock().get(&p.id).map(|k| k.thought_levels.clone()).unwrap_or_default();
let catalog = self.shared.paths.catalogs.cached(&self.shared, &p.id);
info.config = self.shared.config_for_new_task(catalog.config, &p.id);
info.thought_levels = catalog.thought_levels;
info.commands = self.shared.known_commands(&p.id);
info
})
@@ -632,17 +541,20 @@ impl AgentManager {
// merged without duplicating what `availableCommands` or the disk offer.
let mut offered = offered;
if task.is_none()
&& (offered.is_empty() || !self.shared.has_config(&provider) || !self.shared.knows_all_levels(&provider))
&& let Some(spec) = self
.shared
.spec(&provider)
.filter(|s| matches!(s.driver, provider::Driver::Claude | provider::Driver::Acp))
&& let Some(spec) =
provider::load(self.shared.paths.custom_agents_dir.as_deref(), self.shared.env_secrets())
.into_iter()
.find(|spec| spec.id == provider)
{
let claude = spec.driver == provider::Driver::Claude;
self.shared.fetch_settings(spec).await;
if claude {
offered = self.shared.known_commands(&provider);
let (_, flight) = self.shared.paths.catalogs.request(&self.shared, spec, false);
if let Some(mut ready) = flight {
while !*ready.borrow_and_update() {
if ready.changed().await.is_err() {
break;
}
}
}
offered = self.shared.known_commands(&provider);
}
for &(name, description) in builtin_commands(&provider) {
if !offered.iter().any(|c| c.name == name) {
@@ -794,16 +706,18 @@ impl AgentManager {
}
};
// Until the session reports its own, show what the agent offered last time with the choices applied.
let mut config = self.shared.known_config(&spec.id);
let catalog = self.shared.paths.catalogs.cached(&self.shared, &spec.id);
let mut levels = catalog.thought_levels;
let mut config = self.shared.config_for_new_task(catalog.config, &spec.id);
for choice in &req.config {
if let Some(o) = config.iter_mut().find(|o| o.id == choice.id) {
o.current = Some(choice.value.clone());
}
}
if let Some(model) = model_of(&config)
&& let Some(levels) = self.shared.known.lock().get(&spec.id).and_then(|k| k.thought_levels.get(&model))
&& let Some(levels) = levels.remove(&model)
{
config.extend(levels.iter().cloned().map(|mut o| {
config.extend(levels.into_iter().map(|mut o| {
if let Some(choice) =
req.config.iter().find(|c| c.id == o.id && o.values.iter().any(|v| v.value == c.value))
{
+17
Переглянути файл
@@ -538,6 +538,19 @@ pub struct AgentChanges {
pub commits: Vec<AgentCommit>,
}
/// Authoritative choices for new tasks, independent of any running session's settings.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentModelCatalog {
pub provider: String,
pub config: Vec<AgentConfigOption>,
pub thought_levels: BTreeMap<String, Vec<AgentConfigOption>>,
pub refreshing: bool,
pub refreshed_at_ms: Option<f64>,
pub retry_at_ms: Option<f64>,
pub error: Option<String>,
}
/// Streamed to the window that subscribed (`AgentManager::subscribe`).
#[derive(Clone, Debug, Serialize, Type)]
#[serde(tag = "type", rename_all = "camelCase", rename_all_fields = "camelCase")]
@@ -575,6 +588,10 @@ pub enum AgentEvent {
provider: String,
limits: Vec<AgentLimit>,
},
/// A model lookup started or completed, including unsuccessful lookups.
ModelCatalog {
catalog: AgentModelCatalog,
},
/// The installed or configured agent providers changed.
Providers,
}
+2
Переглянути файл
@@ -85,6 +85,8 @@ pub(crate) async fn discover(spec: ProviderSpec, cwd: PathBuf) -> Result<Discove
let (spec, cwd) = spec.await.map_err(|e| e.to_string())?;
let mut args = spec.args;
args.extend(Start::New { id: uuid::Uuid::new_v4().to_string() }.args());
// Metadata discovery must not launch project-configured MCP servers.
args.extend(["--strict-mcp-config".into(), "--mcp-config".into(), "{\"mcpServers\":{}}".into()]);
let launch = Launch { program: spec.command, args, cwd, env: spec.env };
let mut io = cli::Process.launch(&launch, Arc::new(|_| {})).map_err(|e| e.to_string())?;
let _stop = io.kill.clone().drop_guard();
+98 -37
Переглянути файл
@@ -44,6 +44,92 @@ pub(crate) fn connect(
})
}
/// A model lookup uses app-server directly: no thread, task, prompt or project MCP.
pub(crate) async fn discover(
spec: ProviderSpec,
cwd: std::path::PathBuf,
) -> Result<crate::catalogs::Discovered, String> {
let spec = tokio::task::spawn_blocking({
let cwd = cwd.clone();
move || spec.resolve_for_launch(&cwd)
})
.await
.map_err(|e| e.to_string())?;
let peer = Peer::spawn_framed(
&spec.command,
&spec.args,
&cwd,
&spec.env,
Arc::new(ide_jsonrpc::NoopHandler),
Box::new(|_| {}),
Framing::Lines,
)
.map_err(|e| e.to_string())?;
let _stop = crate::catalogs::KillPeer(peer.clone());
tokio::time::timeout(Duration::from_secs(30), async {
let _: Value = peer.request("initialize", initialize_params(), None).await.map_err(|e| e.to_string())?;
peer.notify("initialized", Value::Null).map_err(|e| e.to_string())?;
let mut models = list_models(&peer).await.map_err(|e| e.message)?;
models.retain(|model| !model.hidden);
let mut settings = Settings::of(&json!({}), &models);
if settings.model.is_empty()
&& let Some(model) = models.first()
{
settings.model = model.id.clone();
settings.fit_effort(&models);
}
let options = translate::config_options(&models, &settings);
let config = crate::config::options_of(&json!({ "configOptions": options }));
let levels = models
.iter()
.map(|model| {
let mut settings = settings.clone();
settings.model = model.id.clone();
settings.fit_effort(&models);
let options = translate::config_options(&models, &settings);
let levels = crate::config::options_of(&json!({ "configOptions": options }))
.into_iter()
.filter(|o| o.category.as_deref() == Some("thought_level"))
.collect();
(model.id.clone(), levels)
})
.collect();
Ok(crate::catalogs::Discovered { config, commands: None, levels })
})
.await
.map_err(|_| "Codex model discovery timed out".to_owned())?
}
fn initialize_params() -> Value {
json!({
"clientInfo": { "name": "openidea", "title": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") },
"capabilities": { "experimentalApi": true, "optOutNotificationMethods": OPT_OUT },
})
}
async fn list_models(peer: &Peer) -> Result<Vec<ModelInfo>, ResponseError> {
let mut all = Vec::new();
let mut cursor = Value::Null;
let mut seen = HashSet::new();
loop {
let page: Value = peer.request("model/list", json!({ "cursor": cursor }), None).await.map_err(from_codex)?;
let Some(data) = page.get("data").and_then(Value::as_array) else {
return Err(error(codes::INTERNAL_ERROR, "Codex returned an invalid model list"));
};
let models = translate::models_of(&page);
if models.len() != data.len() {
return Err(error(codes::INTERNAL_ERROR, "Codex returned an invalid model list"));
}
all.extend(models);
match page.get("nextCursor") {
None | Some(Value::Null) => return Ok(all),
Some(Value::String(next)) if next.is_empty() => return Ok(all),
Some(Value::String(next)) if seen.insert(next.clone()) => cursor = json!(next),
_ => return Err(error(codes::INTERNAL_ERROR, "Codex returned an invalid model-list cursor")),
}
}
}
/// Connects `client` to a shim driving the codex connection `connect_codex` opens (it
/// gets the handler for what codex sends). Each side's end closes the other.
fn attach(
@@ -250,15 +336,7 @@ impl CodexShim {
}
async fn initialize(&self) -> Result<Value, ResponseError> {
let info = self
.call(
"initialize",
json!({
"clientInfo": { "name": "openidea", "title": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") },
"capabilities": { "experimentalApi": true, "optOutNotificationMethods": OPT_OUT },
}),
)
.await?;
let info = self.call("initialize", initialize_params()).await?;
self.codex().notify("initialized", Value::Null).map_err(from_codex)?;
self.state.lock().initialized = true;
// `<client>/<codex version> (<platform>) …`
@@ -282,32 +360,15 @@ impl CodexShim {
}))
}
/// The models codex offers; empty when it can't list them.
async fn models(&self) -> Arc<Vec<ModelInfo>> {
/// Cache only complete, successful listings; a failed page must never become availability.
async fn models(&self) -> Result<Arc<Vec<ModelInfo>>, ResponseError> {
let mut cached = self.models.lock().await;
if let Some(m) = cached.as_ref() {
return m.clone();
if let Some(models) = cached.as_ref() {
return Ok(models.clone());
}
let mut all = Vec::new();
let mut cursor = Value::Null;
loop {
match self.call("model/list", json!({ "cursor": cursor })).await {
Ok(page) => {
all.extend(translate::models_of(&page));
match page.get("nextCursor").filter(|c| c.is_string()) {
Some(next) if *next != cursor => cursor = next.clone(),
_ => break,
}
}
Err(e) => {
tracing::warn!("codex: listing models failed: {}", e.message);
return Arc::new(all);
}
}
}
let models = Arc::new(all);
let models = Arc::new(list_models(self.codex()).await?);
*cached = Some(models.clone());
models
Ok(models)
}
async fn publish_skills(&self, session: &str) {
@@ -341,7 +402,7 @@ impl CodexShim {
}
async fn new_session(&self, params: &Value) -> Result<Value, ResponseError> {
let models = self.models().await;
let models = self.models().await?;
let mut request = json!({ "cwd": params.get("cwd") });
if let Some(config) = translate::mcp_config(params) {
request["config"] = config;
@@ -358,7 +419,7 @@ impl CodexShim {
/// Resumes the thread and replays its history before answering.
async fn load_session(&self, params: &Value) -> Result<Value, ResponseError> {
let id = str_param(params, "sessionId")?;
let models = self.models().await;
let models = self.models().await?;
let mut request = json!({ "threadId": id, "cwd": params.get("cwd") });
if let Some(config) = translate::mcp_config(params) {
request["config"] = config;
@@ -376,7 +437,7 @@ impl CodexShim {
async fn fork_session(&self, params: &Value) -> Result<Value, ResponseError> {
let source = str_param(params, "sessionId")?;
let models = self.models().await;
let models = self.models().await?;
let settings = self.state.lock().threads.get(source).and_then(|t| t.settings.clone());
let mut request = json!({ "threadId": source, "cwd": params.get("cwd") });
if let Some(config) = translate::mcp_config(params) {
@@ -397,7 +458,7 @@ impl CodexShim {
let session = str_param(params, "sessionId")?;
let option = str_param(params, "configId")?;
let value = str_param(params, "value")?;
let models = self.models().await;
let models = self.models().await?;
let mut state = self.state.lock();
let settings = state
.threads
@@ -414,7 +475,7 @@ impl CodexShim {
if input.is_empty() {
return Err(invalid("empty prompt"));
}
let models = self.models().await;
let models = self.models().await?;
let (done, ended) = oneshot::channel();
let settings = {
let mut state = self.state.lock();
+149
Переглянути файл
@@ -11,6 +11,155 @@ use std::collections::VecDeque;
use std::path::PathBuf;
use tokio::io::{AsyncBufReadExt, AsyncRead, AsyncWrite, AsyncWriteExt, BufReader};
struct ModelPages {
pages: Mutex<VecDeque<Result<Value, (i64, String)>>>,
cursors: Mutex<Vec<Value>>,
}
impl Handler for ModelPages {
fn on_notification(&self, _: &str, _: Value) {}
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
assert_eq!(method, "model/list", "listing models must never start a thread or turn");
self.cursors.lock().push(params["cursor"].clone());
self.pages.lock().pop_front().expect("another expected page")
}
}
fn model_peer(pages: Arc<ModelPages>) -> (Peer, Peer) {
let (ours, theirs) = tokio::io::duplex(1 << 16);
let (read, write) = tokio::io::split(theirs);
let server = Peer::connect_framed(read, write, pages, Framing::Lines);
let (read, write) = tokio::io::split(ours);
(Peer::connect_framed(read, write, Arc::new(ide_jsonrpc::NoopHandler), Framing::Lines), server)
}
#[tokio::test]
async fn model_pagination_failure_does_not_cache_an_incomplete_catalog() {
let first = json!({ "data": [{ "id": "first" }], "nextCursor": "next" });
let pages = Arc::new(ModelPages {
pages: Mutex::new(VecDeque::from([
Ok(first.clone()),
Err((-32000, "authentication expired".into())),
Ok(first),
Ok(json!({ "data": [{ "id": "second" }], "nextCursor": null })),
])),
cursors: Mutex::default(),
});
let (peer, server) = model_peer(pages.clone());
let shim = CodexShim {
runtime: tokio::runtime::Handle::current(),
codex: OnceLock::from(peer.clone()),
client: OnceLock::new(),
state: Mutex::default(),
models: tokio::sync::Mutex::new(None),
skills: tokio::sync::Mutex::new(()),
};
assert_eq!(shim.models().await.unwrap_err().message, "authentication expired");
let models = shim.models().await.unwrap();
assert_eq!(models.iter().map(|m| m.id.as_str()).collect::<Vec<_>>(), ["first", "second"]);
assert_eq!(shim.models().await.unwrap().len(), 2);
assert_eq!(*pages.cursors.lock(), [Value::Null, json!("next"), Value::Null, json!("next")]);
peer.kill();
server.kill();
}
#[tokio::test]
async fn repeated_or_invalid_model_cursors_and_malformed_pages_are_failures() {
for pages in [
vec![json!({ "data": [], "nextCursor": "a" }), json!({ "data": [], "nextCursor": "a" })],
vec![
json!({ "data": [], "nextCursor": "a" }),
json!({ "data": [], "nextCursor": "b" }),
json!({ "data": [], "nextCursor": "a" }),
],
vec![json!({ "data": [], "nextCursor": 7 })],
vec![json!({ "nextCursor": null })],
vec![json!({ "data": [{}], "nextCursor": null })],
] {
let pages =
Arc::new(ModelPages { pages: Mutex::new(pages.into_iter().map(Ok).collect()), cursors: Mutex::default() });
let (peer, server) = model_peer(pages);
assert!(list_models(&peer).await.is_err());
peer.kill();
server.kill();
}
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn native_discovery_only_initializes_and_pages_models_then_stops_the_process() {
use std::os::unix::fs::PermissionsExt;
let tmp = tempfile::tempdir().unwrap();
let script = tmp.path().join("fake-codex");
std::fs::write(
&script,
r#"#!/usr/bin/env python3
import json, os, pathlib, sys
pathlib.Path('pid').write_text(str(os.getpid()))
for line in sys.stdin:
request = json.loads(line)
with pathlib.Path('requests').open('a') as f:
f.write(json.dumps(request) + '\n')
method = request['method']
if method == 'initialized':
continue
if method == 'initialize':
result = {}
elif method == 'model/list':
cursor = request['params']['cursor']
model = 'one' if cursor is None else 'two'
result = {'data': [{'id': model, 'isDefault': cursor is None,
'defaultReasoningEffort': 'low', 'supportedReasoningEfforts': [{'reasoningEffort': 'low'}]}],
'nextCursor': 'second-page' if cursor is None else None}
else:
raise RuntimeError('must not create a thread or prompt: ' + method)
print(json.dumps({'jsonrpc': '2.0', 'id': request['id'], 'result': result}), flush=True)
"#,
)
.unwrap();
std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap();
let spec = ProviderSpec {
id: "codex".into(),
name: "Codex".into(),
driver: crate::provider::Driver::Codex,
command: script.to_string_lossy().into_owned(),
args: Vec::new(),
env: Vec::new(),
login_hint: String::new(),
source: crate::AgentSource::Builtin,
};
let found = discover(spec, tmp.path().to_path_buf()).await.unwrap();
let models: Vec<_> = found
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
.collect();
assert_eq!(models, ["one", "two"]);
assert_eq!(found.levels.keys().map(String::as_str).collect::<Vec<_>>(), ["one", "two"]);
let requests = std::fs::read_to_string(tmp.path().join("requests")).unwrap();
let requests: Vec<Value> = requests.lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert_eq!(
requests.iter().map(|r| r["method"].as_str().unwrap()).collect::<Vec<_>>(),
["initialize", "initialized", "model/list", "model/list"]
);
assert_eq!(requests[2]["params"]["cursor"], Value::Null);
assert_eq!(requests[3]["params"]["cursor"], "second-page");
let pid = std::fs::read_to_string(tmp.path().join("pid")).unwrap();
tokio::time::timeout(Duration::from_secs(5), async {
loop {
let alive = std::process::Command::new("kill").args(["-0", pid.trim()]).output().unwrap().status.success();
if !alive {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.expect("discovery must kill and reap its child");
}
// ---- Pure translations -------------------------------------------------------------
fn models() -> Vec<ModelInfo> {
+91 -1
Переглянути файл
@@ -120,10 +120,18 @@ impl ProviderSpec {
}
/// Resolve once before connecting, so authentication guidance names that same install.
pub(crate) fn resolve_for_launch(mut self, cwd: &Path) -> Self {
pub(crate) fn resolve_for_launch(self, cwd: &Path) -> Self {
let strip = prefer_stored_copilot_auth();
self.resolve_for_launch_with_copilot_env(cwd, strip)
}
pub(crate) fn resolve_for_launch_with_copilot_env(mut self, cwd: &Path, strip_ambient_tokens: bool) -> Self {
if self.driver != Driver::Acp {
self.command = crate::native::newest_install(&self.command);
}
if is_copilot_provider(&self.id) {
self.env = sanitized_copilot_env(&self.env, strip_ambient_tokens);
}
let program = ide_jsonrpc::resolve_spawn_program(&self.command, &self.env);
self.command =
if program.components().count() > 1 && !program.is_absolute() { cwd.join(program) } else { program }
@@ -133,6 +141,69 @@ impl ProviderSpec {
}
}
const AMBIENT_COPILOT_AUTH_VARS: [&str; 3] = ["COPILOT_GITHUB_TOKEN", "GH_TOKEN", "GITHUB_TOKEN"];
static PREFER_STORED_COPILOT_AUTH: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
pub(crate) fn is_copilot_provider(id: &str) -> bool {
id == "copilot" || id.starts_with("copilot-")
}
pub(crate) fn prefer_stored_copilot_auth() -> bool {
PREFER_STORED_COPILOT_AUTH.load(std::sync::atomic::Ordering::Acquire)
}
pub(crate) fn mark_prefer_stored_copilot_auth() {
PREFER_STORED_COPILOT_AUTH.store(true, std::sync::atomic::Ordering::Release);
}
pub(crate) fn has_unconfigured_copilot_ambient_token(spec_env: &[(String, String)]) -> bool {
AMBIENT_COPILOT_AUTH_VARS.iter().any(|&key| {
!spec_env.iter().any(|(k, v)| k == key && !v.is_empty())
&& ambient_copilot_env_value(key).is_some_and(|v| !v.is_empty())
})
}
fn ambient_copilot_env_value(key: &str) -> Option<String> {
ide_core::process::shell_env()
.find(|(k, _)| *k == key)
.map(|(_, v)| v.to_string_lossy().into_owned())
.or_else(|| ide_core::process::host_var_os(key).map(|v| v.to_string_lossy().into_owned()))
}
/// Builds child environment overrides for `copilot`:
/// - Always clears ambient classic PATs (`ghp_…`), which Copilot CLI unconditionally rejects
/// (`"Classic PATs are not supported"`) instead of falling back to Keychain / `gh` login.
/// - When `strip_ambient_tokens` is true, clears unconfigured ambient `GITHUB_TOKEN` / `GH_TOKEN` /
/// `COPILOT_GITHUB_TOKEN` so Copilot uses its stored login (`~/.copilot` or `gh auth` keyring),
/// while preserving any token explicitly configured in OpenIDEA's agent `spec_env`.
pub(crate) fn sanitized_copilot_env(
spec_env: &[(String, String)],
strip_ambient_tokens: bool,
) -> Vec<(String, String)> {
sanitize_copilot_env_with(spec_env, strip_ambient_tokens, ambient_copilot_env_value)
}
pub(crate) fn sanitize_copilot_env_with(
spec_env: &[(String, String)],
strip_ambient_tokens: bool,
lookup: impl Fn(&str) -> Option<String>,
) -> Vec<(String, String)> {
let mut out = Vec::with_capacity(spec_env.len() + AMBIENT_COPILOT_AUTH_VARS.len());
for key in AMBIENT_COPILOT_AUTH_VARS {
let explicit = spec_env.iter().any(|(k, v)| k == key && !v.is_empty());
if explicit {
continue;
}
if let Some(val) = lookup(key).filter(|v| !v.is_empty())
&& (strip_ambient_tokens || val.trim().starts_with("ghp_"))
{
out.push((key.to_owned(), String::new()));
}
}
out.extend(spec_env.iter().cloned());
out
}
fn render_login_hint(spec: &ProviderSpec, program: &Path, on_path: Option<&Path>, windows: bool) -> String {
let Some(builtin) = builtin_spec(&spec.id) else { return spec.login_hint.clone() };
let prefix = format!("Run `{}", builtin.command);
@@ -1101,4 +1172,23 @@ mod tests {
import_acp(dir.path(), None, "not json").unwrap_err();
assert!(import_acp(dir.path(), None, "{}").unwrap().is_empty());
}
#[test]
fn sanitizes_classic_pats_and_ambient_tokens_while_preserving_explicit_agent_env() {
let lookup = |k: &str| match k {
"GITHUB_TOKEN" => Some("ghp_classic_pat_123".into()),
"GH_TOKEN" => Some("gho_oauth_token_456".into()),
_ => None,
};
assert_eq!(sanitize_copilot_env_with(&[], false, lookup), vec![("GITHUB_TOKEN".into(), String::new())]);
assert_eq!(
sanitize_copilot_env_with(&[], true, lookup),
vec![("GH_TOKEN".into(), String::new()), ("GITHUB_TOKEN".into(), String::new())]
);
let explicit = vec![("GH_TOKEN".into(), "github_pat_custom".into())];
assert_eq!(
sanitize_copilot_env_with(&explicit, true, lookup),
vec![("GITHUB_TOKEN".into(), String::new()), ("GH_TOKEN".into(), "github_pat_custom".into())]
);
}
}
+1
Переглянути файл
@@ -25,6 +25,7 @@ fn paths(dir: &Path) -> AgentPaths {
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
catalogs: Arc::default(),
}
}
+18 -62
Переглянути файл
@@ -405,6 +405,7 @@ fn manager_at(dir: &Path, models_file: &Path, agent: Arc<FakeAgent>) -> AgentMan
// user's profile, when temp lives under it) leaks into the test.
home_dir: Some(dir.to_path_buf()),
env_secrets: None,
catalogs: Arc::default(),
},
tokio::runtime::Handle::current(),
connector,
@@ -442,7 +443,7 @@ async fn saved_conversation_reads_do_not_start_or_load_the_agent() {
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn claude_commands_before_session_are_fetched_once_and_saved() {
async fn claude_commands_share_catalog_discovery_and_persist_results() {
use std::os::unix::fs::PermissionsExt;
for outcome in ["success", "session_report", "empty", "settings_timeout", "error", "exit", "timeout", "missing"] {
@@ -502,14 +503,18 @@ for line in sys.stdin:
std::fs::write(root.join(".claude/skills/triage/SKILL.md"), "---\nname: triage\ndescription: Triage\n---\n")
.unwrap();
let mgr = manager(root, Arc::new(FakeAgent::default()));
{
let mut providers = mgr.shared.providers.write();
let spec = providers.iter_mut().find(|p| p.id == "claude").unwrap();
if outcome != "missing" {
spec.command = script.to_string_lossy().into_owned();
}
spec.env = vec![("OUTCOME".into(), outcome.into())];
if outcome != "missing" {
provider::set_builtin_command(&root.join("agents"), None, "claude", Some(&script.to_string_lossy()))
.unwrap();
}
provider::set_builtin_env(
&root.join("agents"),
None,
"claude",
BTreeMap::from([("OUTCOME".into(), outcome.into())]),
)
.unwrap();
mgr.reload_providers();
let concurrent = async {
if outcome == "session_report" {
tokio::time::timeout(Duration::from_secs(10), async {
@@ -553,7 +558,6 @@ for line in sys.stdin:
effort.unwrap().current.as_deref(),
Some(match outcome {
"settings_timeout" => "default",
"session_report" => "low",
_ => "high",
})
);
@@ -571,6 +575,8 @@ for line in sys.stdin:
assert_eq!(launches.lines().count(), 1, "{outcome}");
let args: Vec<String> = serde_json::from_str(launches.trim()).unwrap();
assert!(args.windows(2).any(|a| a == ["--input-format", "stream-json"]));
assert!(args.iter().any(|a| a == "--strict-mcp-config"));
assert!(args.windows(2).any(|a| a == ["--mcp-config", "{\"mcpServers\":{}}"]));
let requests = std::fs::read_to_string(root.join("requests")).unwrap();
let subtypes: Vec<String> = requests
.lines()
@@ -1431,6 +1437,7 @@ async fn real_claude_task_in_a_worktree() {
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
catalogs: Arc::default(),
},
tokio::runtime::Handle::current(),
);
@@ -2265,6 +2272,7 @@ async fn reload_providers_updates_registry_install_clears_stale_probes_and_notif
custom_agents_dir: Some(agents_dir.clone()),
home_dir: None,
env_secrets: None,
catalogs: Arc::default(),
},
tokio::runtime::Handle::current(),
connector,
@@ -2597,6 +2605,7 @@ async fn delete_running_agent(no_load: bool) {
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
catalogs: Arc::default(),
},
tokio::runtime::Handle::current(),
connector,
@@ -2659,34 +2668,6 @@ fn mcp_servers_follow_server_state_and_agent_capabilities() {
);
}
#[tokio::test(flavor = "multi_thread")]
async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
for discovery_first in [false, true] {
let tmp = tempfile::tempdir().unwrap();
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
let options = |model: &str, effort: &str| {
config::options_of(&json!({ "configOptions": [
{ "id": "model", "category": "model", "type": "select", "currentValue": model,
"options": [{ "value": "saved" }, { "value": "discovered" }] },
{ "id": "effort", "category": "thought_level", "type": "select", "currentValue": effort,
"options": [{ "value": "low" }, { "value": "high" }] }
] }))
};
let saved = options("saved", "low");
let discovered = options("discovered", "high");
if discovery_first {
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
}
mgr.shared.remember("claude", Some(&saved), None, Some("saved"));
mgr.shared.remember_discovered("claude", &discovered, &[], &[]);
let restored = manager(tmp.path(), Arc::new(FakeAgent::default()));
for mgr in [&mgr, &restored] {
assert_eq!(model_of(&mgr.shared.known_config("claude")).as_deref(), Some("saved"));
assert_eq!(mgr.shared.known.lock()["claude"].thought_levels["saved"], saved[1..]);
}
}
}
#[tokio::test(flavor = "multi_thread")]
async fn provider_reload_does_not_start_acp_discovery() {
let agent = Arc::new(FakeAgent::default());
@@ -2773,31 +2754,6 @@ async fn discovery_times_out_when_session_new_hangs() {
assert!(!cursor.config.iter().any(|o| o.category.as_deref() == Some("model")));
}
#[tokio::test(flavor = "multi_thread")]
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
let tmp = tempfile::tempdir().unwrap();
let mgr = manager(tmp.path(), Arc::new(FakeAgent::default()));
let effort = |values: &[&str], current: &str| {
config::options_of(&json!({ "configOptions": [
{ "id": "effort", "category": "thought_level", "type": "select", "currentValue": current,
"options": values.iter().map(|v| json!({ "value": v })).collect::<Vec<_>>() }
] }))
};
let model = config::options_of(&json!({ "configOptions": [
{ "id": "model", "category": "model", "type": "select", "currentValue": "sonnet",
"options": [{ "value": "sonnet" }, { "value": "opus" }] }
] }));
// A session already reported sonnet's levels; discovery adds opus's.
let reported = [model.clone(), effort(&["low", "high"], "high")].concat();
mgr.shared.remember("claude", Some(&reported), None, Some("sonnet"));
let levels =
[("sonnet".to_owned(), effort(&["low"], "low")), ("opus".to_owned(), effort(&["low", "max"], "default"))];
mgr.shared.remember_discovered("claude", &model, &[], &levels);
let known = mgr.shared.known.lock();
assert_eq!(known["claude"].thought_levels["sonnet"], effort(&["low", "high"], "high"), "the session's report wins");
assert_eq!(known["claude"].thought_levels["opus"], effort(&["low", "max"], "default"));
}
#[test]
fn streamed_text_is_sent_as_appends() {
let mut streamed = HashMap::new();
+51 -1
Переглянути файл
@@ -53,6 +53,8 @@ pub enum BuildSystem {
TypeScript,
#[serde(rename = "cmake")]
CMake,
#[serde(rename = "dotnet")]
Dotnet,
}
/// One command of a build.
@@ -165,6 +167,8 @@ fn detect_dir(dir: &Path, model: &BuildModel, root: &Path) -> Option<BuildSystem
|| has(dir, &["build.gradle", "build.gradle.kts", "settings.gradle", "settings.gradle.kts"])
{
Some(BuildSystem::Gradle)
} else if dotnet_target(dir).is_some() {
Some(BuildSystem::Dotnet)
} else if has(dir, &["Cargo.toml"]) {
Some(BuildSystem::Cargo)
} else if has(dir, &["go.mod", "go.work"]) {
@@ -225,7 +229,7 @@ pub fn plan(
.or((!model.gradle.is_empty()).then_some(BuildSystem::Gradle))
.or_else(|| model.maven.as_ref().filter(|m| !m.projects.is_empty()).map(|_| BuildSystem::Maven))
.ok_or_else(|| {
IdeError::not_found("no supported build system (Maven, Gradle, Cargo, Go, TypeScript, CMake) found")
IdeError::not_found("no supported build system (Maven, Gradle, .NET, Cargo, Go, TypeScript, CMake) found")
})?;
let module = module_dir.as_deref().filter(|d| *d != root);
let title = match module {
@@ -244,6 +248,7 @@ pub fn plan(
BuildSystem::Go => go_steps(file.and_then(Path::parent).unwrap_or(&dir), kind)?,
BuildSystem::TypeScript => ts_steps(&dir, kind),
BuildSystem::CMake => cmake_steps(&dir, kind, cmake.and_then(<[_]>::first)),
BuildSystem::Dotnet => dotnet_steps(&dir, kind)?,
};
Ok(BuildPlan { system, title, steps })
}
@@ -451,6 +456,33 @@ fn has_build_script(dir: &Path) -> bool {
.is_some_and(|v| v.pointer("/scripts/build").is_some_and(serde_json::Value::is_string))
}
/// The solution (`.sln` / `.slnx`) or project (`.csproj` / `.fsproj` / `.vbproj`) file `dir`
/// builds: a solution first, then a project; the first by name when there are several.
fn dotnet_target(dir: &Path) -> Option<PathBuf> {
let mut files: Vec<PathBuf> =
std::fs::read_dir(dir).ok()?.flatten().map(|e| e.path()).filter(|p| p.is_file()).collect();
files.sort();
let ext = |p: &Path| p.extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
let solution = files.iter().find(|p| matches!(ext(p).as_str(), "sln" | "slnx"));
solution.or_else(|| files.iter().find(|p| matches!(ext(p).as_str(), "csproj" | "fsproj" | "vbproj"))).cloned()
}
/// `dotnet build` of the directory's solution or project; Rebuild skips incremental builds,
/// Build Artifacts publishes. The summary that repeats every message is left out.
fn dotnet_steps(dir: &Path, kind: BuildKind) -> IdeResult<Vec<BuildStep>> {
let target = dotnet_target(dir).ok_or_else(|| IdeError::not_found("no .NET solution or project found"))?;
let target = target.file_name().map_or_else(String::new, |n| n.to_string_lossy().into_owned());
let verb = if kind == BuildKind::Artifacts { "publish" } else { "build" };
let mut step = BuildStep::new("dotnet", &[verb, &target, "-nologo", "-clp:NoSummary"], dir, Format::Msbuild);
if kind == BuildKind::Rebuild {
step.args.push("--no-incremental".into());
}
// The classic console logger prints the canonical `file(line,col): error` lines; the
// terminal logger (.NET 8+) only appears on a terminal, but rules it out for older SDKs too.
step.env.insert("MSBUILDTERMINALLOGGER".into(), "off".into());
Ok(vec![step])
}
fn ts_steps(dir: &Path, kind: BuildKind) -> Vec<BuildStep> {
let script = has_build_script(dir);
if script && (kind == BuildKind::Artifacts || !dir.join("tsconfig.json").is_file()) {
@@ -1141,6 +1173,24 @@ mod tests {
("pnpm", &["run".to_owned(), "build".to_owned()][..])
);
// .NET: the solution, else the project; a module build is the file's project. An ASP.NET
// project's package.json (its front end) doesn't make it a TypeScript build.
let net = tmp.path().join("net");
write(&net, "App.sln", "");
write(&net, "src/Api/Api.csproj", "<Project/>");
write(&net, "src/Api/package.json", "{}");
write(&net, "src/Api/Program.cs", "");
let p = plan(&net, &BuildModel::default(), BuildKind::Build, None, None).unwrap();
assert_eq!(p.system, BuildSystem::Dotnet);
assert_eq!(p.steps[0].args, ["build", "App.sln", "-nologo", "-clp:NoSummary"]);
assert_eq!(p.steps[0].env["MSBUILDTERMINALLOGGER"], "off");
let p = plan(&net, &BuildModel::default(), BuildKind::Rebuild, Some(&net.join("src/Api/Program.cs")), None)
.unwrap();
assert_eq!(p.steps[0].args, ["build", "Api.csproj", "-nologo", "-clp:NoSummary", "--no-incremental"]);
assert_eq!(p.steps[0].cwd, net.join("src/Api"));
let p = plan(&net, &BuildModel::default(), BuildKind::Artifacts, None, None).unwrap();
assert_eq!(p.steps[0].args[0], "publish");
let cm = tmp.path().join("cm");
write(&cm, "CMakeLists.txt", "project(app)");
let args = |p: &BuildPlan| p.steps.iter().map(|s| s.args.join(" ")).collect::<Vec<_>>();
+61
Переглянути файл
@@ -25,6 +25,8 @@ pub(crate) enum Format {
Go,
/// GCC / Clang (through `cmake --build`): `src/a.cpp:12:5: error: msg`.
Gcc,
/// MSBuild (`dotnet build`): `C:\p\A.cs(12,5): error CS1002: msg [C:\p\App.csproj]`.
Msbuild,
}
/// What a printed column counts.
@@ -268,6 +270,43 @@ fn tsc_line(line: &str) -> Option<RawMessage> {
Some(m)
}
/// MSBuild's canonical form: `path(12,5): error CS1002: msg [project]`, `path(12,5,12,9): …`,
/// or `path : error NU1101: msg [project]` for a project-level message (no position).
/// The path may contain spaces (`C:\Users\Jane Doe\…`); "error" / "warning" are never localized.
fn msbuild_line(line: &str) -> Option<RawMessage> {
let line = line.trim();
// The project the message came from, appended by the console logger.
let line = match line.rfind(" [") {
Some(i) if line.ends_with(']') => &line[..i],
_ => line,
};
let (head, severity, rest) =
[(" error ", Severity::Error), (" warning ", Severity::Warning)].into_iter().find_map(|(word, severity)| {
let i = line.find(&format!(":{word}"))?;
Some((&line[..i], severity, &line[i + word.len() + 1..]))
})?;
let (code, msg) = rest.split_once(':')?;
let code = code.trim();
if code.is_empty() || code.contains(' ') {
return None;
}
let head = head.trim_end();
let (path, pos) = match head.strip_suffix(')').and_then(|h| h.rsplit_once('(')) {
Some((path, pos)) => (path, pos.split(',').map(num).collect::<Vec<_>>()),
None => (head, Vec::new()),
};
let at = |i: usize| pos.get(i).copied().flatten();
// `MSBUILD : error MSB1009` and the like name no file.
let file = path.rsplit(['/', '\\']).next().is_some_and(|n| n.contains('.'));
let mut m = RawMessage::new(severity, msg.trim(), ColUnit::Utf16);
if file {
m.path = Some(path.trim().to_owned());
(m.line, m.col, m.end_line, m.end_col) = (at(0), at(1), at(2), at(3));
}
m.code = Some(code.to_owned());
Some(m)
}
/// go: `./a.go:12:5: msg` (errors only; `go vet` style `a.go:12: msg` too).
fn go_line(line: &str) -> Option<RawMessage> {
let (path, ln, col, rest) = path_line_col(line)?;
@@ -404,6 +443,7 @@ impl OutputParser {
Format::Tsc => tsc_line(&line),
Format::Go => go_line(&line),
Format::Gcc => gcc_line(&line),
Format::Msbuild => msbuild_line(&line),
Format::Cargo => None,
};
if let Some(msg) = parsed {
@@ -551,6 +591,27 @@ mod tests {
assert!(plain.messages.is_empty() && plain.console.is_none());
}
#[test]
fn msbuild_canonical_messages() {
let out = parse(
Format::Msbuild,
" Determining projects to restore...\n\
C:\\Users\\Jane Doe\\App\\Program.cs(12,5): error CS1002: ; expected [C:\\Users\\Jane Doe\\App\\App.csproj]\n\
/p/App/Models/User.cs(3,8,3,12): warning CS0168: The variable 'e' is declared but never used [/p/App/App.csproj]\n\
/p/App/App.csproj : error NU1101: Unable to find package Foo. [/p/App/App.csproj]\n\
MSBUILD : error MSB1009: Project file does not exist.\n\
Build FAILED.\n",
);
assert_eq!(out.len(), 4, "{out:?}");
assert_eq!(pos(&out[0]), (Severity::Error, "C:\\Users\\Jane Doe\\App\\Program.cs", 12, Some(5)));
assert_eq!((out[0].code.as_deref(), out[0].text.as_str()), (Some("CS1002"), "; expected"));
assert_eq!(pos(&out[1]), (Severity::Warning, "/p/App/Models/User.cs", 3, Some(8)));
assert_eq!((out[1].end_line, out[1].end_col), (Some(3), Some(12)));
assert_eq!((out[2].path.as_deref(), out[2].line), (Some("/p/App/App.csproj"), None));
assert_eq!(out[2].text, "Unable to find package Foo.");
assert_eq!((out[3].path.as_deref(), out[3].code.as_deref()), (None, Some("MSB1009")));
}
#[test]
fn tsc_plain_and_pretty() {
let out = parse(
+4
Переглянути файл
@@ -118,6 +118,10 @@ pub enum IdeEvent {
LanguageServer {
server: ServerId,
name: String,
/// What the user sees as the server when another program serves in its place.
#[serde(default)]
#[specta(optional)]
label: Option<String>,
status: ServerStatus,
message: Option<String>,
},
+9 -1
Переглянути файл
@@ -1,8 +1,10 @@
//! JSON with comments (`devcontainer.json`, `deno.jsonc`, `tsconfig.json`): turned
//! into plain JSON for `serde_json`.
/// Strips `//` and `/* */` comments and trailing commas so JSONC parses as JSON.
/// Strips `//` and `/* */` comments, trailing commas and a leading byte order mark (Visual
/// Studio writes one into `launchSettings.json`) so JSONC parses as JSON.
pub fn strip(src: &str) -> String {
let src = src.strip_prefix('\u{feff}').unwrap_or(src);
let mut out = String::with_capacity(src.len());
let mut chars = src.chars().peekable();
let mut in_str = false;
@@ -87,4 +89,10 @@ mod tests {
assert_eq!(v["url"], "https://x.dev//y /* no */");
assert_eq!(v["q"], "say \"hi\",");
}
#[test]
fn drops_a_byte_order_mark() {
let v: serde_json::Value = serde_json::from_str(&strip("\u{feff}{ \"a\": 1 }")).unwrap();
assert_eq!(v["a"], 1);
}
}
+23 -1
Переглянути файл
@@ -1092,7 +1092,8 @@ impl Default for ExternalFormatterSettings {
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct PythonSettings {
/// Project interpreter executable; empty = auto-detect (project venv, else `python3`).
/// Project interpreter executable; empty = auto-detect (project venv, else `python3`);
/// `target:<run-target-id>` = the Python of a Docker or Compose run target.
pub interpreter: String,
/// Activate the interpreter's virtualenv in new terminal sessions.
pub activate_virtualenv: bool,
@@ -1121,6 +1122,17 @@ impl Default for PythonSettings {
}
}
impl PythonSettings {
/// What `interpreter` starts with when the project interpreter is a run target's.
pub const TARGET_PREFIX: &str = "target:";
/// The run target whose Python is the project interpreter (`target:<id>`), if one is.
pub fn run_target(&self) -> Option<&str> {
let id = self.interpreter.trim().strip_prefix(Self::TARGET_PREFIX)?.trim();
(!id.is_empty()).then_some(id)
}
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct PhpSettings {
@@ -1542,6 +1554,16 @@ impl Settings {
mod tests {
use super::*;
#[test]
fn python_interpreter_may_name_a_run_target() {
let python = |interpreter: &str| PythonSettings { interpreter: interpreter.into(), ..Default::default() };
assert_eq!(python("target:web").run_target(), Some("web"));
assert_eq!(python(" target: web ").run_target(), Some("web"));
for local in ["", "target:", "/usr/bin/python3", ".venv/bin/python", "targets/python"] {
assert_eq!(python(local).run_target(), None, "{local}");
}
}
#[test]
fn removed_one_themes_read_as_islands_themes() {
let settings: Settings = serde_json::from_str(
+3
Переглянути файл
@@ -260,6 +260,8 @@ pub struct DebuggerSettings {
pub show_on_breakpoint: bool,
pub focus_app_on_breakpoint: bool,
pub hide_on_termination: bool,
/// A breakpoint or exception hit brings the Debug tool window's Threads & Variables tab to the front.
pub variables_on_breakpoint: bool,
pub breakpoints_any_file: bool,
/// While suspended, hovering the start of a line shows a Run to Cursor button.
pub run_to_cursor_inlay: bool,
@@ -285,6 +287,7 @@ impl Default for DebuggerSettings {
show_on_breakpoint: true,
focus_app_on_breakpoint: false,
hide_on_termination: false,
variables_on_breakpoint: true,
breakpoints_any_file: false,
run_to_cursor_inlay: true,
inline_values: true,
+2
Переглянути файл
@@ -7,6 +7,8 @@ license.workspace = true
[dependencies]
futures.workspace = true
tokio-tungstenite.workspace = true
base64.workspace = true
tempfile.workspace = true
ide-core.workspace = true
ide-language.workspace = true
+528
Переглянути файл
@@ -0,0 +1,528 @@
//! Replacing the responses a debugged page receives. js-debug hands out a WebSocket that
//! speaks the browser's own protocol (`requestCDPProxy`); over it the browser is told to hold
//! the requests whose URL matches a rule (`Fetch.enable`) and each held request is answered
//! here, with the rule's response or by letting it through.
//!
//! A held request stays held until it is answered, so interception is always switched off
//! (`Fetch.disable`) before the connection is given up: a page whose requests nobody answers hangs.
use crate::{DebugManager, DebugSessionId, REQUEST_TIMEOUT, js_debug};
use base64::Engine;
use futures::{SinkExt, StreamExt};
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use specta::Type;
use std::collections::HashMap;
use tokio::sync::{oneshot, watch};
use tokio_tungstenite::tungstenite::{Message, http::StatusCode};
/// A response given in place of the server's to every request whose URL contains `url_contains`.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JsResponseRule {
pub url_contains: String,
pub status: u16,
pub content_type: String,
pub body: String,
}
/// The browser's URL pattern (`*` any text, `?` one character, `\` escapes) for "contains `part`".
pub(crate) fn url_pattern(part: &str) -> String {
let mut pattern = String::with_capacity(part.len() + 2);
pattern.push('*');
for c in part.chars() {
if matches!(c, '*' | '?' | '\\') {
pattern.push('\\');
}
pattern.push(c);
}
pattern.push('*');
pattern
}
pub(crate) fn rule_for<'a>(rules: &'a [JsResponseRule], url: &str) -> Option<&'a JsResponseRule> {
rules.iter().find(|rule| url.contains(&rule.url_contains))
}
/// What to tell the browser about a request it holds: the first matching rule's response, or
/// to send the request on. Every held request gets one of the two.
pub(crate) fn answer(held: &Value, rules: &[JsResponseRule]) -> (&'static str, Value) {
let request_id = held.get("requestId").and_then(Value::as_str).unwrap_or_default();
let url = held.pointer("/request/url").and_then(Value::as_str).unwrap_or_default();
let Some(rule) = rule_for(rules, url) else {
return ("Fetch.continueRequest", json!({ "requestId": request_id }));
};
let header = |name: &str, value: &str| json!({ "name": name, "value": value });
// A page on another origin only reads a response that names its origin. The server would
// say so; the response given in its place has to.
let mut cross_origin = Vec::new();
if let Some(origin) = request_header(held, "Origin") {
cross_origin.push(header("Access-Control-Allow-Origin", origin));
cross_origin.push(header("Access-Control-Allow-Credentials", "true"));
}
let method = held.pointer("/request/method").and_then(Value::as_str).unwrap_or_default();
// The browser's question before such a request: allowed here, the server may not exist.
if method == "OPTIONS"
&& let Some(asked) = request_header(held, "Access-Control-Request-Method")
{
cross_origin.push(header("Access-Control-Allow-Methods", asked));
if let Some(asked) = request_header(held, "Access-Control-Request-Headers") {
cross_origin.push(header("Access-Control-Allow-Headers", asked));
}
let allowed = json!({
"requestId": request_id,
"responseCode": 204,
"responsePhrase": phrase(204),
"responseHeaders": cross_origin,
});
return ("Fetch.fulfillRequest", allowed);
}
let mut headers = vec![header("Content-Type", &rule.content_type)];
headers.extend(cross_origin);
let replaced = json!({
"requestId": request_id,
"responseCode": rule.status,
"responsePhrase": phrase(rule.status),
"responseHeaders": headers,
"body": base64::engine::general_purpose::STANDARD.encode(rule.body.as_bytes()),
});
("Fetch.fulfillRequest", replaced)
}
/// A header of the held request; the browser reports names as the page wrote them.
fn request_header<'a>(held: &'a Value, name: &str) -> Option<&'a str> {
held.pointer("/request/headers")?.as_object()?.iter().find(|(key, _)| key.eq_ignore_ascii_case(name))?.1.as_str()
}
/// The browser refuses a response without a phrase when it has none of its own for the status.
fn phrase(status: u16) -> &'static str {
StatusCode::from_u16(status).ok().and_then(|status| status.canonical_reason()).unwrap_or("Unknown")
}
fn interception(rules: &[JsResponseRule]) -> Value {
let patterns: Vec<Value> = rules
.iter()
.map(|rule| json!({ "urlPattern": url_pattern(&rule.url_contains), "requestStage": "Request" }))
.collect();
json!({ "patterns": patterns })
}
/// Applies `rules` to the page behind js-debug's CDP proxy at `ws_url` until their sender is
/// dropped (the session is gone) or the proxy closes the connection. The browser holds
/// requests only while there are rules. `ready` hears whether the proxy reports held
/// requests at all; nothing is held before it has agreed to.
pub(crate) async fn serve(
ws_url: String,
mut rules: watch::Receiver<Vec<JsResponseRule>>,
ready: oneshot::Sender<Result<(), String>>,
) {
let socket = match tokio_tungstenite::connect_async(ws_url.as_str()).await {
Ok((socket, _)) => socket,
Err(e) => {
let _ = ready.send(Err(e.to_string()));
return;
}
};
let (mut write, mut read) = socket.split();
let mut last_id = 0u64;
let mut call = |method: &str, params: Value| {
last_id += 1;
(last_id, Message::text(json!({ "id": last_id, "method": method, "params": params }).to_string()))
};
let (subscription, subscribe) = call("JsDebug.subscribe", json!({ "events": ["Fetch.requestPaused"] }));
let subscribed = tokio::time::timeout(REQUEST_TIMEOUT, async {
write.send(subscribe).await.map_err(|e| e.to_string())?;
loop {
let reply = read.next().await.ok_or("the connection closed")?.map_err(|e| e.to_string())?;
let Some(reply) = reply.to_text().ok().and_then(|text| serde_json::from_str::<Value>(text).ok()) else {
continue;
};
if reply.get("id").and_then(Value::as_u64) != Some(subscription) {
continue;
}
break match reply.get("error") {
Some(error) => Err(error.get("message").and_then(Value::as_str).unwrap_or("refused").to_owned()),
None => Ok(()),
};
}
})
.await
.unwrap_or_else(|_| Err("no answer".to_owned()));
if let Err(why) = subscribed {
let _ = ready.send(Err(why));
let _ = write.close().await;
return;
}
let _ = ready.send(Ok(()));
let mut current = rules.borrow_and_update().clone();
let mut holding = !current.is_empty();
if holding && write.send(call("Fetch.enable", interception(&current)).1).await.is_err() {
return;
}
// The request each sent replacement is for, by command id, until the browser has taken it.
let mut replacing: HashMap<u64, String> = HashMap::new();
loop {
tokio::select! {
changed = rules.changed() => {
if changed.is_err() {
break;
}
current = rules.borrow_and_update().clone();
let (_, command) = match (current.is_empty(), holding) {
(true, false) => continue,
(true, true) => call("Fetch.disable", json!({})),
(false, _) => call("Fetch.enable", interception(&current)),
};
holding = !current.is_empty();
if write.send(command).await.is_err() {
return;
}
}
message = read.next() => {
// The proxy closed (the page or the session ended): nothing is held any more.
let Some(Ok(message)) = message else { return };
let Ok(text) = message.to_text() else { continue };
let Ok(message) = serde_json::from_str::<Value>(text) else { continue };
if let Some(id) = message.get("id").and_then(Value::as_u64) {
// The browser did not take the replacement: let the request through, or
// it stays held and the page waits for it.
if let Some(request_id) = replacing.remove(&id)
&& message.get("error").is_some()
{
let (_, go_on) = call("Fetch.continueRequest", json!({ "requestId": request_id }));
if write.send(go_on).await.is_err() {
return;
}
}
continue;
}
if message.get("method").and_then(Value::as_str) != Some("Fetch.requestPaused") {
continue;
}
let Some(held) = message.get("params") else { continue };
let Some(request_id) = held.get("requestId").and_then(Value::as_str) else { continue };
let (method, params) = answer(held, &current);
let (id, command) = call(method, params);
if method == "Fetch.fulfillRequest" {
replacing.insert(id, request_id.to_owned());
}
if write.send(command).await.is_err() {
return;
}
}
}
}
if holding {
let _ = write.send(call("Fetch.disable", json!({})).1).await;
}
let _ = write.close().await;
}
impl DebugManager {
/// Makes `rules` the page's response rules; none switches the replacing off. The first
/// rules of a session open the connection to js-debug's CDP proxy, later ones go over it.
/// An error means the rules are not applied.
pub async fn js_set_response_overrides(&self, s: DebugSessionId, rules: Vec<JsResponseRule>) -> IdeResult<()> {
let session = self.session(s)?;
if !js_debug::is_page(&session) {
return Err(IdeError::unavailable("this debug session is not a page in a browser"));
}
// Held to the end: two callers at once would each open a connection.
let mut applied = session.js_overrides.lock().await;
if applied.as_ref().is_some_and(|sender| sender.send(rules.clone()).is_ok()) {
return Ok(());
}
// No connection yet, or the proxy closed it.
applied.take();
if rules.is_empty() {
return Ok(());
}
let refused =
|why: String| IdeError::unavailable(format!("the JavaScript debugger has no browser connection: {why}"));
if !js_debug::launched(&session).await {
return Err(refused("the page's debug session did not finish starting".into()));
}
let proxy = session
.conn
.request_timeout("requestCDPProxy", json!({}), REQUEST_TIMEOUT)
.await
.map_err(|e| refused(e.to_string()))?;
let (Some(host), Some(port)) =
(proxy.get("host").and_then(Value::as_str), proxy.get("port").and_then(Value::as_u64))
else {
return Err(refused("no address".into()));
};
let path = proxy.get("path").and_then(Value::as_str).unwrap_or_default();
let (sender, receiver) = watch::channel(rules);
let (ready, is_ready) = oneshot::channel();
tokio::spawn(serve(format!("ws://{host}:{port}{path}"), receiver, ready));
match is_ready.await {
Ok(Ok(())) => {
*applied = Some(sender);
Ok(())
}
Ok(Err(why)) => Err(refused(why)),
Err(_) => Err(refused("no answer".into())),
}
}
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
use futures::{SinkExt, StreamExt};
use serde_json::json;
use tokio::sync::mpsc;
use tokio_tungstenite::tungstenite::Message;
pub(crate) fn rule(url_contains: &str, status: u16, body: &str) -> JsResponseRule {
JsResponseRule {
url_contains: url_contains.into(),
status,
content_type: "application/json".into(),
body: body.into(),
}
}
#[test]
fn a_part_of_a_url_becomes_a_pattern_that_matches_it_anywhere() {
assert_eq!(url_pattern("api/items"), "*api/items*");
// The pattern's own wildcards and escape character are taken literally.
assert_eq!(url_pattern(r"a*b?c\d"), r"*a\*b\?c\\d*");
}
#[test]
fn the_first_rule_whose_text_is_in_the_url_answers() {
let rules = [rule("api/items", 201, "{\"a\":\"é\"}"), rule("api", 500, "later")];
assert_eq!(rule_for(&rules, "http://h/api/items?x=1").map(|r| r.status), Some(201));
assert_eq!(rule_for(&rules, "http://h/api/users").map(|r| r.status), Some(500));
assert!(rule_for(&rules, "http://h/logo.png").is_none());
let (command, params) = answer(&held("7", "http://h/api/items")["params"], &rules);
assert_eq!(command, "Fetch.fulfillRequest");
assert_eq!(
params,
json!({
"requestId": "7",
"responseCode": 201,
"responsePhrase": "Created",
"responseHeaders": [{ "name": "Content-Type", "value": "application/json" }],
// base64 of the UTF-8 text `{"a":"é"}`
"body": "eyJhIjoiw6kifQ==",
})
);
// A request the browser held that no rule is for goes on to the network.
assert_eq!(
answer(&held("8", "http://h/logo.png")["params"], &rules),
("Fetch.continueRequest", json!({ "requestId": "8" }))
);
}
#[test]
fn a_status_the_browser_has_no_phrase_for_is_given_one() {
// The browser refuses a response whose status it cannot name.
let (_, params) = answer(&held("1", "http://h/api")["params"], &[rule("api", 419, "")]);
assert_eq!((&params["responseCode"], &params["responsePhrase"]), (&json!(419), &json!("Unknown")));
}
fn header<'a>(params: &'a Value, name: &str) -> Option<&'a str> {
params["responseHeaders"].as_array()?.iter().find(|h| h["name"] == name)?["value"].as_str()
}
#[test]
fn a_replaced_response_passes_the_pages_cross_origin_check() {
let rules = [rule("api/items", 200, "[]")];
let from_page = |method: &str, headers: Value| json!({ "requestId": "3", "request": { "url": "http://localhost:8080/api/items", "method": method, "headers": headers } });
// A page on another origin: the response names that origin, as the real server would.
let (_, params) = answer(&from_page("GET", json!({ "origin": "http://localhost:5173" })), &rules);
assert_eq!(header(&params, "Access-Control-Allow-Origin"), Some("http://localhost:5173"));
assert_eq!(header(&params, "Access-Control-Allow-Credentials"), Some("true"));
assert_eq!(params["body"], "W10=");
// The browser's question before such a request is answered too: the server may not exist.
let preflight = from_page(
"OPTIONS",
json!({
"Origin": "http://localhost:5173",
"Access-Control-Request-Method": "PUT",
"Access-Control-Request-Headers": "content-type,x-token",
}),
);
let (command, params) = answer(&preflight, &rules);
assert_eq!((command, &params["responseCode"]), ("Fetch.fulfillRequest", &json!(204)));
assert_eq!(header(&params, "Access-Control-Allow-Origin"), Some("http://localhost:5173"));
assert_eq!(header(&params, "Access-Control-Allow-Methods"), Some("PUT"));
assert_eq!(header(&params, "Access-Control-Allow-Headers"), Some("content-type,x-token"));
assert_eq!(params.get("body"), None);
// Same origin: nothing but the rule's own header.
let (_, params) = answer(&from_page("GET", json!({})), &rules);
assert_eq!(params["responseHeaders"], json!([{ "name": "Content-Type", "value": "application/json" }]));
}
/// A stand-in for js-debug's CDP proxy: what it receives, and a way to send it events.
pub(crate) async fn proxy() -> (String, mpsc::UnboundedReceiver<Value>, mpsc::UnboundedSender<Value>) {
proxy_refusing(|_| false).await
}
/// A proxy that answers the commands `refused` picks with an error, like the real one does
/// for what it or the browser does not accept.
pub(crate) async fn proxy_refusing(
refused: fn(&Value) -> bool,
) -> (String, mpsc::UnboundedReceiver<Value>, mpsc::UnboundedSender<Value>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("ws://127.0.0.1:{}/session", listener.local_addr().unwrap().port());
let (received_tx, received) = mpsc::unbounded_channel();
let (events, mut events_rx) = mpsc::unbounded_channel::<Value>();
tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let (mut write, mut read) = tokio_tungstenite::accept_async(stream).await.unwrap().split();
loop {
tokio::select! {
message = read.next() => {
let Some(Ok(message)) = message else { break };
let Ok(text) = message.to_text() else { continue };
let Ok(request) = serde_json::from_str::<Value>(text) else { continue };
let reply = if refused(&request) {
json!({ "id": request["id"], "error": { "code": -32602, "message": "refused" } })
} else {
json!({ "id": request["id"], "result": {} })
};
let _ = write.send(Message::text(reply.to_string())).await;
let _ = received_tx.send(request);
}
Some(event) = events_rx.recv() => {
let _ = write.send(Message::text(event.to_string())).await;
}
}
}
});
(url, received, events)
}
pub(crate) async fn next(received: &mut mpsc::UnboundedReceiver<Value>) -> (String, Value) {
let request = tokio::time::timeout(std::time::Duration::from_secs(5), received.recv())
.await
.expect("the proxy got a request in time")
.expect("the connection is open");
(request["method"].as_str().unwrap().to_owned(), request["params"].clone())
}
/// The browser's event for a request it holds.
fn held(request_id: &str, url: &str) -> Value {
json!({ "method": "Fetch.requestPaused", "params": { "requestId": request_id, "request": { "url": url } } })
}
type Serving = (tokio::task::JoinHandle<()>, oneshot::Receiver<Result<(), String>>);
fn serving(url: String, rules: watch::Receiver<Vec<JsResponseRule>>) -> Serving {
let (ready, is_ready) = oneshot::channel();
(tokio::spawn(serve(url, rules, ready)), is_ready)
}
async fn ended(task: tokio::task::JoinHandle<()>) {
tokio::time::timeout(std::time::Duration::from_secs(5), task).await.unwrap().unwrap();
}
#[tokio::test]
async fn held_requests_are_answered_and_interception_ends_before_the_connection() {
let (url, mut received, events) = proxy().await;
let (rules, watched) = tokio::sync::watch::channel(vec![rule("api/items", 200, "[]")]);
let (serving, ready) = serving(url, watched);
assert_eq!(ready.await.unwrap(), Ok(()));
assert_eq!(
next(&mut received).await,
("JsDebug.subscribe".into(), json!({ "events": ["Fetch.requestPaused"] }))
);
assert_eq!(
next(&mut received).await,
(
"Fetch.enable".into(),
json!({ "patterns": [{ "urlPattern": "*api/items*", "requestStage": "Request" }] })
)
);
events.send(held("1", "http://h/api/items")).unwrap();
let (command, params) = next(&mut received).await;
assert_eq!(
(command.as_str(), &params["requestId"], &params["responseCode"]),
("Fetch.fulfillRequest", &json!("1"), &json!(200))
);
events.send(held("2", "http://h/other")).unwrap();
assert_eq!(next(&mut received).await, ("Fetch.continueRequest".into(), json!({ "requestId": "2" })));
// New rules replace the browser's patterns.
rules.send(vec![rule("api/users", 404, "{}")]).unwrap();
assert_eq!(
next(&mut received).await,
(
"Fetch.enable".into(),
json!({ "patterns": [{ "urlPattern": "*api/users*", "requestStage": "Request" }] })
)
);
// No rules left: the browser stops holding requests. The connection stays for the
// rules that come next, and goes with the session.
rules.send(Vec::new()).unwrap();
assert_eq!(next(&mut received).await.0, "Fetch.disable");
rules.send(vec![rule("api/items", 200, "[]")]).unwrap();
assert_eq!(next(&mut received).await.0, "Fetch.enable");
rules.send(Vec::new()).unwrap();
assert_eq!(next(&mut received).await.0, "Fetch.disable");
drop(rules);
ended(serving).await;
// Already off: not switched off a second time.
assert!(received.try_recv().is_err());
}
#[tokio::test]
async fn a_session_that_goes_away_switches_interception_off() {
let (url, mut received, _events) = proxy().await;
let (rules, watched) = tokio::sync::watch::channel(vec![rule("api", 200, "[]")]);
let (serving, _ready) = serving(url, watched);
assert_eq!(next(&mut received).await.0, "JsDebug.subscribe");
assert_eq!(next(&mut received).await.0, "Fetch.enable");
drop(rules);
assert_eq!(next(&mut received).await.0, "Fetch.disable");
ended(serving).await;
}
#[tokio::test]
async fn a_response_the_browser_refuses_lets_the_request_through() {
let (url, mut received, events) = proxy_refusing(|request| request["method"] == "Fetch.fulfillRequest").await;
let (_rules, watched) = tokio::sync::watch::channel(vec![rule("api", 200, "[]")]);
let (_serving, _ready) = serving(url, watched);
assert_eq!(next(&mut received).await.0, "JsDebug.subscribe");
assert_eq!(next(&mut received).await.0, "Fetch.enable");
events.send(held("9", "http://h/api")).unwrap();
assert_eq!(next(&mut received).await.0, "Fetch.fulfillRequest");
// Unanswered, the request would hang the page.
assert_eq!(next(&mut received).await, ("Fetch.continueRequest".into(), json!({ "requestId": "9" })));
}
#[tokio::test]
async fn a_proxy_that_does_not_report_held_requests_is_not_asked_to_hold_any() {
let (url, mut received, _events) = proxy_refusing(|request| request["method"] == "JsDebug.subscribe").await;
let (_rules, watched) = tokio::sync::watch::channel(vec![rule("api", 200, "[]")]);
let (serving, ready) = serving(url, watched);
assert_eq!(next(&mut received).await.0, "JsDebug.subscribe");
assert_eq!(ready.await.unwrap(), Err("refused".to_owned()));
ended(serving).await;
assert!(received.try_recv().is_err());
}
#[tokio::test]
async fn no_connection_to_the_proxy_is_reported() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("ws://127.0.0.1:{}/session", listener.local_addr().unwrap().port());
drop(listener);
let (_rules, watched) = tokio::sync::watch::channel(vec![rule("api", 200, "[]")]);
let (serving, ready) = serving(url, watched);
assert!(ready.await.unwrap().is_err());
ended(serving).await;
}
}
+41 -1
Переглянути файл
@@ -11,6 +11,8 @@ use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use specta::Type;
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::atomic::{AtomicU32, Ordering};
pub(crate) const ADAPTER: &str = "js-debug";
@@ -25,7 +27,7 @@ pub(crate) fn is_page(session: &Session) -> bool {
/// Waits until the adapter answered the `launch` of `session`. js-debug sets a page's session
/// up between `configurationDone` and that answer; a request of its own that arrives then is
/// answered and has no effect.
async fn launched(session: &Session) -> bool {
pub(crate) async fn launched(session: &Session) -> bool {
let mut launched = session.launched.subscribe();
let answered = tokio::time::timeout(REQUEST_TIMEOUT, launched.wait_for(|answered| *answered)).await;
matches!(answered, Ok(Ok(_)))
@@ -59,6 +61,44 @@ impl DebugManager {
self.js_page_request(s, "setCustomBreakpoints", json!({ "ids": ids, "xhr": xhr })).await?;
Ok(())
}
/// Starts recording the page's CPU profile; [`Self::js_stop_profile`] ends it.
pub async fn js_start_profile(&self, s: DebugSessionId) -> IdeResult<()> {
self.js_page_request(s, "startProfile", json!({ "type": "cpu" })).await?;
Ok(())
}
/// Ends the recording and returns a new directory holding it as `page.cpuprofile`
/// (what `ide-profiler` reads a directory of).
pub async fn js_stop_profile(&self, s: DebugSessionId) -> IdeResult<PathBuf> {
let session = self.session(s)?;
// js-debug names the file it records into when the recording starts.
let Some(recorded) = session.js_profile.lock().take() else {
return Err(IdeError::invalid("no CPU profile is being recorded"));
};
self.js_page_request(s, "stopProfile", json!({})).await?;
static HANDED_OVER: AtomicU32 = AtomicU32::new(0);
let dir = std::env::temp_dir().join(format!(
"openidea-js-profile-{}-{}-{}",
std::process::id(),
s.0,
HANDED_OVER.fetch_add(1, Ordering::Relaxed)
));
let copied = tokio::task::spawn_blocking({
let dir = dir.clone();
move || {
std::fs::create_dir_all(&dir)?;
std::fs::copy(&recorded, dir.join("page.cpuprofile"))?;
// The adapter's own copy is in the system's temporary directory: no use keeping two.
let _ = std::fs::remove_file(&recorded);
Ok::<_, std::io::Error>(())
}
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?;
copied.map_err(|e| IdeError::unavailable(format!("the recorded CPU profile cannot be read: {e}")))?;
Ok(dir)
}
}
/// The browser events js-debug is asked to pass on (`enableNetworking`'s `mirrorEvents`).
+2
Переглянути файл
@@ -13,6 +13,7 @@ pub mod attach;
mod cargo;
pub mod connection;
pub mod jdwp;
mod js_cdp;
mod js_debug;
mod jvm_memory;
pub mod kotlin;
@@ -24,6 +25,7 @@ use connection::{Connection, DapError};
use futures::future::join_all;
use ide_core::{DebugSessionId, IdeError, IdeResult, SharedSink};
use ide_language::LanguageRegistry;
pub use js_cdp::JsResponseRule;
pub use js_debug::JsResponseBody;
pub use jvm_memory::*;
use parking_lot::Mutex;
+72 -8
Переглянути файл
@@ -94,6 +94,11 @@ pub(crate) struct Session {
/// The `type` of the launch or attach request (js-debug's `pwa-node`, `pwa-chrome`); a child
/// session's comes from the configuration the adapter started it with.
pub launch_type: std::sync::OnceLock<String>,
/// The response rules of a page once a connection applies them (`js_cdp::serve`).
/// Dropped with the session, which switches the browser's interception off.
pub js_overrides: tokio::sync::Mutex<Option<tokio::sync::watch::Sender<Vec<crate::JsResponseRule>>>>,
/// The file js-debug records the page's CPU profile into, while one is being recorded.
pub js_profile: parking_lot::Mutex<Option<std::path::PathBuf>>,
pub jvm_inspector: parking_lot::Mutex<Option<Arc<dyn ide_jvm_inspect::JvmInspector>>>,
/// One inspector request per session at a time.
pub inspect_lock: tokio::sync::Mutex<()>,
@@ -255,6 +260,8 @@ impl Session {
progress: Mutex::new(HashMap::new()),
jdwp_link: parking_lot::Mutex::new(None),
launch_type: std::sync::OnceLock::new(),
js_overrides: tokio::sync::Mutex::new(None),
js_profile: parking_lot::Mutex::new(None),
jvm_inspector: parking_lot::Mutex::new(None),
inspect_lock: tokio::sync::Mutex::default(),
ended: ide_core::CancellationToken::new(),
@@ -474,10 +481,7 @@ impl Session {
let _ = c.start_kill();
}
if let Some(mut program) = self.program.lock().take() {
let _ = program.child.kill();
let _ = std::thread::Builder::new().name("openidea-debug-reap".into()).spawn(move || {
let _ = program.child.wait();
});
program.kill();
}
}
}
@@ -574,7 +578,7 @@ fn launch_groups(s: &Session) -> Vec<libc::pid_t> {
for child in s.debuggees.lock().iter() {
push(child.id());
}
push(s.program.lock().as_ref().and_then(|p| p.child.process_id()));
push(s.program.lock().as_ref().and_then(ProgramTerminal::process_id));
groups.retain(|g| *g > 0);
groups
}
@@ -845,6 +849,10 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
s.emit(DebugEvent::JsNetwork { event });
}
}
// js-debug, after `startProfile`: where it records the profile.
"profileStarted" => {
*s.js_profile.lock() = str_("file").map(std::path::PathBuf::from);
}
_ => tracing::trace!("dap: ignored event {event}"),
}
}
@@ -874,7 +882,11 @@ fn on_reverse_request(mgr: &Arc<DebugManager>, s: &Arc<Session>, seq: i64, comma
/// A `runInTerminal` program and the pseudo-terminal it runs in.
pub(crate) struct ProgramTerminal {
writer: ide_core::input::InputWriter,
child: Box<dyn portable_pty::Child + Send + Sync>,
killer: Box<dyn portable_pty::ChildKiller + Send + Sync>,
#[cfg_attr(not(unix), allow(dead_code))]
pid: Option<u32>,
/// Set once the program has exited and been reaped.
exited: Arc<AtomicBool>,
_master: Box<dyn portable_pty::MasterPty + Send>,
}
@@ -882,6 +894,29 @@ impl ProgramTerminal {
pub(crate) fn write(&mut self, text: &str) -> std::io::Result<()> {
self.writer.write(text.as_bytes())
}
/// The program's process id while it runs.
#[cfg_attr(not(unix), allow(dead_code))]
pub(crate) fn process_id(&self) -> Option<u32> {
if self.exited.load(Ordering::Acquire) { None } else { self.pid }
}
pub(crate) fn kill(&mut self) {
// After the reap the pid may belong to another process.
if !self.exited.load(Ordering::Acquire) {
let _ = self.killer.kill();
}
}
}
/// Reaps the program as soon as it exits. Until then it stays a zombie that `kill(pid, 0)` still
/// finds: js-debug polls the `processId` it was given and keeps the session (and its terminal)
/// open while that process looks alive, and Stop would wait for it as for a running program.
fn reap_on_exit(mut child: Box<dyn portable_pty::Child + Send + Sync>, exited: Arc<AtomicBool>) {
let _ = std::thread::Builder::new().name("openidea-debug-reap".into()).spawn(move || {
let _ = child.wait();
exited.store(true, Ordering::Release);
});
}
/// `runInTerminal`: starts the debuggee on a pseudo-terminal, as an IDE terminal would (CodeLLDB
@@ -968,8 +1003,12 @@ pub(crate) fn run_in_terminal(s: &Arc<Session>, args: &Value) -> Result<Value, S
#[cfg(unix)]
wait_readable,
);
if let Some(mut previous) = s.program.lock().replace(ProgramTerminal { writer, child, _master: pair.master }) {
let _ = previous.child.kill();
let killer = child.clone_killer();
let exited = Arc::new(AtomicBool::new(false));
reap_on_exit(child, exited.clone());
let program = ProgramTerminal { writer, killer, pid, exited, _master: pair.master };
if let Some(mut previous) = s.program.lock().replace(program) {
previous.kill();
}
s.owner.sink.emit(IdeEvent::Debug { session: s.id, event: DebugEvent::ProgramInput });
Ok(json!({ "processId": pid }))
@@ -1338,4 +1377,29 @@ mod tests {
assert_eq!(parse_jdwp_port("Listening for transport dt_socket at address: [::1]:51234"), Some(51234));
assert_eq!(parse_jdwp_port("unrelated output"), None);
}
/// js-debug polls the `runInTerminal` pid with `kill(pid, 0)`; an unreaped zombie answers it as
/// a live process and keeps the session open after the program ends.
#[cfg(unix)]
#[test]
fn a_terminal_program_is_reaped_when_it_exits() {
use portable_pty::{CommandBuilder, PtySize, native_pty_system};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
let pair =
native_pty_system().openpty(PtySize { rows: 24, cols: 80, pixel_width: 0, pixel_height: 0 }).unwrap();
let mut cmd = CommandBuilder::new("/bin/sh");
cmd.args(["-c", "exit 0"]);
let child = pair.slave.spawn_command(cmd).unwrap();
let pid = child.process_id().unwrap() as libc::pid_t;
let exited = Arc::new(AtomicBool::new(false));
super::reap_on_exit(child, exited.clone());
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
while !exited.load(Ordering::Acquire) {
assert!(std::time::Instant::now() < deadline, "program was not reaped");
std::thread::sleep(std::time::Duration::from_millis(10));
}
// SAFETY: signal 0 only checks whether the process exists.
assert_eq!(unsafe { libc::kill(pid, 0) }, -1);
}
}
+136
Переглянути файл
@@ -1104,6 +1104,142 @@ async fn a_page_whose_requests_the_adapter_does_not_report_says_so() {
h.wait_event(|e| matches!(e, DebugEvent::JsNetworkUnavailable)).await;
}
/// Where the fake adapter says its CDP proxy listens (`requestCDPProxy`).
static CDP_PROXY_PORT: std::sync::atomic::AtomicU16 = std::sync::atomic::AtomicU16::new(0);
fn proxy_adapter(msg: &Value) -> Option<CustomReply> {
let port = CDP_PROXY_PORT.load(std::sync::atomic::Ordering::Relaxed);
(msg["command"] == "requestCDPProxy")
.then(|| (json!({ "host": "127.0.0.1", "port": port, "path": "/session" }), Vec::new()))
}
#[tokio::test]
async fn response_overrides_reach_the_page_through_the_adapters_proxy() {
use crate::js_cdp::tests::{next, proxy, rule};
let (url, mut received, _events) = proxy().await;
let port: u16 = url.trim_start_matches("ws://127.0.0.1:").trim_end_matches("/session").parse().unwrap();
CDP_PROXY_PORT.store(port, std::sync::atomic::Ordering::Relaxed);
let h = harness(Script {
start_debugging: true,
custom: Some(proxy_adapter),
child_launch_delay: Duration::from_millis(150),
..Script::default()
});
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
let page = child_of(&h, browser).await;
// Rules that are there as the page starts wait until its session takes requests.
tokio::time::sleep(Duration::from_millis(50)).await;
h.manager.js_set_response_overrides(page, vec![rule("api/items", 200, "[]")]).await.unwrap();
let dropped = h.log.lock().iter().filter(|m| m["type"] == "dropped").count();
assert_eq!(dropped, 0, "the proxy was asked for before the page took requests");
assert_eq!(next(&mut received).await.0, "JsDebug.subscribe");
assert_eq!(
next(&mut received).await,
("Fetch.enable".into(), json!({ "patterns": [{ "urlPattern": "*api/items*", "requestStage": "Request" }] }))
);
// Changed rules go over the connection that is open: the proxy is asked for once.
h.manager.js_set_response_overrides(page, vec![rule("api/users", 200, "[]")]).await.unwrap();
assert_eq!(next(&mut received).await.1["patterns"][0]["urlPattern"], "*api/users*");
assert_eq!(requests(&h, "requestCDPProxy").len(), 1);
h.manager.js_set_response_overrides(page, Vec::new()).await.unwrap();
assert_eq!(next(&mut received).await.0, "Fetch.disable");
// A rule switched back on.
h.manager.js_set_response_overrides(page, vec![rule("api/items", 200, "[]")]).await.unwrap();
assert_eq!(next(&mut received).await.0, "Fetch.enable");
assert_eq!(requests(&h, "requestCDPProxy").len(), 1);
let not_a_page = h.manager.js_set_response_overrides(browser, vec![rule("api", 200, "")]).await.unwrap_err();
assert_eq!(not_a_page.kind, ide_core::error::ErrorKind::Unavailable);
}
#[tokio::test]
async fn response_overrides_are_unavailable_without_the_adapters_proxy() {
use crate::js_cdp::tests::rule;
let h = harness(Script { start_debugging: true, ..Script::default() });
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
let page = child_of(&h, browser).await;
let refused = h.manager.js_set_response_overrides(page, vec![rule("api", 200, "")]).await.unwrap_err();
assert_eq!(refused.kind, ide_core::error::ErrorKind::Unavailable);
// Nothing to switch off is not an error.
h.manager.js_set_response_overrides(page, Vec::new()).await.unwrap();
}
/// A port nothing listens on, which the fake adapter gives as its CDP proxy.
static DEAD_PROXY_PORT: std::sync::atomic::AtomicU16 = std::sync::atomic::AtomicU16::new(0);
fn dead_proxy_adapter(msg: &Value) -> Option<CustomReply> {
let port = DEAD_PROXY_PORT.load(std::sync::atomic::Ordering::Relaxed);
(msg["command"] == "requestCDPProxy")
.then(|| (json!({ "host": "127.0.0.1", "port": port, "path": "/session" }), Vec::new()))
}
#[tokio::test]
async fn response_overrides_that_cannot_reach_the_browser_are_an_error() {
use crate::js_cdp::tests::rule;
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
DEAD_PROXY_PORT.store(listener.local_addr().unwrap().port(), std::sync::atomic::Ordering::Relaxed);
drop(listener);
let h = harness(Script { start_debugging: true, custom: Some(dead_proxy_adapter), ..Script::default() });
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
let page = child_of(&h, browser).await;
let unreachable = h.manager.js_set_response_overrides(page, vec![rule("api", 200, "")]).await.unwrap_err();
assert_eq!(unreachable.kind, ide_core::error::ErrorKind::Unavailable);
// The next rules try again instead of going to a connection that never was.
let again = h.manager.js_set_response_overrides(page, vec![rule("api", 200, "")]).await.unwrap_err();
assert_eq!(again.kind, ide_core::error::ErrorKind::Unavailable);
assert_eq!(requests(&h, "requestCDPProxy").len(), 2);
}
/// The file the fake adapter says it records a CPU profile into (`profileStarted`).
static PROFILE_FILE: parking_lot::Mutex<String> = parking_lot::Mutex::new(String::new());
fn profiling_adapter(msg: &Value) -> Option<CustomReply> {
match msg["command"].as_str()? {
"startProfile" => {
Some((json!({}), vec![("profileStarted", json!({ "file": *PROFILE_FILE.lock(), "type": "cpu" }))]))
}
"stopProfile" => Some((json!({}), vec![("profilerStateUpdate", json!({ "label": "", "running": false }))])),
_ => None,
}
}
#[tokio::test]
async fn a_pages_cpu_profile_is_handed_over_as_a_cpuprofile_file() {
let recorded = std::env::temp_dir().join(format!("openidea-dap-profile-{}", std::process::id()));
std::fs::write(&recorded, r#"{"nodes":[],"samples":[]}"#).unwrap();
*PROFILE_FILE.lock() = recorded.to_string_lossy().into_owned();
let h = harness(Script { start_debugging: true, custom: Some(profiling_adapter), ..Script::default() });
let browser = h.start(js_config("pwa-chrome")).await.unwrap();
let page = child_of(&h, browser).await;
// Nothing is being recorded yet.
let idle = h.manager.js_stop_profile(page).await.unwrap_err();
assert_eq!(idle.kind, ide_core::error::ErrorKind::InvalidInput);
h.manager.js_start_profile(page).await.unwrap();
assert_eq!(requests(&h, "startProfile"), [json!({ "type": "cpu" })]);
let dir = h.manager.js_stop_profile(page).await.unwrap();
assert_eq!(requests(&h, "stopProfile").len(), 1);
let files: Vec<_> = std::fs::read_dir(&dir).unwrap().map(|e| e.unwrap().file_name()).collect();
assert_eq!(files, ["page.cpuprofile"]);
assert_eq!(std::fs::read_to_string(dir.join("page.cpuprofile")).unwrap(), r#"{"nodes":[],"samples":[]}"#);
// One recording is handed over once.
assert_eq!(h.manager.js_stop_profile(page).await.unwrap_err().kind, ide_core::error::ErrorKind::InvalidInput);
// The adapter's file isn't there when it is asked for (the hand-over above took the first one).
assert!(!recorded.exists());
h.manager.js_start_profile(page).await.unwrap();
assert_eq!(h.manager.js_stop_profile(page).await.unwrap_err().kind, ide_core::error::ErrorKind::Unavailable);
assert_eq!(h.manager.js_start_profile(browser).await.unwrap_err().kind, ide_core::error::ErrorKind::Unavailable);
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn a_child_session_is_named_by_the_adapter_unless_it_is_a_blank_page() {
assert_eq!(child_name(&json!({ "name": "server.js [4120]" }), "Server"), "server.js [4120]");
+18
Переглянути файл
@@ -542,6 +542,24 @@ pub fn compose_config(ep: &DockerEndpoint, dir: &Path, files: &[String]) -> IdeR
if out.success { Ok(out.stdout) } else { Err(failure(&args, &out.stderr, &out.stdout)) }
}
/// Runs the docker CLI with `args` in `dir` and returns its standard output; a non-zero
/// exit is an error with docker's message.
pub fn cli(
ep: &DockerEndpoint,
dir: Option<&Path>,
args: &[String],
timeout: Duration,
cancel: Option<&CancellationToken>,
) -> IdeResult<String> {
let args: Vec<&str> = args.iter().map(String::as_str).collect();
let mut cmd = command(ep, &args)?;
if let Some(dir) = dir {
cmd.current_dir(dir);
}
let out = process::run(cmd, timeout, cancel)?;
if out.success { Ok(out.stdout) } else { Err(failure(&args, &out.stderr, &out.stdout)) }
}
#[cfg(test)]
mod local_tests {
use super::*;
+2
Переглянути файл
@@ -66,6 +66,7 @@ pub struct Host {
pub mcp_requests: mcp::Requests,
/// Commands agents run in the Run tool window (`run_command`).
pub mcp_agent_runs: mcp::AgentRuns,
pub(crate) agent_catalogs: Arc<ide_agent::AgentCatalogs>,
pub trust: trust::TrustStore,
pub(crate) mcp_approvals: Arc<trust::McpServerApprovals>,
pub config_dir: PathBuf,
@@ -152,6 +153,7 @@ impl Host {
mcp_reports: Default::default(),
mcp_requests: Default::default(),
mcp_agent_runs: Default::default(),
agent_catalogs: Arc::default(),
trust: trust::TrustStore::load(&config_dir, &data_dir),
mcp_approvals: Arc::new(trust::McpServerApprovals::load(&data_dir)),
proxy_password: ide_ssh::secrets::SecretFile::new(data_dir.join("proxy-password.json")),
+11 -2
Переглянути файл
@@ -7,8 +7,8 @@ use crate::rpc::Dispatcher;
use ide_agent::registry::{AgentRegistry, RegistryPaths};
use ide_agent::{
AcpImportOutcome, AgentAcceptOutcome, AgentChange, AgentChanges, AgentChangesView, AgentCommand, AgentEntry,
AgentEvent, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask, CustomAgentInput, ExternalSession,
McpScope, McpServer, McpServerInput, NewAgentTask,
AgentEvent, AgentModelCatalog, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask, CustomAgentInput,
ExternalSession, McpScope, McpServer, McpServerInput, NewAgentTask,
};
use ide_core::error::ErrorKind;
use ide_core::{IdeError, IdeEvent, IdeResult, IpcPath, RequestId, SharedSink, TaskId};
@@ -28,6 +28,12 @@ pub fn providers(host: &Host, window: &str) -> IdeResult<Vec<AgentProvider>> {
Ok(providers)
}
/// Refreshes the selected agent's model catalog without sending a prompt.
pub fn refresh_models(host: &Host, window: &str, provider: String, force: bool) -> IdeResult<AgentModelCatalog> {
host.require_trust(window)?;
host.project(window)?.agents.refresh_models(&provider, force)
}
/// Overrides built-in agent `id`'s command with `command` (its agent file keeps the
/// built-in's driver and arguments), or returns it to auto-detection with `None`.
pub fn set_command(host: &Host, window: &str, id: String, command: Option<String>) -> IdeResult<()> {
@@ -557,6 +563,9 @@ pub fn mcp_import(host: &Host, window: &str) -> IdeResult<Vec<McpServer>> {
pub(crate) fn register(d: &mut Dispatcher) {
d.add("agent_subscribe", |c| async move { subscribe(&c.host, &c.window, c.channel("onEvent")?) });
d.add("agent_providers", |c| async move { providers(&c.host, &c.window) });
d.add("agent_refresh_models", |c| async move {
refresh_models(&c.host, &c.window, c.arg("provider")?, c.arg("force")?)
});
d.add("agent_set_command", |c| async move { set_command(&c.host, &c.window, c.arg("id")?, c.arg("command")?) });
d.add("agent_set_env", |c| async move { set_env(&c.host, &c.window, c.arg("id")?, c.arg("env")?) });
d.add("agent_registry_list", |c| async move { registry_list(&c.host, &c.window, c.arg("request")?).await });
+45 -6
Переглянути файл
@@ -12,6 +12,7 @@ use ide_db::{
};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::PathBuf;
/// One statement of a console run.
#[derive(Clone, Serialize, Deserialize, Type)]
@@ -179,6 +180,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
apply_edits(&c.host, &c.window, &id, &schema, &table, edits).await
});
d.add("db_console_path", |c| async move { console_path(&c.host, &c.arg::<String>("id")?) });
d.add("db_consoles", |c| async move { consoles(&c.host, &c.arg::<String>("id")?, c.arg("create")?) });
}
/// The data sources configured for the project (the tree's roots).
@@ -472,11 +474,52 @@ pub async fn apply_edits(
/// created on first use. Consoles are files of the project, so a remote
/// window resolves it on the host the project runs on.
pub fn console_path(host: &Host, id: &str) -> IdeResult<IpcPath> {
let file = console_dir(host, id)?.join("console.sql");
if !file.exists() {
std::fs::write(&file, "")?;
}
Ok(file.into())
}
/// The query consoles of data source `id`: the default `console.sql`, then
/// `console_1.sql`, `console_2.sql`… in order. `create` adds the next one first
/// (DataGrip's New › Query Console).
pub fn consoles(host: &Host, id: &str, create: bool) -> IdeResult<Vec<IpcPath>> {
let dir = console_dir(host, id)?;
let default = dir.join("console.sql");
let mut numbered: Vec<(u32, PathBuf)> = std::fs::read_dir(&dir)?
.filter_map(|e| {
let path = e.ok()?.path();
let n = path.file_name()?.to_str()?.strip_prefix("console_")?.strip_suffix(".sql")?.parse().ok()?;
Some((n, path))
})
.collect();
numbered.sort();
if create {
// The default console comes first; once it exists, the next number.
if !default.exists() {
std::fs::write(&default, "")?;
} else {
let n = numbered.last().map_or(1, |(n, _)| n + 1);
let file = dir.join(format!("console_{n}.sql"));
std::fs::write(&file, "")?;
numbered.push((n, file));
}
}
let mut out: Vec<IpcPath> = Vec::new();
if default.exists() {
out.push(default.into());
}
out.extend(numbered.into_iter().map(|(_, p)| p.into()));
Ok(out)
}
/// `<dataDir>/consoles/db/<id>`: `<id>` for the default database, `<id>.<database>` for the others.
fn console_dir(host: &Host, id: &str) -> IdeResult<PathBuf> {
let (source, database) = ide_db::split_target(id);
if source.is_empty() || !source.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_') {
return Err(IdeError::invalid("invalid data source id"));
}
// One console per database: `<id>` for the default one, `<id>.<database>` for the others.
let name = match database {
Some(db) => {
let safe: String =
@@ -487,9 +530,5 @@ pub fn console_path(host: &Host, id: &str) -> IdeResult<IpcPath> {
};
let dir = host.data_dir.join("consoles").join("db").join(&name);
std::fs::create_dir_all(&dir)?;
let file = dir.join("console.sql");
if !file.exists() {
std::fs::write(&file, "")?;
}
Ok(file.into())
Ok(dir)
}
+32
Переглянути файл
@@ -382,6 +382,15 @@ pub async fn debug_js_custom_breakpoints(
host.debugger.js_set_custom_breakpoints(session, ids, xhr).await
}
pub async fn debug_js_response_overrides(
host: &Host,
_window: &str,
session: DebugSessionId,
rules: Vec<ide_dap::JsResponseRule>,
) -> IdeResult<()> {
host.debugger.js_set_response_overrides(session, rules).await
}
pub async fn debug_js_response_body(
host: &Host,
_window: &str,
@@ -391,6 +400,18 @@ pub async fn debug_js_response_body(
host.debugger.js_response_body(session, request_id).await
}
pub async fn debug_js_start_profile(host: &Host, _window: &str, session: DebugSessionId) -> IdeResult<()> {
host.debugger.js_start_profile(session).await
}
pub async fn debug_js_stop_profile(
host: &Host,
_window: &str,
session: DebugSessionId,
) -> IdeResult<ide_core::IpcPath> {
host.debugger.js_stop_profile(session).await.map(Into::into)
}
pub fn debug_capabilities(host: &Host, _window: &str, session: DebugSessionId) -> IdeResult<DebugCapabilities> {
host.debugger.capabilities(session)
}
@@ -607,12 +628,23 @@ pub(crate) fn register(d: &mut Dispatcher) {
debug_restart_frame(&c.host, &c.window, c.arg("session")?, c.arg("frameId")?).await
});
d.add("debug_capabilities", |c| async move { debug_capabilities(&c.host, &c.window, c.arg("session")?) });
d.add("debug_js_response_overrides", |c| async move {
debug_js_response_overrides(&c.host, &c.window, c.arg("session")?, c.arg("rules")?).await
});
d.add("debug_js_response_body", |c| async move {
debug_js_response_body(&c.host, &c.window, c.arg("session")?, c.arg("requestId")?).await
});
d.add("debug_js_custom_breakpoints", |c| async move {
debug_js_custom_breakpoints(&c.host, &c.window, c.arg("session")?, c.arg("ids")?, c.arg("xhr")?).await
});
d.add(
"debug_js_start_profile",
|c| async move { debug_js_start_profile(&c.host, &c.window, c.arg("session")?).await },
);
d.add(
"debug_js_stop_profile",
|c| async move { debug_js_stop_profile(&c.host, &c.window, c.arg("session")?).await },
);
d.add("debug_set_function_breakpoints", |c| async move {
debug_set_function_breakpoints(&c.host, &c.window, c.arg("breakpoints")?).await
});
+22
Переглянути файл
@@ -21,6 +21,28 @@ pub fn close(host: &Host, window: &str, doc: DocId) -> IdeResult<()> {
Ok(())
}
/// Opens an editable untitled buffer (a new-file tab); it reaches the disk
/// through [`save_as`], which turns it into a regular file document. `name`
/// re-references an already minted buffer instead of creating one.
pub async fn open_untitled(host: &Host, window: &str, name: Option<String>) -> IdeResult<DocumentInfo> {
let project = host.project(window)?;
host.blocking(move || project.documents.open_untitled(name.as_deref())).await
}
/// Save As for an untitled buffer ([`Documents::save_as`]); LightEdit windows
/// start watching the saved file's folder.
pub async fn save_as(host: &Host, window: &str, doc: DocId, path: PathBuf) -> IdeResult<DocumentInfo> {
let project = host.project(window)?;
host.blocking(move || {
let info = project.documents.save_as(doc, &path)?;
if let Some(dir) = path.parent().filter(|_| project.light) {
project.watch_light_dir(dir);
}
Ok(info)
})
.await
}
/// Applies a CodeMirror change set; returns the new version (`outOfSync` when `base_version` is stale).
pub fn edit(host: &Host, window: &str, doc: DocId, base_version: u32, changes: ChangeSet) -> IdeResult<u32> {
host.project(window)?.documents.edit(doc, base_version, changes)
+14 -1
Переглянути файл
@@ -50,7 +50,10 @@ pub async fn doc_set_encoding(
reload: bool,
) -> IdeResult<DocumentEncoding> {
let project = host.project(window)?;
if reload && let Some(uri) = project.documents.virtual_uri(doc)? {
if reload
&& !project.documents.is_untitled(doc)?
&& let Some(uri) = project.documents.virtual_uri(doc)?
{
let vd = if uri.starts_with("git-revision:") {
crate::ops::vcs::revision_virtual_document(host, window, &uri, Some(&encoding)).await?
} else {
@@ -159,6 +162,15 @@ pub async fn doc_highlights(host: &Host, window: &str, doc: DocId) -> IdeResult<
.await
}
/// Python class facts the inheritance inspections read: base names as written, a method's
/// parameters, decorator semantics and its body shape. Tree-sitter answers; no LSP.
pub async fn doc_python_facts(host: &Host, window: &str, doc: DocId) -> IdeResult<ide_language::PythonFacts> {
let project = host.project(window)?;
let syntax = host.syntax();
let snap = project.documents.text_snapshot(doc)?;
blocking(move || Ok(syntax.python_facts(&snap.language, &snap.text))).await
}
/// Extend Selection (⌥↑): syntax ranges around `offset`, innermost first.
/// Large documents fall back to word / line / document ranges.
pub async fn doc_selection_ranges(host: &Host, window: &str, doc: DocId, offset: u32) -> IdeResult<Vec<Range>> {
@@ -208,6 +220,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("doc_outline", |c| async move { doc_outline(&c.host, &c.window, c.arg("doc")?).await });
d.add("doc_folding", |c| async move { doc_folding(&c.host, &c.window, c.arg("doc")?).await });
d.add("doc_highlights", |c| async move { doc_highlights(&c.host, &c.window, c.arg("doc")?).await });
d.add("doc_python_facts", |c| async move { doc_python_facts(&c.host, &c.window, c.arg("doc")?).await });
d.add("doc_selection_ranges", |c| async move {
doc_selection_ranges(&c.host, &c.window, c.arg("doc")?, c.arg("offset")?).await
});
+12
Переглянути файл
@@ -1472,6 +1472,15 @@ pub async fn lsp_type_hierarchy(
host.project(window)?.lsp.type_hierarchy(doc, pos, direction).await
}
pub async fn lsp_prepare_type_hierarchy(
host: &Host,
window: &str,
doc: DocId,
pos: Pos,
) -> IdeResult<Option<HierarchyItem>> {
host.project(window)?.lsp.prepare_type_hierarchy(doc, pos).await
}
pub fn lsp_servers(host: &Host, window: &str) -> IdeResult<Vec<ServerInfo>> {
Ok(host.project(window)?.lsp.servers())
}
@@ -1856,6 +1865,9 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("lsp_type_hierarchy", |c| async move {
lsp_type_hierarchy(&c.host, &c.window, c.arg("doc")?, c.arg("pos")?, c.arg("direction")?).await
});
d.add("lsp_prepare_type_hierarchy", |c| async move {
lsp_prepare_type_hierarchy(&c.host, &c.window, c.arg("doc")?, c.arg("pos")?).await
});
d.add("lsp_servers", |c| async move { lsp_servers(&c.host, &c.window) });
d.add("lsp_resend_diagnostics", |c| async move { lsp_resend_diagnostics(&c.host, &c.window) });
d.add("lsp_restart", |c| async move { lsp_restart(&c.host, &c.window, c.arg("server")?).await });
+87 -1
Переглянути файл
@@ -2,7 +2,7 @@ use super::doc::blocking;
use crate::Host;
use crate::Project;
use crate::rpc::Dispatcher;
use ide_core::{DocId, IdeResult, IpcPath, Location, Pos};
use ide_core::{DocId, IdeResult, IpcPath, LanguageId, Location, Pos};
use ide_index::{ExtensionCount, FileMatch, IndexStatus, SymbolFilter, SymbolMatch};
use ide_language::{FileReference, WebReference};
use ide_workspace::file_refs::{self, PathCompletion};
@@ -22,6 +22,23 @@ pub struct WebGoto {
const MAX_WEB_TARGETS: usize = 200;
/// A Python class definition and the base names written in its source. The UI builds a
/// name → locations index from every class in the project (attached dependency roots
/// included) to compute ↑ overrides / ↓ overridden gutter markers without a language
/// server's type-hierarchy provider.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ClassInheritance {
/// Start of the class definition's selection range (0-based line, UTF-16 column).
pub path: IpcPath,
pub line: u32,
pub col: u32,
pub name: String,
/// Base names as written in source (bare name, e.g. `AllToAllTokenDispatcher`; a
/// qualified name keeps only its last segment).
pub bases: Vec<String>,
}
/// The project root and its resource roots (served at `/`), for file references.
fn reference_roots(project: &Project) -> (PathBuf, Vec<PathBuf>) {
let resources = project.roots().resource_dirs().iter().map(|d| project.root.join(d)).collect();
@@ -174,6 +191,74 @@ pub async fn nav_index_definitions(
.await
}
/// Every Python class in the project (attached dependency roots included) and its base
/// names. The index covers content roots, so this is the whole-project answer the UI uses
/// for inheritance gutter markers when no type-hierarchy provider exists.
pub async fn nav_inheritance(host: &Host, window: &str) -> IdeResult<Vec<ClassInheritance>> {
let project = host.project(window)?;
let syntax = host.syntax();
blocking(move || {
let python = LanguageId::new("python");
let files = project.index.files_with_extensions(&["py".into(), "pyi".into()], 20_000);
let mut cache = CLASS_BASES.lock();
let mut out = Vec::new();
for path in files {
// The open (possibly unsaved) buffer is keyed by its text, a file on disk by its
// size and modification time, so only changed files are parsed again.
let open = project.documents.text_of(&path);
let stamp = match &open {
Some(text) => Stamp::Text(text_hash(text)),
None => match std::fs::metadata(&path) {
Ok(m) => Stamp::Disk(m.len(), m.modified().ok()),
Err(_) => continue,
},
};
let classes = match cache.get(&path) {
Some((cached, classes)) if *cached == stamp => classes,
_ => {
let text = match open {
Some(text) => text,
None => match project.text_of(&path) {
Ok(text) => text,
Err(_) => continue,
},
};
let classes = syntax.class_bases(&python, &text);
&cache.entry(path.clone()).insert_entry((stamp, classes)).into_mut().1
}
};
out.extend(classes.iter().map(|class| ClassInheritance {
path: IpcPath::from(path.clone()),
line: class.line,
col: class.col,
name: class.name.clone(),
bases: class.bases.clone(),
}));
}
Ok(out)
})
.await
}
#[derive(Clone, PartialEq, Eq)]
enum Stamp {
Text(u64),
Disk(u64, Option<std::time::SystemTime>),
}
fn text_hash(text: &str) -> u64 {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
text.hash(&mut h);
h.finish()
}
type ClassBasesCache = std::collections::HashMap<PathBuf, (Stamp, Vec<ide_language::ClassBases>)>;
/// Class bases per Python file, reused while the file is unchanged.
static CLASS_BASES: std::sync::LazyLock<parking_lot::Mutex<ClassBasesCache>> =
std::sync::LazyLock::new(Default::default);
/// The existing file or directory a path written in `from` names (`href`, `url()`, a string literal).
pub async fn fs_resolve_reference(
host: &Host,
@@ -249,6 +334,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("nav_index_definitions", |c| async move {
nav_index_definitions(&c.host, &c.window, c.arg("name")?, c.arg("from")?).await
});
d.add("nav_inheritance", |c| async move { nav_inheritance(&c.host, &c.window).await });
d.add("fs_resolve_reference", |c| async move {
fs_resolve_reference(&c.host, &c.window, c.arg("from")?, c.arg("value")?).await
});
+6
Переглянути файл
@@ -239,6 +239,12 @@ pub fn set_settings(
let toolchain = ["languages", "tools", "languageServers", "projectStructure"]
.iter()
.any(|k| key == *k || key.starts_with(&format!("{k}.")));
// A run target chosen as the Python interpreter (or changed) needs a snapshot of its libraries.
if ["languages", "languages.python", "languages.python.interpreter", "runTargets"].contains(&key) {
for project in affected() {
crate::ops::sdk::container_python::ensure(host, &project);
}
}
// Code style only reaches servers as settings (e.g. jdtls' formatter profile); builds don't depend on it.
if code_style || toolchain {
for project in affected() {
+1 -1
Переглянути файл
@@ -508,7 +508,7 @@ pub fn list(host: &Host) -> Vec<RunInfo> {
}
mod commands;
mod container;
pub(crate) mod container;
pub use commands::*;
#[cfg(test)]
+26 -15
Переглянути файл
@@ -33,23 +33,11 @@ pub(super) async fn prepare(host: &Host, root: &Path, target: RunTarget, config:
let root = root.to_path_buf();
let data_dir = host.data_dir.clone();
host.blocking(move || {
if !ide_docker::endpoint_is_local(&endpoint)? {
return Err(IdeError::invalid(format!(
"Run target '{}' uses a Docker daemon on another machine: the project can only be mounted from a local one",
target.name
)));
}
require_local(&endpoint, &target)?;
let docker = ide_docker::docker_executable().ok_or_else(|| {
IdeError::unavailable("docker executable not found; install Docker and make sure it is on PATH")
})?;
let container_root = match target.container_root.trim() {
"" if target.kind == RunTargetKind::Compose => {
let json = ide_docker::compose_config(&endpoint, &root, &target.compose_files)?;
compose_container_root(&json, target.service.trim(), &root)?
}
"" => default_container_root(&root),
folder => folder.to_owned(),
};
let container_root = container_root(&endpoint, &root, &target)?;
let file_env = env_files(&root, &config)?;
let name = container_name(&config.id, &unique());
let debugging = if debug { Some(Debugging::prepare(&data_dir)?) } else { None };
@@ -75,6 +63,29 @@ pub(super) async fn prepare(host: &Host, root: &Path, target: RunTarget, config:
.await
}
/// The project can only be bind-mounted from a daemon on this machine.
pub(crate) fn require_local(endpoint: &DockerEndpoint, target: &RunTarget) -> IdeResult<()> {
if !ide_docker::endpoint_is_local(endpoint)? {
return Err(IdeError::invalid(format!(
"Run target '{}' uses a Docker daemon on another machine: the project can only be mounted from a local one",
target.name
)));
}
Ok(())
}
/// Where the project is in the target's containers.
pub(crate) fn container_root(endpoint: &DockerEndpoint, root: &Path, target: &RunTarget) -> IdeResult<String> {
Ok(match target.container_root.trim() {
"" if target.kind == RunTargetKind::Compose => {
let json = ide_docker::compose_config(endpoint, root, &target.compose_files)?;
compose_container_root(&json, target.service.trim(), root)?
}
"" => default_container_root(root),
folder => folder.to_owned(),
})
}
/// What a debug run mounts and publishes.
struct Debugging {
port: u16,
@@ -104,7 +115,7 @@ fn free_port() -> IdeResult<u16> {
}
/// The target's Docker connection (Services tool window), by id; none = the CLI's daemon.
fn endpoint(data_dir: &Path, target: &RunTarget) -> IdeResult<DockerEndpoint> {
pub(crate) fn endpoint(data_dir: &Path, target: &RunTarget) -> IdeResult<DockerEndpoint> {
let id = target.docker_connection.trim();
if id.is_empty() {
return Ok(DockerEndpoint::Default);
+12 -2
Переглянути файл
@@ -10,6 +10,9 @@ use std::path::PathBuf;
use std::sync::Mutex;
use std::sync::atomic::{AtomicUsize, Ordering};
pub(crate) mod container_python;
pub use container_python::{ContainerPython, sdk_container_python, sdk_container_python_refresh};
fn join_err(e: impl std::fmt::Display) -> IdeError {
IdeError::internal(e.to_string())
}
@@ -232,12 +235,17 @@ pub async fn sdk_project_libraries(
let project = host.project(window)?;
let (root, cache, settings) = (project.root.clone(), project.build_cache_dir(), project.settings.effective());
let cancel = host.begin_project_request(window, request);
let container = container_python::library_group(host, &root, &settings);
let libraries = {
let (root, settings, cancel, on_groups) = (root.clone(), settings.clone(), cancel.clone(), on_groups.clone());
tokio::task::spawn_blocking(move || {
ide_sdk::libraries::project_libraries_stream(&root, &settings, &[], &cancel, &|groups| {
if let Some(group) = &container {
let _ = on_groups.send(vec![group.clone()]);
}
let groups = ide_sdk::libraries::project_libraries_stream(&root, &settings, &[], &cancel, &|groups| {
let _ = on_groups.send(groups);
})
});
container.into_iter().chain(groups).collect::<Vec<_>>()
})
};
let dependencies = {
@@ -377,6 +385,8 @@ pub(crate) fn register(d: &mut Dispatcher) {
);
d.add("sdk_python_effective", |c| async move { sdk_python_effective(&c.host, &c.window, c.arg("near")?) });
d.add("sdk_php_effective", |c| async move { sdk_php_effective(&c.host, &c.window) });
d.add("sdk_container_python", |c| async move { sdk_container_python(&c.host, &c.window) });
d.add("sdk_container_python_refresh", |c| async move { sdk_container_python_refresh(&c.host, &c.window).await });
d.add("sdk_php_xdebug", |c| async move { sdk_php_xdebug(&c.host, &c.window, c.arg("php")?, c.arg("cwd")?).await });
d.add(
"sdk_php_language_level",
+298
Переглянути файл
@@ -0,0 +1,298 @@
//! A Docker or Compose run target's Python as the project interpreter: snapshots of the
//! container's library folders for the language server (see `ide_sdk::container_python`).
//!
//! A snapshot runs the target's Python once in a new container (the image's entrypoint
//! and the service's dependencies skipped), then copies its `sys.path` folders out of
//! the stopped container with `docker cp`. Nothing is installed in the container.
use crate::Host;
use crate::project::Project;
use ide_core::event::NotifyLevel;
use ide_core::remote_settings::{RunTarget, RunTargetKind};
use ide_core::settings::Settings;
use ide_core::{IdeError, IdeEvent, IdeResult, IpcPath, TaskId};
use ide_sdk::container_python::{self as snapshots, Snapshot};
use ide_sdk::python::PythonPackage;
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::sync::{Arc, LazyLock, Mutex};
use std::time::Duration;
/// Building or pulling the image may come first.
const PROBE_TIMEOUT: Duration = Duration::from_secs(15 * 60);
const COPY_TIMEOUT: Duration = Duration::from_secs(10 * 60);
const QUICK: Duration = Duration::from_secs(30);
/// The project interpreter when it is a run target's.
#[derive(Clone, Debug, serde::Serialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct ContainerPython {
/// The run target's id.
pub target: String,
/// Its name; empty when no run target has the id.
pub name: String,
/// The Python version of the snapshot in use; `None`: no snapshot yet.
pub version: Option<String>,
/// The image the snapshot was copied from.
pub image: Option<String>,
/// The copied library folders on this machine.
pub folders: Vec<IpcPath>,
/// Packages of the snapshot (read-only).
pub packages: Vec<PythonPackage>,
/// A snapshot is being taken.
pub updating: bool,
}
/// Where snapshots are kept, and the Docker connections the targets refer to.
#[derive(Clone)]
struct Dirs {
data: PathBuf,
/// `<local data>/container-python/`: `<project>/<target>/<image>/` and `neutral/`.
cache: PathBuf,
}
impl Dirs {
fn of(host: &Host) -> Self {
Self::new(&host.data_dir, &host.local_data_dir)
}
fn new(data_dir: &Path, local_data_dir: &Path) -> Self {
Self { data: data_dir.to_path_buf(), cache: local_data_dir.join("container-python") }
}
fn target(&self, root: &Path, target: &str) -> PathBuf {
snapshots::target_dir(&self.cache.join(crate::project::project_key(root)), target)
}
}
/// Target folders with a snapshot being taken.
static UPDATING: LazyLock<Mutex<HashSet<PathBuf>>> = LazyLock::new(Default::default);
fn updating(dir: &Path) -> bool {
UPDATING.lock().unwrap_or_else(|e| e.into_inner()).contains(dir)
}
/// The snapshot in use for the project's interpreter, if it is a run target's.
fn current(dirs: &Dirs, root: &Path, settings: &Settings) -> Option<(PathBuf, Snapshot)> {
let target = settings.languages.python.run_target()?;
snapshots::current(&dirs.target(root, target))
}
/// Language server configuration: basedpyright / pyright see the container's libraries.
pub(crate) fn configure_server(
cfg: &mut ide_language::LanguageServerConfig,
local_data_dir: &Path,
root: &Path,
settings: &Settings,
) {
if settings.languages.python.run_target().is_none() {
return;
}
let current = current(&Dirs::new(local_data_dir, local_data_dir), root, settings);
snapshots::configure_server(cfg, current.as_ref().map(|(d, s)| (d.as_path(), s)));
}
/// External Libraries: the snapshot's folders, named after the run target.
pub(crate) fn library_group(host: &Host, root: &Path, settings: &Settings) -> Option<ide_sdk::libraries::LibraryGroup> {
let target = settings.languages.python.run_target()?;
let (dir, snapshot) = current(&Dirs::of(host), root, settings)?;
let name = settings.run_targets.iter().find(|t| t.id == target).map_or(target, |t| t.name.as_str());
snapshots::library_group(name, &dir, &snapshot)
}
/// The project interpreter when it is a run target's, with its snapshot.
pub fn sdk_container_python(host: &Host, window: &str) -> IdeResult<Option<ContainerPython>> {
host.require_trust(window)?;
let project = host.project(window)?;
Ok(describe(host, &project.root, &project.settings.effective()))
}
fn describe(host: &Host, root: &Path, settings: &Settings) -> Option<ContainerPython> {
let target = settings.languages.python.run_target()?;
let name = settings.run_targets.iter().find(|t| t.id == target).map(|t| t.name.clone()).unwrap_or_default();
let dir = Dirs::of(host).target(root, target);
let mut out = ContainerPython {
target: target.to_owned(),
name,
version: None,
image: None,
folders: Vec::new(),
packages: Vec::new(),
updating: updating(&dir),
};
if let Some((dir, snapshot)) = snapshots::current(&dir) {
out.folders = snapshots::local_folders(&dir, &snapshot).into_iter().map(Into::into).collect();
out.packages = snapshots::packages(&dir, &snapshot);
out.version = Some(snapshot.probe.version);
out.image = Some(snapshot.image);
}
Some(out)
}
/// Takes a new snapshot of the project interpreter's container (also of an image that
/// has one already), then the language servers pick it up.
pub async fn sdk_container_python_refresh(host: &Host, window: &str) -> IdeResult<Option<ContainerPython>> {
host.require_trust(window)?;
let project = host.project(window)?;
update(Dirs::of(host), &project, true).await?;
Ok(describe(host, &project.root, &project.settings.effective()))
}
/// Takes a snapshot when the project interpreter is a run target's and the current image
/// has none yet (a new choice, a rebuilt image), in the background. Errors are notified.
pub(crate) fn ensure(host: &Host, project: &Arc<Project>) {
if !project.is_trusted() || project.settings.effective().languages.python.run_target().is_none() {
return;
}
let (dirs, project) = (Dirs::of(host), project.clone());
host.runtime.spawn(async move {
if let Err(e) = update(dirs, &project, false).await
&& e.kind != ide_core::error::ErrorKind::Cancelled
{
project.sink.emit(IdeEvent::Notify {
level: NotifyLevel::Warning,
title: "Cannot copy the container's Python libraries".into(),
body: Some(e.message),
});
}
});
}
async fn update(dirs: Dirs, project: &Arc<Project>, force: bool) -> IdeResult<()> {
let settings = project.settings.effective();
let Some(id) = settings.languages.python.run_target() else { return Ok(()) };
let target = ide_remote::service::run_target(&settings, id)?;
if target.kind == RunTargetKind::Ssh {
return Err(IdeError::invalid(format!(
"Run target '{}' runs over SSH: only a Docker or Compose run target can be the project interpreter",
target.name
)));
}
let dir = dirs.target(&project.root, id);
if !UPDATING.lock().unwrap_or_else(|e| e.into_inner()).insert(dir.clone()) {
return Ok(());
}
let _done = Updating(dir.clone());
let endpoint = crate::ops::run::container::endpoint(&dirs.data, &target)?;
let task = TaskId::next();
let title = format!("Python libraries of {}", target.name);
let (root, sink) = (project.root.clone(), project.sink.clone());
let progress = move |message: &str, done: bool| {
sink.emit(IdeEvent::Task {
id: task,
title: title.clone(),
message: Some(message.to_owned()),
fraction: None,
done,
cancellable: false,
})
};
let report = progress.clone();
let result = tokio::task::spawn_blocking(move || {
take(&endpoint, &root, &target, &dir, &dirs.cache, force, &|message| report(message, false))
})
.await
.map_err(|e| IdeError::internal(e.to_string()))
.and_then(|r| r);
progress(if result.is_ok() { "Done" } else { "Failed" }, true);
if result? {
project.lsp.reconfigure().await;
}
Ok(())
}
struct Updating(PathBuf);
impl Drop for Updating {
fn drop(&mut self) {
UPDATING.lock().unwrap_or_else(|e| e.into_inner()).remove(&self.0);
}
}
/// Removes the probe container, also when copying failed.
struct Container<'a> {
endpoint: &'a ide_docker::DockerEndpoint,
name: String,
}
impl Drop for Container<'_> {
fn drop(&mut self) {
let args = ["rm", "--force", "--volumes", &self.name].map(String::from);
if let Err(e) = ide_docker::cli(self.endpoint, None, &args, QUICK, None) {
tracing::warn!("cannot remove container {}: {}", self.name, e.message);
}
}
}
/// Takes the snapshot in `target_dir` and makes it the one in use; `false` when the image
/// had one already and it is in use (unless `force`).
fn take(
endpoint: &ide_docker::DockerEndpoint,
root: &Path,
target: &RunTarget,
target_dir: &Path,
cache: &Path,
force: bool,
progress: &dyn Fn(&str),
) -> IdeResult<bool> {
crate::ops::run::container::require_local(endpoint, target)?;
let container_root = crate::ops::run::container::container_root(endpoint, root, target)?;
let name = format!("openidea-python-{}", unique());
let args = ide_remote::docker::python_probe_args(target, root, &name, snapshots::PROBE)?;
progress("Starting the container's Python");
let container = Container { endpoint, name };
let out = ide_docker::cli(endpoint, Some(root), &args, PROBE_TIMEOUT, None);
let probe = snapshots::parse_probe(&out?)?;
let inspect = ["inspect".into(), "--format".into(), "{{.Image}}".into(), container.name.clone()];
let image = ide_docker::cli(endpoint, None, &inspect, QUICK, None)?.trim().to_owned();
if image.is_empty() {
return Err(IdeError::internal(format!("docker inspect printed no image for {}", container.name)));
}
let dir = target_dir.join(snapshots::dir_name(&image));
let before = snapshots::current(target_dir).map(|(_, s)| s.image);
if !force && snapshots::read(&dir).is_some() {
snapshots::make_current(target_dir, &image)?;
return Ok(before.as_deref() != Some(image.as_str()));
}
let partial = target_dir.join(format!("{}.partial", snapshots::dir_name(&image)));
if partial.exists() {
std::fs::remove_dir_all(&partial)?;
}
std::fs::create_dir_all(&partial)?;
let mut folders = Vec::new();
for folder in snapshots::folders_to_copy(&probe, &container_root) {
progress(&format!("Copying {folder}"));
let local = snapshots::local_folder(&partial, &folder);
let Some(parent) = local.parent() else { continue };
std::fs::create_dir_all(parent)?;
// Into an existing folder: `docker cp` puts the source folder in it under its own name.
let args = ["cp".into(), format!("{}:{folder}", container.name), format!("{}/", parent.to_string_lossy())];
match ide_docker::cli(endpoint, None, &args, COPY_TIMEOUT, None) {
Ok(_) => folders.push(folder),
Err(e) => tracing::warn!("cannot copy {folder} from {}: {}", container.name, e.message),
}
}
drop(container);
progress("Preparing the language server's interpreter");
let snapshot = Snapshot {
neutral_python: snapshots::neutral_python(cache, &probe.version),
image: image.clone(),
probe,
folders,
};
snapshots::write(&partial, &snapshot)?;
if dir.exists() {
std::fs::remove_dir_all(&dir)?;
}
std::fs::rename(&partial, &dir)?;
snapshots::make_current(target_dir, &image)?;
Ok(true)
}
/// Distinguishes probe containers.
fn unique() -> String {
use std::sync::atomic::{AtomicU32, Ordering};
static COUNT: AtomicU32 = AtomicU32::new(0);
let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0, |d| d.as_secs());
format!("{:x}-{}-{}", secs & 0xff_ffff, std::process::id(), COUNT.fetch_add(1, Ordering::Relaxed))
}
+37 -2
Переглянути файл
@@ -77,9 +77,22 @@ pub(super) fn validate_reply(reply: &ContentTabReply) -> Result<(), HostError> {
Ok(())
}
/// Largest side of a plugin diff, in UTF-16 code units (earlier hosts allowed 1 MiB).
pub(super) const MAX_DIFF_SIDE: usize = 8 << 20;
fn diff_side(value: &str, side: &str) -> Result<(), HostError> {
if value.contains('\0') {
return Err(HostError::Invalid(format!("the {side} side of the diff contains NUL")));
}
if value.encode_utf16().count() > MAX_DIFF_SIDE {
return Err(HostError::Invalid(format!("diff too large: the {side} side is over {} MiB", MAX_DIFF_SIDE >> 20)));
}
Ok(())
}
pub(super) fn validate_diff(diff: &ide_plugin::ContentDiff) -> Result<(), HostError> {
body(&diff.left)?;
body(&diff.right)?;
diff_side(&diff.left, "left")?;
diff_side(&diff.right, "right")?;
for value in [&diff.left_label, &diff.right_label].into_iter().flatten() {
label(value, 128)?;
}
@@ -571,3 +584,25 @@ impl PluginHost {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn diff(left: String, right: String) -> ide_plugin::ContentDiff {
ide_plugin::ContentDiff { left, right, left_label: None, right_label: None, title: None }
}
#[test]
fn diff_sides_up_to_the_limit_open_and_larger_ones_say_why() {
assert!(validate_diff(&diff("x".repeat(5 << 20), "y".repeat(MAX_DIFF_SIDE))).is_ok());
let err = validate_diff(&diff(String::new(), "y".repeat(MAX_DIFF_SIDE + 1))).unwrap_err();
assert!(
matches!(&err, HostError::Invalid(m) if m == "diff too large: the right side is over 8 MiB"),
"{err:?}"
);
assert!(
matches!(validate_diff(&diff("a\0".into(), String::new())), Err(HostError::Invalid(m)) if m.contains("NUL"))
);
}
}
+47 -4
Переглянути файл
@@ -88,6 +88,9 @@ impl Host {
let root = dunce::canonicalize(root)?;
let opening = self.opening(window);
let _opening = opening.lock();
// Untitled buffers belong to the window, not a project: they follow it
// into the project's fresh document registry.
let carried = self.project(window).ok().map(|prev| prev.documents.untitled_buffers()).unwrap_or_default();
// Reject a damaged manifest before replacing the currently open project.
let is_workspace = ide_workspace::workspace::Workspace::load(&root)?.is_some();
// The window reloaded (or asked again) for the project it already has: keep
@@ -174,6 +177,7 @@ impl Host {
let root = member.as_ref().unwrap_or(window_root);
ide_sdk::apply::server_launch_overrides(cfg, &effective);
ide_sdk::apply::configure_server(cfg, root, server_root, &effective);
crate::ops::sdk::container_python::configure_server(cfg, &data_dir, root, &effective);
if cfg.name == "jdtls" {
// A workspace member keeps its build model under its own key (Build Project).
let cache = match &member {
@@ -620,6 +624,9 @@ impl Host {
}));
}
self.insert_project(window, project.clone());
for (key, text, dirty) in &carried {
let _ = project.documents.restore_untitled_buffer(key, text, *dirty);
}
project.lsp.prewarm_java_configuration();
if let Some(bridge) = &project.kotlin_bridge {
bridge.start(&self.runtime);
@@ -630,6 +637,7 @@ impl Host {
if let Some(cmd) = post_start {
self.environment.spawn_post_start(cmd, project.root.clone(), project.sink.clone());
}
crate::ops::sdk::container_python::ensure(self, &project);
Ok(project)
}
@@ -664,6 +672,7 @@ impl Host {
.join(project_key)
}),
models_file: self.data_dir.join("agents").join("models.json"),
catalogs: self.agent_catalogs.clone(),
custom_agents_dir: Some(self.config_dir.join("agents")),
home_dir: dirs::home_dir(),
env_secrets: Some(Arc::new(AgentEnvSecrets {
@@ -700,12 +709,29 @@ impl Host {
return Err(IdeError::invalid(format!("{} is a folder", file.display())));
}
let dir = file.parent().map(Path::to_path_buf).ok_or_else(|| IdeError::invalid("no parent folder"))?;
let watch = dir.clone();
self.open_light_in(window, dir, Some(&watch))
}
/// LightEdit without a file (a welcome window's File › New File): rooted
/// at the user's home folder, watching nothing until a buffer is saved
/// into a real folder ([`Project::watch_light_dir`]).
pub fn open_light_empty(&self, window: &str) -> IdeResult<Arc<Project>> {
let dir = dirs::home_dir().unwrap_or_else(std::env::temp_dir);
self.open_light_in(window, dir, None)
}
fn open_light_in(&self, window: &str, dir: PathBuf, watch_dir: Option<&Path>) -> IdeResult<Arc<Project>> {
let opening = self.opening(window);
let _opening = opening.lock();
// Untitled buffers belong to the window: they follow it out of a project.
let carried = self.project(window).ok().map(|prev| prev.documents.untitled_buffers()).unwrap_or_default();
if let Ok(current) = self.project(window)
&& current.light
{
current.watch_light_dir(&dir);
if let Some(dir) = watch_dir {
current.watch_light_dir(dir);
}
return Ok(current);
}
self.close_project(window);
@@ -724,7 +750,16 @@ impl Host {
let syntax = Arc::new(SyntaxCache::new(self.syntax()));
documents.add_observer(Arc::new(SyntaxObserver(syntax.clone())));
documents.add_observer(Arc::new(HistoryObserver(history.clone())));
let watcher = light_watcher(&dir, &sink, &documents, &history)?;
let watcher = match watch_dir {
Some(dir) => light_watcher(dir, &sink, &documents, &history)?,
// A quiet private folder stands in until the first save brings a
// real folder in.
None => {
let idle = self.local_data_dir.join("light-edit");
std::fs::create_dir_all(&idle)?;
light_watcher(&idle, &sink, &documents, &history)?
}
};
let project_key = project_key(&dir);
let (db, http_history, agents) = self.project_services(&dir, &project_key, &settings, &sink);
let notebooks = crate::notebook::NotebookService::new(
@@ -770,8 +805,11 @@ impl Host {
light: true,
root: dir,
});
tracing::info!("window {window} edits {} without a project", file.display());
tracing::info!("window {window} edits {} without a project", project.root.display());
self.insert_project(window, project.clone());
for (key, text, dirty) in &carried {
let _ = project.documents.restore_untitled_buffer(key, text, *dirty);
}
Ok(project)
}
}
@@ -1455,7 +1493,7 @@ fn content_root_watcher(
) -> Option<Watcher> {
let (kotlin_bridge, phpstan) = (kotlin_bridge.cloned(), phpstan.cloned());
let (index, documents, history, lsp) = (index.clone(), documents.clone(), history.clone(), lsp.clone());
let root = dir.to_path_buf();
let (root, events) = (dir.to_path_buf(), sink.clone());
Watcher::start(
dir,
excludes.to_vec(),
@@ -1477,6 +1515,11 @@ fn content_root_watcher(
}
documents.on_disk_changed(&[&changed[..], &ignored].concat());
record_external_changes(&history, &changed, &batch.removed);
// A member outside the project folder: its repository changes (commit, checkout,
// fetch, pull) reach the UI only through this watcher.
if batch.vcs.iter().any(|p| vcs_path_changed(p)) {
events.emit(IdeEvent::VcsChanged { root: root.clone().into() });
}
}),
)
.map_err(|e| tracing::warn!("content root watcher {}: {e}", dir.display()))
+7
Переглянути файл
@@ -218,6 +218,13 @@ impl Dispatcher {
})
.add("fs_read_text", |c| async move { ops::fs::read_text(&c.host, &c.window, c.arg("path")?).await })
.add("doc_open", |c| async move { ops::doc::open(&c.host, &c.window, c.arg("path")?).await })
.add("doc_open_untitled", |c| async move {
ops::doc::open_untitled(&c.host, &c.window, c.arg::<Option<String>>("name")?).await
})
.add(
"doc_save_as",
|c| async move { ops::doc::save_as(&c.host, &c.window, c.arg("doc")?, c.arg("path")?).await },
)
.add("doc_close", |c| async move {
ops::doc::blocking(move || ops::doc::close(&c.host, &c.window, c.arg("doc")?)).await
})
+3
Переглянути файл
@@ -289,6 +289,7 @@ impl crate::Host {
}
}
project.lsp.reconfigure().await;
crate::ops::sdk::container_python::ensure(self, &project);
project.sink.emit(ide_core::IdeEvent::ProjectTrustChanged { root: project.root.clone().into(), trusted });
}
Ok(())
@@ -357,6 +358,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "docker_set_registry_password"
| "docker_run_env"
| "db_console_path"
| "db_consoles"
| "db_export_file"
| "vcs_write_patch"
| "app_metrics"
@@ -399,6 +401,7 @@ pub fn restricted_command_allowed(cmd: &str) -> bool {
| "web_unresolved_references"
| "project_rename"
| "light_edit_open"
| "light_edit_open_untitled"
| "workspace_create"
| "workspace_discover_projects"
| "workspace_update"
+98
Переглянути файл
@@ -0,0 +1,98 @@
//! Separate project windows share discovery and its backend-enforced cooldown.
#![cfg(unix)]
use ide_agent::AgentEvent;
use ide_core::NullSink;
use ide_host::{Host, HostConfig, ops};
use std::sync::Arc;
use std::time::Duration;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn windows_share_model_discovery_and_manual_refresh_cooldown() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("project");
let config = tmp.path().join("config");
std::fs::create_dir_all(&root).unwrap();
std::fs::create_dir_all(config.join("agents")).unwrap();
let script = tmp.path().join("catalog.sh");
std::fs::write(
&script,
r#"#!/bin/sh
printf 'launch\n' >> "${0%/*}/launches"
while IFS= read -r line; do
case "$line" in
*'"id":'*) rest=${line#*\"id\":}; id=${rest%%,*}; id=${id%%\}*} ;;
*) continue ;;
esac
case "$line" in
*'"method":"initialize"'*)
printf '{"jsonrpc":"2.0","id":%s,"result":{"protocolVersion":1,"agentCapabilities":{}}}\n' "$id" ;;
*'"method":"session/new"'*)
while [ ! -f "${0%/*}/release" ]; do sleep 0.01; done
printf '{"jsonrpc":"2.0","id":%s,"result":{"sessionId":"catalog","configOptions":[{"id":"model","name":"Model","category":"model","type":"select","currentValue":"new-model","options":[{"value":"new-model","name":"New model"}]}]}}\n' "$id" ;;
esac
done
"#,
)
.unwrap();
std::fs::write(
config.join("agents/catalog-test.toml"),
format!(
"name = \"Catalog test\"\ncommand = \"/bin/sh\"\nargs = [{}]\n",
serde_json::to_string(&script).unwrap()
),
)
.unwrap();
let host = Host::new(HostConfig {
config_dir: config,
data_dir: tmp.path().join("data"),
local_data_dir: tmp.path().join("local"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |window| window,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
host.trust.decide(&root, true, false).unwrap();
host.open_project("first", &root).unwrap();
host.open_project("second", &root).unwrap();
let (send, mut events) = tokio::sync::mpsc::unbounded_channel();
for window in ["first", "second"] {
let send = send.clone();
host.project(window).unwrap().agents.subscribe(Arc::new(move |event| {
if let AgentEvent::ModelCatalog { catalog } = event {
let _ = send.send((window, catalog));
}
}));
}
for _ in 0..20 {
for window in ["first", "second"] {
let state = ops::agent::refresh_models(&host, window, "catalog-test".into(), true).unwrap();
assert!(state.refreshing);
}
}
std::fs::write(tmp.path().join("release"), "").unwrap();
let completed = tokio::time::timeout(Duration::from_secs(10), async {
let mut windows = std::collections::HashSet::new();
while windows.len() < 2 {
let (window, catalog) = events.recv().await.expect("catalog event stream closed");
if !catalog.refreshing && catalog.refreshed_at_ms.is_some() {
assert!(catalog.error.is_none(), "{:?}", catalog.error);
windows.insert(window);
}
}
})
.await;
assert!(completed.is_ok(), "both windows must receive the completed catalog");
for _ in 0..20 {
for window in ["first", "second"] {
let catalog = ops::agent::refresh_models(&host, window, "catalog-test".into(), true).unwrap();
assert!(!catalog.refreshing);
assert!(catalog.retry_at_ms.is_some());
let models = catalog.config.iter().find(|o| o.category.as_deref() == Some("model")).unwrap();
assert_eq!(models.values[0].value, "new-model");
}
}
assert_eq!(std::fs::read_to_string(tmp.path().join("launches")).unwrap(), "launch\n");
}
+174
Переглянути файл
@@ -0,0 +1,174 @@
//! A Compose run target as the Python interpreter, against a real daemon and the real
//! basedpyright. Ignored by default: `OPENIDEA_DOCKER_TEST_DIR` names an empty folder the
//! daemon can bind-mount (on this machine, or at the same path in a VM such as OrbStack's),
//! `docker` with the Compose plugin and `basedpyright-langserver` must be on `PATH`, and
//! this machine's Python must not have Django installed.
#![cfg(unix)]
use ide_core::IdeEvent;
use ide_host::{Host, HostConfig, ops};
use ide_run::{OutputChunk, RunConfig};
use std::path::{Path, PathBuf};
use std::process::Command;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
const WINDOW: &str = "main";
type Events = Arc<Mutex<Vec<IdeEvent>>>;
fn host(base: &Path, events: &Events) -> Arc<Host> {
Arc::new(Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new({
let events = events.clone();
move |_| {
let events = events.clone();
Arc::new(move |e: IdeEvent| events.lock().unwrap().push(e))
}
}),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}))
}
/// basedpyright's latest diagnostics of `path` once `done` accepts them.
async fn diagnostics(events: &Events, path: &Path, timeout: Duration, done: impl Fn(&[String]) -> bool) -> Vec<String> {
let deadline = Instant::now() + timeout;
loop {
let latest = events.lock().unwrap().iter().rev().find_map(|e| match e {
IdeEvent::Diagnostics { path: p, items, source } if **p == *path && source.contains("basedpyright") => {
Some(items.iter().map(|d| d.message.clone()).collect::<Vec<_>>())
}
_ => None,
});
if let Some(messages) = &latest
&& done(messages)
{
return messages.clone();
}
assert!(Instant::now() < deadline, "timed out; last diagnostics: {latest:?}");
tokio::time::sleep(Duration::from_millis(500)).await;
}
}
fn unresolved(messages: &[String]) -> bool {
messages.iter().any(|m| m.contains("\"django") && m.contains("could not be resolved"))
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "requires a Docker daemon, basedpyright and OPENIDEA_DOCKER_TEST_DIR"]
async fn compose_target_is_the_python_interpreter() {
let root = PathBuf::from(std::env::var("OPENIDEA_DOCKER_TEST_DIR").expect("OPENIDEA_DOCKER_TEST_DIR"));
let root = dunce::canonicalize(root).unwrap();
let files = [
("compose.yaml", "services:\n web:\n build: .\n volumes:\n - .:/app\n working_dir: /app\n"),
(
"Dockerfile",
"FROM python:3.12-slim\nCOPY requirements.txt /tmp/requirements.txt\n\
RUN pip install --no-cache-dir -r /tmp/requirements.txt\n",
),
("requirements.txt", "Django==5.1.3\n"),
(
"app/main.py",
"import django\nfrom django.http import HttpResponse\n\nprint(django.get_version(), HttpResponse)\n",
),
];
for (path, text) in files {
std::fs::create_dir_all(root.join(path).parent().unwrap()).unwrap();
std::fs::write(root.join(path), text).unwrap();
}
let main = root.join("app/main.py");
let state = tempfile::tempdir().unwrap();
let events: Events = Arc::default();
let host = host(state.path(), &events);
host.trust.decide(&root, true, false).unwrap();
host.open_project(WINDOW, &root).unwrap();
let set = |key: &str, value: serde_json::Value| {
ops::project::set_settings(
&host,
WINDOW,
ide_workspace::settings::SettingsScope::Project,
key,
&value.to_string(),
)
.unwrap();
};
set("runTargets", serde_json::json!([{ "kind": "compose", "id": "web", "name": "Compose web", "service": "web" }]));
// A local interpreter: Django is only in the image.
ops::doc::open(&host, WINDOW, main.clone()).await.unwrap();
let before = diagnostics(&events, &main, Duration::from_secs(120), unresolved).await;
eprintln!("local interpreter: {before:?}");
// The run target as the interpreter: a snapshot is taken in the background, then the
// server restarts with the copied site-packages as `extraPaths`.
set("languages.python.interpreter", serde_json::json!("target:web"));
let after = diagnostics(&events, &main, Duration::from_secs(900), |m| !unresolved(m)).await;
eprintln!("run target interpreter: {after:?}");
assert!(after.iter().all(|m| !m.contains("could not be resolved")), "{after:?}");
let info = ops::sdk::sdk_container_python(&host, WINDOW).unwrap().expect("a run target interpreter");
eprintln!("snapshot: version {:?}, image {:?}, folders {:?}", info.version, info.image, info.folders);
assert_eq!(info.name, "Compose web");
assert!(info.version.as_deref().is_some_and(|v| v.starts_with("3.12.")), "{info:?}");
assert!(info.packages.iter().any(|p| p.name == "Django" && p.version == "5.1.3"), "{:?}", info.packages);
let site = info.folders.iter().find(|f| f.0.ends_with("site-packages")).expect("site-packages copied");
assert!(site.0.join("django/__init__.py").is_file());
assert!(site.0.starts_with(state.path().join("data/container-python")), "{site:?}");
let servers = ops::lsp::lsp_servers(&host, WINDOW).unwrap();
eprintln!("servers: {:?}", servers.iter().map(|s| (&s.name, s.status)).collect::<Vec<_>>());
// Run: a Python configuration without its own target runs in the container.
let out = Arc::new(Mutex::new(String::new()));
let sink = out.clone();
let config = RunConfig {
id: "main".into(),
name: "main".into(),
kind: "python".into(),
command: "python3".into(),
args: vec!["app/main.py".into()],
cwd: None,
env: Default::default(),
detected: false,
is_test: false,
debug_adapter: None,
allow_parallel: false,
extras: Default::default(),
};
ops::run::start(
&host,
WINDOW,
config,
Arc::new(move |chunk| match chunk {
OutputChunk::Stdout { text } | OutputChunk::Stderr { text } | OutputChunk::System { text } => {
sink.lock().unwrap().push_str(&text)
}
}),
)
.await
.unwrap();
for _ in 0..1200 {
if out.lock().unwrap().contains("Process finished") {
break;
}
tokio::time::sleep(Duration::from_millis(100)).await;
}
let out = out.lock().unwrap().clone();
eprintln!("run output: {out}");
assert!(out.contains("5.1.3") && out.contains("exit code 0"), "{out}");
// Refresh takes a new snapshot of the same image; nothing is left running.
let refreshed = ops::sdk::sdk_container_python_refresh(&host, WINDOW).await.unwrap().unwrap();
assert_eq!(refreshed.image, info.image);
let probes = Command::new("docker")
.args(["ps", "--all", "--quiet", "--filter", "label=openidea.python-probe=true"])
.output()
.unwrap();
assert_eq!(String::from_utf8_lossy(&probes.stdout).trim(), "", "probe containers are removed");
Command::new("docker").args(["compose", "down", "--rmi", "local"]).current_dir(&root).status().unwrap();
}
+37
Переглянути файл
@@ -0,0 +1,37 @@
use ide_core::NullSink;
use ide_host::ops::db;
use ide_host::{Host, HostConfig};
use std::sync::Arc;
#[tokio::test]
async fn new_query_consoles_are_numbered_after_the_default_one() {
let tmp = tempfile::tempdir().unwrap();
let base = tmp.path();
let host = Arc::new(Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
let names = |paths: Vec<ide_core::IpcPath>| -> Vec<String> {
paths.iter().map(|p| p.0.file_name().unwrap().to_string_lossy().into_owned()).collect()
};
assert!(db::consoles(&host, "pg", false).unwrap().is_empty());
// The first new console is the default one, then console_1, console_2…
assert_eq!(names(db::consoles(&host, "pg", true).unwrap()), ["console.sql"]);
assert_eq!(names(db::consoles(&host, "pg", true).unwrap()), ["console.sql", "console_1.sql"]);
let all = db::consoles(&host, "pg", true).unwrap();
assert_eq!(names(all.clone()), ["console.sql", "console_1.sql", "console_2.sql"]);
// Numbers sort numerically, and the default console path is the first one.
std::fs::write(all[0].0.with_file_name("console_10.sql"), "").unwrap();
assert_eq!(names(db::consoles(&host, "pg", false).unwrap())[3], "console_10.sql");
assert_eq!(db::console_path(&host, "pg").unwrap().0, all[0].0);
// Each database of a source has its own consoles.
assert!(db::consoles(&host, "pg:other", false).unwrap().is_empty());
assert!(db::consoles(&host, "../x", true).is_err());
}
+70
Переглянути файл
@@ -179,3 +179,73 @@ async fn light_notebooks_open_without_interpreter_detection_or_execution() {
assert!(!bin.join("python.probed").exists());
host.shutdown().await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn untitled_buffers_reach_the_disk_through_save_as() {
let tmp = tempfile::tempdir().unwrap();
let tmp_root = dunce::canonicalize(tmp.path()).unwrap();
let dir = tmp_root.join("notes");
let host = host(&tmp_root.join("app"));
// A window with no file (a welcome screen's File › New File): rooted at the
// home folder, no watcher yet, doc commands work.
let empty = host.open_light_empty("main").unwrap();
assert!(empty.light);
let untitled = ops::doc::open_untitled(&host, "main", None).await.unwrap();
assert_eq!(untitled.path, std::path::PathBuf::from("untitled:Untitled-1"));
assert_eq!(untitled.language.as_str(), "plaintext");
let version = ops::doc::edit(&host, "main", untitled.id, untitled.version, untitled_edit("# notes\n")).unwrap();
assert!(ops::doc::save(&host, "main", untitled.id).await.is_ok(), "saving nowhere is a no-op");
assert!(host.project("main").unwrap().documents.is_dirty(untitled.id).unwrap());
// Save As: the buffer lands on the disk and turns into a file document.
let target = dir.join("deep/notes.md");
let saved = ops::doc::save_as(&host, "main", untitled.id, target.clone()).await.unwrap();
assert_eq!(saved.path, target);
assert_eq!(saved.language.as_str(), "markdown");
assert!(!saved.dirty);
// The platform's default line separator applies to new buffers.
assert_eq!(std::fs::read_to_string(&target).unwrap().replace("\r\n", "\n"), "# notes\n");
ops::doc::edit(&host, "main", untitled.id, version, untitled_edit("x\n")).unwrap();
ops::doc::save(&host, "main", untitled.id).await.unwrap();
assert_eq!(std::fs::read_to_string(&target).unwrap().replace("\r\n", "\n"), "x\n# notes\n");
// The saved file's folder is followed for external changes now.
host.shutdown().await;
}
fn untitled_edit(insert: &str) -> ChangeSet {
ChangeSet { changes: vec![TextChange { from: 0, to: 0, insert: insert.into() }] }
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn untitled_buffers_follow_the_window_into_a_project() {
let tmp = tempfile::tempdir().unwrap();
let tmp_root = dunce::canonicalize(tmp.path()).unwrap();
let host = host(&tmp_root.join("app"));
// A LightEdit window with an edited untitled buffer (and a clean one).
host.open_light_empty("main").unwrap();
let draft = ops::doc::open_untitled(&host, "main", None).await.unwrap();
ops::doc::edit(&host, "main", draft.id, draft.version, untitled_edit("{\"draft\": true}")).unwrap();
let clean = ops::doc::open_untitled(&host, "main", None).await.unwrap();
// The window switches to a project: the buffers follow, contents and dirty flags kept.
let project_root = tmp_root.join("proj");
std::fs::create_dir_all(&project_root).unwrap();
let project = host.open_project("main", &project_root).unwrap();
assert!(!project.light);
let carried =
ops::doc::open_untitled(&host, "main", Some(draft.path.to_string_lossy().into_owned())).await.unwrap();
assert_eq!(carried.path, draft.path, "same key, fresh registry: a new doc id carries the content");
let snap = ops::doc::snapshot(&host, "main", carried.id).await.unwrap();
assert_eq!(snap.text, "{\"draft\": true}");
assert!(host.project("main").unwrap().documents.is_dirty(carried.id).unwrap());
assert_eq!(snap.language.as_str(), "plaintext");
let clean_carried =
ops::doc::open_untitled(&host, "main", Some(clean.path.to_string_lossy().into_owned())).await.unwrap();
assert!(!host.project("main").unwrap().documents.is_dirty(clean_carried.id).unwrap());
host.shutdown().await;
}
+141
Переглянути файл
@@ -0,0 +1,141 @@
//! The native `nav_inheritance` command: it must return every Python class definition
//! in the project — attached dependency roots included — together with the base names
//! written in its source, so the UI can compute inheritance gutter markers without a
//! language server's type-hierarchy provider.
use ide_host::{Host, HostConfig, ops};
use std::path::PathBuf;
use std::sync::Arc;
use std::time::{Duration, Instant};
const WINDOW: &str = "main";
struct Opened {
host: Arc<Host>,
root: PathBuf,
_tmp: tempfile::TempDir,
}
/// Opens a synthetic project whose Python files carry known class / base shapes, then
/// waits for the index to settle (it walks content roots, so dependency roots are covered).
async fn open() -> Opened {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap().join("proj");
let src = root.join("src");
let dep = root.join("vendor");
std::fs::create_dir_all(&src).unwrap();
std::fs::create_dir_all(&dep).unwrap();
// Own root: a plain class, a class with two bases + a keyword arg, a qualified base,
// and a subscripted generic base. A comment / docstring that mentions `class X(Y)`
// must not be reported.
std::fs::write(
src.join("app.py"),
concat!(
"class Base:\n",
" pass\n",
"\n",
"class Mixin:\n",
" pass\n",
"\n",
"class Child(Base, Mixin, metaclass=Meta):\n",
" # class Ghost(Spook): must be ignored\n",
" \"\"\"class Phantom(Apparition): must be ignored\"\"\"\n",
" pass\n",
"\n",
"class Qualified(pkg.mod.Base):\n",
" pass\n",
"\n",
"class Generic(Generic[T]):\n",
" pass\n",
),
)
.unwrap();
// A second own-root file to prove the command scans more than one file.
std::fs::write(src.join("extra.pyi"), "class Stub(Base):\n ...\n").unwrap();
// An attached dependency root (content root) — must be covered by the index.
std::fs::write(dep.join("dep.py"), "class DepBase:\n pass\n").unwrap();
let app = tmp.path().join("app");
let host = Arc::new(Host::new(HostConfig {
config_dir: app.join("config"),
data_dir: app.join("data"),
local_data_dir: app.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(|_| {})),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
}));
host.trust.decide(&root, true, false).unwrap();
ops::project::open(&host, WINDOW, &root).await.unwrap();
let deadline = Instant::now() + Duration::from_secs(30);
loop {
let s = ops::nav::index_status(&host, WINDOW).unwrap();
if !s.indexing && s.files > 0 {
break;
}
assert!(Instant::now() < deadline, "indexing timed out; status: {s:?}");
tokio::time::sleep(Duration::from_millis(100)).await;
}
Opened { host, root, _tmp: tmp }
}
fn find<'a>(all: &'a [ide_host::ops::nav::ClassInheritance], name: &str) -> &'a ide_host::ops::nav::ClassInheritance {
all.iter().find(|c| c.name == name).unwrap_or_else(|| panic!("no class {name:?} in {all:#?}"))
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn returns_all_python_classes_and_bases() {
let p = open().await;
let all = ops::nav::nav_inheritance(&p.host, WINDOW).await.unwrap();
// Both own-root Python files are scanned (src/app.py + src/extra.pyi).
let base = find(&all, "Base");
assert!(base.path.0.ends_with("src/app.py"), "Base path: {:?}", base.path);
// The location is the class name identifier (selection-range start), i.e. after `class `.
assert_eq!(base.line, 0);
assert_eq!(base.col, 6);
assert!(base.bases.is_empty(), "a bare class has no bases: {:?}", base.bases);
let child = find(&all, "Child");
assert_eq!(child.line, 6);
assert_eq!(child.col, 6);
// The `metaclass=Meta` keyword argument is not a base.
assert!(child.bases.contains(&"Base".to_string()), "Child bases: {:?}", child.bases);
assert!(child.bases.contains(&"Mixin".to_string()), "Child bases: {:?}", child.bases);
assert!(!child.bases.iter().any(|b| b == "Meta"), "metaclass must be skipped: {:?}", child.bases);
// A qualified base keeps only its last segment.
let qualified = find(&all, "Qualified");
assert_eq!(qualified.bases, vec!["Base".to_string()], "qualified base: {:?}", qualified.bases);
// A subscripted generic base keeps the identifier.
let generic = find(&all, "Generic");
assert_eq!(generic.bases, vec!["Generic".to_string()], "generic base: {:?}", generic.bases);
// Comments / docstrings are not mistaken for class definitions.
assert!(!all.iter().any(|c| c.name == "Ghost" || c.name == "Phantom"), "parsed a comment: {all:#?}");
// The .pyi stub in the second own-root file is included.
let stub = find(&all, "Stub");
assert!(stub.path.0.ends_with("src/extra.pyi"), "Stub path: {:?}", stub.path);
assert_eq!(stub.bases, vec!["Base".to_string()]);
// The attached dependency root is covered by the project index.
let dep = find(&all, "DepBase");
assert!(dep.path.0.starts_with(&p.root), "DepBase path: {:?}", dep.path);
// Every returned entry is a Python file.
assert!(all.iter().all(|c| { matches!(c.path.0.extension().and_then(|e| e.to_str()), Some("py" | "pyi")) }));
// A file changed on disk is parsed again; the others come from the cache.
let extra = p.root.join("src/extra.pyi");
std::fs::write(&extra, "class Stub(Child): ...\nclass Added(Base): ...\n").unwrap();
let again = ops::nav::nav_inheritance(&p.host, WINDOW).await.unwrap();
assert_eq!(find(&again, "Stub").bases, vec!["Child".to_string()]);
assert_eq!(find(&again, "Added").bases, vec!["Base".to_string()]);
assert_eq!(find(&again, "Child").bases, child.bases);
}
+4 -1
Переглянути файл
@@ -82,7 +82,9 @@ async fn serves_projects_files_documents_search_and_runs() {
invoke(&peer, "lsp_resend_diagnostics", json!({})).await.unwrap();
let files = invoke(&peer, "find_files", json!({"query": "main", "limit": 10})).await.unwrap();
assert!(files.is_array());
for command in ["build_sync", "run_start", "sdk_locate_tools", "notebook_execute", "debug_start"] {
for command in
["build_sync", "run_start", "sdk_locate_tools", "notebook_execute", "debug_start", "agent_refresh_models"]
{
let error = invoke(&peer, command, json!({})).await.unwrap_err();
assert!(error.message.contains("Restricted Mode"), "{command}: {error:?}");
}
@@ -365,6 +367,7 @@ async fn local_deployment_runs_on_the_project_host() {
"db_table_page",
"db_apply_edits",
"db_console_path",
"db_consoles",
"ssh_test",
"ssh_set_secret",
"ssh_password_saved",
+2
Переглянути файл
@@ -66,6 +66,7 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"docker_set_registry_password",
"docker_run_env",
"db_console_path",
"db_consoles",
"db_export_file",
"vcs_write_patch",
"remote_bootstrap_list",
@@ -92,6 +93,7 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"vcs_git_version",
"docker_compose_op",
"db_execute",
"agent_refresh_models",
"remote_delete",
"remote_mkdir",
"remote_rename",
+22
Переглянути файл
@@ -599,6 +599,28 @@ impl Store {
txn.commit().map_err(err)
}
/// [`Self::apply`] for records already serialized (by the parse pool).
pub(crate) fn apply_raw(&self, puts: &[(String, Vec<u8>)], deletes: &[String], durable: bool) -> Result<()> {
self.ensure_db()?;
let db = self.db.read();
let mut txn = db.as_ref().expect("ensured").begin_write().map_err(err)?;
if !durable {
txn.set_durability(Durability::None).map_err(err)?;
}
{
let mut table = txn.open_table(FILES).map_err(err)?;
for (rel, bytes) in puts {
table.insert(rel.as_str(), bytes.as_slice()).map_err(err)?;
}
for rel in deletes {
table.remove(rel.as_str()).map_err(err)?;
}
}
// Any file change makes the columnar start-up snapshot stale.
let _ = std::fs::remove_file(&self.snapshot);
txn.commit().map_err(err)
}
/// Reclaims free pages in the database file; `true` if anything was compacted.
pub(crate) fn compact(&self) -> Result<bool> {
self.ensure_db()?;
+164 -48
Переглянути файл
@@ -17,6 +17,8 @@ use std::time::{Duration, Instant, SystemTime};
/// Files larger than this are listed but not parsed for symbols.
pub const MAX_SYMBOL_FILE: u64 = 2 * 1024 * 1024;
const CHUNK: usize = 512;
/// Insert/persist batch of the initial scan's streaming pass.
const SCAN_BATCH: usize = 2048;
const PROGRESS_EVERY: Duration = Duration::from_millis(250);
#[cfg(not(test))]
const SNAPSHOT_DELAY: Duration = Duration::from_secs(2);
@@ -436,10 +438,13 @@ impl Shared {
});
let filter = self.filter();
let walk_started = Instant::now();
let found = prewalked.unwrap_or_else(|| self.walk_roots(dirs));
let walk = walk_started.elapsed();
if self.is_cancelled() {
return self.end_scan(task);
}
let classify_started = Instant::now();
let seen: HashSet<&Path> = found.iter().map(|f| f.path.as_path()).collect();
let (todo, stale): (Vec<&Found>, Vec<PathBuf>) = {
let d = self.data.read();
@@ -452,6 +457,7 @@ impl Shared {
.collect();
(todo, stale)
};
let classify = classify_started.elapsed();
if !stale.is_empty() {
let mut d = self.data.write();
for path in &stale {
@@ -462,33 +468,76 @@ impl Shared {
self.persist(&[], &stale, false);
let total = todo.len();
let mut done = 0usize;
let mut last_progress = Instant::now();
for chunk in todo.chunks(CHUNK) {
if self.is_cancelled() {
return self.end_scan(task);
}
self.index_files(chunk.iter().copied(), false);
done += chunk.len();
if last_progress.elapsed() >= PROGRESS_EVERY {
last_progress = Instant::now();
self.sink.emit(IdeEvent::Task {
id: task,
title: "Indexing".into(),
message: Some(format!("{done} / {total} files")),
fraction: Some(done as f32 / total.max(1) as f32),
done: false,
cancellable: false,
let mut laps = ChunkLaps::default();
{
// One streaming parallel pass: parse on the rayon pool without
// per-chunk joins (a straggler file at a join's tail idles every
// other core), while parsed files flow to this thread to be
// inserted and persisted in batches. `filter` moves into the
// parse thread; this thread keeps the outer one.
let filter = self.filter();
// Bounded: when inserting and persisting fall behind, the parse waits instead
// of holding the whole project's records (and their bytes) in the channel.
let (tx, rx) = std::sync::mpsc::sync_channel::<ParsedFile>(2 * SCAN_BATCH);
let cancelled = &self.cancelled;
let mut done = 0usize;
let mut last_progress = Instant::now();
let pass_started = Instant::now();
std::thread::scope(|scope| -> () {
scope.spawn(move || {
todo.par_iter().for_each(|f| {
if cancelled.load(Ordering::Relaxed) {
return;
}
let Some(out) = self.parse_found(f, &filter, false) else { return };
let _ = tx.send(out);
});
// `tx` drops here: the consumer's loop ends with the channel.
});
}
let mut batch: Vec<ParsedFile> = Vec::with_capacity(SCAN_BATCH);
for out in rx {
if self.is_cancelled() {
break;
}
batch.push(out);
if batch.len() < SCAN_BATCH {
continue;
}
done += batch.len();
laps.add(self.publish(batch.drain(..)));
if last_progress.elapsed() >= PROGRESS_EVERY {
last_progress = Instant::now();
self.sink.emit(IdeEvent::Task {
id: task,
title: "Indexing".into(),
message: Some(format!("{done} / {total} files")),
fraction: Some(done as f32 / total.max(1) as f32),
done: false,
cancellable: false,
});
}
}
if !batch.is_empty() && !self.is_cancelled() {
done += batch.len();
laps.add(self.publish(batch.drain(..)));
}
});
laps.extract = pass_started.elapsed();
}
if self.is_cancelled() {
return self.end_scan(task);
}
let durable_started = Instant::now();
self.persist(&[], &[], true);
let durable = durable_started.elapsed();
if let Some(store) = &self.store {
let recent: Vec<String> = self.recent.lock().iter().filter_map(|p| filter.store_key(p)).collect();
let d = self.data.read();
let snapshot_started = Instant::now();
if let Err(e) = store.save_snapshot(&d, &filter, guard) {
tracing::warn!("index snapshot: {e}");
}
tracing::info!("index snapshot: written in {:?}", snapshot_started.elapsed());
if let Err(e) = store.save_recent_file(&recent) {
tracing::warn!("index recent snapshot: {e}");
}
@@ -506,54 +555,85 @@ impl Shared {
self.emit_state();
let d = self.data.read();
tracing::info!(
"indexed {} files ({} parsed, {} symbols) in {:?}",
"indexed {} files ({} parsed, {} symbols) in {:?} (walk {:?}, classify {:?}, extract {:?}, insert {:?}, persist {:?}, durable {:?})",
d.file_count,
total,
d.symbol_count,
started.elapsed()
started.elapsed(),
walk,
classify,
laps.extract,
laps.insert,
laps.persist,
durable
);
}
/// Extracts records in parallel, then publishes them to memory and disk.
/// `force` re-parses files whose content is unchanged (their language changed).
fn index_files<'a>(&self, files: impl Iterator<Item = &'a Found>, force: bool) {
fn index_files<'a>(&self, files: impl Iterator<Item = &'a Found>, force: bool) -> ChunkLaps {
let mut laps = ChunkLaps::default();
let files: Vec<&Found> = files.collect();
let filter = self.filter();
let extracted: Vec<(String, FileRecord, PathBuf)> = files
.par_iter()
.filter_map(|f| {
if self.is_cancelled() {
return None;
}
let rel = filter.rel(&f.path)?;
let prev_hash = if force { None } else { self.data.read().get(&f.path).map(|e| e.hash) };
match self.extract(f, prev_hash) {
Extracted::Unchanged => {
// Same content, new mtime: keep the symbols, refresh metadata.
let mut rec = self.data.read().record_of(&f.path)?;
rec.mtime = f.mtime;
rec.size = f.size;
rec.ignored = f.ignored;
Some((rel, rec, f.path.clone()))
}
Extracted::New(rec) => Some((rel, rec, f.path.clone())),
}
})
.collect();
let started = Instant::now();
let extracted: Vec<ParsedFile> = files.par_iter().filter_map(|f| self.parse_found(f, &filter, force)).collect();
laps.extract = started.elapsed();
if extracted.is_empty() {
return;
return laps;
}
laps.add(self.publish(extracted.into_iter()));
laps
}
/// Reads and parses one file (parallel-safe), serializing its record for
/// the cache alongside the parse. `None` when the file is outside the
/// scope or the scan was cancelled.
fn parse_found(&self, f: &Found, filter: &std::sync::Arc<PathFilter>, force: bool) -> Option<ParsedFile> {
if self.is_cancelled() {
return None;
}
let rel = filter.rel(&f.path)?;
let prev_hash = if force { None } else { self.data.read().get(&f.path).map(|e| e.hash) };
let rec = match self.extract(f, prev_hash) {
Extracted::Unchanged => {
// Same content, new mtime: keep the symbols, refresh metadata.
let mut rec = self.data.read().record_of(&f.path)?;
rec.mtime = f.mtime;
rec.size = f.size;
rec.ignored = f.ignored;
rec
}
Extracted::New(rec) => rec,
};
let key = filter.store_key(&f.path);
let bytes = postcard::to_allocvec(&rec).ok();
Some(ParsedFile { rel, rec, key, bytes, path: f.path.clone() })
}
/// Inserts a parsed batch into the in-memory index and persists the records.
fn publish(&self, batch: impl Iterator<Item = ParsedFile>) -> ChunkLaps {
let mut laps = ChunkLaps::default();
let started = Instant::now();
let mut entries: Vec<(FileEntry, Vec<StoredSymbol>)> = Vec::new();
let mut puts: Vec<(String, Vec<u8>)> = Vec::new();
for out in batch {
let ParsedFile { rel, rec, key, bytes, path } = out;
if let (Some(key), Some(bytes)) = (key, bytes) {
puts.push((key, bytes));
}
entries.push(FileEntry::new(path, rel, rec));
}
let puts: Vec<(String, FileRecord)> =
extracted.iter().filter_map(|(_, rec, path)| Some((filter.store_key(path)?, rec.clone()))).collect();
{
let entries: Vec<(FileEntry, Vec<StoredSymbol>)> =
extracted.into_iter().map(|(rel, rec, path)| FileEntry::new(path, rel, rec)).collect();
let mut d = self.data.write();
for (e, symbols) in entries {
d.insert(e, symbols);
}
}
self.persist(&puts, &[], false);
laps.insert = started.elapsed();
let started = Instant::now();
self.persist_raw(&puts, &[], false);
laps.persist = started.elapsed();
laps
}
fn schedule_snapshot(&self) {
@@ -594,6 +674,15 @@ impl Shared {
}
}
/// [`Self::persist`] for records already serialized by the parse pool.
fn persist_raw(&self, puts: &[(String, Vec<u8>)], deletes: &[String], durable: bool) {
if let Some(store) = &self.store
&& let Err(e) = store.apply_raw(puts, deletes, durable)
{
tracing::warn!("index cache write failed: {e}");
}
}
fn extract(&self, f: &Found, prev_hash: Option<[u8; 32]>) -> Extracted {
let mut rec = FileRecord { mtime: f.mtime, size: f.size, ignored: f.ignored, ..Default::default() };
if f.ignored || f.size > MAX_SYMBOL_FILE {
@@ -793,6 +882,33 @@ enum Extracted {
New(FileRecord),
}
/// Wall-time laps of one `index_files` chunk, accumulated by the scan.
#[derive(Default)]
struct ChunkLaps {
extract: Duration,
insert: Duration,
persist: Duration,
}
impl ChunkLaps {
fn add(&mut self, other: ChunkLaps) {
self.extract += other.extract;
self.insert += other.insert;
self.persist += other.persist;
}
}
/// One file parsed off the index thread: its record ready to publish, already
/// serialized for the cache (in parallel with the parse).
struct ParsedFile {
rel: String,
rec: FileRecord,
/// Cache key and postcard bytes; `None` when outside the persisted root.
key: Option<String>,
bytes: Option<Vec<u8>>,
path: PathBuf,
}
#[cfg(test)]
mod tests {
use crate::data::safe_rel;
+48 -4
Переглянути файл
@@ -80,18 +80,47 @@ pub fn trim_line_end(line: &[u8]) -> &[u8] {
&line[..end]
}
/// The longest line kept in full; the rest of a longer one is dropped (a server that writes a
/// progress bar with `\r` only, or never a newline, must not grow memory without bound).
pub const MAX_LINE: usize = 8 * 1024;
/// Reads `stream` to its end, calling `on_line` with each line decoded by [`decode`].
/// Unlike `AsyncBufReadExt::lines`, a line that is not UTF-8 does not end the stream.
/// A line longer than [`MAX_LINE`] bytes is cut there and ends with `…`.
pub async fn read_lines<R: AsyncRead + Unpin>(stream: R, mut on_line: impl FnMut(String)) {
let mut reader = BufReader::new(stream);
let mut line = Vec::new();
let mut cut = false;
loop {
line.clear();
match reader.read_until(b'\n', &mut line).await {
Ok(0) | Err(_) => break,
Ok(_) => on_line(decode(trim_line_end(&line))),
let (used, complete) = match reader.fill_buf().await {
Ok([]) | Err(_) => break,
Ok(buf) => {
let end = buf.iter().position(|&b| b == b'\n');
let piece = &buf[..end.unwrap_or(buf.len())];
let room = MAX_LINE.saturating_sub(line.len());
cut |= piece.len() > room;
line.extend_from_slice(&piece[..piece.len().min(room)]);
(end.map_or(buf.len(), |i| i + 1), end.is_some())
}
};
reader.consume(used);
if complete {
emit(&mut line, &mut cut, &mut on_line);
}
}
if !line.is_empty() || cut {
emit(&mut line, &mut cut, &mut on_line);
}
}
fn emit(line: &mut Vec<u8>, cut: &mut bool, on_line: &mut impl FnMut(String)) {
let mut text = decode(trim_line_end(line));
if *cut {
text.push('…');
}
line.clear();
*cut = false;
on_line(text);
}
#[cfg(test)]
@@ -133,4 +162,19 @@ mod tests {
assert!(lines[1].ends_with(" broken"));
assert_eq!(lines[2], "last");
}
#[tokio::test]
async fn a_very_long_line_is_cut_and_the_next_one_is_intact() {
let mut input = vec![b'x'; MAX_LINE * 10 + 3];
input.extend_from_slice(b"\nnext\r\n");
// No newline at all after this one: still bounded.
input.extend_from_slice(&vec![b'y'; MAX_LINE * 3]);
let mut lines = Vec::new();
read_lines(&input[..], |l| lines.push(l)).await;
assert_eq!(lines.len(), 3);
assert_eq!(lines[0].chars().count(), MAX_LINE + 1);
assert!(lines[0].ends_with('…') && lines[0].starts_with("xxx"));
assert_eq!(lines[1], "next");
assert_eq!(lines[2].chars().count(), MAX_LINE + 1);
}
}
+56
Переглянути файл
@@ -0,0 +1,56 @@
; Shell scripts: drawn over the stream mode, which knows only fixed word lists.
; The last matching pattern wins.
[
(string)
(raw_string)
(ansi_c_string)
(translated_string)
(heredoc_body)
(heredoc_start)
] @string
[
(command_substitution)
(process_substitution)
(expansion)
(simple_expansion)
] @embedded
(comment) @comment
[
"case"
"do"
"done"
"elif"
"else"
"esac"
"fi"
"for"
"function"
"if"
"in"
"select"
"then"
"until"
"while"
"declare"
"export"
"local"
"readonly"
"typeset"
"unset"
"unsetenv"
] @keyword
(command_name) @function.command
(function_definition name: (word) @function.definition)
(variable_name) @variable
(special_variable_name) @variable
[
(number)
(file_descriptor)
] @number
+10 -5
Переглянути файл
@@ -22,12 +22,13 @@ pub use markers::{arduino_sketch_root, is_arduino_sketch_dir, marked_root};
pub use plugin_point::PluginLanguages;
pub use registry::{FileTypes, LanguageRegistry};
pub use syntax::{
CSS_CLASS, CSS_ID, CSS_VARIABLE, ConfigurationKey, ExternalFold, ExternalSymbol, ExternalSyntax, ExternalToken,
FileReference, Literal, LiteralKind, MAX_CACHED_TREES, MAX_PARSE_BYTES, ParsedTree, SpellingKind, SpellingRegion,
Syntax, SyntaxCache, SyntaxCacheStats, SyntaxHighlightKind, SyntaxHighlights, TreeError, VueComponentImport,
VueRefAccess, VueRefElement, WebReference, WebReferenceKind, merge_visibility, set_external_syntax, shift_pos,
shift_range,
CSS_CLASS, CSS_ID, CSS_VARIABLE, ClassBases, ConfigurationKey, ExternalFold, ExternalSymbol, ExternalSyntax,
ExternalToken, FileReference, Literal, LiteralKind, MAX_CACHED_TREES, MAX_PARSE_BYTES, ParsedTree, SpellingKind,
SpellingRegion, Syntax, SyntaxCache, SyntaxCacheStats, SyntaxHighlightKind, SyntaxHighlights, TreeError,
VueComponentImport, VueRefAccess, VueRefElement, WebReference, WebReferenceKind, merge_visibility,
set_external_syntax, shift_pos, shift_range,
};
pub use syntax::{MethodBody, PythonClassFacts, PythonFacts, PythonMethodFacts};
/// Parsed form of `languages/<id>.toml`. Backend-only; the UI gets [`LanguageInfo`].
#[derive(Clone, Debug, Serialize, Deserialize)]
@@ -198,6 +199,10 @@ pub struct LanguageServerConfig {
/// it parses a document.
#[serde(default)]
pub resync_on_configuration: bool,
/// Set by the settings hook: what the user sees as the server when another program serves
/// in its place (TypeScript 7's own `tsc --lsp` for vtsls).
#[serde(skip)]
pub display_name: Option<String>,
/// Set on plugin languages: start exactly this approved program.
#[serde(skip)]
pub pinned: Option<ProgramPin>,
+1 -1
Переглянути файл
@@ -314,7 +314,7 @@ pub(crate) static SPECS: &[Spec] = &[
name: "bash",
language: tree_sitter_bash::LANGUAGE,
outline: outline!("bash"),
highlights: None,
highlights: Some(Highlights { query: highlights!("bash"), first_wins: false }),
fold_nodes: &["do_group", "if_statement", "case_statement", "heredoc_body"],
fold_bodies: NONE,
imports: NONE,
+29 -1
Переглянути файл
@@ -43,11 +43,13 @@ pub enum SyntaxHighlightKind {
Operator,
Punctuation,
Parameter,
/// A shell command name (`pgrep`, a script's own function).
Command,
}
impl SyntaxHighlightKind {
/// Every kind, in `as u32` order: the legend for [`SyntaxHighlights::data`].
pub const ALL: [Self; 25] = [
pub const ALL: [Self; 26] = [
Self::Keyword,
Self::String,
Self::StringSpecial,
@@ -73,6 +75,7 @@ impl SyntaxHighlightKind {
Self::Operator,
Self::Punctuation,
Self::Parameter,
Self::Command,
];
/// The kebab-case name plugins use (`doc-comment`, `function-call`, …).
@@ -153,6 +156,7 @@ const CAPTURES: &[(&str, Paint)] = {
("function.builtin", Kind(K::Builtin)),
("function.macro", Kind(K::Macro)),
("function.definition", Kind(K::Function)),
("function.command", Kind(K::Command)),
("function", Kind(K::FunctionCall)),
("method", Kind(K::FunctionCall)),
("constructor", Kind(K::Type)),
@@ -408,6 +412,30 @@ mod tests {
}
}
#[test]
fn shell_scripts_highlight_declarations_commands_and_function_names() {
use SyntaxHighlightKind as K;
let src = "this::check_process() {\n local process\n process=\"${1:?}\"\n if [ -x \"$(command -v pgrep)\" ]; then\n if pgrep -f -q \"${process}\"; then\n functions::exit 1 \"'${process}' is running\"\n fi\n fi\n}\n# done\n";
let got = spans("shellscript", src);
for (text, expected) in [
("this::check_process", K::Function),
("local", K::Keyword),
("if", K::Keyword),
("command", K::Command),
("pgrep", K::Command),
("functions::exit", K::Command),
("process", K::Variable),
("# done", K::Comment),
] {
assert!(
got.iter().any(|(word, kind)| word == text && *kind == expected),
"missing {text}: {expected:?} in {got:?}"
);
}
// An expansion inside a string is not string-coloured.
assert!(!got.iter().any(|(word, kind)| word.contains("${process}") && *kind == K::String), "{got:?}");
}
#[test]
fn non_kotlin_parameters_keep_variable_highlighting() {
for (lang, src) in [
+176
Переглянути файл
@@ -0,0 +1,176 @@
//! Extraction of Python class base names, used to build an inheritance graph
//! without an LSP `typeHierarchyProvider`.
use super::lines::LineIndex;
use crate::{LanguageId, Syntax};
use tree_sitter::{Node, Tree};
/// A Python class definition and the base names written in its source.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ClassBases {
/// The class name (bare identifier).
pub name: String,
/// 0-based line of the class name (selection-range start).
pub line: u32,
/// 0-based UTF-16 column of the class name (selection-range start).
pub col: u32,
/// Source-written base names (last segment of a qualified name).
pub bases: Vec<String>,
}
impl Syntax {
/// Class definitions of `text` with their base names, in source order.
/// Only Python is supported; every other language returns an empty vector.
pub fn class_bases(&self, lang: &LanguageId, text: &str) -> Vec<ClassBases> {
self.analyze(lang, text, |g, tree, text| (g.spec.name == "python").then(|| extract(tree, text)))
.unwrap_or_default()
}
}
fn extract(tree: &Tree, text: &str) -> Vec<ClassBases> {
let idx = LineIndex::new(text);
let mut out = Vec::new();
let mut cursor = tree.walk();
visit(&mut cursor, &mut |node| {
if node.kind() != "class_definition" {
return;
}
let Some(name_node) = node.child_by_field_name("name") else { return };
let name = text.get(name_node.byte_range()).unwrap_or_default().to_owned();
if name.is_empty() {
return;
}
// Selection-range start of the class: its name identifier (`class Foo`
// -> `Foo`), matching `outline.rs` and the gutter marker candidates.
let range = idx.range(name_node.start_byte(), name_node.end_byte());
let bases = class_base_names(node, text);
out.push(ClassBases { name, line: range.start.line, col: range.start.col, bases });
});
out
}
/// Depth-first pre-order traversal of all descendants of `cursor`'s node.
fn visit(cursor: &mut tree_sitter::TreeCursor, f: &mut impl FnMut(Node)) {
loop {
f(cursor.node());
if cursor.goto_first_child() {
continue;
}
loop {
if cursor.goto_next_sibling() {
break;
}
if !cursor.goto_parent() {
return;
}
}
}
}
/// The bare base names a `class_definition` node writes (keyword arguments left out).
pub(super) fn class_base_names(node: Node, text: &str) -> Vec<String> {
node.child_by_field_name("superclasses").map(|args| base_names(args, text)).unwrap_or_default()
}
fn base_names(args: Node, text: &str) -> Vec<String> {
let mut out = Vec::new();
let mut cursor = args.walk();
for arg in args.named_children(&mut cursor) {
if let Some(name) = base_name(arg, text) {
out.push(name);
}
}
out
}
/// Bare name of one base expression, or `None` for keyword arguments and
/// expressions that are not a (possibly qualified) name.
fn base_name(arg: Node, text: &str) -> Option<String> {
match arg.kind() {
"identifier" => text.get(arg.byte_range()).map(str::to_owned),
// `pkg.mod.Base` -> `Base`.
"attribute" => {
let attr = arg.child_by_field_name("attribute")?;
(attr.kind() == "identifier").then(|| text.get(attr.byte_range()).unwrap_or_default().to_owned())
}
// `Generic[T]` -> `Generic`.
"subscript" => {
let value = arg.child_by_field_name("value")?;
(value.kind() == "identifier").then(|| text.get(value.byte_range()).unwrap_or_default().to_owned())
}
// `metaclass=Meta`, `**kwargs`, `*bases` and other expressions.
_ => None,
}
}
#[cfg(test)]
mod tests {
use super::*;
fn bases(src: &str) -> Vec<ClassBases> {
static S: std::sync::OnceLock<Syntax> = std::sync::OnceLock::new();
S.get_or_init(Syntax::new).class_bases(&LanguageId::new("python"), src)
}
fn shape(src: &str) -> Vec<(String, u32, u32, Vec<String>)> {
bases(src).into_iter().map(|c| (c.name, c.line, c.col, c.bases)).collect()
}
#[test]
fn no_bases() {
assert_eq!(shape("class A: pass"), vec![("A".to_owned(), 0, 6, vec![])]);
}
#[test]
fn multiple_bases_skips_metaclass() {
assert_eq!(
shape("class B(A, C, metaclass=Meta): pass"),
vec![("B".to_owned(), 0, 6, vec!["A".to_owned(), "C".to_owned()])]
);
}
#[test]
fn qualified_name_takes_last_segment() {
assert_eq!(shape("class D(pkg.mod.Base): pass"), vec![("D".to_owned(), 0, 6, vec!["Base".to_owned()])]);
}
#[test]
fn subscript_takes_value() {
assert_eq!(shape("class E(Generic[T]): pass"), vec![("E".to_owned(), 0, 6, vec!["Generic".to_owned()])]);
}
#[test]
fn multiline_bases() {
let src = "class F(\n A,\n B,\n):\n pass";
assert_eq!(shape(src), vec![("F".to_owned(), 0, 6, vec!["A".to_owned(), "B".to_owned()])]);
}
#[test]
fn nested_classes() {
let src = "class Outer:\n class Inner(Y):\n pass";
assert_eq!(
shape(src),
vec![("Outer".to_owned(), 0, 6, vec![]), ("Inner".to_owned(), 1, 10, vec!["Y".to_owned()]),]
);
}
#[test]
fn comments_and_strings_are_ignored() {
let src = r#"
# class Fake(Nope): pass
doc = """
class InDocstring(X): pass
"""
s = 'class InString(Y): pass'
class Real(Base): pass
"#;
assert_eq!(shape(src), vec![("Real".to_owned(), 6, 6, vec!["Base".to_owned()])]);
}
#[test]
fn other_languages_are_empty() {
static S: std::sync::OnceLock<Syntax> = std::sync::OnceLock::new();
let s = S.get_or_init(Syntax::new);
assert!(s.class_bases(&LanguageId::new("rust"), "struct A;").is_empty());
}
}
+4
Переглянути файл
@@ -6,9 +6,11 @@ mod external;
mod folding;
mod grammars;
mod highlight;
mod inheritance;
pub(crate) mod lines;
mod literals;
mod outline;
mod python_facts;
mod rst;
mod selection;
mod sfc;
@@ -22,9 +24,11 @@ pub use external::{ExternalFold, ExternalSymbol, ExternalSyntax, ExternalToken,
use grammars::{SPECS, Spec};
pub use highlight::{SyntaxHighlightKind, SyntaxHighlights};
use ide_core::{LanguageId, OutlineItem, Pos, Range};
pub use inheritance::ClassBases;
pub use lines::{shift_pos, shift_range};
pub use literals::{Literal, LiteralKind, SpellingKind, SpellingRegion};
pub use outline::merge_visibility;
pub use python_facts::{MethodBody, PythonClassFacts, PythonFacts, PythonMethodFacts};
use std::cell::RefCell;
use std::collections::HashMap;
use std::sync::OnceLock;
+438
Переглянути файл
@@ -0,0 +1,438 @@
//! Python class facts the inheritance inspections need: base names as written, parameter
//! lists, decorator semantics and what a method body does. They come from the syntax tree in
//! one pass, so the frontend reasons about the inheritance graph without parsing source text.
//!
//! Deliberately small: a fact exists here only when a check can decide it without guessing.
use super::lines::LineIndex;
use crate::{LanguageId, Syntax};
use serde::{Deserialize, Serialize};
use specta::Type;
use tree_sitter::{Node, Tree};
/// One Python module's classes, in source order.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PythonFacts {
pub classes: Vec<PythonClassFacts>,
}
/// A class definition and the methods it declares.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PythonClassFacts {
pub name: String,
/// The class name identifier (selection range).
pub selection: crate::Range,
/// Base names as written, a qualified name reduced to its last segment.
pub bases: Vec<String>,
pub methods: Vec<PythonMethodFacts>,
}
/// A method definition the inspections can reason about.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PythonMethodFacts {
pub name: String,
/// The method name identifier (selection range).
pub selection: crate::Range,
/// Parameter names in order; `None` for a form whose forwarding cannot be compared
/// (`*args`, `**kwargs`, keyword-only, positional-only).
pub params: Option<Vec<String>>,
/// Decorated with `abstractmethod` (`abc.abstractmethod` included).
pub abstract_method: bool,
/// Decorated with `override` from `typing` / `typing_extensions`.
pub override_decorated: bool,
pub body: MethodBody,
}
/// What a method body does, for the shapes a check can decide without guessing.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", tag = "kind")]
pub enum MethodBody {
/// `pass`, `...`, a docstring, or nothing at all.
Empty,
/// Exactly one `super().name(args)` call, optionally `return`ed.
SuperCall { returns: bool, target: String, args: Vec<String> },
/// Anything else.
Other,
}
impl Syntax {
/// The classes of a Python `text` with the facts the inheritance checks need.
/// Every other language returns an empty result.
pub fn python_facts(&self, lang: &LanguageId, text: &str) -> PythonFacts {
self.analyze(lang, text, |g, tree, text| (g.spec.name == "python").then(|| extract(tree, text)))
.unwrap_or_default()
}
}
fn extract(tree: &Tree, text: &str) -> PythonFacts {
let idx = LineIndex::new(text);
let imports = ImportBindings::collect(tree, text);
let mut classes = Vec::new();
let mut cursor = tree.walk();
visit(&mut cursor, &mut |node| {
if node.kind() != "class_definition" {
return;
}
let Some(name_node) = node.child_by_field_name("name") else { return };
let name = text_of(name_node, text);
if name.is_empty() {
return;
}
classes.push(PythonClassFacts {
name,
selection: idx.range(name_node.start_byte(), name_node.end_byte()),
bases: super::inheritance::class_base_names(node, text),
methods: methods_of(node, text, &idx, &imports),
});
});
PythonFacts { classes }
}
/// The methods declared directly in `class`.
fn methods_of(class: Node, text: &str, idx: &LineIndex, imports: &ImportBindings) -> Vec<PythonMethodFacts> {
let Some(body) = class.child_by_field_name("body") else { return Vec::new() };
let mut out = Vec::new();
let mut cursor = body.walk();
for member in body.named_children(&mut cursor) {
// A decorated method is a `decorated_definition` wrapping the `function_definition`.
let method = match member.kind() {
"function_definition" => member,
"decorated_definition" => match member.child_by_field_name("definition") {
Some(inner) if inner.kind() == "function_definition" => inner,
_ => continue,
},
_ => continue,
};
let Some(name_node) = method.child_by_field_name("name") else { continue };
let name = text_of(name_node, text);
if name.is_empty() {
continue;
}
let decorators = decorators_of(method, text);
out.push(PythonMethodFacts {
name,
selection: idx.range(name_node.start_byte(), name_node.end_byte()),
params: parameters_of(method, text),
abstract_method: decorators.iter().any(|d| last_segment(d) == "abstractmethod"),
override_decorated: decorators.iter().any(|d| imports.is_typing_override(d)),
body: body_of(method, text),
});
}
out
}
/// The parameter names of a `function_definition`, or `None` when the signature contains a
/// form whose forwarding cannot be compared without guessing.
fn parameters_of(function: Node, text: &str) -> Option<Vec<String>> {
let params = function.child_by_field_name("parameters")?;
let mut names = Vec::new();
let mut cursor = params.walk();
for param in params.named_children(&mut cursor) {
let name = match param.kind() {
"identifier" => text_of(param, text),
"typed_parameter" | "default_parameter" | "typed_default_parameter" => {
let named = param
.child_by_field_name("name")
.or_else(|| param.child_by_field_name("pattern"))
.or_else(|| param.named_child(0))?;
if named.kind() != "identifier" {
return None;
}
text_of(named, text)
}
// `*args`, `**kwargs`, `/`, `*`: the call form cannot be compared textually.
_ => return None,
};
if !name.is_empty() {
names.push(name);
}
}
Some(names)
}
/// A `function_definition`'s body, classified into the shapes a check can decide.
fn body_of(function: Node, text: &str) -> MethodBody {
let Some(body) = function.child_by_field_name("body") else { return MethodBody::Empty };
// A docstring is not a statement the checks care about.
let statements: Vec<Node> = statements_of(body).into_iter().filter(|s| !is_string(*s)).collect();
if statements.is_empty() || statements.iter().all(|s| is_noop(*s)) {
return MethodBody::Empty;
}
let [only] = statements.as_slice() else { return MethodBody::Other };
let call = match only.kind() {
"expression_statement" => (only.named_child(0), false),
"return_statement" => (only.named_child(0), true),
_ => (None, false),
};
match call {
(Some(call), returns) if is_super_call(call, text) => {
let target = call
.child_by_field_name("function")
.and_then(|f| f.child_by_field_name("attribute"))
.map(|a| text_of(a, text))
.unwrap_or_default();
let args = call
.child_by_field_name("arguments")
.map(|args| {
let mut cursor = args.walk();
args.named_children(&mut cursor).map(|a| text_of(a, text)).collect()
})
.unwrap_or_default();
MethodBody::SuperCall { returns, target, args }
}
_ => MethodBody::Other,
}
}
/// Whether `node` is a `super().name(...)` call.
fn is_super_call(node: Node, text: &str) -> bool {
if node.kind() != "call" {
return false;
}
let Some(function) = node.child_by_field_name("function") else { return false };
if function.kind() != "attribute" {
return false;
}
function.child_by_field_name("object").is_some_and(|object| {
object.kind() == "call"
&& object
.child_by_field_name("function")
.is_some_and(|f| f.kind() == "identifier" && text_of(f, text) == "super")
})
}
/// Whether `statement` is `pass` or a bare `...`.
fn is_noop(statement: Node) -> bool {
match statement.kind() {
"pass_statement" => true,
"expression_statement" => statement.named_child(0).is_some_and(|child| child.kind() == "ellipsis"),
_ => false,
}
}
/// A string expression statement: a docstring.
fn is_string(statement: Node) -> bool {
statement.kind() == "expression_statement" && statement.named_child(0).is_some_and(|child| child.kind() == "string")
}
/// The statements of a `block`, comments excluded.
fn statements_of(block: Node) -> Vec<Node> {
let mut cursor = block.walk();
block.named_children(&mut cursor).filter(|child| child.kind() != "comment").collect()
}
/// The decorators written on a definition, without the `@` or the call arguments.
fn decorators_of(definition: Node, text: &str) -> Vec<String> {
let Some(parent) = definition.parent() else { return Vec::new() };
if parent.kind() != "decorated_definition" {
return Vec::new();
}
let mut out = Vec::new();
let mut cursor = parent.walk();
for decorator in parent.named_children(&mut cursor) {
if decorator.kind() != "decorator" {
continue;
}
let Some(inner) = decorator.named_child(0) else { continue };
let target =
if inner.kind() == "call" { inner.child_by_field_name("function").unwrap_or(inner) } else { inner };
let name = text_of(target, text).trim().to_owned();
if !name.is_empty() {
out.push(name);
}
}
out
}
/// The last segment of a dotted name (`typing.override` -> `override`).
fn last_segment(name: &str) -> &str {
name.rsplit('.').next().unwrap_or(name)
}
fn text_of(node: Node, text: &str) -> String {
text.get(node.byte_range()).unwrap_or_default().to_owned()
}
/// The module-level `from ... import ...` bindings, so `override` can be told apart from a
/// same-named decorator of another library.
struct ImportBindings {
/// (local name, module, exported name)
bindings: Vec<(String, String, String)>,
}
impl ImportBindings {
fn collect(tree: &Tree, text: &str) -> Self {
let mut bindings = Vec::new();
let mut cursor = tree.walk();
visit(&mut cursor, &mut |node| {
if node.kind() != "import_from_statement" {
return;
}
let Some(module) = node.child_by_field_name("module_name") else { return };
let module = text_of(module, text);
let mut names = node.walk();
for name in node.children_by_field_name("name", &mut names) {
let (local, exported) = match name.kind() {
"aliased_import" => (
name.child_by_field_name("alias").map(|n| text_of(n, text)),
name.child_by_field_name("name").map(|n| text_of(n, text)),
),
_ => {
let written = text_of(name, text);
(Some(written.clone()), Some(written))
}
};
if let (Some(local), Some(exported)) = (local, exported) {
bindings.push((local, module.clone(), exported));
}
}
});
Self { bindings }
}
/// Whether `decorator` names `override` from `typing` / `typing_extensions`.
fn is_typing_override(&self, decorator: &str) -> bool {
const MODULES: [&str; 2] = ["typing", "typing_extensions"];
let segments: Vec<&str> = decorator.split('.').filter(|s| !s.is_empty()).collect();
match segments.as_slice() {
// `@typing.override`, or `@t.override` after `import typing as t`.
[prefix, "override"] => {
MODULES.contains(prefix)
|| self
.bindings
.iter()
.find(|(local, _, _)| local == prefix)
.is_some_and(|(_, module, _)| MODULES.contains(&module.as_str()))
}
// `@override`, or `@ov` after `from typing import override as ov`.
[local] => match self.bindings.iter().find(|(bound, _, _)| bound == local) {
Some((_, module, exported)) => exported == "override" && MODULES.contains(&module.as_str()),
// Nothing contradicts a bare `@override`: the usual `from typing import *`.
None => *local == "override",
},
_ => false,
}
}
}
/// Depth-first pre-order traversal of all descendants of `cursor`'s node.
fn visit(cursor: &mut tree_sitter::TreeCursor, f: &mut impl FnMut(Node)) {
loop {
f(cursor.node());
if cursor.goto_first_child() {
continue;
}
loop {
if cursor.goto_next_sibling() {
break;
}
if !cursor.goto_parent() {
return;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
static SYNTAX: std::sync::OnceLock<Syntax> = std::sync::OnceLock::new();
fn facts(src: &str) -> PythonFacts {
SYNTAX.get_or_init(Syntax::new).python_facts(&LanguageId::new("python"), src)
}
fn method<'a>(facts: &'a PythonFacts, class: &str, method: &str) -> &'a PythonMethodFacts {
facts
.classes
.iter()
.find(|c| c.name == class)
.and_then(|c| c.methods.iter().find(|m| m.name == method))
.expect("method not found")
}
#[test]
fn classes_carry_bases_and_methods() {
let f = facts(
"class Base(ABC):\n pass\n\nclass Child(Base, pkg.Mixin):\n def run(self, x=1):\n pass\n",
);
assert_eq!(f.classes.len(), 2);
assert_eq!(f.classes[0].name, "Base");
assert_eq!(f.classes[0].bases, ["ABC"]);
assert_eq!(f.classes[1].bases, ["Base", "Mixin"]);
assert_eq!(f.classes[1].methods.len(), 1);
assert_eq!(f.classes[1].selection.start.line, 3);
assert_eq!(f.classes[1].selection.start.col, 6);
}
#[test]
fn parameters_are_names_in_order_and_none_for_splats() {
let f = facts(
"class C:\n def a(self, x: int, y=2):\n pass\n\n def b(self, *args):\n pass\n\n def c(self, *, x):\n pass\n",
);
assert_eq!(
method(&f, "C", "a").params.as_deref(),
Some(["self".to_string(), "x".to_string(), "y".to_string()].as_slice())
);
assert_eq!(method(&f, "C", "b").params, None);
assert_eq!(method(&f, "C", "c").params, None);
}
#[test]
fn body_shapes_are_classified() {
let f = facts(
"class C:\n def noop(self):\n pass\n\n def dots(self):\n ...\n\n def doc(self):\n \"\"\"text\"\"\"\n\n def forward(self, x):\n return super().run(x)\n\n def call(self):\n super().run()\n\n def other(self):\n return 1\n",
);
assert_eq!(method(&f, "C", "noop").body, MethodBody::Empty);
assert_eq!(method(&f, "C", "dots").body, MethodBody::Empty);
assert_eq!(method(&f, "C", "doc").body, MethodBody::Empty);
assert_eq!(
method(&f, "C", "forward").body,
MethodBody::SuperCall { returns: true, target: "run".into(), args: vec!["x".into()] }
);
assert_eq!(
method(&f, "C", "call").body,
MethodBody::SuperCall { returns: false, target: "run".into(), args: vec![] }
);
assert_eq!(method(&f, "C", "other").body, MethodBody::Other);
}
#[test]
fn decorators_are_their_names_and_semantics() {
let f = facts(
"from abc import abstractmethod\nfrom typing import override\n\nclass C:\n @abstractmethod\n def a(self):\n pass\n\n @override\n def b(self):\n pass\n\n @functools.cache\n def c(self):\n pass\n",
);
assert!(method(&f, "C", "a").abstract_method);
assert!(!method(&f, "C", "a").override_decorated);
assert!(method(&f, "C", "b").override_decorated);
assert!(!method(&f, "C", "c").override_decorated);
}
#[test]
fn override_of_another_library_is_not_typing_override() {
let f = facts("from mylib import override\n\nclass C:\n @override\n def a(self):\n pass\n");
assert!(!method(&f, "C", "a").override_decorated);
}
#[test]
fn aliased_override_and_qualified_forms_are_recognised() {
let f = facts(
"from typing import override as ov\n\nclass C:\n @ov\n def a(self):\n pass\n\nclass D:\n @typing_extensions.override\n def b(self):\n pass\n",
);
assert!(method(&f, "C", "a").override_decorated);
assert!(method(&f, "D", "b").override_decorated);
}
#[test]
fn nested_classes_and_other_languages() {
let f = facts("class Outer:\n class Inner:\n def m(self):\n pass\n");
let names: Vec<&str> = f.classes.iter().map(|c| c.name.as_str()).collect();
assert_eq!(names, ["Outer", "Inner"]);
let other = SYNTAX.get_or_init(Syntax::new).python_facts(&LanguageId::new("rust"), "class C {}");
assert_eq!(other.classes.len(), 0);
}
}
+36 -6
Переглянути файл
@@ -41,20 +41,44 @@ pub struct ClientHandler {
generation: u64,
}
/// The diagnostics in `items` (a JSON array) that deserialize. A server's malformed one is
/// skipped with a warning, so the rest of the file's problems still show.
pub(crate) fn lenient_diagnostics(server: &str, items: Value) -> Vec<lsp::Diagnostic> {
let Value::Array(items) = items else { return Vec::new() };
let mut skipped = 0usize;
let parsed: Vec<lsp::Diagnostic> = items
.into_iter()
.filter_map(|mut item| {
crate::virtual_doc::normalize_uris(&mut item);
match serde_json::from_value(item) {
Ok(diagnostic) => Some(diagnostic),
Err(_) => {
skipped += 1;
None
}
}
})
.collect();
if skipped > 0 {
tracing::warn!("{server}: skipped {skipped} malformed diagnostic(s)");
}
parsed
}
impl ClientHandler {
pub fn new(mgr: Weak<LspManager>, server: Arc<Server>, generation: u64) -> Self {
Self { mgr, server, generation }
}
fn encoding(&self) -> Encoding {
self.server.running_gen(self.generation).map(|r| r.caps.read().encoding).unwrap_or_default()
self.server.initialized_gen(self.generation).map(|r| r.caps.read().encoding).unwrap_or_default()
}
/// Cancelling task `task` asks this server instance to stop the work of progress `token`.
fn cancel_handle(&self, task: TaskId, token: &Value) -> ide_core::task::CancelHandle {
let (server, generation, token) = (self.server.clone(), self.generation, token.clone());
ide_core::task::on_cancel(task, move || {
if let Some(run) = server.running_gen(generation)
if let Some(run) = server.initialized_gen(generation)
&& let Err(e) = run.peer.notify("window/workDoneProgress/cancel", json!({ "token": token }))
{
tracing::debug!("{}: cannot cancel progress: {e}", server.name);
@@ -176,7 +200,7 @@ impl ClientHandler {
loop {
tokio::time::sleep(PROGRESS_SWEEP).await;
let Some(mgr) = weak.upgrade() else { return };
if server.running_gen(generation).is_none() || !server.has_progress(&token) {
if !server.is_live(generation) || !server.has_progress(&token) {
return; // the instance is gone, or the task ended normally
}
if server.progress_silent(&token, PROGRESS_STALE) {
@@ -251,18 +275,24 @@ impl ClientHandler {
fn publish_diagnostics(&self, mgr: &LspManager, params: Value) {
// Problems come only from servers used for `diagnostics`.
if self.server.running_gen(self.generation).is_some_and(|run| !run.caps.read().restriction.diagnostics()) {
if self.server.initialized_gen(self.generation).is_some_and(|run| !run.caps.read().restriction.diagnostics()) {
return;
}
let mut params = params;
crate::virtual_doc::normalize_uris(&mut params);
let params: lsp::PublishDiagnosticsParams = match serde_json::from_value(params) {
// One malformed diagnostic must not take the others of the file with it.
let raw = params.get_mut("diagnostics").map(Value::take);
if let Some(fields) = params.as_object_mut() {
fields.insert("diagnostics".into(), json!([]));
}
let mut params: lsp::PublishDiagnosticsParams = match serde_json::from_value(params) {
Ok(p) => p,
Err(e) => {
tracing::debug!("{}: bad publishDiagnostics: {e}", self.server.name);
return;
}
};
params.diagnostics = lenient_diagnostics(&self.server.name, raw.unwrap_or(Value::Null));
let Some(path) = uri_to_path(params.uri.as_str())
.or_else(|| crate::notebook::is_cell_uri(params.uri.as_str()).then(|| PathBuf::from(params.uri.as_str())))
else {
@@ -480,7 +510,7 @@ impl Handler for ClientHandler {
crate::kotlin_lsp::record_import_state(&self.server, self.generation, &params)
}
"language/status" if self.server.name == "jdtls" && params["type"] == "ServiceReady" => {
if self.server.running_gen(self.generation).is_some() {
if self.server.initialized_gen(self.generation).is_some() {
self.server.set_java_ready();
}
mgr.java_import_ready(self.server.clone(), self.generation);
+75 -25
Переглянути файл
@@ -103,7 +103,7 @@ impl Launcher for ProcessLauncher {
// A plugin's server: the approved program, unchanged since.
Some(pin) => Some(pin.verify().map_err(LaunchError::Failed)?.to_path_buf()),
None => ide_core::tool_path::find_command(&config.command, &search, root)
.and_then(|b| resolve_rustup_proxy(&config.command, b))
.and_then(|b| resolve_rustup_proxy(&config.command, b, root))
.and_then(|b| reject_dead_shim(b, &search, root)),
};
let mut env: Vec<(String, String)> = vec![("PATH".into(), search.to_string_lossy().into_owned())];
@@ -208,34 +208,16 @@ fn reject_dead_shim(binary: PathBuf, search: &OsStr, root: &Path) -> Option<Path
/// toolchain lacks the component (common for rust-analyzer). In that case use
/// the newest installed toolchain that ships the binary; with none, the tool is
/// missing (an install hint, not a server that crashes on every restart).
fn resolve_rustup_proxy(command: &str, binary: PathBuf) -> Option<PathBuf> {
/// The probe runs in the project's root, where a `rust-toolchain.toml` picks the toolchain.
fn resolve_rustup_proxy(command: &str, binary: PathBuf, root: &Path) -> Option<PathBuf> {
let is_proxy = binary.parent().is_some_and(|d| d.ends_with(".cargo/bin"));
if !is_proxy || command.contains('/') {
return Some(binary);
}
let works = ide_core::process::prepare(&mut std::process::Command::new(&binary))
.arg("--version")
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.is_ok_and(|s| s.success());
if works {
if proxy_works(&binary, root) {
return Some(binary);
}
let toolchains = std::env::var_os("RUSTUP_HOME")
.map(PathBuf::from)
.or_else(|| std::env::var_os("HOME").map(|h| PathBuf::from(h).join(".rustup")))
.map(|h| h.join("toolchains"));
let mut candidates: Vec<(std::time::SystemTime, PathBuf)> = toolchains
.and_then(|dir| std::fs::read_dir(dir).ok())
.into_iter()
.flatten()
.flatten()
.map(|e| e.path().join("bin").join(command))
.filter(|p| p.is_file())
.filter_map(|p| Some((p.metadata().ok()?.modified().ok()?, p)))
.collect();
let mut candidates = rustup_home().map(|h| toolchain_binaries(&h.join("toolchains"), command)).unwrap_or_default();
// Prefer stable-looking toolchains, then the most recently installed.
candidates.sort_by_key(|(mtime, p)| (!p.to_string_lossy().contains("nightly"), *mtime));
match candidates.pop() {
@@ -250,6 +232,57 @@ fn resolve_rustup_proxy(command: &str, binary: PathBuf) -> Option<PathBuf> {
}
}
/// Whether the proxy runs for the toolchain active in `root`. A probe that hangs or can't
/// be waited for counts as working: the real start decides.
fn proxy_works(binary: &Path, root: &Path) -> bool {
let Ok(mut child) = ide_core::process::prepare(&mut std::process::Command::new(binary))
.arg("--version")
.current_dir(root)
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.spawn()
else {
return true;
};
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
loop {
match child.try_wait() {
Ok(Some(status)) => return status.success(),
Ok(None) if std::time::Instant::now() < deadline => {
std::thread::sleep(std::time::Duration::from_millis(20))
}
_ => {
let _ = child.kill();
return true;
}
}
}
}
/// Where rustup keeps its toolchains: `RUSTUP_HOME`, else `.rustup` in the user's home
/// (`HOME`, which a GUI app on Windows doesn't have, or `USERPROFILE`).
fn rustup_home() -> Option<PathBuf> {
std::env::var_os("RUSTUP_HOME").map(PathBuf::from).or_else(|| {
let home = std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE"))?;
Some(PathBuf::from(home).join(".rustup"))
})
}
/// The `command` binary (with the platform's executable suffix) of every toolchain in
/// `toolchains`, with its modification time.
fn toolchain_binaries(toolchains: &Path, command: &str) -> Vec<(std::time::SystemTime, PathBuf)> {
let file = format!("{command}{}", std::env::consts::EXE_SUFFIX);
std::fs::read_dir(toolchains)
.into_iter()
.flatten()
.flatten()
.map(|e| e.path().join("bin").join(&file))
.filter(|p| p.is_file())
.filter_map(|p| Some((p.metadata().ok()?.modified().ok()?, p)))
.collect()
}
/// Workspace root for `file`: the outermost ancestor within `project_root`
/// containing a root marker (so a Cargo/npm workspace gets one server, not one
/// per member), else `project_root`. `None` when `file` is outside the project.
@@ -363,7 +396,24 @@ mod proxy_tests {
#[test]
fn non_proxy_binaries_are_kept() {
let p = PathBuf::from("/usr/bin/true");
assert_eq!(resolve_rustup_proxy("true", p.clone()), Some(p));
assert_eq!(resolve_rustup_proxy("true", p.clone(), Path::new(".")), Some(p));
}
/// The binary is `rust-analyzer.exe` on Windows: a lookup of the bare name found nothing there.
#[test]
fn toolchain_binaries_carry_the_platform_executable_suffix() {
let tmp = tempfile::tempdir().unwrap();
let file = format!("rust-analyzer{}", std::env::consts::EXE_SUFFIX);
for toolchain in ["stable-x86_64", "nightly-x86_64"] {
std::fs::create_dir_all(tmp.path().join(toolchain).join("bin")).unwrap();
}
std::fs::write(tmp.path().join("stable-x86_64/bin").join(&file), "").unwrap();
// Another tool, and a bare name without the suffix where one is required.
std::fs::write(tmp.path().join("nightly-x86_64/bin").join("rustfmt"), "").unwrap();
let found = toolchain_binaries(tmp.path(), "rust-analyzer");
assert_eq!(found.len(), 1);
assert_eq!(found[0].1, tmp.path().join("stable-x86_64/bin").join(&file));
assert!(toolchain_binaries(&tmp.path().join("missing"), "rust-analyzer").is_empty());
}
/// Exercises the real machine setup when a broken rustup proxy is present.
@@ -372,7 +422,7 @@ mod proxy_tests {
fn broken_rustup_proxy_falls_back_to_a_toolchain() {
let home = PathBuf::from(std::env::var_os("HOME").unwrap());
let proxy = home.join(".cargo/bin/rust-analyzer");
let resolved = resolve_rustup_proxy("rust-analyzer", proxy.clone()).unwrap();
let resolved = resolve_rustup_proxy("rust-analyzer", proxy.clone(), Path::new(".")).unwrap();
eprintln!("resolved: {}", resolved.display());
assert_ne!(resolved, proxy);
}
+110 -36
Переглянути файл
@@ -343,6 +343,10 @@ impl<T> Outcome<T> {
/// How often a rename the server withdrew (`ContentModified`) is asked, and the pause between.
const RENAME_ATTEMPTS: u32 = 5;
const RENAME_RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(300);
/// How often a semantic tokens, inlay hints or code lens request the server withdrew
/// (`ContentModified`) is asked again, and the pause between.
const STALE_RETRIES: u32 = 2;
const STALE_RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(150);
/// Implementation requests of one line-marker pass in flight at once.
const IMPLEMENTATION_CONCURRENCY: usize = 4;
@@ -654,8 +658,9 @@ impl LspManager {
}
/// The servers serving `path`: every regular one, and each `fallback` one
/// only while all servers listed before it are missing. One that requires a
/// root marker serves only files below one, and stands in for no other.
/// only while all servers listed before it have given up (missing, not
/// launchable, or crashed for good). One that requires a root marker serves
/// only files below one, and stands in for no other.
fn servers_for(&self, language: &LanguageId, path: &Path) -> Vec<Arc<Server>> {
if let Some(servers) = self.arduino_servers(language, path) {
return servers;
@@ -672,7 +677,7 @@ impl LspManager {
continue;
}
let server = self.server_for(sc, language, path);
available_before |= !sc.require_root_marker && server.status() != ide_core::event::ServerStatus::Missing;
available_before |= !sc.require_root_marker && !server.gave_up();
out.push(server);
}
out
@@ -748,9 +753,14 @@ impl LspManager {
if self.shutting_down.load(Ordering::Acquire) || !self.execution_allowed.load(Ordering::Acquire) {
return;
}
// A server dropped meanwhile (disabled in settings, shadowed by another) must not come
// back as a process nothing lists, serves or stops: a crash backoff can outlive it.
if !self.servers.lock().iter().any(|s| s.id == server.id) {
return;
}
let workspace = is_local_jdtls(&server.config).then(|| self.jdtls_workspace_state());
if let Some(state) = &workspace {
if state.transitioning || !self.servers.lock().iter().any(|s| s.id == server.id) {
if state.transitioning {
return;
}
let (shared, roots) = self.desired_jdtls_workspace();
@@ -861,24 +871,28 @@ impl LspManager {
}
Err(LaunchError::Config(message)) => {
tracing::warn!("{}: {message}", server.name);
server.fail_start(
if server.fail_start(
generation,
Conn::Failed,
&self.sink,
ide_core::event::ServerStatus::Misconfigured,
Some(message),
);
) {
self.attach_fallbacks();
}
return;
}
Err(LaunchError::Failed(message)) => {
tracing::warn!("{}: {message}", server.name);
server.fail_start(
if server.fail_start(
generation,
Conn::Failed,
&self.sink,
ide_core::event::ServerStatus::Crashed,
Some(message),
);
) {
self.attach_fallbacks();
}
return;
}
};
@@ -920,9 +934,18 @@ impl LspManager {
}
Err(e) => {
tracing::warn!("{}: initialize failed: {e}", server.name);
if matches!(e, RpcError::Closed) {
match e {
// The server is exiting: let it, so the crash report has its exit code and last output.
let _ = tokio::time::timeout(Duration::from_secs(3), peer.closed()).await;
RpcError::Closed => {
let _ = tokio::time::timeout(Duration::from_secs(3), peer.closed()).await;
}
RpcError::Timeout => {
server.set_start_error(
generation,
format!("it did not answer initialize within {} s", INITIALIZE_TIMEOUT.as_secs()),
);
}
e => server.set_start_error(generation, format!("initialize failed ({e})")),
}
Self::kill_peer(&server, &peer).await;
// Treated like a crash so transient failures retry with backoff.
@@ -1008,11 +1031,15 @@ impl LspManager {
self.forget(std::slice::from_ref(&server));
return;
}
if let Some(delay) = delay {
tokio::time::sleep(delay).await;
if server.is_idle_gen(generation) {
self.ensure_started(&server);
match delay {
Some(delay) => {
tokio::time::sleep(delay).await;
if server.is_idle_gen(generation) {
self.ensure_started(&server);
}
}
// Gave up (crashed repeatedly, or its build expired): the fallbacks it hid take over.
None => self.attach_fallbacks(),
}
}
@@ -1458,6 +1485,28 @@ impl LspManager {
result
}
/// [`Self::call`], asked again when the server withdrew the answer because the text changed
/// (`ContentModified`, which arrives as a cancellation as no cancel token is passed).
/// The client advertises `retryOnContentModified` for the methods that use it.
async fn call_retrying<P: Serialize + Clone, R: DeserializeOwned>(
&self,
t: &Target,
method: &str,
params: P,
timeout: Duration,
) -> IdeResult<R> {
let mut attempt = 0;
loop {
match self.call(t, method, params.clone(), None, timeout).await {
Err(e) if e.kind == ErrorKind::Cancelled && attempt < STALE_RETRIES => {
attempt += 1;
tokio::time::sleep(STALE_RETRY_DELAY).await;
}
result => return result,
}
}
}
// ---- features ----
pub async fn completion(
&self,
@@ -2892,11 +2941,10 @@ impl LspManager {
async move {
(
t,
self.call::<_, Option<Vec<lsp::InlayHint>>>(
self.call_retrying::<_, Option<Vec<lsp::InlayHint>>>(
t,
"textDocument/inlayHint",
params,
None,
DEFAULT_TIMEOUT,
)
.await,
@@ -2927,7 +2975,8 @@ impl LspManager {
self.ready_targets(&snap.servers, |c| c.semantic_tokens_full() && c.semantic_legend.is_some()).await;
let Some(t) = targets.first() else { return Ok(None) };
let params = json!({ "textDocument": snap.ident() });
let r: Option<Value> = self.call(t, "textDocument/semanticTokens/full", params, None, DEFAULT_TIMEOUT).await?;
let r: Option<Value> =
self.call_retrying(t, "textDocument/semanticTokens/full", params, DEFAULT_TIMEOUT).await?;
let Some(data) = r.as_ref().and_then(|v| v.get("data")).and_then(Value::as_array) else { return Ok(None) };
let data: Vec<i64> = data.iter().map(|n| n.as_i64().unwrap_or(0)).collect();
let (enc, legend) = {
@@ -2948,7 +2997,7 @@ impl LspManager {
let params = json!({ "textDocument": snap.ident() });
async move {
let lenses = self
.call::<_, Option<Vec<lsp::CodeLens>>>(t, "textDocument/codeLens", params, None, DEFAULT_TIMEOUT)
.call_retrying::<_, Option<Vec<lsp::CodeLens>>>(t, "textDocument/codeLens", params, DEFAULT_TIMEOUT)
.await?;
let lenses = lenses.unwrap_or_default();
if !t.caps().has_sub("codeLensProvider", "resolveProvider") {
@@ -3097,6 +3146,21 @@ impl LspManager {
Ok(CallHierarchy { items: out, incomplete })
}
/// The type at `pos` as the server's type hierarchy names it (with its `detail`, the
/// package for Java); `None` when there is no type there.
pub async fn prepare_type_hierarchy(&self, doc: DocId, pos: Pos) -> IdeResult<Option<HierarchyItem>> {
let (snap, targets) = self.doc_targets(doc, "type hierarchy", |c| c.has("typeHierarchyProvider")).await?;
let t = &targets[0];
let items: Option<Vec<serde_json::Value>> =
self.call(t, "textDocument/prepareTypeHierarchy", snap.pos_params(t, pos), None, DEFAULT_TIMEOUT).await?;
let conv = self.conv(t.enc());
Ok(items
.unwrap_or_default()
.iter()
.filter_map(convert::type_item_from_json)
.find_map(|i| convert::type_item(&conv, &i)))
}
pub async fn type_hierarchy(
&self,
doc: DocId,
@@ -3855,31 +3919,41 @@ impl LspManager {
languages: s.languages(),
status: s.status(),
pid: s.running().and_then(|r| r.peer.pid()),
label: s.display_name(),
})
.collect()
}
pub async fn restart(&self, server: ide_core::ServerId) -> IdeResult<()> {
let _reconfigure = self.reconfigure_lock.lock().await;
let s = self
.servers
.lock()
.iter()
.find(|s| s.id == server)
.cloned()
.ok_or_else(|| IdeError::not_found(format!("unknown language server {server}")))?;
if !s.root.is_dir() {
return Err(IdeError::unavailable(format!(
"the workspace root of {} is gone: {}",
s.name,
s.root.display()
)));
}
self.stop_instance(&s, false).await;
s.reset_for_restart();
self.ensure_started(&s);
let s = {
// Only the restart itself holds the lock: a start can take minutes (a Gradle or
// Maven import), and `reconfigure` and `shutdown_all` wait for it.
let _reconfigure = self.reconfigure_lock.lock().await;
let s = self
.servers
.lock()
.iter()
.find(|s| s.id == server)
.cloned()
.ok_or_else(|| IdeError::not_found(format!("unknown language server {server}")))?;
if !s.root.is_dir() {
return Err(IdeError::unavailable(format!(
"the workspace root of {} is gone: {}",
s.name,
s.root.display()
)));
}
self.stop_instance(&s, false).await;
s.reset_for_restart();
self.ensure_started(&s);
// Started again: it shadows the fallbacks that took over while it was down.
self.detach_shadowed_fallbacks().await;
s
};
match s.wait_ready(Duration::from_secs(60)).await {
Some(_) => Ok(()),
// Still importing the project: the restart worked, the server is just not ready yet.
None if s.is_starting() => Ok(()),
None => {
Err(IdeError::unavailable(s.status_message().unwrap_or_else(|| format!("{} failed to start", s.name))))
}
+72 -11
Переглянути файл
@@ -136,7 +136,8 @@ fn source_entry(sources: &Path, class: &str, encoding: Option<&str>) -> Option<(
return Some((entry.clone(), text));
}
}
// A facade renamed with `@file:JvmName("…")`.
// A facade renamed with `@file:JvmName("…")`, or a class declared in a file named otherwise
// (`UserDto` in `Models.kt`).
let kotlin_files: Vec<String> = zip
.file_names()
.filter_map(|n| n.ok().map(|n| n.into_owned()))
@@ -145,7 +146,7 @@ fn source_entry(sources: &Path, class: &str, encoding: Option<&str>) -> Option<(
let jvm_name = format!("@file:JvmName(\"{outer}\")");
for entry in kotlin_files {
if let Some(text) = read(&entry)
&& text.text.contains(&jvm_name)
&& (text.text.contains(&jvm_name) || declares_type(&text.text, outer))
{
return Some((entry, text));
}
@@ -153,6 +154,26 @@ fn source_entry(sources: &Path, class: &str, encoding: Option<&str>) -> Option<(
None
}
/// Whether `text` declares a type called `name` (`class`, `interface`, `object`, `typealias`):
/// how a Kotlin file that is not named after its class is recognised.
fn declares_type(text: &str, name: &str) -> bool {
const KINDS: [&str; 4] = ["class", "interface", "object", "typealias"];
text.lines().any(|line| {
let line = line.trim_start();
if line.starts_with("//") || line.starts_with("/*") || line.starts_with('*') {
return false;
}
let mut words =
line.split(|c: char| !(c.is_alphanumeric() || c == '_' || c == '$' || c == '`')).filter(|w| !w.is_empty());
let mut previous = words.next();
words.any(|word| {
let hit = previous.is_some_and(|p| KINDS.contains(&p)) && word.trim_matches('`') == name;
previous = Some(word);
hit
})
})
}
/// Where `name` is declared in a Kotlin or Java source `text` (the start of the
/// name), else the start of the file.
pub(crate) fn declaration_range(text: &str, name: &str) -> Range {
@@ -245,6 +266,22 @@ pub(crate) fn folder_class(uri: &str) -> Option<(String, String)> {
Some((package.replace('.', "/"), name.to_owned()))
}
/// Kotlin sources larger than this are not searched for a class declared in a file named otherwise.
const MAX_SOURCE: u64 = 1 << 20;
/// The files of the project's source trees: build output and ignored files are skipped.
fn walk_sources(root: &Path) -> impl Iterator<Item = ignore::DirEntry> {
ignore::WalkBuilder::new(root)
.filter_entry(|e| {
let output = matches!(e.file_name().to_str(), Some("target" | "build" | "out" | "node_modules"));
// A package may be named `build`: only outside source trees is it output.
!output || e.path().components().any(|c| c.as_os_str() == "src")
})
.build()
.flatten()
.filter(|e| e.file_type().is_some_and(|t| t.is_file()))
}
/// The project's source of `class` (`Name`, `Outer$Inner`, `FileKt`) in `package`
/// (`/`-separated): a `.kt` / `.java` file of that name declaring the package. Build
/// output and ignored files are skipped.
@@ -255,19 +292,24 @@ pub(crate) fn project_class_source(root: &Path, package: &str, class: &str) -> O
names.push(format!("{file}.kt"));
}
let package = package.replace('/', ".");
let mut found: Vec<PathBuf> = ignore::WalkBuilder::new(root)
.filter_entry(|e| {
let output = matches!(e.file_name().to_str(), Some("target" | "build" | "out" | "node_modules"));
// A package may be named `build`: only outside source trees is it output.
!output || e.path().components().any(|c| c.as_os_str() == "src")
})
.build()
.flatten()
.filter(|e| e.file_type().is_some_and(|t| t.is_file()))
let mut found: Vec<PathBuf> = walk_sources(root)
.filter(|e| e.file_name().to_str().is_some_and(|n| names.iter().any(|name| name == n)))
.map(|e| e.into_path())
.filter(|p| std::fs::read_to_string(p).is_ok_and(|text| declared_package(&text) == package))
.collect();
// A class declared in a file named otherwise (`UserDto` in `Models.kt`).
if found.is_empty() {
found = walk_sources(root)
.filter(|e| {
e.path().extension().is_some_and(|x| x == "kt") && e.metadata().is_ok_and(|m| m.len() <= MAX_SOURCE)
})
.map(|e| e.into_path())
.filter(|p| {
std::fs::read_to_string(p)
.is_ok_and(|text| declared_package(&text) == package && declares_type(&text, outer))
})
.collect();
}
// The directory matching the package first, then by name order.
found.sort_by_key(|p| {
let dir = p.parent().map(|d| d.to_string_lossy().replace('\\', "/")).unwrap_or_default();
@@ -358,6 +400,10 @@ mod tests {
"@file:JvmName(\"BootUtils\")\npackage org.springframework.boot\nfun util() {}",
),
("org/springframework/boot/Builder.kt", "class Builder"),
(
"org/springframework/boot/Models.kt",
"package org.springframework.boot\n\n// class NotMe\ndata class UserDto(val id: Int)\nenum class Role { A }\ninterface Repo\nobject Registry\ntypealias Ids = List<Int>\n",
),
("Root.java", "class Root {}"),
];
jar(&m2.join("spring-boot-3.5.16.jar"), &[]);
@@ -373,6 +419,11 @@ mod tests {
("org/springframework/boot/SpringApplication$Running.class", "SpringApplication.java"),
("org/springframework/boot/BootUtils.class", "Renamed.kt"),
("org/springframework/boot/Builder.class", "Builder.kt"),
// Declared in a file that is not named after them.
("org/springframework/boot/UserDto.class", "Models.kt"),
("org/springframework/boot/Role$A.class", "Models.kt"),
("org/springframework/boot/Repo.class", "Models.kt"),
("org/springframework/boot/Registry.class", "Models.kt"),
("Root.class", "Root.java"),
] {
let (source_uri, text) =
@@ -385,6 +436,7 @@ mod tests {
);
}
assert!(class_source(&uri("org/springframework/boot/Missing.class"), None).is_none());
assert!(class_source(&uri("org/springframework/boot/NotMe.class"), None).is_none(), "only in a comment");
}
// No sources jar: nothing to show from here.
let lone = tmp.path().join("lone/x-1.jar");
@@ -513,6 +565,11 @@ mod tests {
write("src/main/kotlin/com/x/build/Tool.kt", "package com.x.build\n\nclass Tool\n");
write("src/main/kotlin/flat/Elsewhere.kt", "");
write("src/main/kotlin/Loose.kt", "package flat\n\nobject Loose\n");
write(
"src/main/kotlin/test001/Models.kt",
"package test001\n\n// class NotMe\ndata class UserDto(val id: Int)\n",
);
write("src/main/kotlin/other/Models.kt", "package other\n\ndata class UserDto(val name: String)\n");
let source = |package: &str, class: &str| {
project_class_source(root, package, class).map(|p| p.strip_prefix(root).unwrap().to_owned())
};
@@ -523,6 +580,10 @@ mod tests {
assert_eq!(source("com/x/build", "Tool"), Some(PathBuf::from("src/main/kotlin/com/x/build/Tool.kt")));
assert_eq!(source("flat", "Loose"), Some(PathBuf::from("src/main/kotlin/Loose.kt")), "any directory");
assert_eq!(source("test001", "Missing"), None);
// Declared in a file named otherwise: the one of the package, not another package's.
assert_eq!(source("test001", "UserDto"), Some(PathBuf::from("src/main/kotlin/test001/Models.kt")));
assert_eq!(source("other", "UserDto"), Some(PathBuf::from("src/main/kotlin/other/Models.kt")));
assert_eq!(source("test001", "NotMe"), None, "only in a comment");
assert_eq!(declared_package("package a.b.c;\n"), "a.b.c");
assert_eq!(declared_package("class A"), "");
}
+4 -7
Переглянути файл
@@ -166,13 +166,10 @@ impl LspManager {
let result_id = report.get("resultId").and_then(Value::as_str).map(str::to_owned);
let items = match report.get("kind").and_then(Value::as_str) {
Some("unchanged") => None,
_ => match serde_json::from_value(report.get("items").cloned().unwrap_or(json!([]))) {
Ok(items) => Some(items),
Err(e) => {
tracing::debug!("{}: bad textDocument/diagnostic report: {e}", t.server.name);
return;
}
},
_ => Some(crate::handler::lenient_diagnostics(
&t.server.name,
report.get("items").cloned().unwrap_or(json!([])),
)),
};
let shown = t.server.set_pulled(t.path.clone(), result_id, items);
let enc = t.run.caps.read().encoding;
+78 -7
Переглянути файл
@@ -112,8 +112,11 @@ pub struct Server {
status: Mutex<(ServerStatus, Option<String>)>,
/// Consecutive crashes of short-lived instances.
crashes: Mutex<usize>,
/// When the current instance was launched.
/// When the current instance became ready: only time spent running counts as uptime,
/// not the time spent starting (an `initialize` that hangs for minutes is not a stable run).
started: Mutex<Option<Instant>>,
/// Why the current instance's `initialize` failed, for the crash report.
start_error: Mutex<Option<(u64, String)>>,
stderr: Mutex<VecDeque<String>>,
pub progress: Mutex<HashMap<String, Progress>>,
/// What the editor shows per path: the pushed and the pulled diagnostics together.
@@ -177,6 +180,7 @@ impl Server {
status: Mutex::new((ServerStatus::Stopped, None)),
crashes: Mutex::new(0),
started: Mutex::new(None),
start_error: Mutex::new(None),
stderr: Mutex::new(VecDeque::new()),
progress: Mutex::new(HashMap::new()),
diagnostics: Mutex::new(HashMap::new()),
@@ -213,6 +217,11 @@ impl Server {
config
}
/// The configured server's display name for the current (or last) instance.
pub fn display_name(&self) -> Option<String> {
self.effective.lock().as_ref().and_then(|c| c.display_name.clone())
}
pub fn set_effective_config(&self, generation: u64, config: LanguageServerConfig) {
let st = self.state.lock();
if st.generation == generation && matches!(st.conn, Conn::Starting | Conn::Ready(_)) {
@@ -252,7 +261,8 @@ impl Server {
st.conn = Conn::Starting;
st.restarting = false;
*self.start_cancel.lock() = CancellationToken::new();
*self.started.lock() = Some(Instant::now());
*self.started.lock() = None;
*self.start_error.lock() = None;
// A crash report quotes only this instance's output, not its predecessors'.
self.stderr.lock().clear();
self.import_state.send_replace(None);
@@ -300,9 +310,15 @@ impl Server {
}
st.conn = Conn::Ready(run);
*self.initializing.lock() = None;
*self.started.lock() = Some(Instant::now());
true
}
/// Records why instance `generation` failed to initialize, so its crash report says so.
pub fn set_start_error(&self, generation: u64, reason: String) {
*self.start_error.lock() = Some((generation, reason));
}
/// Wakes requests waiting for the instance (after status and documents are in place).
pub fn announce_ready(&self, generation: u64) {
let st = self.state.lock();
@@ -322,6 +338,13 @@ impl Server {
matches!(self.state.lock().conn, Conn::Starting)
}
/// The server will not run until something changes: its binary is missing, its
/// configuration can't launch it, or it crashed too often (or its build expired).
/// Servers that crashed and are about to restart are not given up on.
pub fn gave_up(&self) -> bool {
matches!(self.state.lock().conn, Conn::Missing | Conn::Failed)
}
/// The instance told `initialized`, ready or not.
pub fn instance(&self) -> Option<Running> {
self.running().or_else(|| self.initializing.lock().clone())
@@ -346,6 +369,13 @@ impl Server {
self.running().filter(|r| r.generation == generation)
}
/// Instance `generation` is starting or running: not stopped, crashed or replaced.
/// Work it reports before it is ready (a Gradle import answers `initialize` late) still counts.
pub fn is_live(&self, generation: u64) -> bool {
let st = self.state.lock();
st.generation == generation && matches!(st.conn, Conn::Starting | Conn::Ready(_))
}
/// Waits through startup or a planned restart; returns the instance once ready.
pub async fn wait_ready(&self, timeout: Duration) -> Option<Running> {
let mut rx = self.phase.subscribe();
@@ -436,10 +466,12 @@ impl Server {
}
let tail = self.stderr_tail(5);
let cfg = self.effective_config();
let mut message = match (code, &cfg.connect) {
(Some(c), _) => format!("{} exited with code {c}", self.name),
(None, Some(addr)) => format!("{}: the connection to the server ({addr}) closed", self.name),
(None, None) => format!("{} terminated unexpectedly", self.name),
let start_error = self.start_error.lock().take().filter(|(g, _)| *g == generation).map(|(_, r)| r);
let mut message = match (&start_error, code, &cfg.connect) {
(Some(reason), _, _) => format!("{} failed to start: {reason}", self.name),
(None, Some(c), _) => format!("{} exited with code {c}", self.name),
(None, None, Some(addr)) => format!("{}: the connection to the server ({addr}) closed", self.name),
(None, None, None) => format!("{} terminated unexpectedly", self.name),
};
if !tail.is_empty() {
message.push_str(":\n");
@@ -472,6 +504,7 @@ impl Server {
/// Records a crash; returns the backoff before restarting, or `None` once
/// [`MAX_RESTARTS`] instances in a row died before running [`STABLE_UPTIME`]
/// (a server that crashes on every start, however slowly, is not restarted forever).
/// An instance that never became ready has run for no time at all.
pub fn record_crash(&self) -> Option<Duration> {
let uptime = self.started.lock().map(|t| t.elapsed()).unwrap_or_default();
let mut crashes = self.crashes.lock();
@@ -507,7 +540,13 @@ impl Server {
return;
}
*cur = (status, message.clone());
st.events.push_back(IdeEvent::LanguageServer { server: self.id, name: self.name.clone(), status, message });
st.events.push_back(IdeEvent::LanguageServer {
server: self.id,
name: self.name.clone(),
label: self.display_name(),
status,
message,
});
}
fn queue_cleanup(&self, st: &mut State) {
@@ -1028,6 +1067,38 @@ mod tests {
assert!(s.is_busy(), "a reporting task keeps the server busy");
}
#[test]
fn only_time_spent_ready_counts_as_uptime() {
let sink: SharedSink = Arc::new(|_: IdeEvent| {});
let s = server();
for n in 1..=MAX_RESTARTS + 1 {
// Never ready: crashes after a slow start count however long the start took.
let generation = s.begin_start().unwrap();
assert!(s.started.lock().is_none());
s.set_start_error(generation, "it did not answer initialize within 300 s".into());
let delay = s.on_closed(generation, &sink, None).unwrap();
assert_eq!(delay.is_some(), n <= MAX_RESTARTS, "crash {n}");
let message = s.status_message().unwrap();
assert!(
message.starts_with(&format!("{} failed to start: it did not answer initialize", s.name)),
"{message}"
);
s.state.lock().conn = Conn::Idle;
}
assert!(matches!(s.state.lock().conn, Conn::Idle));
}
#[test]
fn an_instance_is_live_from_its_start_until_it_is_retired() {
let sink: SharedSink = Arc::new(|_: IdeEvent| {});
let s = server();
let generation = s.begin_start().unwrap();
assert!(s.is_live(generation), "still starting: its progress (an import) is being watched");
assert!(!s.is_live(generation + 1));
assert!(s.fail_start(generation, Conn::Failed, &sink, ServerStatus::Crashed, None));
assert!(!s.is_live(generation));
}
#[test]
fn crashes_of_short_lived_instances_exhaust_the_budget_whatever_the_spacing() {
let s = server();
+4
Переглянути файл
@@ -341,6 +341,10 @@ pub struct ServerInfo {
pub status: ServerStatus,
/// OS process of the running instance (for memory attribution).
pub pid: Option<u32>,
/// What the user sees as the server when another program serves in its place.
#[serde(default)]
#[specta(optional)]
pub label: Option<String>,
}
/// Read-only contents a language server provides for a non-`file:` URI
+17 -1
Переглянути файл
@@ -32,7 +32,7 @@ pub(crate) fn normalize_uris(value: &mut serde_json::Value) {
if matches!(key.as_str(), "data" | "arguments") {
continue;
}
if matches!(key.as_str(), "uri" | "targetUri" | "oldUri" | "newUri")
if matches!(key.as_str(), "uri" | "targetUri" | "oldUri" | "newUri" | "target")
&& let Some(uri) = value.as_str().and_then(escaped_uri)
{
*value = uri.into();
@@ -247,6 +247,22 @@ mod tests {
assert!(encoded.parse::<lsp_types::Uri>().is_ok());
}
/// `DocumentLink.target` is a URI too: one unescaped link fails the whole `documentLink` response.
#[test]
fn document_link_targets_are_escaped() {
let range = serde_json::json!({ "start": { "line": 0, "character": 0 }, "end": { "line": 0, "character": 1 } });
let mut links = serde_json::json!([
{ "range": range, "target": "file:///D:/My Game/项目/a.xsd" },
{ "range": range, "target": "https://example.com/a b" }
]);
assert!(serde_json::from_value::<Vec<lsp_types::DocumentLink>>(links.clone()).is_err());
normalize_uris(&mut links);
assert_eq!(links[0]["target"], "file:///D:/My%20Game/%E9%A1%B9%E7%9B%AE/a.xsd");
assert_eq!(links[1]["target"], "https://example.com/a b", "other schemes are not touched");
links[1]["target"] = "https://example.com/ab".into();
assert_eq!(serde_json::from_value::<Vec<lsp_types::DocumentLink>>(links).unwrap().len(), 2);
}
/// #3089: jdtls's Introduce Variable edit for a project under a Chinese path names the file
/// with the characters unescaped, and the resolved code action failed to deserialize.
#[test]
+182
Переглянути файл
@@ -146,6 +146,9 @@ impl Handler for MockHandler {
if let Some(gate) = gate {
gate.hold();
}
if let Some(error) = s.errors.lock().get("initialize").cloned() {
return Err(error);
}
Ok(json!({ "capabilities": s.caps.lock().clone(), "serverInfo": { "name": "mock" } }))
}
"shutdown" => Ok(Value::Null),
@@ -268,6 +271,7 @@ fn server_cfg(name: &str) -> LanguageServerConfig {
tsserver_project: false,
tsserver_project_extras: Default::default(),
resync_on_configuration: false,
display_name: None,
pinned: None,
}
}
@@ -2072,6 +2076,29 @@ async fn crash_restarts_and_reopens_documents() {
assert_eq!(hover.contents.value, "hi");
}
/// A restart during a long start (a project import) returns when the instance is up and
/// running, and does not hold up `reconfigure` and `shutdown_all` meanwhile.
#[tokio::test(flavor = "multi_thread")]
async fn restart_does_not_hold_the_reconfigure_lock_while_the_server_starts() {
let h = harness(&[("mock", incremental_caps())]);
h.open("a.mk", "x");
h.wait_status("mock", ServerStatus::Ready).await;
let s = h.launcher.state("mock");
let gate = Arc::new(Gate::default());
*s.initialize_gate.lock() = Some(gate.clone());
let id = h.mgr.servers()[0].id;
let restart = tokio::spawn({
let mgr = h.mgr.clone();
async move { mgr.restart(id).await }
});
eventually(|| gate.entered()).await;
let reconfigured = tokio::time::timeout(Duration::from_secs(2), h.mgr.reconfigure()).await;
gate.open();
reconfigured.expect("a restart still waiting for the server to start must not block reconfigure");
restart.await.unwrap().unwrap();
assert_eq!(h.mgr.servers()[0].status, ServerStatus::Ready);
}
#[tokio::test(flavor = "multi_thread")]
async fn gives_up_after_repeated_crashes_and_manual_restart_works() {
let h = harness(&[("mock", incremental_caps())]);
@@ -2099,6 +2126,118 @@ async fn gives_up_after_repeated_crashes_and_manual_restart_works() {
assert_eq!(h.mgr.servers()[0].status, ServerStatus::Ready);
}
/// One malformed diagnostic costs only itself, not the file's other problems.
#[tokio::test(flavor = "multi_thread")]
async fn a_malformed_diagnostic_does_not_drop_the_others() {
let h = harness(&[("mock", incremental_caps())]);
let (_, path) = h.open("a.mk", "one");
h.wait_status("mock", ServerStatus::Ready).await;
let uri = uri_of(&path);
let range = json!({"start":{"line":0,"character":0},"end":{"line":0,"character":1}});
h.launcher
.state("mock")
.peer()
.notify(
"textDocument/publishDiagnostics",
json!({ "uri": uri, "diagnostics": [
{ "range": range, "message": "first" },
{ "message": "no range" },
{ "range": range, "message": 7 },
{ "range": range, "message": "last" }] }),
)
.unwrap();
eventually(|| {
h.events.lock().iter().any(|e| match e {
IdeEvent::Diagnostics { items, .. } => {
items.iter().map(|d| d.message.as_str()).collect::<Vec<_>>() == ["first", "last"]
}
_ => false,
})
})
.await;
}
/// The `initialize` error is the crash report's reason, not "terminated unexpectedly", and
/// a server that never gets past `initialize` gives up like any other crash loop.
#[tokio::test(flavor = "multi_thread")]
async fn a_failed_initialize_is_reported_with_its_reason_and_gives_up() {
let h = harness(&[("mock", incremental_caps())]);
let s = h.launcher.state("mock");
s.errors.lock().insert("initialize".into(), (-32603, "no workspace".into()));
h.open("a.mk", "x");
let count = |status| h.statuses("mock").iter().filter(|s| **s == status).count();
eventually(|| count(ServerStatus::Crashed) == 4).await;
let messages: Vec<String> = h
.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::LanguageServer { status: ServerStatus::Crashed, message, .. } => message.clone(),
_ => None,
})
.collect();
assert!(
messages[0].contains("failed to start: initialize failed (server error -32603: no workspace)"),
"{messages:?}"
);
assert!(messages[0].ends_with("Restarting…"));
assert!(messages[3].ends_with("restart it manually."), "{messages:?}");
tokio::time::sleep(Duration::from_millis(300)).await;
assert_eq!(s.launches.load(Ordering::SeqCst), 4, "initial start + 3 restarts");
}
/// The client tells servers it retries semantic tokens, inlay hints and code lens that they
/// withdrew with `ContentModified` (`staleRequestSupport`), so it must.
#[tokio::test(flavor = "multi_thread")]
async fn withdrawn_editor_decoration_requests_are_asked_again() {
let h = harness(&[("mock", incremental_caps())]);
let (doc, _) = h.open("a.mk", "let x = 1");
h.wait_status("mock", ServerStatus::Ready).await;
let s = h.launcher.state("mock");
let range = json!({"start":{"line":0,"character":0},"end":{"line":0,"character":1}});
let cases = [
("textDocument/semanticTokens/full", json!({ "data": [0, 0, 1, 0, 0] })),
("textDocument/inlayHint", json!([{ "position": {"line":0,"character":0}, "label": "x" }])),
("textDocument/codeLens", json!([{ "range": range, "command": { "title": "t", "command": "c" } }])),
];
for (method, answer) in cases {
s.errors.lock().insert(method.into(), (-32801, "content modified".into()));
let mgr = h.mgr.clone();
let request = tokio::spawn(async move {
match method {
"textDocument/semanticTokens/full" => mgr.semantic_tokens(doc).await.map(|r| r.is_some()),
"textDocument/inlayHint" => {
mgr.inlay_hints(doc, Range::new(Pos::new(0, 0), Pos::new(0, 9))).await.map(|v| v.len() == 1)
}
_ => mgr.code_lens(doc).await.map(|v| v.len() == 1),
}
});
eventually(|| !s.requested(method).is_empty()).await;
s.errors.lock().remove(method);
s.respond(method, answer);
let answered = tokio::time::timeout(Duration::from_secs(5), request).await.unwrap().unwrap();
assert!(answered.unwrap_or(false), "{method} was not asked again");
assert_eq!(s.requested(method).len(), 2, "{method}");
}
}
/// A server disabled in settings while its crash backoff is still sleeping stays gone: the
/// pending restart must not bring it back as a process nothing lists or stops.
#[tokio::test(flavor = "multi_thread")]
async fn a_server_disabled_during_its_crash_backoff_does_not_restart() {
let h = harness(&[("mock", incremental_caps()), ("other", incremental_caps())]);
h.open("a.mk", "x");
h.wait_status("mock", ServerStatus::Ready).await;
let s = h.launcher.state("mock");
s.peer().kill();
eventually(|| h.statuses("mock").contains(&ServerStatus::Crashed)).await;
h.mgr.set_servers_hook(Box::new(|_, configs| configs.iter().filter(|c| c.name == "other").cloned().collect()));
h.mgr.reconfigure().await;
eventually(|| h.removed("mock")).await;
tokio::time::sleep(Duration::from_millis(1200)).await;
assert_eq!(s.launches.load(Ordering::SeqCst), 1, "no restart after the server was dropped");
}
/// The whole workspace root deleted: the server can never start again
/// (its cwd is gone), so the crash drops the entry instead of restarting.
#[tokio::test(flavor = "multi_thread")]
@@ -2997,6 +3136,49 @@ async fn fallback_servers_only_run_when_the_preferred_one_is_missing() {
assert!(h.mgr.hover(doc, Pos::new(0, 0), Default::default()).await.is_ok());
}
/// A server that is installed but cannot run (crashed for good, or its configuration
/// can't launch it) hides its fallbacks no longer than one that is missing.
#[tokio::test(flavor = "multi_thread")]
async fn fallback_servers_take_over_from_a_preferred_one_that_crashed_for_good() {
let h = harness_with_fallbacks(&[("main", incremental_caps()), ("alt", incremental_caps())], &["alt"]);
let (doc, path) = h.open("a.mk", "x");
let main = h.launcher.state("main");
let count = |status| h.statuses("main").iter().filter(|s| **s == status).count();
for n in 1..=4 {
eventually(|| count(ServerStatus::Ready) == n).await;
// Restarting after a crash is not giving up: the fallback stays away.
assert!(h.statuses("alt").is_empty(), "alt started after crash {}", n - 1);
main.peer().kill();
eventually(|| count(ServerStatus::Crashed) == n).await;
}
h.wait_status("alt", ServerStatus::Ready).await;
let uri = uri_of(&path);
eventually(|| h.launcher.state("alt").docs.lock().contains_key(&uri)).await;
assert!(h.mgr.hover(doc, Pos::new(0, 0), Default::default()).await.is_ok());
// Restarting the preferred one by hand gives the documents back to it.
let id = h.mgr.servers().into_iter().find(|s| s.name == "main").unwrap().id;
h.mgr.restart(id).await.unwrap();
eventually(|| h.removed("alt")).await;
assert_eq!(h.mgr.servers().iter().filter(|s| s.name == "alt").count(), 0);
}
#[tokio::test(flavor = "multi_thread")]
async fn fallback_servers_take_over_from_a_preferred_one_that_cannot_be_launched() {
let h = harness_with_fallbacks(&[("main", incremental_caps()), ("alt", incremental_caps())], &["alt"]);
h.mgr.set_config_hook(Box::new(|cfg, _| {
if cfg.name == "main" {
cfg.launch_error = Some("needs JDK 21".into());
}
}));
let (doc, path) = h.open("a.mk", "x");
h.wait_status("main", ServerStatus::Misconfigured).await;
h.wait_status("alt", ServerStatus::Ready).await;
let uri = uri_of(&path);
eventually(|| h.launcher.state("alt").docs.lock().contains_key(&uri)).await;
assert!(h.mgr.hover(doc, Pos::new(0, 0), Default::default()).await.is_ok());
}
#[tokio::test(flavor = "multi_thread")]
async fn a_server_requiring_a_root_marker_serves_only_files_below_one() {
// `framework` next to `main` (one instance for the whole project), like the
+3
Переглянути файл
@@ -426,6 +426,7 @@ async fn missing_binary_is_reported_with_install_hint() {
tsserver_project: false,
tsserver_project_extras: Default::default(),
resync_on_configuration: false,
display_name: None,
pinned: None,
};
let (mgr, events) = manager(dir.path(), LanguageRegistry::from_configs(vec![lang("nope", "nope", server)]));
@@ -698,6 +699,7 @@ async fn typescript_language_server_end_to_end() {
tsserver_project: false,
tsserver_project_extras: Default::default(),
resync_on_configuration: false,
display_name: None,
pinned: None,
};
let (mgr, events) = manager(&root, LanguageRegistry::from_configs(vec![lang("typescript", "ts", server)]));
@@ -750,6 +752,7 @@ fn npm_server(tools: &Path, name: &str, bin: &str, init: Option<serde_json::Valu
tsserver_project: false,
tsserver_project_extras: Default::default(),
resync_on_configuration: false,
display_name: None,
pinned: None,
}
}
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
e616b87b24371581ad648f9c7072259de7732d122b5887b488f7bd480ebd69c2
0b77e7a414ec3b0dd1f41e5658b981f059dd842743dbe2f06e5d428f5b9570f9
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
75eeecc467bff141bda14f1f080633348af348f7259e793be81e6548d2008669
6f7d6527837c8e5c86c377479cceef5116decd61666b28f36dd11332e36e9934
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
4aaf7754e895f022d34dad05bd2b8018e3a61ba18771b8c95d9f070cb212530c
c1ac86a5b4d45b55013e6e00dcb83deda33b759ecc366c3fdf57739bdd22cbff
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
e573144879a7caa0975e17b233476eb317ff54bef23fd8fa64280d3649bad700
e4d8d8f5f5849b912b2c0ebee6b91d60796c3070929d7698323e8ce4f200a38a
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
36dd40ef751a617f1ad59491312889c12481b05666e580f21f7c2552df76edbc
e2e2fc44ad9cae9cf26a5bf706768d3786502f23db6994dc29695deb249ed73f
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
724ed3bbd828e4fc1925c1b555bdaec9268633cb587806a661f8838e5a540545
61e1f0c43bbfddc35dcf40685169eb273777ddca13da534626e1699b47e8eac3
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
1a3685234f1b3b0957184c44c603c0eefe7e3084b88d0df32da268b29292ee92
2ea1c1693db8638db996c9c47597f0c1635ef0a9c7dfb33685c515b7af95e6a6
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
4283de821d69a18438f58062e4bc28b63ba5025cc5f9fa51f21ff475b677a6a5
3c788f6092b229484a5d94fd0e6f4651afc88dd0ab1b22b067d84614a6a0c85e
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
0f25487b8e488e5ec01fabd2951bdb03d80bc26e4e9aa508394a7f9cd09d1759
18273599cc459833d6a30185a5f0ba127cdb4e22bc054a973571e530eb3a85ca
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
80f9dc28e4f168dd22ccee1c53e1f6b08c446622eb10f07de29be5fc45e9fb1c
c9fafba362fa0745d7ee1685efe5582d432510003c04c98100eddd557bd1e2cf
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
5355cb4ef373eba3c54806eb00294fcd5fccb3cf05dae3e1c2a4ad45ab958f03
4121009913709edeca8c5cb52d88dfcc2b16867bc06859768f73c17cf9c456a2
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
fc75ede42e6db7619fa28fc8999f091d39e3c1ed401356db3a3f07de54287c99
b55bd202a06f5643c4f70abb6fb473fbe53af657f4e21a6d13581a23a726e085
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
8793601496f1727cf912e4cdf86ec48fc340a17585098e98dee52a125cb8a16d
11a12228df67f9b94bd52375b5dddb2d955f7e5f88d7f9ea0a430b1db128f6ca
Бінарний файл не відображається.
+1 -1
Переглянути файл
@@ -1 +1 @@
fb2db14a645cad42941d20a25a3ff87ca6fdee635fdb40953383ff30ce16af8a
553ce6c57052f8ff2ea2526e88762d5a0aee2489cda192fc7ab708766078ccf7
Loaded 100 of 452 files, more files were not shown because too many files have changed in this diff. Show more