Порівняти коміти

...
Автор SHA1 Повідомлення Дата
Gabriel Gondim e8bc577e9d test(deployment): cover Remote Host Download to… wiring (#2763)
Covers Download to… in the Remote Host view: the Project view selection preselects the folder picker (its parent folder for a selected file), cancelling the picker downloads nothing, and the chosen folder receives the remote paths.

Co-authored-by: GGVidal <[email protected]>
2026-11-12 02:18:54 +08:00
rohitkumardandaтаrohitdanda b1348ee03a fix(lsp): open network-share files and upper-case drive letters from language servers (#2794)
On Windows `uri_to_path` rejected any `file:` URI host other than `localhost`, so network-share results (`file://server/share/a.ts`) from language servers were dropped from diagnostics and renames and opened as read-only virtual documents; such hosts now map to `\\server\share\...`, and `path_to_uri` writes UNC paths in the same RFC 8089 form. Lower-case or escaped drive letters (`file:///c%3A/x`) are upper-cased in `uri_to_path` and `to_ipc`, so the UI no longer treats `c:/x` and `C:/x` as different files. Non-Windows behaviour is unchanged.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:17:35 +08:00
rohitkumardandaтаrohitdanda e99d05d7a8 fix(vfs): retry saving on Windows while another process briefly holds the file (#2791)
The atomic save renames a temp file over the target, and Windows refuses that while another process (antivirus or indexer scan, sync client, a program reading its config) has the file open without delete sharing, so the save failed with "Access is denied" or a sharing violation. The rename is now retried for up to about a second on Windows errors 5, 32 and 33; other errors and other platforms fail at once.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:14:29 +08:00
rohitkumardandaтаrohitdanda 0e4cf1059a fix(workspace): keep repeated inserts at one position in workspace edits (#2793)
`WorkspaceEdit::deduplicate` dropped any edit already present for the file, including repeats inside one `TextDocumentEdit`, so a server edit inserting the same text twice at one position (two closing brackets, two blank lines) lost one insert. It now only drops edits an earlier alias entry for the same file already contributed and keeps every edit of one entry; the result stays idempotent.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:13:19 +08:00
rohitkumardandaтаrohitdanda b4bcd8cce7 fix(editor): keep a chosen file encoding when the file changes on disk (#2792)
Reloads after an external change (`reload_from_disk`) and after undoing a workspace edit (`reload_resource_buffers`) always re-ran encoding detection, so a file the user had reloaded or converted to a legacy encoding came back as mojibake and later saves used the guessed encoding. Documents now record that the encoding was chosen and both reload paths decode with it; files whose encoding was never chosen are still detected again.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:12:13 +08:00
rohitkumardandaтаrohitdanda cb5e7081fa fix(platform): keep delivering IDE events after one handler throws (#2790)
Broadcasts called their listeners in a bare loop, so one throwing handler kept the rest (including the catch-all "*" IDE event handlers) from seeing the event. The IDE event bus, local diagnostics, endpoint-change and navigation-modifier broadcasts now go through `notifyEach`, which logs a throwing listener and moves on. `onIdeEvent` wraps each subscription so unsubscribing one of two identical handlers keeps the other.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:09:24 +08:00
rohitkumardandaтаrohitdanda 034b2b9f65 fix(app): keep the window responsive while local history, shelves and run files load (#2789)
Tauri runs commands declared without `async` on the main thread, so Local History, shelf, change-list and run-configuration reads and writes, `.env` loading and the line diff of two arbitrary texts blocked input, painting and every other IPC call while they ran. They are now async and run on the blocking pool; IPC signatures are unchanged. `vcs_changelists_save` stays synchronous so its fire-and-forget saves land in order. A guard test keeps these commands async.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:06:22 +08:00
l0ng-ai 7ea3c59d05 fix(vcs): list local changes when the work tree has a directory git can't read (#2800)
gitoxide's status stops at the first IO error, so one unreadable directory failed the whole status and the Commit window showed no changes, where git warns and lists the rest. Status now asks git (porcelain v2) when gitoxide fails; a test checks the parser against gitoxide on a full status matrix.

Fixes #2769
2026-11-12 01:59:27 +08:00
l0ng-ai 6717b0948a fix(vcs): open a log commit's file diffs in one reusable preview tab (#2798)
Double-click, Enter or Show Diff on a file in the Git log's changes opened a new editor tab each time. It now opens the editor's preview tab, as the Commit window does, and while that tab shows one of the pane's diffs, selecting another file shows it there. Pinned tabs are left alone.

Fixes #2768
2026-11-12 01:54:10 +08:00
l0ng-ai 8302cd1d6f fix(debugger): clean up Delve's build and hide Run/Debug when their last tab closes (#2797)
Delve in DAP mode builds the debuggee as __debug_bin<digits> on every platform; the post-session sweep only accepted that form on Windows, so the binary stayed in the project on macOS and Linux. Closing the last session tab also left an empty Debug (or Run) tool window open; both now hide with their last tab.

Fixes #2766
2026-11-12 01:49:21 +08:00
l0ng-ai c05670f4fd test: add framework_files to the LanguageConfig literals in ide-dap and ide-lsp tests (#2796)
LanguageConfig gained framework_files in #2781; the test files that spell the struct out no longer compiled.
2026-11-12 01:42:20 +08:00
l0ng-ai 2f9658c367 fix(vcs): smooth-scroll again when moving between differences (#2795)
Next/Previous Difference also dispatched a scrollIntoView effect, which set scrollTop in the same measure as the smooth scroll and cut the animation off. The smooth scroll already centres the caret, so the effect is removed.

Fixes #2764
2026-11-12 01:38:03 +08:00
l0ng-ai 40aa7e4899 fix(java): resolve navigation and usages against the text just typed (#2759)
Right after typing in a Java file, Show Usages answered for the wrong symbol and Go to Declaration could do nothing. Usages, Find Usages and hierarchy now wait for the editor's edits to sync before sending the caret position, and the java-usages commands (usages, implementations, hierarchy, configuration) wait for jdtls to reconcile queued document changes before resolving a position (bundle 1.13.0, rebuilt reproducibly from source). File-system events (e.g. the build writing target/classes) now only drop cached navigation answers instead of discarding an in-flight Go to Declaration.
2026-11-12 01:33:25 +08:00
arookieofcтаClaude Code fba1f02375 feat(hotkeys): warn when the OS grabbed an IDE shortcut (#1876) (#2683)
* feat(hotkeys): warn when the OS grabbed an IDE shortcut (#1876)

Probe single-stroke Alt bindings at startup with RegisterHotKey and
notify once per session when another app owns one, with an action that
opens Keymap for reassignment.

Co-Authored-By: Claude Code <[email protected]>

* fix(hotkeys): satisfy clippy::collapsible_if on the F-key parse

Co-Authored-By: Claude Code <[email protected]>

---------

Co-authored-by: Claude Code <[email protected]>
2026-11-12 01:33:04 +08:00
l0ng-ai 053ca43368 fix(usages): say at once that Show Usages is searching (#2760)
Show Usages showed nothing until its popup opened (300 ms after the first usages, or when the whole answer came), so a slower search looked like the shortcut had done nothing. It now shows "Searching for usages…" at the caret until the popup or the answer replaces it, and pending hints (such as Go to Implementation's) appear after 50 ms instead of 250 ms.
2026-11-12 01:30:18 +08:00
Vadim LazukтаInf1mum 8fedc2efba fix(vcs): stop filtered logs from fanning one dashed lane per merge path (#2788)
A filtered log connects each visible commit to the nearest matching ancestor along every merge path, and in merge-heavy histories one row could collect dozens of dashed connections, each getting its own lane, so the graph column stretched the log horizontally. pruneFilteredEdges now drops a dashed connection whose target the commit's other connections already reach (shallowest first), so reachability is unchanged with one lane instead of one per hidden merge path. Real parents, collapsed fragments and unfiltered histories are untouched; parent/child navigation follows the same pruned graph.

Fixes #2670

Co-authored-by: Inf1mum <[email protected]>
2026-11-12 01:28:32 +08:00
zhangdagouтаPLZY 2f2fa79cbe fix(build-tools): select a clicked completion in Execute Maven Goal / Gradle Task instead of appending it (#2785)
Clicking a completion appended it to the command line and re-filtered the list, so repeated clicks kept accumulating goals. A click now selects the row and shows its command line in the input; another click switches to that row; the list filters only on typing; Enter or a double-click runs it. Goals and tasks already on the command line are no longer offered again.

Co-authored-by: PLZY <+[email protected]>
2026-11-12 01:26:36 +08:00
Deng Liuтаdengliu 65d49619dd fix(terminal): invalidate selection highlight cache on scrollback trim (#2784)
The selection-occurrence cache keyed matches by absolute buffer row and relied on onRender to invalidate edited lines, so a full scrollback trimming from row 0, CSI 3 J or clear() shifted cached margin and scrollback rows without invalidation, and a selection trimmed off row 0 left its boxes behind. The cache now anchors a marker at row 0 while populated (any top trim disposes it and clears the cache), invalidates active-screen rows across baseY advances on write, and clears the overlay when a trim removes the selection.

Co-authored-by: dengliu <[email protected]>
2026-11-12 01:24:20 +08:00
Andrea Basileтаandreabasile 7f28161869 feat(languages): Django templates (#2781)
Adds a Django Template language: .html/.txt files in a templates folder of a project with manage.py (new [[framework-files]] detection), a Lezer grammar for tags, variables, filters and comments with HTML mounted between them, block folding/indent/matching, {% %} / {{ }} / {# #} pairing, end-tag insertion and completion, per-line {# #} comments, HTML language server and Emmet in the HTML parts, a banner offering the language for HTML files with Django tags elsewhere, a fixture and docs. The built-in HTML syntax check skips Django templates, whose tags the HTML grammar reports as errors.

Refs #2581

Co-authored-by: andreabasile <[email protected]>
2026-11-12 01:22:23 +08:00
l0ng-ai fc1493e1a4 feat(go): report upgrades, vulnerabilities and deprecated modules when go.mod opens (#2787)
gopls reports available upgrades only after its "Check for upgrades" code lens runs, never reports deprecated modules, and checks vulnerabilities only on demand. Opening a go.mod now runs the upgrade check once per file and session, gopls' vulncheck setting is on for imports (Go, go.mod, go.work), and `go list -m -u` marks deprecated requirements on open and save. go.mod highlighting also covers godebug, tool and ignore.
2026-11-12 01:14:56 +08:00
l0ng-ai efda6834ad feat(search): list directories in Go to File and Search Everywhere (#2783)
Go to File and the Files section of Search Everywhere listed only files, so a
folder could not be found by name. Directories from the index's folder search
now rank with the files (a file first on equal scores), with a folder icon, and
choosing one selects it in the Project view. A file type filter or a
File.kt:42 query still lists files only.
2026-11-12 00:54:21 +08:00
zhangdagouтаPLZY e92e867019 feat(vcs): show an added or deleted file's diff as one full-width pane (#2779)
A diff of an added or deleted file showed an empty half next to the existing side. It now shows the side that exists alone, full width, with its lines tinted and the viewer-mode buttons disabled, as IDEA's one-side viewer does. The saved viewer mode is unchanged.

Co-authored-by: PLZY <[email protected]>
2026-11-12 00:47:35 +08:00
zhangdagouтаPLZY 8d8c7a481b fix(vcs): align the diff titles and find bar with the panes, and title revisions by commit and path (#2778)
After the side-by-side divider was dragged, the right half of the title row and of the find bar stayed at 50% and no longer lined up with the right pane. Both now follow the right pane's actual start. A revision side's title shows its short commit id (HEAD resolved) and the file's path in the repository.

Co-authored-by: PLZY <[email protected]>
2026-11-12 00:47:25 +08:00
zhangdagouтаPLZY 0b3ca2fc9a feat(vcs): put the left pane's line numbers next to the divider in side-by-side diffs (#2776)
The left editor of a side-by-side diff is mirrored: its line numbers sit on its right, next to the divider, as in IDEA. CodeMirror's lineNumbers() has no side option, so the viewer adds an equivalent line-number gutter on the after side for that pane.

Co-authored-by: PLZY <[email protected]>
2026-11-12 00:47:15 +08:00
Ernstjan FreriksтаEjaYF 2414773d61 fix(agents): attach files dropped from outside the project (#2767)
Dropping a file from the OS onto the agent message box did nothing when it was outside the project (a screen recording on the Desktop, say): the drop handler only turned project paths into @ references and skipped the rest silently. Such files now show as attachment chips next to the image thumbnails and reach the agent as resource links. A dropped image over the 5 MB inline limit, or one that cannot be read, is attached by path instead of inlined. The box highlights while an OS drag is over it. Remote windows still skip paths outside the project.

Co-authored-by: EjaYF <[email protected]>
2026-11-12 00:45:19 +08:00
rohitkumardandaтаrohitdanda e4181c36ea fix(agents): roll a local task back to where it started when HEAD moved (#2774)
A task working in the project's own checkout compared and rolled back its changes against the current HEAD. Once HEAD moved back or to another branch during the task (git reset --soft, an undone commit, a checkout), the user's own work showed up as the task's changes and Rollback overwrote it; in a repository without commits, Changes and Rollback failed.

Local tasks now use the commit they started from: files that are as they were at the start are not listed, and Rollback and the diff base use the start commit, or HEAD when the user committed on top of it so those commits are kept. In a repository without commits new files are listed and rollback deletes them. Worktree tasks are unchanged.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 00:42:35 +08:00
usong 5063f6756e fix(editor): correct library source encoding and Reload with Encoding (#2775)
Library sources in a non-UTF-8 charset (e.g. GBK) opened garbled, and Reload with Encoding failed on them with a read-only error: library and virtual documents bypassed the editor's encoding pipeline (archive entries read as strict UTF-8, jdt: documents labelled utf-8, set_encoding rejecting every read-only virtual document).

jar: and jdt: sources are now decoded through the same VFS pipeline as files on disk (detection, BOM, UTF-16, explicit reload) and report the charset used. Reload re-decodes virtual documents while keeping them read-only: jar: entries are re-read; for jdt: sources the jdtls source attachment encoding is updated (waiting for Maven's background refresh, retrying once) and the attached entry re-read. Reload is disabled for decompiled and server-generated content and for read-only containers such as the JDK.

Co-authored-by: usongon <[email protected]>
2026-11-12 00:41:22 +08:00
rohitkumardandaтаrohitdanda 1a2f5308a0 fix(editor): keep syncing edits after a document snapshot fails (#2777)
When an edit was rejected as out of sync and the rebase snapshot then failed, the in-flight edit was never cleared, so flush() returned early forever: save, format and refactorings waited and the backend kept the old text. A failed rebase snapshot now resends the edit (bounded by MAX_REBASES, then the existing hard resync), and idle waiters are no longer released while an edit is pending.

When hardResync's own snapshot failed, the in-flight and buffered changes were dropped although the editor still showed them, so later edits were expressed against text the backend never had. The difference between the confirmed and local text is now kept as the buffer and sent with the next flush.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 00:34:03 +08:00
l0ng-ai cd04593d04 refactor(settings): open Project Structure only as its own dialog (#2744)
Settings no longer lists a Project Structure page; File › Project Structure…
(⌘; / Ctrl+Alt+Shift+S) opens the standalone dialog, which already had the
same sections. Arrow keys in the dialog's sidebar follow its navigation order.
2026-11-12 00:31:51 +08:00
l0ng-ai 38241e7fae feat(deployment): download Remote Host items to a chosen folder or by dragging (#2780)
Remote Host › Download to… asks for a local folder (the Project view's selected
folder preselected) and copies only the selected files and folders, each under
its own name, instead of recreating the server path below the project root.
Dragging items from Remote Host onto a folder in the Project view does the same.
Download to Project keeps using the mappings.
2026-11-12 00:29:34 +08:00
l0ng-ai 2e6d6a05b9 fix(plugins): accept empty navigation anchor ranges as positions (#2773)
A document indexer that anchors a whole-file definition at the file's start
(an empty range at 0:0) had the whole document rejected with "invalid
navigation anchors". Empty ranges inside the document are now accepted:
Go to Declaration lands on them, while Find Usages and Rename skip them so
a rename never inserts the new name at that position. The error for a
document that is still rejected names the plugin and the file.
2026-11-12 00:11:15 +08:00
ntbang0901 c73d858895 feat(debugger): debug Kotlin with kotlin-lsp's debugger (#2651)
kotlin-debug-adapter (fwcd) is unmaintained: its evaluate only resolves plain local names and it reports breakpoints verified without checking. Kotlin debugging now uses the debugger inside JetBrains kotlin-lsp once its project import has finished, attached to the JVM the IDE starts through the existing JDWP link. kotlin-debug-adapter stays the fallback, with a notice saying why (not installed, still importing, import failed, cannot start).

kotlin-lsp's DAP quirks (step stop reasons, thread ids from earlier stops, frame names, file: URIs) are handled in one place, and its invalidated events reload the Variables pane and watches.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-12 00:05:41 +08:00
l0ng-ai 6e5486586c fix(i18n): restore the comma missing after the local data directory strings in zh.json (#2771) 2026-11-12 00:03:13 +08:00
arookieofc 93642b3741 feat(settings): custom local data directory with one-click migration (#1722) (#2677) 2026-11-11 23:53:24 +08:00
Gabriel Gondim b06f2e27ea feat(settings): Color Scheme › Console Colors for the terminal's ANSI colors (#2751)
Settings › Editor › Color Scheme gets a Console Colors page, as in IntelliJ IDEA: the console background and the 16 ANSI colors, used by the terminal and consoles. A light IDE theme no longer forces a light terminal. IDEA's CONSOLE_* keys from .icls schemes and IntelliJ theme imports set the same colors.

Closes #2698

Co-authored-by: GGVidal <[email protected]>
2026-11-11 23:40:04 +08:00
Muuтаmuutot 86f69b7fca feat(settings): complete the Python color page as PyCharm's full model (#2761)
The Python page of Settings › Editor › Color Scheme now has PyCharm's 38 attributes in its order and groups, all scoped to Python: an override lands in the editor block for Python files instead of recolouring every language, and unset values fall back to the shared scheme value. The seven attributes that had no default value (parentheses, brackets, braces, nested function, method call, type annotation, f-string format spec) now resolve, and .icls PY.* keys map to their Python attributes. Attribute ids and saved overrides are unchanged.

Co-authored-by: muutot <[email protected]>
2026-11-11 23:38:25 +08:00
Vadim LazukтаInf1mum fefa135f5a feat(docker): pick an SSH configuration for TCP/SSH Docker connections (#2737)
The TCP / SSH host kind of a Docker connection now offers the IDE's SSH Configurations (with … to open Settings › Tools › SSH Configurations), as IntelliJ IDEA does. Picking a configuration with OpenSSH config authentication fills the Engine API URL with ssh://[user@]host[:port], keeping the alias so the system ssh docker runs applies the entry's key, user and jump hosts; an existing ssh:// URL shows its configuration again. Password and key-pair configurations are listed but explained as unusable for now, since docker over SSH goes through the system ssh.

Refs #939

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 23:36:44 +08:00
Vadim LazukтаInf1mum 8fe2d35d62 feat(editor): pair <> where a generic type list starts (#2738)
Under Smart Keys › Insert paired brackets, < now pairs into <> where the syntax tree says a type argument / parameter list starts, as in IntelliJ IDEA: after a type name or generic declaration name in Java, TypeScript, Kotlin and C#, and Rust types, fn names and turbofish ::<. Comparisons (a < b, i <= n, x << 2), tags and literals never pair; > steps over the inserted half and Tab leaves it. Languages without a rule don't pair.

Closes #2700

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 23:34:24 +08:00
rohitkumardandaтаrohitdanda 21046c3047 fix(platform): open only http(s) and mailto URLs externally (#2755)
openExternalUrl handed any string to the system opener, and plugins and rendered content (Markdown, notebook output, editor links) reach it with arbitrary input, so file: or app-launching schemes such as ms-msdt: and search-ms: could be passed to the OS. It now opens only http:, https: and mailto: URLs and ignores anything else, including malformed or whitespace-padded strings.

Co-authored-by: rohitdanda <[email protected]>
2026-11-11 23:32:18 +08:00
l0ng-ai 017561a20b fix(libraries): list the JDK for an .idea project only when it is a Java one (#2765)
A project without a build manifest got a JDK under External Libraries as soon
as it had an .idea folder, which GoLand, PyCharm and WebStorm write too: a Go
project showed < jdk-25 >. The folder now counts only when misc.xml names a
Java SDK or language level, or a module file is a JAVA_MODULE.
2026-11-11 23:31:36 +08:00
l0ng-ai d3a68b2754 fix(lsp): ask again when a server withdraws a rename while analysing (#2762)
A server still analysing an edit may answer textDocument/rename with
ContentModified (the Dart analysis server does when the document changed
under it). That maps to a cancellation, so the rename the user confirmed was
dropped without a word. The rename is now asked again a few times, 300 ms
apart, and if it keeps being withdrawn the error says so.
2026-11-11 23:28:22 +08:00
l0ng-ai e98a560523 fix(run): open .NET Project configurations as the form with a project field (#2757)
The .NET Project type had no way back from a command line to its fields, and a
new configuration starts without params, so the editor fell back to the plain
Command/Arguments form: there was no .NET project field, and Debug failed with
"Select a .NET project". dotnet run command lines the form can express now map
back to its fields, for new configurations and ones typed as a command.

Fixes #2733
2026-11-11 23:21:53 +08:00
l0ng-ai 6983e8ec2f feat(color-scheme): add a C# page to Settings › Editor › Color Scheme (#2756)
C# gets its own page like Go, Java, Kotlin and PHP: every Language Defaults
attribute can be overridden for C# editors only, with a C# preview. Language
sub-page ids are now slugged (C# → csharp): the '#' in editor.colorScheme.c#
made the page unreachable from the settings tree.
2026-11-11 23:17:46 +08:00
l0ng-ai dc355602a9 feat(http-client): go to a {{variable}}'s definition (#2754)
Navigate › Declaration (⌘-click, ⌘B) on {{name}} in a .http file now lists
its definitions in lookup order: the file's @name = … line, then the key in
each environment (and $shared) of the nearest http-client.private.env.json
and http-client.env.json. Dynamic variables ($uuid…) have none.
2026-11-11 23:13:42 +08:00
l0ng-ai 924fa9259d fix(plugins): refuse a command while its plugin waits for an answer in another window (#2753)
A plugin runs one call at a time, and a prompt waits for its answer without a
limit. A command run from another window while the plugin's prompt was open
queued silently behind it and ran only after the prompt was answered. It is
now refused with a message naming the reason, and the window that holds the
prompt is brought to the front.

Fixes #2673
2026-11-11 23:04:11 +08:00
l0ng-ai 24deeb0dbb fix(i18n): drop repeated zh catalog keys and test that every key appears once (#2752)
Merges had added eleven zh.json keys a second time (Docker, Shell, the language
names, Add Service ({kind}), Others, Servers). JSON.parse silently keeps the last
copy, so nothing failed. The catalogs test now reads each catalog file and fails
on a key listed twice.
2026-11-11 22:58:21 +08:00
renyun2таrenyun2 0ce7f6e430 feat(agents): expose Cursor effort and Fast as separate options (#2705)
Cursor's ACP agent only offers thinking effort and a separate Fast switch when the client advertises parameterizedModelPicker. OpenIDEA now advertises it, and when an ACP agent is picked in the Home composer before any session has reported its options, a short discovery session (session/new without MCP servers, killed afterwards or on timeout, once per agent per run) records its config options and effort levels. Effort and Fast show inside the model control, a new Cursor task defaults Fast to off, and config changes show immediately and roll back if the agent rejects them.

Co-authored-by: renyun2 <[email protected]>
2026-11-11 22:54:44 +08:00
misswellтаmisswell 84fd0d0a60 fix(build): skip JaCoCo instrumentation in IDE compile builds (#2686)
Projects that bind jacoco-maven-plugin's offline instrument goal to process-classes had target/classes rewritten by the IDE's own Build/Rebuild and launch builds, so a later Debug/Run without the JaCoCo agent failed with NoClassDefFoundError. The Maven steps of Build and Rebuild now pass -Djacoco.skip=true; artifact (package) builds and coverage runs are unchanged, and Automatic launches stay incremental.

Fixes #2676

Co-authored-by: misswell <[email protected]>
2026-11-11 22:50:57 +08:00
Kamal ElFadlтаelfadl 96c99e0396 feat(agents): add a command execution policy for ACP agents (#1165)
Settings › Agents › Command Execution Policy (global or per project) lists commands an agent may run without asking and commands it is never allowed to run. A permission request for a shell command is answered from the policy: a blocked pattern (glob, checked against the whole command and every part of a chain) rejects it; it is approved only when every part of a &&/||/;/|/& chain matches an allowed entry exactly or up to a word boundary and none uses command substitution, redirection, subshells, here-docs or leading variable assignments. Everything else prompts as before. The allowed list starts empty; the blocked list ships common destructive commands.

Closes #1152

Co-authored-by: elfadl <[email protected]>
2026-11-11 22:50:17 +08:00
arookieofcтаa-rookie-of-C-language fd4838fc36 feat(agents): create, remove and import custom ACP agents (#2690)
Settings › Agents › Coding Agents can add a custom ACP agent (name, id, command, arguments, environment) without writing its agent file by hand, remove one, and import the agent_servers entries of a JetBrains acp.json. Built-in and ACP Registry ids are refused, ids are validated as file stems on save and on remove, and colliding or incomplete acp.json entries are skipped with a reason.

Closes #228

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:45:36 +08:00
arookieofcтаa-rookie-of-C-language 4873905884 feat(agents): accept worktree task changes into the local checkout (#2704)
The Changes pane of a worktree task gets an Accept action next to Rollback: after confirmation it copies the task's changed files (all, or the selected ones) from the task worktree into the local checkout, uncommitted. A file the checkout has changed itself is left alone and listed in the notification instead of being overwritten; running and local tasks are refused.

Closes #1777

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:40:49 +08:00
arookieofcтаa-rookie-of-C-language 6b37788971 test(lsp): go.mod/go.work reach gopls with the language id it understands (#2719)
Pins the go.mod serving contract: opening go.mod through LspManager with the bundled registry sends didOpen to gopls with languageId go.mod, scoped to the module root, and gopls's pushed diagnostics reach Problems. The gomod/gowork gopls entries now use the same initialization options as the go entry, so startup options no longer depend on which file launches the server first.

Refs #2248

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:33:33 +08:00
arookieofcтаa-rookie-of-C-language 44bf648bee fix(tomcat): rebuild the WAR when web resources or dependency modules change (#2722)
The pre-launch up-to-date check only compared the WAR with the WAR module's own src and build files, so a save in a dependency module (mapper XML in a DAO module), in web content outside src (custom warSourceDirectory / webResources), or in any module when the config had no cwd (the check fell back to the project root) left the old WAR deployed. The check now resolves the WAR module from the selected artifact when cwd is missing, includes its web resource directories, and includes the sources and build files of every module in the build model.

Fixes #2685

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:32:36 +08:00
Vadim LazukтаInf1mum a0c3249368 fix(linux): keep the first-map hint under Mutter's auto-maximize share (#2520)
Wayland has no primary monitor, so the first-map size hint fell back to scale 1 with no work-area limit and requested the saved physical size as logical. On a HiDPI output the first mapped area exceeded Mutter's 80% share and the window opened maximized before the saved geometry was restored. The hint now converts with the largest output scale and shrinks proportionally to stay within the tightest output's share; X11 keeps its primary-monitor path.

Fixes #2400

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 22:29:05 +08:00
l0ng-ai 757412cb73 fix(security): reach content roots in plugin quick fixes and keep diagrams local
- A plugin quick fix may change or open files in the project's content
  roots (a project window group's members, extra Project Structure
  folders), still never git's own files; before, it was disabled for
  every file outside the primary root.
- `git init` (New Project › Create Git repository) uses the configured
  git executable again.
- Mermaid output is cleaned before it is shown: remote image sources,
  external `url()`, `@import` and address strings in its CSS are
  removed, so `style`/`classDef` values and HTML labels fetch nothing.
- The reply image e2e test loads a web image only after its button is
  clicked; the post-checkout hook test runs in a trusted scope.
- Capability docs list `project.write` as high risk.
2026-11-11 22:18:48 +08:00
Fabio Nunes Cerqueira a497178b3e fix(security): close the findings of GHSA-442p-gxfg-qxv9
- Restricted Mode git: reads run without a signature verifier, hooks
  (including config-defined ones), an external diff, filters or a lazy
  fetch, also inside submodules; every repository and every git starts
  restricted until the project is trusted.
- project.write and plugin quick fixes cannot reach git's own files under
  any spelling or through links; project.write is a high-risk grant.
- Built-in web server: each served root has its own port and token; a
  file outside the project is served with its folder minus dot names, or
  alone when that folder is the home folder or above it; no fixed port.
- Images, references, Markdown links and language-server links reach no
  network share other than the one already open.
- Marketplace: entries whose id or version no manifest may carry are
  dropped; an index read over the network names nothing on this machine;
  network reads are bounded in size and time.
- Agent replies load remote images on a click; the Markdown preview and
  Mermaid configuration no longer fetch from other hosts.
2026-11-11 22:18:48 +08:00
ntbang0901 8b179992e8 feat(profiler): profile allocations and live objects of JVM runs (#2650)
The Profiler gets two more modes for JVM runs through async-profiler: allocations (where the program allocates, in bytes) and live objects (what is still alive at the end). A live-object profile collects garbage first: run executors get a beforeStop hook, bounded to 12 s, that runs a GC before the stop signal; a program that ends by itself says that unreachable objects may show as live.

Part 3 of #2217.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 22:13:38 +08:00
wangjunboit 85541b3663 feat(vcs): show a diff in a separate window (#2231)
Changes trees get Show Diff in New Window: the diff opens in its own native window (labelled diff-*, with the same capability as comparison windows), with the full diff toolbar, editable working-copy side and a save prompt for unsaved edits on close. Showing the same diff again reuses its window.

Co-authored-by: wangjunboit <[email protected]>
2026-11-11 22:07:04 +08:00
l0ng-ai 0418a2ce01 fix(java): explain a Maven build plugin that doesn't resolve when m2e only logs it (#2745)
When a build plugin such as maven-resources-plugin can't be resolved
(e.g. the default version is missing from a company mirror), m2e fails
the import's last step and every build but puts no marker on pom.xml,
so Problems stays empty and the "Maven project could not be configured"
notification never appeared. The notification now also fires for these
logged failures, once per plugin whichever way jdtls reports it.
2026-11-11 21:39:00 +08:00
l0ng-ai c8639b38ba fix(libraries): write only SDKs in angle brackets under External Libraries (#2743)
Every External Libraries group was shown as < name >, so a Maven or Gradle project listed hundreds of bracketed dependencies. As in IntelliJ IDEA, only SDKs (JDK, Python, Ruby, Go, Rust) keep the brackets; dependency groups such as Maven: g:a:v are shown as they are.

Refs #1977
2026-11-11 21:34:48 +08:00
371 змінених файлів з 18817 додано та 1338 видалено

No files matched your search

+1
Переглянути файл
@@ -12,6 +12,7 @@
"src/editor/core/cm/angular/parser*.ts",
"src/editor/core/cm/jsp/parser*.ts",
"src/editor/core/cm/astro/parser*.ts",
"src/editor/core/cm/django/parser*.ts",
"plugins/themes.md",
"pnpm-lock.yaml",
"**/*.toml",
Створено
+3 -1
Переглянути файл
@@ -4618,6 +4618,7 @@ name = "ide-lsp"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
"futures",
"globset",
"ide-core",
@@ -4710,6 +4711,7 @@ name = "ide-profiler"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-jvm-attach",
"ide-run",
"serde",
"serde_json",
@@ -4963,9 +4965,9 @@ dependencies = [
name = "ide-web"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"ide-core",
"open",
"parking_lot",
"percent-encoding",
"serde",
"specta",
+102
Переглянути файл
@@ -0,0 +1,102 @@
//! A short-lived ACP session to learn models, effort levels and other config options
//! before the user's first task (Cursor and other ACP agents do not expose these in
//! `initialize` alone).
use super::{Conn, Dialect};
use crate::config;
use crate::model::AgentConfigOption;
use crate::provider::ProviderSpec;
use crate::{Shared, client_info};
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::sync::Arc;
use std::time::Duration;
const DISCOVER_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(3) } else { Duration::from_secs(120) };
/// Opens `session/new`, reads `configOptions`, and records per-model `thought_level` choices.
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(), String> {
let root = shared.paths.project_root.clone();
let launch = spec.clone();
let launch = tokio::task::spawn_blocking({
let root = root.clone();
move || launch.resolve_for_launch(&root)
})
.await
.map_err(|e| e.to_string())?;
let cwd = dunce::canonicalize(&root).unwrap_or(root);
let stderr: Arc<Mutex<Vec<String>>> = Arc::default();
let lines = stderr.clone();
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(move |l| lines.lock().push(l));
let client = Dialect::wrap(Arc::new(ide_jsonrpc::NoopHandler));
let peer = (shared.connector)(&launch, &cwd, client.clone(), on_stderr)
.map_err(|e| format!("Couldn't start {} (`{}`): {e}", launch.name, launch.command))?;
let conn = Conn::new(peer.clone(), client);
let work = async {
conn.initialize(&client_info()).await.map_err(|e| e.to_string())?;
let cwd_s = cwd.to_string_lossy().into_owned();
// No project MCP during discovery: Cursor can spend a long time spawning servers.
let new: Value = conn
.request("session/new", json!({ "cwd": cwd_s, "mcpServers": [] }), None)
.await
.map_err(|e| e.to_string())?;
let mut options = config::options_of(&new);
config::add_client_options(&launch.id, &mut options);
let model_option = options.iter().find(|o| o.category.as_deref() == Some("model"));
let effort = levels_for_model_from_slice(&options);
let levels: Vec<(String, Vec<AgentConfigOption>)> = match (model_option, effort) {
(Some(model), Some(effort)) if !model.values.is_empty() => {
model.values.iter().map(|v| (v.value.clone(), effort.clone())).collect()
}
(Some(model), Some(effort)) => model.current.clone().map(|m| vec![(m, effort)]).unwrap_or_default(),
_ => Vec::new(),
};
let option_ids: Vec<_> = options.iter().map(|o| o.id.as_str()).collect();
tracing::info!(
"agent {}: settings discovery got {} config option(s) {:?}, {} model effort list(s)",
launch.id,
options.len(),
option_ids,
levels.len()
);
shared.remember_discovered(&launch.id, &options, &[], &levels);
Ok::<(), String>(())
};
let result = tokio::time::timeout(DISCOVER_TIMEOUT, work).await;
peer.kill();
match result {
Ok(Ok(())) => Ok(()),
Ok(Err(e)) => Err(match stderr.lock().last() {
Some(line) => format!("{e}: {line}"),
None => e,
}),
Err(_) => {
let tail = stderr.lock().join("\n");
if tail.is_empty() {
Err("the agent did not answer in time".into())
} else {
Err(format!("the agent did not answer in time:\n{tail}"))
}
}
}
}
fn is_thought_level(o: &AgentConfigOption) -> bool {
o.category.as_deref() == Some("thought_level")
|| matches!(o.id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
}
fn levels_for_model_from_slice(options: &[AgentConfigOption]) -> Option<Vec<AgentConfigOption>> {
let levels: Vec<_> = options
.iter()
.filter(|o| is_thought_level(o))
.cloned()
.map(|mut o| {
if o.category.as_deref() != Some("thought_level") {
o.category = Some("thought_level".into());
}
o
})
.collect();
(!levels.is_empty()).then_some(levels)
}
+4
Переглянути файл
@@ -14,6 +14,7 @@
//! - Agent → client ([`Dialect`]): `session/update`s are normalized ([`normalize`]) and
//! permission requests get the v1 `toolCall`.
pub(crate) mod discover;
mod normalize;
use ide_jsonrpc::{Handler, Peer, RpcError, codes};
@@ -57,6 +58,9 @@ fn client_capabilities() -> Value {
"terminal": false,
// Subagents as sessions of their own (ACP subagents RFD; v2 baseline).
"subagents": {},
// Cursor exposes model parameters (e.g. Composer `fast`) as separate config options
// only when the client advertises this flag (undocumented Cursor ACP extension).
"_meta": { "parameterizedModelPicker": true },
})
}
+74 -2
Переглянути файл
@@ -93,9 +93,81 @@ fn the_negotiating_request_carries_both_versions() {
assert!(p["capabilities"].is_object());
assert_eq!(p["clientCapabilities"]["subagents"], json!({}));
assert_eq!(p["clientCapabilities"]["terminal"], false);
assert_eq!(
p["clientCapabilities"]["_meta"]["parameterizedModelPicker"], true,
"Cursor and similar agents need parameterized model options (e.g. fast on/off)"
);
// What we sent in the capture is what claude-agent-acp negotiated v2 and v1 from.
let sent = &frames(TURN, "out")[0]["params"];
assert_eq!(sent["clientCapabilities"], p["clientCapabilities"]);
let sent = frames(TURN, "out")[0]["params"]["clientCapabilities"].clone();
let mut expected = sent;
expected["_meta"] = p["clientCapabilities"]["_meta"].clone();
assert_eq!(expected, p["clientCapabilities"]);
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent() {
#[derive(Default)]
struct DiscoveryAgent {
peer: std::sync::OnceLock<Peer>,
calls: parking_lot::Mutex<Vec<(String, Value)>>,
}
impl Handler for DiscoveryAgent {
fn on_notification(&self, _: &str, _: Value) {}
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push((method.to_owned(), params));
match method {
"initialize" => Ok(json!({
"protocolVersion": 1,
"agentCapabilities": { "loadSession": true }
})),
"session/new" => Ok(json!({
"sessionId": "discover",
"configOptions": [
{"id": "model", "category": "model", "type": "select", "currentValue": "m1",
"options": [{"value": "m1", "name": "One"}, {"value": "m2", "name": "Two"}]},
{"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
})),
_ => Ok(json!({})),
}
}
}
let agent = Arc::new(DiscoveryAgent::default());
let tmp = tempfile::tempdir().unwrap();
let agent_for_connector = agent.clone();
let connector: crate::Connector = Arc::new(move |_, _, client, _| {
let (ca, cb) = tokio::io::duplex(1 << 20);
let (car, caw) = tokio::io::split(ca);
let (cbr, cbw) = tokio::io::split(cb);
let agent_peer = Peer::connect_framed(cbr, cbw, agent_for_connector.clone(), Framing::Lines);
let _ = agent_for_connector.peer.set(agent_peer);
Ok(Peer::connect_framed(car, caw, client, Framing::Lines))
});
let mgr = crate::AgentManager::with_connector(
crate::AgentPaths {
project_root: tmp.path().to_path_buf(),
store_dir: tmp.path().join("store"),
worktrees_dir: tmp.path().join("worktrees"),
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert_eq!(calls.iter().map(|(m, _)| m.as_str()).collect::<Vec<_>>(), ["initialize", "session/new"]);
assert_eq!(calls[1].1["mcpServers"], json!([]));
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
let fast = cursor.config.iter().find(|o| o.id == "fast").expect("fast option");
assert_eq!(fast.category.as_deref(), Some("model_config"));
assert_eq!(fast.current.as_deref(), Some("false"));
assert!(cursor.thought_levels.contains_key("m1") && cursor.thought_levels.contains_key("m2"));
assert!(cursor.thought_levels["m1"].iter().any(|o| o.category.as_deref() == Some("thought_level")));
}
#[test]
+24 -1
Переглянути файл
@@ -107,7 +107,15 @@ pub(crate) fn parse_options(options: &Value) -> Vec<AgentConfigOption> {
Some(AgentConfigOption {
name: str_of(o, "name").unwrap_or_else(|| id.clone()),
description: str_of(o, "description"),
category: str_of(o, "category").or_else(|| matches!(id.as_str(), "model" | "mode").then(|| id.clone())),
category: str_of(o, "category").or_else(|| {
matches!(id.as_str(), "model" | "mode")
.then(|| id.clone())
.or_else(|| {
matches!(id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
.then(|| "thought_level".into())
})
.or_else(|| matches!(id.as_str(), "fast").then(|| "model_config".into()))
}),
current: str_of(o, "currentValue"),
values,
via: AgentConfigVia::Option,
@@ -333,6 +341,21 @@ mod tests {
assert!(bypass_approvals(&url_mode));
}
#[test]
fn effort_and_fast_categories_are_inferred_from_ids() {
let c = options_of(&json!({
"configOptions": [
{"id": "effort", "type": "select", "currentValue": "high",
"options": [{"value": "low"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
}));
assert_eq!(c.len(), 2);
assert_eq!(c[0].category.as_deref(), Some("thought_level"));
assert_eq!(c[1].category.as_deref(), Some("model_config"));
}
#[test]
fn command_sources_in_updates_and_stored_tasks() {
let update = json!({"availableCommands": [
+196 -30
Переглянути файл
@@ -16,6 +16,7 @@ mod model;
mod native;
mod provider;
pub mod registry;
pub mod security;
mod skills;
mod timeline;
mod worktree;
@@ -39,6 +40,9 @@ use std::time::Duration;
/// Receives the events of one project's tasks.
pub type AgentSink = Arc<dyn Fn(AgentEvent) + Send + Sync>;
/// Returns the active command execution security policy for the project.
pub type SecurityHook = Arc<dyn Fn() -> ide_core::settings::AgentSecuritySettings + Send + Sync>;
/// Entry changes are batched for this long before they are sent.
const FLUSH_DELAY: Duration = Duration::from_millis(40);
/// Updates that arrive this late after `session/load` returns are still replayed history.
@@ -102,6 +106,7 @@ struct Shared {
runtime: tokio::runtime::Handle,
connector: Connector,
sink: RwLock<Option<AgentSink>>,
security: RwLock<Option<SecurityHook>>,
known: Mutex<HashMap<String, Known>>,
providers: RwLock<Vec<provider::ProviderSpec>>,
/// Last probe of each provider.
@@ -177,9 +182,15 @@ impl Shared {
let shared = self.clone();
// Keep the lookup alive if a composer unmounts while it is waiting.
self.runtime.spawn(async move {
if let Ok(found) =
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await
{
tracing::info!("agent {}: settings discovery started", spec.id);
let discovered = if spec.driver == provider::Driver::Acp {
acp::discover::run(&shared, &spec).await.map(|_| None)
} else if spec.driver == provider::Driver::Claude {
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await.map(Some)
} else {
return;
};
if let Ok(Some(found)) = discovered {
let commands =
config::commands_of(&json!({ "availableCommands": found.commands }), &spec.id);
let options = config::options_of(&json!({ "configOptions": found.options }));
@@ -198,6 +209,8 @@ impl Shared {
shared.remember_discovered(&spec.id, &options, &commands, &levels);
})
.await;
} else if let Err(e) = discovered {
tracing::info!("agent {}: settings discovery failed: {e}", spec.id);
}
let _ = done.send(true);
});
@@ -220,12 +233,17 @@ impl Shared {
fn knows_all_levels(&self, provider: &str) -> bool {
let cache = self.known.lock();
let Some(known) = cache.get(provider) else { return false };
known
let models: Vec<&str> = known
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| &o.values)
.all(|v| known.thought_levels.contains_key(&v.value))
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
.collect();
// No advertised models yet: nothing to attach effort levels to.
if models.is_empty() {
return false;
}
models.iter().all(|id| known.thought_levels.contains_key(*id))
}
fn root_key(&self) -> String {
@@ -434,6 +452,7 @@ impl AgentManager {
runtime,
connector,
sink: RwLock::new(None),
security: RwLock::new(None),
known: Mutex::new(known),
providers,
probes: Mutex::new(HashMap::new()),
@@ -477,6 +496,11 @@ impl AgentManager {
tasks
}
/// Sets the callback used to read the active command execution security policy.
pub fn set_security_hook(&self, hook: SecurityHook) {
*self.shared.security.write() = Some(hook);
}
/// The agents (custom ones re-read), whether each is installed, and what they offered last time.
pub fn providers(&self) -> Vec<AgentProvider> {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -501,7 +525,9 @@ impl AgentManager {
let new = specs.iter().find(|p| &p.id == id);
matches!((old, new), (Some(a), Some(b)) if a.command == b.command && a.args == b.args && a.env == b.env)
});
*prev = specs;
*prev = specs.clone();
drop(probes);
drop(prev);
list
}
@@ -522,6 +548,30 @@ impl AgentManager {
provider::set_builtin_env(&dir, id, env)
}
/// Saves a new custom ACP agent through its agent file.
pub fn save_custom_agent(&self, input: CustomAgentInput) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::save_custom(&dir, &input)
}
/// Removes a custom agent's file (registry installs go through `registry_remove`).
pub fn remove_custom_agent(&self, id: &str) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::remove_custom(&dir, id)
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom agents.
pub fn import_acp_agents(&self, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::import_acp(&dir, text)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -552,10 +602,16 @@ impl AgentManager {
let mut offered = offered;
if task.is_none()
&& (offered.is_empty() || !self.shared.has_config(&provider) || !self.shared.knows_all_levels(&provider))
&& let Some(spec) = self.shared.spec(&provider).filter(|s| s.driver == provider::Driver::Claude)
&& let Some(spec) = self
.shared
.spec(&provider)
.filter(|s| matches!(s.driver, provider::Driver::Claude | provider::Driver::Acp))
{
let claude = spec.driver == provider::Driver::Claude;
self.shared.fetch_settings(spec).await;
offered = self.shared.known_commands(&provider);
if claude {
offered = self.shared.known_commands(&provider);
}
}
for &(name, description) in builtin_commands(&provider) {
if !offered.iter().any(|c| c.name == name) {
@@ -888,14 +944,38 @@ impl AgentManager {
}
o.clone()
};
let previous = target.current.clone();
// Cursor's `session/set_config_option` (Fast, effort) can take seconds. Show the
// choice first so the switch and slider move immediately.
apply_config_choice(&task, option, &value);
task.persist();
task.emit_task();
let live = task.live.lock().clone().filter(|l| !l.peer.is_closed());
let mut model_options =
target.category.as_deref() != Some("model") || target.current.as_deref() == Some(value.as_str());
target.category.as_deref() != Some("model") || previous.as_deref() == Some(value.as_str());
if let Some(live) = live.filter(|_| option != config::CURSOR_APPROVAL) {
let (method, mut params) = config::set_request(&target, &value);
params["sessionId"] = json!(live.session);
let reply: Value =
live.peer.request(method, params, None).await.map_err(|e| IdeError::internal(task.describe(e)))?;
let reply: Value = match live.peer.request(method, params, None).await {
Ok(reply) => reply,
Err(e) => {
{
let mut s = task.state.lock();
s.wanted.retain(|c| c.id != option);
if let Some(prev) = previous.clone() {
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: prev });
}
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = previous.clone();
}
s.task.model = model_of(&s.task.config).or(s.task.model.take());
}
task.persist();
task.emit_task();
return Err(IdeError::internal(task.describe(e)));
}
};
if let Some(options) = reply.get("configOptions") {
config::merge_update(&mut task.state.lock().task.config, options);
model_options = true;
@@ -906,8 +986,6 @@ impl AgentManager {
if !model_options {
s.task.config.retain(|o| o.category.as_deref() != Some("thought_level"));
}
s.wanted.retain(|c| c.id != option);
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: value.clone() });
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = Some(value);
}
@@ -1260,15 +1338,19 @@ impl AgentManager {
/// Files the task changed since it started.
pub async fn changes(&self, id: &str) -> IdeResult<Vec<AgentChange>> {
let task = self.task(id)?;
let (cwd, base) = task.cwd_and_base();
let at = task.base();
let before = task.state.lock().before.clone();
blocking(move || Ok(worktree::without_untouched(worktree::changes(&cwd, &base)?, &before))).await
blocking(move || at.changes(&before)).await
}
/// Content of `path` (in the task's directory) when the task started.
pub async fn base_text(&self, id: &str, path: PathBuf) -> IdeResult<Option<String>> {
let (cwd, base) = self.task(id)?.cwd_and_base();
blocking(move || worktree::file_at(&cwd, &path, &base)).await
let at = self.task(id)?.base();
blocking(move || match at.rev()? {
Some(rev) => worktree::file_at(at.cwd(), &path, &rev),
None => Ok(None),
})
.await
}
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as they
@@ -1278,16 +1360,17 @@ impl AgentManager {
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before rolling back its changes"));
}
let (cwd, base) = task.cwd_and_base();
let at = task.base();
let (before, saved) = {
let s = task.state.lock();
(s.before.clone(), s.saved.as_ref().map(|name| self.shared.paths.store_dir.join(name)))
};
let restamped = blocking(move || {
let changes = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before);
let changes = at.changes(&before)?;
let rev = at.rev()?;
let mut restamped = Vec::new();
for change in changes.iter().filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path))) {
worktree::restore(&cwd, &base, change, &before, saved.as_deref())?;
worktree::restore(at.cwd(), rev.as_deref(), change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
@@ -1306,6 +1389,41 @@ impl AgentManager {
Ok(())
}
/// Accept: copies the task's changed files among `rel_paths` (all with `None`) from its
/// worktree into the local checkout, uncommitted, for the user to review and commit.
/// Files the checkout changed itself are left alone and reported. Only worktree tasks
/// have anything to accept.
pub async fn accept_changes(&self, id: &str, rel_paths: Option<Vec<String>>) -> IdeResult<AgentAcceptOutcome> {
let task = self.task(id)?;
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before accepting its changes"));
}
let (isolation, cwd, base, root, before) = {
let s = task.state.lock();
(
s.task.isolation,
s.task.cwd.0.clone(),
s.task.base_commit.clone(),
s.task.project_root.clone().map(|p| p.0).unwrap_or_else(|| self.shared.paths.project_root.clone()),
(*s.before).clone(),
)
};
if isolation != Isolation::Worktree {
return Err(IdeError::invalid("the task works in the local checkout: there is nothing to accept"));
}
let Some(base) = base else {
return Err(IdeError::invalid("the task has no base revision to accept from"));
};
blocking(move || {
let wanted: Vec<AgentChange> = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before)
.into_iter()
.filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path)))
.collect();
worktree::accept(&root, &cwd, &base, &wanted)
})
.await
}
/// Stops every agent (project closed).
pub fn shutdown(&self) {
for task in self.tasks.read().values() {
@@ -1356,15 +1474,15 @@ impl Task {
})
}
fn cwd_and_base(&self) -> (PathBuf, String) {
fn base(&self) -> worktree::TaskBase {
let s = self.state.lock();
// In the project's own checkout the user commits too: compare against the current HEAD,
// as the Commit tool window does, so commits made since the task started drop out.
let base = match s.task.isolation {
Isolation::Local => None,
Isolation::Worktree => s.task.base_commit.clone(),
};
(s.task.cwd.0.clone(), base.unwrap_or_else(|| "HEAD".into()))
let cwd = s.task.cwd.0.clone();
match s.task.isolation {
Isolation::Local => worktree::TaskBase::Local { cwd, start: s.task.base_commit.clone() },
Isolation::Worktree => {
worktree::TaskBase::Worktree { cwd, base: s.task.base_commit.clone().unwrap_or_else(|| "HEAD".into()) }
}
}
}
fn emit_task(&self) {
@@ -2124,6 +2242,44 @@ impl Task {
.iter()
.map(|question| question.answer_key.clone().unwrap_or_else(|| question.question.clone()))
.collect();
if let Some(hook) = self.shared.security.read().clone()
&& let Some(command) = security::extract_command(params)
{
match security::evaluate_command(&command, &hook()) {
security::PolicyDecision::Blocked(pattern) => {
let reject = options.iter().find(|o| o.kind.starts_with("reject")).cloned();
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
options,
questions: Vec::new(),
chosen: Some(format!("Blocked by policy ({pattern})")),
});
return match reject {
Some(option) => json!({ "outcome": { "outcome": "selected", "optionId": option.id } }),
None => json!({ "outcome": { "outcome": "cancelled" } }),
};
}
security::PolicyDecision::Allowed => {
if let Some(allow) = options
.iter()
.find(|o| o.kind == "allow_once")
.or_else(|| options.iter().find(|o| o.kind.starts_with("allow")))
.cloned()
{
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
options,
questions: Vec::new(),
chosen: Some(format!("{} (whitelisted)", allow.name)),
});
return json!({ "outcome": { "outcome": "selected", "optionId": allow.id } });
}
}
security::PolicyDecision::AskUser => {}
}
}
let (tx, rx) = std::sync::mpsc::channel();
let entry =
self.push_entry(AgentEntry::Permission { id, title, options: options.clone(), questions, chosen: None });
@@ -2437,7 +2593,7 @@ fn with_details(message: String, data: Option<&Value>) -> String {
}
}
fn client_info() -> Value {
pub(crate) fn client_info() -> Value {
json!({ "name": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") })
}
@@ -2496,6 +2652,16 @@ fn retry_of(update: &Value, now: f64) -> Option<AgentRetry> {
})
}
fn apply_config_choice(task: &Task, option: &str, value: &str) {
let mut s = task.state.lock();
s.wanted.retain(|c| c.id != option);
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: value.to_owned() });
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = Some(value.to_owned());
}
s.task.model = model_of(&s.task.config).or(s.task.model.take());
}
fn model_of(config: &[AgentConfigOption]) -> Option<String> {
config.iter().find(|o| o.category.as_deref() == Some("model")).and_then(|o| o.current.clone())
}
+46
Переглянути файл
@@ -602,3 +602,49 @@ pub struct McpServerInput {
fn default_true() -> bool {
true
}
/// Payload for creating a custom ACP agent (always the ACP driver); the id
/// becomes the `<config>/agents/<id>.toml` file name.
#[derive(Clone, Debug, Deserialize, Type)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CustomAgentInput {
pub id: String,
pub name: String,
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: BTreeMap<String, String>,
}
/// What importing one entry of a JetBrains `acp.json` file did.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AcpImportOutcome {
/// The `agent_servers` key the entry was imported from.
pub name: String,
/// The id its agent file would get (also when skipped).
pub id: String,
pub saved: bool,
/// Why the entry was skipped; empty when it was saved.
#[serde(default)]
pub reason: String,
}
/// What accepting a worktree task's changes copied into the local checkout.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptOutcome {
/// Relative paths copied into the checkout (uncommitted), including ones already identical.
pub accepted: Vec<String>,
/// Files left alone because the checkout has its own edits to them.
pub skipped: Vec<AgentAcceptSkipped>,
}
/// A file accept left alone, with why.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptSkipped {
pub path: String,
pub reason: String,
}
+198 -1
Переглянути файл
@@ -2,7 +2,7 @@
//! `<config>/agents/<id>.toml` (written by hand, or by installing one from the ACP
//! Registry, see `registry`).
use crate::{AgentCommandReadOnlyReason, AgentProvider, AgentSource};
use crate::{AcpImportOutcome, AgentCommandReadOnlyReason, AgentProvider, AgentSource, CustomAgentInput};
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -278,6 +278,127 @@ fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAg
Ok(())
}
/// Saves a new custom agent; refuses built-in and ACP Registry ids, taken ids
/// and an empty command. The id is also the agent file's name, so it must be
/// a safe file stem.
pub(crate) fn save_custom(custom_dir: &Path, input: &CustomAgentInput) -> IdeResult<()> {
let id = validated_custom_id(&input.id)?;
if builtin_spec(&id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent; change its command instead")));
}
if input.command.trim().is_empty() {
return Err(IdeError::invalid("the agent command must not be empty"));
}
let file = custom_dir.join(format!("{id}.toml"));
if file.try_exists()? {
if CustomAgent::read(&file).is_ok_and(|a| a.registry.is_some()) {
return Err(IdeError::invalid(format!(
"{id} is installed from the ACP Registry; uninstall it there to replace it"
)));
}
return Err(IdeError::invalid(format!("there is already a custom agent '{id}'")));
}
CustomAgent {
name: Some(input.name.clone()),
driver: Driver::Acp,
command: input.command.clone(),
args: input.args.clone(),
env: input.env.clone(),
registry: None,
}
.write(&file)?;
Ok(())
}
/// Removes a custom agent's file; refuses built-in, ACP Registry and missing ids.
pub(crate) fn remove_custom(custom_dir: &Path, id: &str) -> IdeResult<()> {
let id = validated_custom_id(id)?;
let id = id.as_str();
if builtin_spec(id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent and cannot be removed")));
}
let file = custom_dir.join(format!("{id}.toml"));
let agent =
CustomAgent::read(&file).map_err(|_| IdeError::not_found(format!("there is no custom agent '{id}'")))?;
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry; uninstall it there")));
}
match std::fs::remove_file(&file) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e.into()),
_ => Ok(()),
}
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom
/// agents, one outcome per entry; entries that collide, lack a command or
/// have no usable name are skipped with a reason.
pub(crate) fn import_acp(custom_dir: &Path, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
#[derive(Deserialize)]
struct AcpJson {
#[serde(default)]
agent_servers: BTreeMap<String, AcpServer>,
}
#[derive(Deserialize)]
struct AcpServer {
#[serde(default)]
command: String,
#[serde(default)]
args: Vec<String>,
#[serde(default)]
env: BTreeMap<String, String>,
}
let parsed: AcpJson =
serde_json::from_str(text).map_err(|e| IdeError::invalid(format!("not an acp.json file: {e}")))?;
let mut outcomes = Vec::with_capacity(parsed.agent_servers.len());
for (name, server) in parsed.agent_servers {
let id = slugify_id(&name);
let mut outcome = AcpImportOutcome { name, id: id.clone(), saved: false, reason: String::new() };
if id.is_empty() {
outcome.reason = "the name has no usable characters for an id".to_owned();
} else if builtin_spec(&id).is_some() {
outcome.reason = format!("{id} is a built-in agent");
} else if server.command.trim().is_empty() {
outcome.reason = "the entry has no command".to_owned();
} else if custom_dir.join(format!("{id}.toml")).try_exists()? {
outcome.reason = format!("there is already a custom agent '{id}'");
} else {
CustomAgent {
name: Some(outcome.name.clone()),
driver: Driver::Acp,
command: server.command,
args: server.args,
env: server.env,
registry: None,
}
.write(&custom_dir.join(format!("{id}.toml")))?;
outcome.saved = true;
}
outcomes.push(outcome);
}
Ok(outcomes)
}
/// A custom agent id doubles as a file stem: letters, digits, `-` and `_` only.
fn validated_custom_id(id: &str) -> IdeResult<String> {
if !id.is_empty() && id.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_') {
return Ok(id.to_owned());
}
Err(IdeError::invalid(format!("'{id}' is not a usable agent id (letters, digits, '-' and '_' only)")))
}
/// Turns an `acp.json` server name into a file-stem id (`"My Agent!"` → `"my-agent"`).
fn slugify_id(name: &str) -> String {
let mut id = String::with_capacity(name.len());
for c in name.chars() {
if c.is_ascii_alphanumeric() {
id.push(c.to_ascii_lowercase());
} else if !id.ends_with('-') {
id.push('-');
}
}
id.trim_matches('-').to_owned()
}
/// An agent file, `<config>/agents/<id>.toml`.
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
@@ -753,4 +874,80 @@ mod tests {
assert_eq!(spec.source, AgentSource::Registry { version: "0.10.0".into(), update: None });
assert!(toml::from_str::<CustomAgent>("command = \"x\"\n[registry]\nid = \"x\"\n").is_err());
}
fn custom_input(id: &str) -> CustomAgentInput {
CustomAgentInput {
id: id.into(),
name: "Mine".into(),
command: "/opt/mine".into(),
args: vec!["acp".into()],
env: BTreeMap::from([("K".to_owned(), "v".to_owned())]),
}
}
#[test]
fn custom_agents_save_list_and_remove() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap();
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
assert_eq!(mine.source, AgentSource::Custom);
remove_custom(dir.path(), "mine").unwrap();
assert!(!load(Some(dir.path())).iter().any(|s| s.id == "mine"));
remove_custom(dir.path(), "mine").unwrap_err();
}
#[test]
fn custom_agents_refuse_builtin_registry_and_bad_ids() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("claude")).unwrap_err();
save_custom(dir.path(), &custom_input("../evil")).unwrap_err();
save_custom(dir.path(), &custom_input("")).unwrap_err();
let mut empty_command = custom_input("mine");
empty_command.command = " ".into();
save_custom(dir.path(), &empty_command).unwrap_err();
save_custom(dir.path(), &custom_input("mine")).unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap_err();
remove_custom(dir.path(), "claude").unwrap_err();
// An id that isn't a plain file stem never reaches outside the agents directory.
let outside = dir.path().join("outside.toml");
std::fs::write(&outside, "command = \"/bin/sh\"\n").unwrap();
let agents = dir.path().join("agents");
std::fs::create_dir(&agents).unwrap();
remove_custom(&agents, "../outside").unwrap_err();
assert!(outside.exists());
std::fs::write(
dir.path().join("amp-acp.toml"),
"command = \"/registry/amp\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n",
)
.unwrap();
save_custom(dir.path(), &custom_input("amp-acp")).unwrap_err();
remove_custom(dir.path(), "amp-acp").unwrap_err();
}
#[test]
fn acp_json_imports_servers_and_skips_collisions() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("taken")).unwrap();
let outcomes = import_acp(
dir.path(),
r#"{"agent_servers": {
"My Agent!": {"command": "/opt/agent", "args": ["--acp"], "env": {"K": "v"}},
"taken": {"command": "/opt/taken"},
"claude": {"command": "/opt/claude"},
"!!!": {"command": "/opt/noname"},
"silent": {"args": []}
}}"#,
)
.unwrap();
let saved: Vec<&str> = outcomes.iter().filter(|o| o.saved).map(|o| o.id.as_str()).collect();
assert_eq!(saved, ["my-agent"]);
assert!(outcomes.iter().filter(|o| !o.saved).all(|o| !o.reason.is_empty()));
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "my-agent").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("My Agent!", "/opt/agent"));
assert_eq!(mine.source, AgentSource::Custom);
import_acp(dir.path(), "not json").unwrap_err();
assert!(import_acp(dir.path(), "{}").unwrap().is_empty());
}
}
+487
Переглянути файл
@@ -0,0 +1,487 @@
//! Command execution security policy (Whitelist / Blacklist) for coding agents.
use ide_core::settings::AgentSecuritySettings;
use serde_json::Value;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum PolicyDecision {
/// Command matched a blacklist pattern (`pattern`).
Blocked(String),
/// Command matched the whitelist and no blacklist pattern.
Allowed,
/// Neither matched; prompt the user as usual.
AskUser,
}
/// Extracts the shell command string from an ACP `session/request_permission` payload
/// when the request is for executing a command.
pub fn extract_command(params: &Value) -> Option<String> {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"))?;
let raw_cmd = tool.pointer("/rawInput/command").or_else(|| tool.pointer("/rawInput/cmd"));
if let Some(cmd) = raw_cmd.and_then(command_from_value) {
return Some(cmd);
}
let kind = tool.get("kind").and_then(Value::as_str).unwrap_or_default();
let name = tool.get("name").and_then(Value::as_str).unwrap_or_default();
let title =
tool.get("title").and_then(Value::as_str).or_else(|| params.get("title").and_then(Value::as_str))?.trim();
if let Some(cmd) = title.strip_prefix("Run ").map(str::trim).filter(|s| !s.is_empty()) {
return Some(cmd.to_owned());
}
if (kind == "execute" || matches!(name, "Bash" | "PowerShell" | "Execute" | "Run" | "terminal"))
&& !title.is_empty()
&& title != "Terminal"
&& title != "Run a command"
{
return Some(title.to_owned());
}
None
}
fn command_from_value(val: &Value) -> Option<String> {
match val {
Value::String(s) => {
let trimmed = s.trim();
(!trimmed.is_empty()).then(|| trimmed.to_owned())
}
Value::Array(items) => {
let words: Vec<&str> = items.iter().filter_map(Value::as_str).collect();
if words.is_empty() {
return None;
}
let cmd = match words.as_slice() {
[_, flag, script] if flag.starts_with('-') && flag.ends_with('c') => (*script).trim().to_owned(),
_ => words.join(" ").trim().to_owned(),
};
(!cmd.is_empty()).then_some(cmd)
}
_ => None,
}
}
/// Checks whether a single pattern matches `command` using prefix + glob rules:
/// - If `pattern` contains `*` or `?`, matches using glob wildcards (where `*` matches
/// any sequence of characters). A trailing ` *` in the pattern also matches the bare
/// command without arguments (e.g. `git status *` matches `git status`).
/// - Otherwise, matches when `command` equals `pattern` or starts with `pattern` followed
/// by whitespace (prefix match at word boundary).
pub fn matches_command(pattern: &str, command: &str) -> bool {
let pat = pattern.trim();
let cmd = command.trim();
if pat.is_empty() || cmd.is_empty() {
return false;
}
if pat.contains('*') || pat.contains('?') {
if let Some(stem) = pat.strip_suffix(" *")
&& cmd == stem.trim()
{
return true;
}
wildcard_match(pat.as_bytes(), cmd.as_bytes())
} else {
cmd == pat || cmd.strip_prefix(pat).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
}
fn wildcard_match(pat: &[u8], text: &[u8]) -> bool {
let (mut p, mut t) = (0, 0);
let (mut star_p, mut star_t) = (None, 0);
while t < text.len() {
if p < pat.len() && (pat[p] == b'?' || pat[p] == text[t]) {
p += 1;
t += 1;
} else if p < pat.len() && pat[p] == b'*' {
star_p = Some(p);
p += 1;
star_t = t;
} else if let Some(sp) = star_p {
p = sp + 1;
star_t += 1;
t = star_t;
} else {
return false;
}
}
while p < pat.len() && pat[p] == b'*' {
p += 1;
}
p == pat.len()
}
/// Matches an allowed command entry against a sub-command.
/// An allow entry is a whole command or a prefix only up to a word boundary:
/// e.g. `git status` matches `git status` and `git status -s`, but NOT `git statusx`.
/// Any accidental trailing `*` in the allow entry is stripped so `git status*` behaves
/// cleanly as a prefix at word boundaries instead of a glob wildcard.
pub fn matches_allow_entry(entry: &str, command: &str) -> bool {
let mut entry = entry.trim();
if let Some(stripped) = entry.strip_suffix('*') {
entry = stripped.trim_end();
}
let cmd = command.trim();
if entry.is_empty() || cmd.is_empty() {
return false;
}
cmd == entry || cmd.strip_prefix(entry).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
/// Checks whether a word is an environment variable assignment (`IDENTIFIER=...`).
fn is_env_assignment_word(word: &str) -> bool {
let Some((name, _)) = word.split_once('=') else {
return false;
};
if name.is_empty() {
return false;
}
let mut chars = name.chars();
match chars.next() {
Some(c) if c.is_ascii_alphabetic() || c == '_' => {}
_ => return false,
}
chars.all(|c| c.is_ascii_alphanumeric() || c == '_')
}
/// Returns true if the sub-command starts with an environment variable assignment,
/// e.g. `FOO=bar cmd` or `PORT=3000`.
fn has_leading_env_assignment(cmd: &str) -> bool {
let trimmed = cmd.trim_start();
let bytes = trimmed.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
}
i += 1;
continue;
}
if b == b'\'' || b == b'"' {
quote = Some(b);
i += 1;
continue;
}
if b.is_ascii_whitespace() {
break;
}
i += 1;
}
let first_word = &trimmed[..i];
is_env_assignment_word(first_word)
}
/// Checks whether `command` contains shell constructs that must never be auto-approved:
/// - Command substitution: `$(...)` or backticks
/// - Redirection: `>`, `>>`, `<`
/// - Here-docs: `<<`
/// - Subshells: `(...)`
/// - Leading environment assignments: `VAR=val cmd`
pub fn has_unsafe_shell_syntax(command: &str) -> bool {
let cmd = command.trim();
if cmd.is_empty() {
return false;
}
if has_leading_env_assignment(cmd) {
return true;
}
let bytes = cmd.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
i += 1;
continue;
}
// Inside double quotes: $(...) and `...` are evaluated by the shell.
if q == b'"' {
if b == b'`' {
return true;
}
if b == b'$' && i + 1 < bytes.len() && bytes[i + 1] == b'(' {
return true;
}
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'`' => return true,
b'$' if i + 1 < bytes.len() && bytes[i + 1] == b'(' => return true,
b'<' | b'>' => return true,
b'(' | b')' => return true,
_ => i += 1,
}
}
false
}
/// Splits a shell command into sub-commands separated by `;`, `&&`, `||`, `|`, or `\n`
/// (outside single/double quotes) so compound commands cannot bypass policy checks.
fn split_subcommands(command: &str) -> Vec<&str> {
let bytes = command.as_bytes();
let mut out = Vec::new();
let mut start = 0;
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if let Some(q) = quote {
if b == b'\\' && q == b'"' && i + 1 < bytes.len() {
i += 2;
continue;
}
if b == q {
quote = None;
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'\\' if i + 1 < bytes.len() => {
i += 2;
}
b';' | b'\n' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
i += 1;
start = i;
}
b'&' | b'|' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
if i + 1 < bytes.len() && bytes[i + 1] == b {
i += 2;
} else {
i += 1;
}
start = i;
}
_ => i += 1,
}
}
let tail = command[start..].trim();
if !tail.is_empty() {
out.push(tail);
}
out
}
/// Evaluates `command` against `settings`:
/// 1. Blacklist takes precedence: if the full command or any sub-command matches a
/// `blocked_commands` entry, returns [`PolicyDecision::Blocked`].
/// 2. Whitelist: allowed_commands cannot auto-approve when empty.
/// 3. Every sub-command in a chain must be safe (no command substitution, redirection,
/// subshell, or env assignments) and match at least one `allowed_commands` entry
/// exactly or up to word boundary; otherwise falls back to [`PolicyDecision::AskUser`].
pub fn evaluate_command(command: &str, settings: &AgentSecuritySettings) -> PolicyDecision {
let cmd = command.trim();
if cmd.is_empty() {
return PolicyDecision::AskUser;
}
let subcommands = split_subcommands(cmd);
for pat in &settings.blocked_commands {
let pat = pat.trim();
if pat.is_empty() {
continue;
}
if matches_command(pat, cmd) || subcommands.iter().any(|sub| matches_command(pat, sub)) {
return PolicyDecision::Blocked(pat.to_owned());
}
}
if settings.allowed_commands.is_empty() || subcommands.is_empty() {
return PolicyDecision::AskUser;
}
let all_allowed = subcommands.iter().all(|sub| {
let s = sub.trim();
if s.is_empty() || has_unsafe_shell_syntax(s) {
return false;
}
settings.allowed_commands.iter().any(|entry| matches_allow_entry(entry, s))
});
if all_allowed {
return PolicyDecision::Allowed;
}
PolicyDecision::AskUser
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn prefix_and_glob_matching() {
assert!(matches_command("git status", "git status"));
assert!(matches_command("git status", "git status -s"));
assert!(!matches_command("git status", "git statusbar"));
assert!(matches_command("git status*", "git status"));
assert!(matches_command("git status*", "git status --short"));
assert!(matches_command("npm test *", "npm test"));
assert!(matches_command("npm test *", "npm test unit"));
assert!(matches_command("rm -rf *", "rm -rf /tmp/foo"));
assert!(matches_command("rm -rf *", "rm -rf /"));
}
#[test]
fn blacklist_takes_precedence_and_catches_chained_commands() {
let settings = AgentSecuritySettings::default();
assert_eq!(evaluate_command("rm -rf /tmp/build", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command(":(){ :|:& };:", &settings), PolicyDecision::Blocked(":(){ :|:& };:".into()));
}
#[test]
fn allow_entry_matching_word_boundary() {
assert!(matches_allow_entry("git status", "git status"));
assert!(matches_allow_entry("git status", "git status -s"));
assert!(!matches_allow_entry("git status", "git statusbar"));
assert!(!matches_allow_entry("git status", "git statusx"));
// Trailing asterisk in allow entry normalizes cleanly to word boundary prefix
assert!(matches_allow_entry("git status*", "git status"));
assert!(matches_allow_entry("git status*", "git status --short"));
assert!(!matches_allow_entry("git status*", "git statusbar"));
assert!(matches_allow_entry("ls", "ls"));
assert!(matches_allow_entry("ls", "ls -la"));
assert!(!matches_allow_entry("ls", "lsof"));
assert!(matches_allow_entry("cargo check", "cargo check"));
assert!(matches_allow_entry("cargo check", "cargo check --workspace"));
assert!(!matches_allow_entry("cargo check", "cargo checkmate"));
}
#[test]
fn unsafe_shell_syntax_detection() {
// Command substitutions: $(...) and `...`
assert!(has_unsafe_shell_syntax("echo $(whoami)"));
assert!(has_unsafe_shell_syntax("echo \"user: $(whoami)\""));
assert!(has_unsafe_shell_syntax("echo `whoami`"));
assert!(has_unsafe_shell_syntax("echo \"`whoami`\""));
// Redirections: >, >>, <
assert!(has_unsafe_shell_syntax("git status > /tmp/out"));
assert!(has_unsafe_shell_syntax("git status >> /tmp/out"));
assert!(has_unsafe_shell_syntax("cat < /tmp/in"));
assert!(has_unsafe_shell_syntax("cat << 'EOF'"));
// Subshells: (...)
assert!(has_unsafe_shell_syntax("(cd /tmp && ls)"));
assert!(has_unsafe_shell_syntax("git status && (exit 1)"));
// Environment assignments: VAR=val ...
assert!(has_unsafe_shell_syntax("FOO=bar git status"));
assert!(has_unsafe_shell_syntax("PORT=3000 npm test"));
assert!(has_unsafe_shell_syntax("_SECRET=123 cargo test"));
assert!(has_unsafe_shell_syntax("A=\"b c\" git status"));
// Safe constructs with characters inside quotes or command flags
assert!(!has_unsafe_shell_syntax("git commit -m \"fix <bug> and (issue)\""));
assert!(!has_unsafe_shell_syntax("git commit -m 'fix <bug>'"));
assert!(!has_unsafe_shell_syntax("cargo test --test=unit"));
assert!(!has_unsafe_shell_syntax("git status -s"));
}
#[test]
fn default_settings_has_no_auto_approval_and_blocks_dangerous_defaults() {
let settings = AgentSecuritySettings::default();
assert!(settings.allowed_commands.is_empty(), "allowed_commands must start empty");
// Safe commands ask user by default
assert_eq!(evaluate_command("git status", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("ls", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo check", &settings), PolicyDecision::AskUser);
// Blacklist patterns are still blocked by default
assert_eq!(evaluate_command("rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("rm -rf /tmp/test", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn whitelist_allows_safe_single_and_chained_commands_only() {
let settings = AgentSecuritySettings {
allowed_commands: vec!["git status".into(), "git diff".into(), "cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
};
assert_eq!(evaluate_command("git status -s", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git diff HEAD~1", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git status && cargo test", &settings), PolicyDecision::Allowed);
// Word boundary: git statusx must not match git status
assert_eq!(evaluate_command("git statusx", &settings), PolicyDecision::AskUser);
// Unsafe syntax (redirection, command substitution, env assignment) falls back to AskUser
assert_eq!(evaluate_command("git status > out.txt", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("git status $(whoami)", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("FOO=bar git status", &settings), PolicyDecision::AskUser);
// Chaining an unapproved command must still ask the user.
assert_eq!(evaluate_command("git status && curl https://example.com", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo publish", &settings), PolicyDecision::AskUser);
// Blacklist still takes precedence over whitelist
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn extracts_command_from_v1_v2_and_codex_payloads() {
let claude = json!({
"toolCall": {
"title": "git status",
"kind": "execute",
"rawInput": { "command": "git status --short" }
}
});
assert_eq!(extract_command(&claude).as_deref(), Some("git status --short"));
let codex = json!({
"toolCall": {
"title": "Run cargo test",
"kind": "execute",
"rawInput": { "command": ["/bin/zsh", "-lc", "cargo test --workspace"] }
}
});
assert_eq!(extract_command(&codex).as_deref(), Some("cargo test --workspace"));
let write = json!({
"toolCall": {
"title": "Write foo.txt",
"kind": "edit",
"rawInput": { "file_path": "foo.txt" }
}
});
assert_eq!(extract_command(&write), None);
}
}
+318
Переглянути файл
@@ -43,6 +43,11 @@ struct FakeAgent {
/// Hold the late mode reply until the prompt's permission request is already waiting.
hold_late_mode: bool,
empty_config: AtomicBool,
/// Blocks `session/new` until discovery times out (see `acp::discover` test timeout).
hold_session_new: AtomicBool,
/// Sleeps this long at the start of `session/set_config_option`, after noting the time.
delay_config: Duration,
config_entered: Mutex<Option<std::time::Instant>>,
/// Config options `session/new` reports after the model one (per-model efforts, say).
extra_options: Mutex<Vec<Value>>,
model_options: Mutex<HashMap<String, Vec<Value>>>,
@@ -157,6 +162,9 @@ impl Handler for FakeAgent {
)
}
"session/new" => {
if self.hold_session_new.load(Ordering::Acquire) {
std::thread::sleep(Duration::from_secs(5));
}
if self.empty_config.load(Ordering::Acquire) {
return Ok(json!({"sessionId": "s1"}));
}
@@ -190,6 +198,10 @@ impl Handler for FakeAgent {
}
}
"session/set_config_option" | "session/set_mode" => {
if method == "session/set_config_option" && !self.delay_config.is_zero() {
*self.config_entered.lock() = Some(std::time::Instant::now());
std::thread::sleep(self.delay_config);
}
self.calls.lock().push(format!("{method} {}", params.get("value").or(params.get("modeId")).unwrap()));
if params["configId"] == "model" && !self.model_options.lock().is_empty() {
return Ok(json!({"configOptions": self.options_for(params["value"].as_str().unwrap())}));
@@ -2631,6 +2643,92 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn provider_reload_does_not_start_acp_discovery() {
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.reload_providers();
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(agent.calls.lock().is_empty(), "reload must not spawn cursor-agent");
}
#[tokio::test(flavor = "multi_thread")]
async fn composer_commands_discover_cursor_effort_and_fast() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![
json!({"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]}),
json!({"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}),
]),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert!(calls.iter().any(|m| m.as_str() == "session/new"));
assert_eq!(calls.iter().filter(|m| m.as_str() == "initialize").count(), 1);
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert_eq!(cursor.config.iter().find(|o| o.id == "fast").unwrap().current.as_deref(), Some("false"));
assert!(
cursor
.thought_levels
.values()
.next()
.is_some_and(|levels| { levels.iter().any(|o| o.category.as_deref() == Some("thought_level")) })
);
}
#[tokio::test(flavor = "multi_thread")]
async fn config_choice_shows_before_the_agent_answers() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![json!({
"id": "fast", "category": "model_config", "type": "select", "currentValue": "true",
"options": [{"value": "true", "name": "On"}, {"value": "false", "name": "Off"}]
})]),
delay_config: Duration::from_millis(300),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let shown: Arc<Mutex<Option<std::time::Instant>>> = Arc::default();
let mark = shown.clone();
mgr.subscribe(Arc::new(move |e| {
if let AgentEvent::Task { task } = e
&& task.config.iter().any(|o| o.id == "fast" && o.current.as_deref() == Some("false"))
&& mark.lock().is_none()
{
*mark.lock() = Some(std::time::Instant::now());
}
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
mgr.set_config(&task.id, "fast", "false".into()).await.unwrap();
let shown = (*shown.lock()).expect("the switch moved");
let entered = (*agent.config_entered.lock()).expect("the agent received the change");
assert!(shown < entered, "the composer updates before the agent answers");
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_times_out_when_session_new_hangs() {
let agent = Arc::new(FakeAgent { hold_session_new: AtomicBool::new(true), ..Default::default() });
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let started = std::time::Instant::now();
mgr.commands("cursor", None).await.unwrap();
assert!(started.elapsed() < Duration::from_secs(10), "discovery should time out quickly in tests");
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert!(!cursor.config.iter().any(|o| o.category.as_deref() == Some("model")));
}
#[tokio::test(flavor = "multi_thread")]
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
let tmp = tempfile::tempdir().unwrap();
@@ -2770,3 +2868,223 @@ async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edi
mgr.delete(&task.id, false).await.unwrap();
assert!(!dir.join("store").join(format!("{}.before", task.id)).exists());
}
/// A git repository in a temporary folder, for local tasks whose checkout moves under them.
struct Checkout {
_tmp: tempfile::TempDir,
dir: PathBuf,
}
impl Checkout {
fn new() -> Self {
let tmp = tempfile::tempdir().unwrap();
let dir = dunce::canonicalize(tmp.path()).unwrap();
let checkout = Self { _tmp: tmp, dir };
checkout.git(&["init", "-q", "-b", "main"]);
checkout.git(&["config", "core.autocrlf", "false"]);
checkout.write(".gitignore", "store/\nworktrees/\nagents/\nmodels.json\n");
checkout
}
fn git(&self, args: &[&str]) {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&self.dir)
.status()
.unwrap()
.success();
assert!(ok, "{args:?}");
}
fn commit(&self, message: &str) {
self.git(&["add", "."]);
self.git(&["commit", "-q", "-m", message]);
}
fn write(&self, rel: &str, text: &str) {
std::fs::write(self.dir.join(rel), text).unwrap();
}
fn read(&self, rel: &str) -> String {
std::fs::read_to_string(self.dir.join(rel)).unwrap()
}
/// A local task, stopped so it can be rolled back.
async fn task(&self) -> (AgentManager, AgentTask) {
let mgr = manager(&self.dir, Arc::new(FakeAgent::default()));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Hello"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.cancel(&task.id, None).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
(mgr, task)
}
}
async fn changed(mgr: &AgentManager, id: &str) -> Vec<String> {
mgr.changes(id).await.unwrap().into_iter().map(|c| c.rel_path).collect()
}
#[tokio::test(flavor = "multi_thread")]
async fn work_uncommitted_by_a_reset_during_a_local_task_is_not_rolled_back() {
let repo = Checkout::new();
repo.write("a.txt", "base\n");
repo.write("mine.txt", "base\n");
repo.commit("init");
repo.write("mine.txt", "my work\n");
repo.commit("my work");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The user takes the last commit back into the working tree (`git reset --soft`, an
// undone commit): its changes are the user's, not the agent's.
repo.git(&["reset", "-q", "--soft", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"]);
let base = mgr.base_text(&task.id, repo.dir.join("a.txt")).await.unwrap();
assert_eq!(base.as_deref(), Some("base\n"));
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n");
assert_eq!(repo.read("mine.txt"), "my work\n", "the user's work survives the rollback");
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn a_branch_switched_to_during_a_local_task_is_left_alone() {
let repo = Checkout::new();
repo.write("a.txt", "base\n");
repo.write("other.txt", "base\n");
repo.commit("init");
repo.git(&["checkout", "-q", "-b", "feature"]);
repo.write("other.txt", "feature\n");
repo.write("only-feature.txt", "feature\n");
repo.commit("feature");
repo.git(&["checkout", "-q", "main"]);
repo.write("only-main.txt", "main\n");
repo.commit("main");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The agent's uncommitted edit comes along to a branch that doesn't contain the task's start.
repo.git(&["checkout", "-q", "feature"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"]);
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n");
assert_eq!(repo.read("other.txt"), "feature\n");
assert_eq!(repo.read("only-feature.txt"), "feature\n");
assert!(!repo.dir.join("only-main.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn commits_made_during_a_local_task_are_kept_by_its_rollback() {
let repo = Checkout::new();
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.write("b.txt", "agent\n");
// The user commits what the agent did to `a.txt` so far: it drops out of the task's changes.
repo.git(&["add", "a.txt"]);
repo.git(&["commit", "-q", "-m", "keep"]);
assert_eq!(changed(&mgr, &task.id).await, ["b.txt"]);
repo.write("a.txt", "agent\nagain\n");
assert_eq!(changed(&mgr, &task.id).await, ["a.txt", "b.txt"]);
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "back to the commit, not to before it");
assert!(!repo.dir.join("b.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_changes() {
let repo = Checkout::new();
repo.write("mine.txt", "mine\n");
let (mgr, task) = repo.task().await;
repo.write("new.txt", "agent\n");
assert_eq!(changed(&mgr, &task.id).await, ["new.txt"]);
assert_eq!(mgr.base_text(&task.id, repo.dir.join("new.txt")).await.unwrap(), None);
mgr.rollback_changes(&task.id, None).await.unwrap();
assert!(!repo.dir.join("new.txt").exists());
assert_eq!(repo.read("mine.txt"), "mine\n");
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn security_policy_auto_approves_whitelist_and_rejects_blacklist() {
use ide_core::settings::AgentSecuritySettings;
// 1. Whitelist auto-approves "cargo test" without waiting for manual respond().
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent);
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run tests"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
let entries = mgr.entries(&task.id, None).unwrap();
assert!(
entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose yes"))),
"{entries:?}"
);
assert!(
entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("whitelisted")
)),
"{entries:?}"
);
// 2. Blacklist auto-rejects "cargo test" when matched by blocked_commands.
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["cargo test*".into()],
}));
let blocked_task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run blocked"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &blocked_task.id, AgentStatus::Ready).await;
let blocked_entries = mgr.entries(&blocked_task.id, None).unwrap();
assert!(
blocked_entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose no"))),
"{blocked_entries:?}"
);
assert!(
blocked_entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("Blocked by policy")
)),
"{blocked_entries:?}"
);
}
+237 -12
Переглянути файл
@@ -14,8 +14,18 @@ pub(crate) struct Created {
pub base_commit: String,
}
/// The repository of `path`, free to execute: the host runs every agent command that
/// reaches a repository only in a trusted project.
fn discover(path: &Path) -> IdeResult<Option<Repo>> {
let repo = Repo::discover(path)?;
if let Some(repo) = &repo {
repo.set_execution_allowed(true);
}
Ok(repo)
}
fn repo_at(path: &Path) -> IdeResult<Repo> {
Repo::discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
}
/// The current branch (or commit when detached) and its commit.
@@ -91,7 +101,7 @@ fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Res
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
/// Outside a repository there is nothing to compare against: no changes.
pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(Vec::new()) };
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let stats: std::collections::HashMap<String, (u32, u32)> =
repo.numstat_with_worktree(base)?.into_iter().map(|(p, a, r)| (p, (a, r))).collect();
@@ -107,14 +117,136 @@ pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
let tracked: std::collections::HashSet<String> = out.iter().map(|c| c.rel_path.clone()).collect();
for s in repo.status()? {
if s.unstaged == Some(ChangeKind::Untracked) && !tracked.contains(&s.rel_path) {
let added = std::fs::read_to_string(&*s.path).map_or(0, |t| t.lines().count() as u32);
out.push(AgentChange { path: s.path, rel_path: s.rel_path, kind: ChangeKind::Added, added, removed: 0 });
out.push(added(s.path, s.rel_path));
}
}
out.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
Ok(out)
}
fn added(path: IpcPath, rel_path: String) -> AgentChange {
let added = std::fs::read_to_string(&*path).map_or(0, |t| t.lines().count() as u32);
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0 }
}
/// Where a task works and what its changes are measured against.
pub(crate) enum TaskBase {
/// A worktree of its own: everything since the commit it was created from, the agent's
/// own commits included.
Worktree { cwd: PathBuf, base: String },
/// The project's checkout, where the user works and commits too. `start` is the commit it
/// started from (`None` in a repository without commits, or an imported session).
Local { cwd: PathBuf, start: Option<String> },
}
impl TaskBase {
pub(crate) fn cwd(&self) -> &Path {
match self {
Self::Worktree { cwd, .. } | Self::Local { cwd, .. } => cwd,
}
}
/// The files the task changed, less those changed before it started and untouched since.
pub(crate) fn changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<Vec<AgentChange>> {
let (cwd, start) = match self {
Self::Worktree { cwd, base } => return Ok(without_untouched(changes(cwd, base)?, before)),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
// What is uncommitted, as the Commit tool window lists it: commits made since the task
// started drop out.
let head = repo.branch_info()?.head_commit;
let listed = match &head {
Some(head) => changes(cwd, head)?,
None => {
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
all
}
};
let listed = without_untouched(listed, before);
let Some(start) = start.filter(|s| head.as_deref() != Some(*s)) else { return Ok(listed) };
// HEAD moved (a reset, a checkout, a pull): a file that is as it was when the task
// started is the user's doing, not the task's.
Ok(listed
.into_iter()
.filter(|c| {
before.contains_key(&c.rel_path)
|| !repo.file_bytes_at(&c.path.0, Some(start)).is_ok_and(|at| at == std::fs::read(&c.path.0).ok())
})
.collect())
}
/// The revision the task's changed files go back to on rollback, and are compared with:
/// the commit the task started from, or the current `HEAD` once the user committed on top
/// of it, so those commits stay. `None` while the repository has no commits.
pub(crate) fn rev(&self) -> IdeResult<Option<String>> {
let (cwd, start) = match self {
Self::Worktree { base, .. } => return Ok(Some(base.clone())),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(None) };
let head = repo.branch_info()?.head_commit;
// A start commit git no longer knows (garbage-collected) can't be gone back to.
let start = start.filter(|s| repo.merge_base(s, s).ok().flatten().is_some());
Ok(match (start, head) {
(Some(start), Some(head)) if start != head && repo.merge_base(start, &head)?.as_deref() != Some(start) => {
Some(start.to_owned())
}
(start, head) => head.or_else(|| start.map(str::to_owned)),
})
}
}
/// Copies the task's `changes` (against `base`) from the worktree `cwd` into the
/// local checkout `root`, uncommitted: the user reviews and commits them with the
/// VCS tools. Files the checkout changed itself are left alone and reported, so
/// accepting never destroys the user's own edits. Returns accepted and skipped paths.
pub(crate) fn accept(
root: &Path,
cwd: &Path,
base: &str,
changes: &[AgentChange],
) -> IdeResult<crate::AgentAcceptOutcome> {
let repo = repo_at(cwd)?;
let mut accepted = Vec::with_capacity(changes.len());
let mut skipped = Vec::new();
for change in changes {
let local = root.join(&change.rel_path);
let base_bytes = repo.file_bytes_at(&change.path.0, Some(base))?;
let current = std::fs::read(&change.path.0).ok();
let local_bytes = std::fs::read(&local).ok();
if local_bytes == current {
accepted.push(change.rel_path.clone());
} else if local_bytes == base_bytes {
match current {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&local, bytes)?;
}
None => match std::fs::remove_file(&local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
accepted.push(change.rel_path.clone());
} else {
skipped.push(crate::AgentAcceptSkipped {
path: change.rel_path.clone(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
});
}
}
Ok(crate::AgentAcceptOutcome { accepted, skipped })
}
/// Size and modification time of a file, `None` once it is gone.
pub(crate) type Stamp = Option<(u64, u128)>;
@@ -166,11 +298,11 @@ pub(crate) fn save_before(cwd: &Path, before: &BTreeMap<String, Stamp>, dir: &Pa
}
/// Rolls `change` back: to the copy saved when the task started when the file had changes
/// then (`before`, `saved` holding the copies), otherwise to `base` (removed when `base`
/// lacks it).
/// then (`before`, `saved` holding the copies), otherwise to `rev` (removed when `rev` lacks
/// it, or when there is no `rev`: the repository had no commits).
pub(crate) fn restore(
cwd: &Path,
base: &str,
rev: Option<&str>,
change: &AgentChange,
before: &BTreeMap<String, Stamp>,
saved: Option<&Path>,
@@ -188,10 +320,13 @@ pub(crate) fn restore(
})?;
Some(std::fs::read(copy)?)
}
None => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(base))?
}
None => match rev {
Some(rev) => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(rev))?
}
None => None,
},
};
match content {
Some(bytes) => {
@@ -209,7 +344,7 @@ pub(crate) fn restore(
}
pub(crate) fn file_at(cwd: &Path, path: &Path, rev: &str) -> IdeResult<Option<String>> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(None) };
let Some(repo) = discover(cwd)? else { return Ok(None) };
let encoding = match ide_vfs::load_text(path) {
Ok(loaded) => encoding_rs::Encoding::for_label(loaded.encoding.as_bytes()).unwrap_or(encoding_rs::UTF_8),
Err(_) => ide_vfs::detect_encoding(&repo.file_bytes_at(path, Some(rev))?.unwrap_or_default(), None),
@@ -396,4 +531,94 @@ mod tests {
assert!(remove(&repo, &tmp.path().join("wt"), &elsewhere).is_err());
assert!(elsewhere.join("keep.txt").exists());
}
fn accept_repo() -> (tempfile::TempDir, PathBuf, Created) {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
std::fs::write(repo.join("gone.txt"), "bye\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
(tmp, repo, created)
}
fn listed(cwd: &Path, base: &str) -> Vec<AgentChange> {
without_untouched(changes(cwd, base).unwrap(), &BTreeMap::new())
}
#[test]
fn accept_copies_new_modified_and_deleted_files_into_the_checkout() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("new.txt"), "x\n").unwrap();
std::fs::remove_file(created.path.join("gone.txt")).unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(outcome.skipped.is_empty());
assert_eq!(outcome.accepted, ["a.txt", "gone.txt", "new.txt"]);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert_eq!(std::fs::read_to_string(repo.join("new.txt")).unwrap(), "x\n");
assert!(!repo.join("gone.txt").exists());
// Accepting again changes nothing: the checkout already matches.
let again =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(again.skipped.is_empty());
assert_eq!(again.accepted, ["a.txt", "gone.txt", "new.txt"]);
}
#[test]
fn a_local_task_goes_back_to_its_start_unless_the_user_committed_on_top_of_it() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
let local = |start: Option<&str>| TaskBase::Local { cwd: repo.clone(), start: start.map(str::to_owned) };
let head = || head_of(&repo).unwrap().1;
let commit = |name: &str| {
std::fs::write(repo.join(name), "x\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", name]);
head_of(&repo).unwrap().1
};
assert_eq!(local(None).rev().unwrap(), None, "no commits yet");
let first = commit("a.txt");
assert_eq!(local(None).rev().unwrap(), Some(first.clone()), "an imported session: HEAD");
let start = commit("b.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "HEAD where it started");
let ahead = commit("c.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(ahead), "committed on top: HEAD");
git(&repo, &["reset", "-q", "--soft", &first]);
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "moved back: the start");
git(&repo, &["checkout", "-q", "-b", "side"]);
commit("d.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "another branch: the start");
let gone = "0123456789abcdef0123456789abcdef01234567";
assert_eq!(local(Some(gone)).rev().unwrap(), Some(head()), "an unknown start: HEAD");
let worktree = TaskBase::Worktree { cwd: repo.clone(), base: start.clone() };
assert_eq!(worktree.rev().unwrap(), Some(start));
}
#[test]
fn accept_leaves_files_the_checkout_changed_itself() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("gone.txt"), "bye\nbye\n").unwrap();
std::fs::write(repo.join("a.txt"), "one\nMINE\n").unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert_eq!(outcome.accepted, ["gone.txt"]);
assert_eq!(outcome.skipped.len(), 1);
assert_eq!(outcome.skipped[0].path, "a.txt");
assert!(!outcome.skipped[0].reason.is_empty());
// The user's own edit survives; the other file comes over.
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nMINE\n");
assert_eq!(std::fs::read_to_string(repo.join("gone.txt")).unwrap(), "bye\nbye\n");
}
}
+31 -1
Переглянути файл
@@ -266,6 +266,11 @@ pub(super) fn maven_steps(
BuildKind::Rebuild => &["clean", "test-compile"],
BuildKind::Artifacts => &["package", "-DskipTests"],
};
// The IDE's compile builds are not coverage runs. JaCoCo's offline `instrument`
// rewrites target/classes in `process-classes`; `jacoco.skip` prevents that in this
// build. It does not restore classes instrumented by an earlier build: Rebuild's
// `clean` removes those, while an incremental Build can still reuse them.
let skip_jacoco = matches!(kind, BuildKind::Build | BuildKind::Rebuild);
let tops: Vec<&maven::MavenProject> = m.projects.iter().filter(|p| !p.aggregated).collect();
if tops.is_empty() {
return Err(IdeError::not_found("no Maven project found"));
@@ -278,6 +283,9 @@ pub(super) fn maven_steps(
Format::Maven,
);
s.args.extend(p.options.iter().cloned());
if skip_jacoco {
s.args.push("-Djacoco.skip=true".into());
}
s.args.extend(extra.iter().cloned());
s.args.extend(goals.iter().map(|g| (*g).to_owned()));
s.env = p.env.clone();
@@ -1041,6 +1049,22 @@ mod tests {
std::fs::write(p, text).unwrap();
}
#[test]
fn maven_compile_steps_skip_jacoco_but_artifacts_do_not() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("demo");
write(&root, "pom.xml", "<project><artifactId>demo</artifactId></project>");
let model = crate::load(&root, &tmp.path().join("cache"), None);
for (kind, expected) in [(BuildKind::Build, true), (BuildKind::Rebuild, true), (BuildKind::Artifacts, false)] {
let plan = plan(&root, &model, kind, None, None).unwrap();
assert_eq!(
plan.steps[0].args.iter().any(|arg| arg == "-Djacoco.skip=true"),
expected,
"{kind:?}: {plan:?}"
);
}
}
#[test]
fn plans_per_build_system() {
let tmp = tempfile::tempdir().unwrap();
@@ -1057,7 +1081,10 @@ mod tests {
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "test-compile"]);
assert_eq!(
p.steps[0].args,
["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "-Djacoco.skip=true", "test-compile"]
);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(
p.steps[0].args,
@@ -1065,6 +1092,7 @@ mod tests {
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-Djacoco.skip=true",
"-pl",
"core",
"-am",
@@ -1353,6 +1381,7 @@ mod tests {
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
@@ -1369,6 +1398,7 @@ mod tests {
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
+1
Переглянути файл
@@ -536,6 +536,7 @@ mod tests {
crate::maven::LOCAL_REPO_TRACKING_ARG,
"-o",
"-Peditor",
"-Djacoco.skip=true",
"test-compile"
]
);
+13
Переглянути файл
@@ -275,6 +275,11 @@ pub enum DebugEvent {
},
/// The session's capabilities changed after it started: fetch them again.
Capabilities,
/// The adapter's `invalidated` event: what it shows for `areas` (`all`, `stacks`, `threads`,
/// `variables`; empty means all) is stale and has to be fetched again.
Invalidated {
areas: Vec<String>,
},
JvmBackend {
state: JvmBackendState,
},
@@ -330,4 +335,12 @@ mod tests {
);
assert_eq!(serde_json::to_value(DebugEvent::Capabilities).unwrap(), json!({"kind": "capabilities"}));
}
#[test]
fn invalidated_event_json_shape() {
assert_eq!(
serde_json::to_value(DebugEvent::Invalidated { areas: vec!["variables".into()] }).unwrap(),
json!({"kind": "invalidated", "areas": ["variables"]})
);
}
}
+10 -2
Переглянути файл
@@ -12,11 +12,16 @@ use std::path::{Path, PathBuf};
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// `p` as the UI sees it: `C:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
/// `p` as the UI sees it: `C:\a\b`, `c:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
pub fn to_ipc(p: &Path) -> String {
#[cfg(windows)]
{
dunce::simplified(p).to_string_lossy().replace('\\', "/")
let mut s = dunce::simplified(p).to_string_lossy().replace('\\', "/");
// The UI compares path strings; a drive letter is case-insensitive.
if s.as_bytes().get(1) == Some(&b':') {
s[..1].make_ascii_uppercase();
}
s
}
#[cfg(not(windows))]
{
@@ -144,5 +149,8 @@ mod tests {
assert_eq!(to_ipc(Path::new(r"\\?\C:\a\b")), "C:/a/b");
assert_eq!(to_ipc(&Path::new("C:/a").join("b")), "C:/a/b");
assert_eq!(serde_json::to_string(&IpcPath::from(r"C:\a\b")).unwrap(), r#""C:/a/b""#);
assert_eq!(to_ipc(Path::new(r"c:\a\b")), "C:/a/b");
assert_eq!(to_ipc(Path::new(r"\\?\d:\a")), "D:/a");
assert_eq!(to_ipc(Path::new(r"\\server\share\a")), "//server/share/a");
}
}
+3
Переглянути файл
@@ -63,6 +63,8 @@ pub struct Settings {
pub agents: AgentSettings,
/// Agents › MCP Server (global only).
pub mcp_server: McpServerSettings,
/// Tools › Agents: Command Execution Policy.
pub agent_security: AgentSecuritySettings,
/// Plugin settings (`[contributes.settings]`): plugin id → its values by key.
#[specta(type = BTreeMap<String, BTreeMap<String, specta_typescript::Unknown>>)]
pub plugins: BTreeMap<String, serde_json::Map<String, serde_json::Value>>,
@@ -411,6 +413,7 @@ impl Default for Settings {
cmake: CMakeSettings::default(),
agents: AgentSettings::default(),
mcp_server: McpServerSettings::default(),
agent_security: AgentSecuritySettings::default(),
plugins: BTreeMap::new(),
}
}
+35 -2
Переглянути файл
@@ -138,8 +138,11 @@ impl WorkspaceEdit {
continue;
}
*expected = expected.or(version);
// Repeats within one entry are intended (LSP inserts at one position
// apply in order); only an alias entry repeats what is already merged.
let merged = own.len();
for edit in edits {
if !own.contains(&edit) {
if !own[..merged].contains(&edit) {
own.push(edit);
}
}
@@ -277,10 +280,40 @@ pub struct OutlineItem {
/// Fuzzy-match highlight: indices (in chars of the displayed string) that matched.
pub type MatchIndices = Vec<u32>;
#[cfg(all(test, unix))]
#[cfg(test)]
mod workspace_edit_tests {
use super::*;
#[test]
fn repeated_edits_within_one_entry_survive_but_repeats_across_entries_do_not() {
let path = PathBuf::from("missing-dir/main.ts");
let at = |line, text: &str| TextEdit {
range: Range::new(Pos::new(line, 0), Pos::new(line, 0)),
new_text: text.into(),
};
let entry = |edits: Vec<TextEdit>| FileEdit::Edit { path: path.clone().into(), edits, version: None };
let (nl, brace, other) = (at(0, "\n"), at(1, "}"), at(2, "x"));
for (entries, expected) in [
(vec![vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![brace.clone(), nl.clone(), brace.clone()]], vec![brace.clone(), nl.clone(), brace.clone()]),
(vec![vec![nl.clone(), nl.clone()], vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![nl.clone()], vec![nl.clone(), other.clone()]], vec![nl.clone(), other.clone()]),
(
vec![vec![nl.clone()], vec![other.clone(), other.clone()]],
vec![nl.clone(), other.clone(), other.clone()],
),
] {
let mut workspace =
WorkspaceEdit { files: entries.clone().into_iter().map(entry).collect(), ..Default::default() };
workspace.deduplicate();
assert_eq!(workspace.files, [entry(expected.clone())], "{entries:?}");
let once = workspace.clone();
workspace.deduplicate();
assert_eq!(workspace, once, "{entries:?}");
}
}
#[cfg(unix)]
#[test]
fn aliases_are_one_edit_but_resource_operations_preserve_order() {
let dir = tempfile::tempdir().unwrap();
+27
Переглянути файл
@@ -412,3 +412,30 @@ impl Default for ExternalTool {
}
}
}
/// Tools › Agents: Command Execution Policy (whitelist and blacklist patterns).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentSecuritySettings {
/// Commands or patterns (`git status*`, `pwd`) automatically approved without prompting.
pub allowed_commands: Vec<String>,
/// Commands or patterns (`rm -rf *`, `mkfs*`) automatically rejected.
pub blocked_commands: Vec<String>,
}
impl Default for AgentSecuritySettings {
fn default() -> Self {
Self {
allowed_commands: Vec::new(),
blocked_commands: vec![
"rm -rf *".into(),
"rm -rf /".into(),
"mkfs*".into(),
"dd *".into(),
":(){ :|:& };:".into(),
"shutdown*".into(),
"reboot*".into(),
],
}
}
}
+7
Переглянути файл
@@ -72,6 +72,13 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install Kotlin Debug Adapter in Settings › Languages & Frameworks › Language Servers",
),
// Inside the language server: reached through `debugServer`, never started.
"kotlin-lsp" => cfg(
"kotlin-lsp",
&[],
"tcp",
"Install JetBrains Kotlin LSP in Settings › Languages & Frameworks › Language Servers",
),
// Windows installs `python.exe`; `python3.exe` there is at most a Store alias.
"debugpy" => cfg(
if cfg!(windows) { "python" } else { "python3" },
+1 -1
Переглянути файл
@@ -29,7 +29,7 @@ pub(crate) use scan::watch_for_attached_jvm;
pub(crate) use scan::{JvmFinder, attach_target, find_attached_jvm, find_local_jvm, local_jdwp_port};
use snapshot::{AttachProbe, JdkProbe, take_snapshot};
pub const JVM_MEMORY_ADAPTERS: &[&str] = &["java-debug", crate::kotlin::ADAPTER];
pub const JVM_MEMORY_ADAPTERS: &[&str] = &["java-debug", crate::kotlin::ADAPTER, crate::kotlin_lsp::ADAPTER];
pub const HISTOGRAM_TIMEOUT: Duration = Duration::from_secs(10);
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
+9 -5
Переглянути файл
@@ -9,11 +9,15 @@ use std::{
pub const ADAPTER: &str = "kotlin-debug-adapter";
/// The adapter attaches to a JVM the IDE starts, through the JDWP link. It resumes the VM while
/// constructing its debuggee, before its breakpoint hooks exist, so the link holds resumes back
/// until the attach is done.
pub(crate) fn link_options() -> ide_jdwp::LinkOptions {
ide_jdwp::LinkOptions { hold_resumes: true }
/// fwcd's adapter or kotlin-lsp's: both debug a JVM the IDE starts, through the JDWP link.
pub fn is_kotlin(adapter: &str) -> bool {
adapter == ADAPTER || adapter == crate::kotlin_lsp::ADAPTER
}
/// fwcd's adapter resumes the VM while constructing its debuggee, before its breakpoint hooks
/// exist, so only its link holds resumes back until the attach is done.
pub(crate) fn link_options(adapter: &str) -> ide_jdwp::LinkOptions {
ide_jdwp::LinkOptions { hold_resumes: adapter == ADAPTER }
}
pub struct SourceMirror {
+147
Переглянути файл
@@ -0,0 +1,147 @@
//! JetBrains kotlin-lsp's debugger: a DAP server inside the language server, reached through
//! the `debugServer` port it opens once the project is imported.
//!
//! Known limits, with no guard here: an evaluate that calls a function with a breakpoint, right
//! after a step over a suspension point, loses every later stop at that breakpoint; while
//! stepping, breakpoints hit by other coroutines are skipped.
use ide_core::{IdeError, IdeResult};
use parking_lot::Mutex;
use std::collections::HashSet;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
pub const ADAPTER: &str = "kotlin-lsp";
/// The `adapterID` its debug server expects in `initialize`.
pub const DAP_ADAPTER_ID: &str = "intellij_jvm";
/// The path of a `file:` URI, as kotlin-lsp reports some sources (percent-encoded); `None` for
/// anything else, including a URI naming another host.
pub(crate) fn source_path(path: &str) -> Option<PathBuf> {
let rest = path.strip_prefix("file:")?;
let path = match rest.strip_prefix("//") {
Some(authority) => {
let slash = authority.find('/')?;
if !matches!(&authority[..slash], "" | "localhost") {
return None;
}
&authority[slash..]
}
None => rest,
};
let decoded = percent_encoding::percent_decode_str(path).decode_utf8().ok()?;
// `/C:/p/App.kt` on Windows.
let decoded = match decoded.strip_prefix('/') {
Some(drive) if cfg!(windows) && drive.as_bytes().get(1) == Some(&b':') => drive,
_ => &decoded,
};
Some(PathBuf::from(decoded))
}
pub(crate) fn frame_name(name: &str) -> &str {
name.strip_prefix("at ").unwrap_or(name)
}
/// What kotlin-lsp's DAP server does unlike other adapters.
///
/// It never answers a request that carries a thread id from an earlier stop (K1 G4), so such a
/// request would wait out `REQUEST_TIMEOUT`: [`Self::check_thread`] refuses it first. Only thread
/// ids were measured; a stale `frameId` or `variablesReference` may wait the same way. A step
/// ends with `reason: "breakpoint"`; a real breakpoint hit during a step is then reported as a
/// step too, since the stop carries neither `hitBreakpointIds` nor a description.
#[derive(Default)]
pub(crate) struct Quirks {
step_pending: AtomicBool,
/// The stop (a `Suspension` ticket) and the thread ids seen in it.
threads: Mutex<(u64, HashSet<i64>)>,
}
impl Quirks {
/// Before `next`, `stepIn`, `stepOut` or a forced step.
pub fn step_sent(&self) {
self.step_pending.store(true, Ordering::Release);
}
/// Before `continue` or run to cursor, or after a step request failed.
pub fn resume_sent(&self) {
self.step_pending.store(false, Ordering::Release);
}
/// The reason to report for a `stopped` event.
pub fn stop_reason(&self, reason: String) -> String {
let stepped = self.step_pending.swap(false, Ordering::AcqRel);
if stepped && reason == "breakpoint" { "step".into() } else { reason }
}
/// Thread ids seen in stop `stop`: a `stopped` event's, or a `threads` reply to a request sent
/// during that stop. A reply from an older stop changes nothing.
pub fn saw_threads(&self, stop: u64, ids: impl IntoIterator<Item = i64>) {
let mut threads = self.threads.lock();
if stop > threads.0 {
*threads = (stop, ids.into_iter().collect());
} else if stop == threads.0 {
threads.1.extend(ids);
}
}
/// Refuses a thread id that is not one of current stop `stop`'s, once some are known.
pub fn check_thread(&self, stop: u64, suspended: bool, id: i64) -> IdeResult<()> {
let threads = self.threads.lock();
if suspended && threads.0 == stop && !threads.1.is_empty() && !threads.1.contains(&id) {
return Err(IdeError::invalid(format!("thread {id} belongs to an earlier stop")));
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
#[test]
fn file_uris_become_paths() {
assert_eq!(source_path("file:///p/a%20b/App.kt").as_deref(), Some(Path::new("/p/a b/App.kt")));
assert_eq!(source_path("file://localhost/p/x.kt").as_deref(), Some(Path::new("/p/x.kt")));
assert_eq!(source_path("/p/x.kt"), None);
}
#[test]
fn frame_names_lose_the_at_prefix() {
assert_eq!(frame_name("at com.x.AppKt.main(App.kt:13)"), "com.x.AppKt.main(App.kt:13)");
assert_eq!(frame_name("main"), "main");
}
#[test]
fn only_a_pending_step_turns_a_breakpoint_into_a_step() {
let quirks = Quirks::default();
quirks.step_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "step");
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
quirks.step_sent();
quirks.resume_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
}
#[test]
fn threads_are_checked_against_the_current_stop() {
let quirks = Quirks::default();
quirks.saw_threads(3, [18]);
assert!(quirks.check_thread(3, true, 18).is_ok());
assert!(quirks.check_thread(3, true, 6).is_err());
assert!(quirks.check_thread(4, true, 6).is_ok(), "nothing known about stop 4 yet");
assert!(quirks.check_thread(3, false, 6).is_ok());
}
#[test]
fn an_older_threads_reply_does_not_change_a_newer_stop() {
let quirks = Quirks::default();
quirks.saw_threads(4, [2]);
quirks.saw_threads(3, [1]);
assert!(quirks.check_thread(4, true, 1).is_err());
quirks.saw_threads(4, [7]);
assert!(quirks.check_thread(4, true, 2).is_ok() && quirks.check_thread(4, true, 7).is_ok());
quirks.saw_threads(5, [9]);
assert!(quirks.check_thread(5, true, 2).is_err());
}
}
+55 -19
Переглянути файл
@@ -15,6 +15,7 @@ pub mod connection;
pub mod jdwp;
mod jvm_memory;
pub mod kotlin;
pub mod kotlin_lsp;
mod session;
use adapter::{AdapterLauncher, AdapterSpec, ProcessLauncher};
@@ -610,9 +611,19 @@ fn unverified(enabled: impl Iterator<Item = bool>) -> Vec<BreakpointState> {
.collect()
}
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (on Windows
/// `__debug_bin.exe` plus `__debug_bin.exe<digits>` in DAP mode) and removes it on a
/// graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// A step request's outcome; a failed one leaves no step pending, so the next stop keeps its reason.
fn step_settled(session: &Session, sent: Result<Value, DapError>) -> Result<Value, DapError> {
if sent.is_err()
&& let Some(quirks) = session.kotlin_lsp_quirks()
{
quirks.resume_sent();
}
sent
}
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (in DAP mode a
/// temp-file name, `__debug_bin<digits>`; on Windows with `.exe` before the digits) and
/// removes it on a graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// the leftovers once the session is down. Runs after `session_closed`, so Delve
/// has already exited or been killed and holds no file locks; a leftover of a
/// still-building run is removed on that run's own shutdown.
@@ -621,14 +632,10 @@ fn remove_delve_binaries(cwd: &Path) {
for entry in entries.flatten() {
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
// Windows writes `__debug_bin.exe` (and `.exe<digits>` in DAP mode); other
// platforms keep the bare name. Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin");
let leftover = suffix.is_some_and(|s| {
s.is_empty()
|| s == ".exe"
|| (s.starts_with(".exe") && s[".exe".len()..].chars().all(|c| c.is_ascii_digit()))
});
// `__debug_bin`, then `.exe` on Windows, then DAP mode's temp-file digits.
// Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin").map(|s| s.strip_prefix(".exe").unwrap_or(s));
let leftover = suffix.is_some_and(|s| s.chars().all(|c| c.is_ascii_digit()));
if !leftover || entry.file_type().is_ok_and(|t| !t.is_file()) {
continue;
}
@@ -981,6 +988,7 @@ impl DebugManager {
arguments["projectRoot"] = json!(mirror.root());
*s.kotlin_mirror.lock() = Some(mirror);
}
let is_kotlin_lsp = s.adapter == kotlin_lsp::ADAPTER;
let caps = s
.conn
.request_timeout(
@@ -988,7 +996,7 @@ impl DebugManager {
json!({
"clientID": "openidea",
"clientName": "OpenIDEA",
"adapterID": s.adapter,
"adapterID": if is_kotlin_lsp { kotlin_lsp::DAP_ADAPTER_ID } else { s.adapter.as_str() },
"locale": "en",
"linesStartAt1": true,
"columnsStartAt1": true,
@@ -998,7 +1006,8 @@ impl DebugManager {
"supportsRunInTerminalRequest": true,
"supportsStartDebuggingRequest": true,
"supportsProgressReporting": true,
"supportsInvalidatedEvent": false,
// kotlin-lsp sends `invalidated` either way.
"supportsInvalidatedEvent": is_kotlin_lsp,
"supportsMemoryReferences": false,
"supportsArgsCanBeInterpretedByShell": false,
}),
@@ -1008,7 +1017,7 @@ impl DebugManager {
*s.caps.lock() = if caps.is_object() { caps } else { json!({}) };
// The Kotlin adapters ignore the resolved classpath and project runtime on launch.
let request = if s.adapter == kotlin::ADAPTER && s.request == "launch" {
let request = if kotlin::is_kotlin(&s.adapter) && s.request == "launch" {
session::start_kotlin_debuggee(s, &mut arguments).await?;
"attach"
} else {
@@ -1032,7 +1041,7 @@ impl DebugManager {
jvm_memory::find_attached_jvm(s, port, finder).await;
}
}
if s.adapter == kotlin::ADAPTER {
if kotlin::is_kotlin(&s.adapter) {
let host = arguments
.get("hostName")
.and_then(Value::as_str)
@@ -1044,7 +1053,7 @@ impl DebugManager {
.and_then(|p| u16::try_from(p).ok())
.filter(|p| *p != 0)
.ok_or_else(|| IdeError::invalid("Invalid Kotlin JDWP port"))?;
let (link, port) = ide_jdwp::JdwpLink::start(host, port, kotlin::link_options()).await?;
let (link, port) = ide_jdwp::JdwpLink::start(host, port, kotlin::link_options(&s.adapter)).await?;
*s.jvm_inspector.lock() = Some(Arc::new(ide_jdwp::LinkInspector::new(link.clone())));
jvm_memory::watch_link(s, &link);
*s.jdwp_link.lock() = Some(link);
@@ -1381,6 +1390,8 @@ impl DebugManager {
pub async fn threads(&self, s: DebugSessionId) -> IdeResult<Vec<ThreadInfo>> {
let session = self.session(s)?;
// Before sending: a stop that overtakes the reply must not take the reply's ids as its own.
let stop = session.suspended.ticket();
let body = session.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
let threads: Vec<ThreadInfo> = body
.get("threads")
@@ -1394,11 +1405,15 @@ impl DebugManager {
.collect()
})
.unwrap_or_default();
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.saw_threads(stop, threads.iter().map(|t| t.id));
}
Ok(threads)
}
pub async fn stack_trace(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<Vec<StackFrame>> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
let body = session
.conn
.request_timeout(
@@ -1407,11 +1422,15 @@ impl DebugManager {
REQUEST_TIMEOUT,
)
.await?;
let kotlin_lsp = session.kotlin_lsp_quirks().is_some();
let frame = |f: &Value| {
let mut frame = frame_from(f);
if let Some(client) = frame.path.as_deref().and_then(|p| session.client_path(&p.to_string_lossy())) {
frame.path = Some(client.into());
}
if kotlin_lsp {
frame.name = kotlin_lsp::frame_name(&frame.name).to_owned();
}
frame
};
Ok(body.get("stackFrames").and_then(Value::as_array).map(|f| f.iter().map(frame).collect()).unwrap_or_default())
@@ -1523,6 +1542,10 @@ impl DebugManager {
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.resume_sent();
}
let ticket = session.suspended.ticket();
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
// DAP: a response without the flag continued every thread.
@@ -1567,8 +1590,13 @@ impl DebugManager {
StepKind::Out => "stepOut",
};
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
let sent = session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await;
step_settled(&session, sent)?;
// A step runs its thread; other threads stopped by their own events stay stopped.
session.suspended.resumed(ticket, thread_id, false);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
@@ -1587,6 +1615,8 @@ impl DebugManager {
force: bool,
) -> IdeResult<()> {
let session = self.session(s)?;
// Before touching breakpoints: a refused `continue` would leave them replaced.
session.check_thread(thread_id)?;
if force {
self.suspend_breakpoints(&session, Some(&path)).await?;
}
@@ -1661,6 +1691,7 @@ impl DebugManager {
/// Force Step Over: `next` with every breakpoint suspended until the step ends.
pub async fn force_step_over(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
self.suspend_breakpoints(&session, None).await?;
self.step(s, thread_id, StepKind::Over).await
}
@@ -1668,11 +1699,16 @@ impl DebugManager {
/// Step into a specific call of the current line (Smart Step Into).
pub async fn step_in_target(&self, s: DebugSessionId, thread_id: i64, target_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
session
let sent = session
.conn
.request_timeout("stepIn", json!({ "threadId": thread_id, "targetId": target_id }), REQUEST_TIMEOUT)
.await?;
.await;
step_settled(&session, sent)?;
session.suspended.resumed(ticket, thread_id, false);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
+34 -1
Переглянути файл
@@ -3,6 +3,7 @@
use crate::adapter::AdapterSpec;
use crate::connection::{Connection, DapError, Incoming};
use crate::kotlin_lsp;
use crate::{Breakpoint, BreakpointState, DataBreakpoint, DebugManager, FunctionBreakpoint, SessionOwner};
use ide_core::event::DebugEvent;
#[cfg(unix)]
@@ -103,6 +104,8 @@ pub(crate) struct Session {
pub jvm_located: watch::Sender<bool>,
pub kotlin_mirror: Mutex<Option<crate::kotlin::SourceMirror>>,
pub jvm_memory: tokio::sync::Mutex<crate::JvmMemoryState>,
/// Used only by a kotlin-lsp session: see [`Session::kotlin_lsp_quirks`].
pub(crate) quirks: kotlin_lsp::Quirks,
}
/// Which threads of the program are suspended: each from its `stopped` event until a
@@ -252,6 +255,19 @@ impl Session {
jvm_located: watch::channel(false).0,
kotlin_mirror: Mutex::new(None),
jvm_memory: tokio::sync::Mutex::default(),
quirks: kotlin_lsp::Quirks::default(),
}
}
pub fn kotlin_lsp_quirks(&self) -> Option<&kotlin_lsp::Quirks> {
(self.adapter == kotlin_lsp::ADAPTER).then_some(&self.quirks)
}
/// Call before sending a request for thread `id`.
pub fn check_thread(&self, id: i64) -> ide_core::IdeResult<()> {
match self.kotlin_lsp_quirks() {
Some(quirks) => quirks.check_thread(self.suspended.ticket(), self.suspended.get(), id),
None => Ok(()),
}
}
@@ -283,6 +299,11 @@ impl Session {
/// A path from the adapter as the IDE knows it (undoes [`Session::adapter_path`]).
pub fn client_path(&self, path: &str) -> Option<PathBuf> {
if self.adapter == kotlin_lsp::ADAPTER
&& let Some(path) = kotlin_lsp::source_path(path)
{
return Some(path);
}
if let Some(real) = self.kotlin_mirror.lock().as_ref().and_then(|m| m.client_path(Path::new(path))) {
return Some(real);
}
@@ -697,7 +718,11 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
"stopped" => {
let all = body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false);
s.suspended.stopped(int("threadId"), all);
let reason = str_("reason").unwrap_or_else(|| "pause".into());
let mut reason = str_("reason").unwrap_or_else(|| "pause".into());
if let Some(quirks) = s.kotlin_lsp_quirks() {
quirks.saw_threads(s.suspended.ticket(), int("threadId"));
reason = quirks.stop_reason(reason);
}
mgr.restore_breakpoints(s).await;
s.emit(DebugEvent::Stopped {
thread_id: int("threadId").unwrap_or(0),
@@ -799,6 +824,14 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
change_type: crate::hot_code_replace_type(body.get("changeType")),
message: str_("message").unwrap_or_default(),
}),
// Sent only by adapters told `supportsInvalidatedEvent` (kotlin-lsp).
"invalidated" => s.emit(DebugEvent::Invalidated {
areas: body
.get("areas")
.and_then(Value::as_array)
.map(|a| a.iter().filter_map(Value::as_str).map(str::to_owned).collect())
.unwrap_or_default(),
}),
_ => tracing::trace!("dap: ignored event {event}"),
}
}
+291 -6
Переглянути файл
@@ -126,8 +126,22 @@ struct Script {
continued_after_stop: bool,
/// Close the connection when a `pause` request arrives (the adapter dies).
hang_up_on_pause: bool,
/// Close the connection at the first request after the first `stopped` event.
hang_up_after_stop: bool,
/// Answer `continue` with this body and keep the program running, instead of letting it exit.
continue_reply: Option<Value>,
/// Answer `continue` and stop at a breakpoint again, instead of letting the program exit.
stop_after_continue: bool,
/// The `reason` of the stop a `next` ends with (`step` when unset; kotlin-lsp says `breakpoint`).
step_reason: Option<&'static str>,
/// Refuse `next`, then stop at a breakpoint (one the program hit meanwhile).
refuse_next: bool,
/// Send `invalidated { areas: ["variables"] }` right after the first `stopped` event.
invalidated_after_stop: bool,
/// On `threads`: stop again on thread 2 first, then answer with thread 1 of the earlier stop.
stop_during_threads: bool,
/// Name of the first stack frame (`main` when unset).
frame_name: Option<&'static str>,
/// Answer `initialize` only after this long (a slow adapter keeps the session connecting).
delay_initialize: Option<Duration>,
initialized_delay: Duration,
@@ -156,7 +170,14 @@ impl Default for Script {
hold_set_breakpoints: 0,
continued_after_stop: false,
hang_up_on_pause: false,
hang_up_after_stop: false,
continue_reply: None,
stop_after_continue: false,
step_reason: None,
refuse_next: false,
invalidated_after_stop: false,
stop_during_threads: false,
frame_name: None,
delay_initialize: None,
initialized_delay: Duration::ZERO,
launch_delay: Duration::ZERO,
@@ -202,6 +223,7 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
let mut seq = 1000;
let mut pending_launch: Option<i64> = None;
let mut held: Vec<Value> = Vec::new();
let mut stopped = false;
let mut running = false;
macro_rules! send {
($v:expr) => {{
@@ -222,6 +244,9 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
if msg["type"] == "response" {
continue;
}
if script.hang_up_after_stop && stopped {
break;
}
match msg["command"].as_str().unwrap_or_default() {
"initialize" => {
if let Some(delay) = script.delay_initialize {
@@ -320,9 +345,18 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
ok!(msg, Value::Null);
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "restart", "threadId": 1}}));
}
"next" if script.refuse_next => {
send!(
json!({"type": "response", "request_seq": msg["seq"], "command": "next", "success": false, "message": "refused"})
);
// Lets the client handle the refusal before the stop.
tokio::time::sleep(Duration::from_millis(50)).await;
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1}}));
}
"next" => {
ok!(msg, Value::Null);
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "step", "threadId": 1}}));
let reason = script.step_reason.unwrap_or("step");
send!(json!({"type": "event", "event": "stopped", "body": {"reason": reason, "threadId": 1}}));
}
"setVariable" => {
ok!(msg, json!({"value": format!("{}!", msg["arguments"]["value"].as_str().unwrap()), "type": "int"}))
@@ -358,6 +392,12 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
stopped = true;
if script.invalidated_after_stop {
send!(
json!({"type": "event", "event": "invalidated", "body": {"areas": ["variables"], "threadId": 1}})
);
}
if script.continued_after_stop {
send!(json!({"type": "event", "event": "continued", "body": {"threadId": 1}}));
}
@@ -365,11 +405,19 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
json!({"type": "event", "event": "breakpoint", "body": {"reason": "changed", "breakpoint": {"id": 1, "verified": false, "message": "moved", "line": 9, "source": {"path": script.source_path}}}})
);
}
"threads" if script.stop_during_threads => {
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 2, "allThreadsStopped": true}})
);
// Lets the client handle the stop before it reads the reply.
tokio::time::sleep(Duration::from_millis(50)).await;
ok!(msg, json!({"threads": [{"id": 1, "name": "\"main\"@1"}]}));
}
"threads" => ok!(msg, json!({"threads": [{"id": 1, "name": "MainThread"}]})),
"stackTrace" => ok!(
msg,
json!({"stackFrames": [
{"id": 10, "name": "main", "line": 5, "column": 3, "source": {"path": script.source_path.as_deref().unwrap_or("/p/a.py")}},
{"id": 10, "name": script.frame_name.unwrap_or("main"), "line": 5, "column": 3, "source": {"path": script.source_path.as_deref().unwrap_or("/p/a.py")}},
{"id": 11, "name": "lib", "line": 1, "column": 1, "source": {"path": "/usr/lib/x.py", "presentationHint": "deemphasize"}},
]})
),
@@ -393,6 +441,12 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
]})
),
"continue" if script.continue_reply.is_some() => ok!(msg, script.continue_reply.clone().unwrap()),
"continue" if script.stop_after_continue => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
}
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
@@ -1418,6 +1472,7 @@ fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegist
indent: None,
ui_mode: None,
framework_ui_modes: Vec::new(),
framework_files: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
@@ -2041,20 +2096,20 @@ async fn stop_kills_the_program_started_by_the_reported_launcher() {
fn delve_leftovers_are_removed_without_touching_other_files() {
let dir = std::env::temp_dir().join(format!("openidea-delve-bin-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
for name in ["__debug_bin", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
for name in ["__debug_bin", "__debug_bin833767175", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
std::fs::write(dir.join(name), b"bin").unwrap();
}
// Lookalikes that are not Delve's own binaries stay: sources, a future
// in-progress build, and a directory that happens to share the name.
for name in ["__debug_bin.go", "__debug_bin.go.exe", "main.go"] {
for name in ["__debug_bin.go", "__debug_bin.go.exe", "__debug_bin12.go", "main.go"] {
std::fs::write(dir.join(name), b"keep").unwrap();
}
std::fs::create_dir(dir.join("__debug_bin.dir")).unwrap();
remove_delve_binaries(&dir);
for name in ["__debug_bin", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
for name in ["__debug_bin", "__debug_bin833767175", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
assert!(!dir.join(name).exists(), "{name} should be gone");
}
for name in ["__debug_bin.go", "__debug_bin.go.exe", "main.go", "__debug_bin.dir"] {
for name in ["__debug_bin.go", "__debug_bin.go.exe", "__debug_bin12.go", "main.go", "__debug_bin.dir"] {
assert!(dir.join(name).exists(), "{name} should stay");
}
std::fs::remove_dir_all(&dir).unwrap();
@@ -2401,6 +2456,13 @@ fn kotlin_attach(project: &std::path::Path, host: &str, port: u16) -> LaunchConf
}
}
#[test]
fn link_options_hold_only_for_fwcd() {
assert!(kotlin::link_options(kotlin::ADAPTER).hold_resumes);
assert!(!kotlin::link_options(kotlin_lsp::ADAPTER).hold_resumes);
assert!(!kotlin::link_options("x").hold_resumes);
}
#[tokio::test]
async fn kotlin_session_installs_link_inspector() {
let h = harness(Script::default());
@@ -2472,6 +2534,229 @@ async fn closing_the_session_drops_the_link() {
assert!(session.jvm_inspector.lock().is_none());
}
/// kotlin-lsp's debugger, reached through the language server's `debugServer` port (the fake ignores it).
fn kotlin_lsp_attach(project: &std::path::Path) -> LaunchConfig {
LaunchConfig {
adapter: kotlin_lsp::ADAPTER.into(),
request: "attach".into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(
json!({"debugServer": 4711, "hostName": "127.0.0.1", "port": 5005, "projectRoot": project}).to_string(),
),
..config()
}
}
#[tokio::test]
async fn kotlin_lsp_initializes_as_intellij_jvm() {
let project = tempfile::tempdir().unwrap();
for (config, adapter_id, invalidated) in [
(kotlin_lsp_attach(project.path()), "intellij_jvm", true),
(kotlin_attach(project.path(), "127.0.0.1", 5005), "kotlin-debug-adapter", false),
] {
let h = harness(Script::default());
let id = h.start(config).await.unwrap();
let init = &h.request("initialize")["arguments"];
assert_eq!(init["adapterID"], adapter_id);
assert_eq!(init["supportsInvalidatedEvent"], invalidated, "{adapter_id}");
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn kotlin_lsp_gets_real_paths_and_its_link_holds_nothing() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let path = project.path().join("Main.kt");
std::fs::write(&path, "package org.example\nfun main() {}\n").unwrap();
h.manager.set_breakpoints(path.clone(), vec![bp(1)]).await.unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
assert_eq!(h.request("setBreakpoints")["arguments"]["source"]["path"], json!(path));
let sent = &h.request("attach")["arguments"];
assert_eq!(sent["projectRoot"], json!(project.path()));
assert!(sent.get("debugServer").is_none());
assert_eq!(sent["hostName"], "127.0.0.1");
let link_port = sent["port"].as_u64().unwrap();
assert!(link_port != 0 && link_port != 5005);
let session = h.manager.session(id).unwrap();
assert!(session.kotlin_mirror.lock().is_none());
assert!(session.jdwp_link.lock().is_some() && session.jvm_inspector.lock().is_some());
assert_eq!(session.attach_target.get(), Some(&("127.0.0.1".to_owned(), 5005)));
assert!(!kotlin::link_options(&session.adapter).hold_resumes);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn kotlin_lsp_waits_for_the_configuration_done_reply() {
let project = tempfile::tempdir().unwrap();
// fwcd never answers it (`kotlin_configures_without_configuration_done_response`).
let h = harness(Script { ignore_configuration_done: true, ..Default::default() });
let fwcd = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005));
let id = tokio::time::timeout(Duration::from_secs(2), fwcd).await.unwrap().unwrap();
h.manager.stop(id).await.unwrap();
tokio::time::pause();
let h = harness(Script { ignore_configuration_done: true, ..Default::default() });
let started = tokio::time::Instant::now();
let err = h.start(kotlin_lsp_attach(project.path())).await.unwrap_err();
assert!(started.elapsed() >= REQUEST_TIMEOUT, "{:?}", started.elapsed());
assert!(err.message.contains("configurationDone"), "{err:?}");
assert!(h.manager.sessions(None).is_empty());
}
#[tokio::test]
async fn kotlin_lsp_sessions_have_the_memory_tab() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_memory);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn losing_the_kotlin_lsp_connection_ends_the_session() {
let h = harness(Script { hang_up_after_stop: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some());
let _ = session.conn.request_timeout("threads", Value::Null, Duration::from_secs(2)).await;
h.wait_event(|e| matches!(e, DebugEvent::Terminated)).await;
assert!(h.manager.sessions(None).is_empty());
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
/// The reasons of the first `count` stops, once that many arrived.
async fn stop_reasons(h: &Harness, count: usize) -> Vec<String> {
for _ in 0..200 {
let reasons: Vec<String> = h
.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event: DebugEvent::Stopped { reason, .. }, .. } => Some(reason.clone()),
_ => None,
})
.collect();
if reasons.len() >= count {
return reasons;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("{count} stops not received; got {:?}", h.events.lock());
}
#[tokio::test]
async fn a_step_stop_is_reported_as_a_step() {
let project = tempfile::tempdir().unwrap();
for (config, reason) in [(kotlin_lsp_attach(project.path()), "step"), (config(), "breakpoint")] {
let h = harness(Script { step_reason: Some("breakpoint"), ..Script::default() });
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.step(id, 1, StepKind::Over).await.unwrap();
assert_eq!(stop_reasons(&h, 2).await, ["breakpoint", reason]);
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn a_breakpoint_after_continue_stays_a_breakpoint() {
let h = harness(Script { step_reason: Some("breakpoint"), stop_after_continue: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.step(id, 1, StepKind::Over).await.unwrap();
stop_reasons(&h, 2).await;
// The fake ends `stepIn` with no stop: the `continue` after it decides the next one.
h.manager.step(id, 1, StepKind::Into).await.unwrap();
h.manager.resume(id, 1).await.unwrap();
assert_eq!(stop_reasons(&h, 3).await, ["breakpoint", "step", "breakpoint"]);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn a_breakpoint_after_a_refused_step_stays_a_breakpoint() {
let h = harness(Script { refuse_next: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert!(h.manager.step(id, 1, StepKind::Over).await.is_err());
assert_eq!(stop_reasons(&h, 2).await, ["breakpoint", "breakpoint"]);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn a_thread_from_an_earlier_stop_is_refused() {
let project = tempfile::tempdir().unwrap();
for (config, kotlin_lsp) in [(kotlin_lsp_attach(project.path()), true), (config(), false)] {
let h = harness(Script::default());
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(h.manager.threads(id).await.unwrap().len(), 1);
let started = tokio::time::Instant::now();
let stale = h.manager.stack_trace(id, 99).await;
let asked =
|command: &str| h.log.lock().iter().any(|m| m["command"] == command && m["arguments"]["threadId"] == 99);
if kotlin_lsp {
assert!(stale.is_err() && started.elapsed() < Duration::from_millis(100), "{stale:?}");
assert!(h.manager.step(id, 99, StepKind::Over).await.is_err());
assert!(h.manager.resume(id, 99).await.is_err());
assert!(!asked("stackTrace") && !asked("next") && !asked("continue"));
} else {
assert!(stale.is_ok() && asked("stackTrace"), "{stale:?}");
}
assert!(h.manager.stack_trace(id, 1).await.is_ok());
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn a_threads_reply_from_before_a_stop_does_not_vouch_for_its_ids() {
let h = harness(Script { stop_during_threads: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let threads = h.manager.threads(id).await.unwrap();
// M3 maps kotlin-lsp's threads by these names.
assert_eq!((threads[0].id, threads[0].name.as_str()), (1, "\"main\"@1"));
stop_reasons(&h, 2).await;
assert!(h.manager.stack_trace(id, 1).await.is_err());
assert!(h.manager.stack_trace(id, 2).await.is_ok());
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn kotlin_lsp_frames_get_real_paths() {
let project = tempfile::tempdir().unwrap();
let at = "at com.x.AppKt.main(App.kt:13)";
for (config, path, name) in [
(kotlin_lsp_attach(project.path()), "/p/App.kt", "com.x.AppKt.main(App.kt:13)"),
(java_config("launch"), "file:///p/App.kt", at),
] {
let script = Script { source_path: Some("file:///p/App.kt".into()), frame_name: Some(at), ..Script::default() };
let h = harness(script);
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let frames = h.manager.stack_trace(id, 1).await.unwrap();
assert_eq!(frames[0].path.as_ref().map(|p| p.0.clone()), Some(PathBuf::from(path)));
assert_eq!(frames[0].name, name);
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn invalidated_events_reach_the_client() {
let h = harness(Script { invalidated_after_stop: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
match h.wait_event(|e| matches!(e, DebugEvent::Invalidated { .. })).await {
DebugEvent::Invalidated { areas } => assert_eq!(areas, ["variables"]),
other => panic!("{other:?}"),
}
h.manager.stop(id).await.unwrap();
}
#[cfg(unix)]
#[tokio::test]
async fn rdbg_drains_test_output_after_dap_closes_before_killing_the_process() {
+17 -2
Переглянути файл
@@ -71,7 +71,6 @@ impl Capability {
match self {
Self::ProjectRead | Self::EditorRead | Self::Storage | Self::Events | Self::VcsRead => Risk::Low,
Self::EditorWrite
| Self::ProjectWrite
| Self::DatabaseSources
| Self::VcsWrite
| Self::Net(_)
@@ -80,7 +79,14 @@ impl Capability {
| Self::Secrets
| Self::InputEvents
| Self::Clipboard => Risk::Medium,
Self::IdeCommands | Self::Ipc(_) | Self::Process(_) | Self::FsRead(_) | Self::FsWrite(_) => Risk::High,
// A trusted project's own files name the programs the IDE starts (its settings, its
// build scripts), so writing them is running code.
Self::ProjectWrite
| Self::IdeCommands
| Self::Ipc(_)
| Self::Process(_)
| Self::FsRead(_)
| Self::FsWrite(_) => Risk::High,
}
}
@@ -402,6 +408,15 @@ mod tests {
assert!(Capability::Storage.auto_migrates() && !Capability::EditorWrite.auto_migrates());
}
/// A trusted project's files name programs the IDE starts by itself: the git and language
/// server executables in `.openidea/settings.toml`, its build scripts. Writing them is
/// running code, and the install dialog says so.
#[test]
fn writing_project_files_is_a_high_risk_grant() {
assert_eq!(Capability::ProjectWrite.risk(), Risk::High);
assert!(!Capability::ProjectWrite.auto_migrates());
}
#[test]
fn host_patterns() {
assert!(host_matches("crates.io", "crates.io"));
+70 -3
Переглянути файл
@@ -405,10 +405,31 @@ impl Index {
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let (index, dropped) = Self::parse_dropping(text)?;
if let Some(first) = dropped.first() {
// One line per index, and a bounded one: whoever wrote the index chose the text.
let id: String = first.id.chars().take(80).collect();
tracing::warn!(
"index: {} entries are not offered, their id or version is not one a manifest may carry (first: {id:?})",
dropped.len()
);
}
Ok(index)
}
/// [`Index::parse`], with the entries it does not offer: those whose id or
/// version no manifest may carry, so no IDE could install them.
pub fn parse_dropping(text: &str) -> Result<(Self, Vec<IndexEntry>), String> {
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
let mut index: Index = serde_json::from_value(value).map_err(|e| format!("index: {e}"))?;
// What no manifest may carry is not offered: the id and version also name files on disk.
let (offered, dropped) = std::mem::take(&mut index.plugins)
.into_iter()
.partition(|e| crate::validate_plugin_id(&e.id).is_ok() && e.semver().is_some());
index.plugins = offered;
return Ok((index, dropped));
}
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
@@ -417,13 +438,14 @@ impl Index {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
let index = Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
};
Ok((index, Vec::new()))
}
/// Written in a format newer than this IDE reads: it lists no plugins.
@@ -908,6 +930,51 @@ mod tests {
}
}
#[test]
fn an_index_offers_only_the_ids_and_versions_a_manifest_may_carry() {
let invalid_ids = [
"../x",
"../../escape",
"acme/../x",
r"acme.\..\x",
"acme./../../../escape",
"/abs",
r"C:\x",
"ACME.x",
"acme.",
"acme..x",
"acme.x y",
"",
];
let invalid_versions =
["../1.0.0", "1.0.0/../../../escape", r"1.0.0\..\x", "1.0.0-..", " 1.0.0", "1.0", "latest", ""];
let valid = [("acme.x", "1.0.0"), ("acme.tools-2_b", "1.0.0-rc.1+b.2")];
let mut listed: Vec<(&str, &str)> = valid.to_vec();
listed.extend(invalid_ids.iter().map(|id| (*id, "1.0.0")));
listed.extend(invalid_versions.iter().map(|version| ("acme.x", *version)));
let plugins: Vec<serde_json::Value> = listed
.iter()
.map(|(id, version)| {
serde_json::json!({
"id": id, "name": "X", "version": version, "publisher": "acme",
"download": "x.oiplugin", "digest": "d", "engine": "^1.0",
})
})
.collect();
let text = serde_json::json!({ "format": INDEX_FORMAT, "plugins": plugins }).to_string();
let pairs = |entries: &[IndexEntry]| -> Vec<(String, String)> {
entries.iter().map(|e| (e.id.clone(), e.version.clone())).collect()
};
let owned = |pairs: &[(&str, &str)]| -> Vec<(String, String)> {
pairs.iter().map(|(id, version)| ((*id).to_owned(), (*version).to_owned())).collect()
};
assert_eq!(pairs(&Index::parse(&text).unwrap().plugins), owned(&valid));
let (index, dropped) = Index::parse_dropping(&text).unwrap();
assert_eq!(pairs(&index.plugins), owned(&valid));
assert_eq!(pairs(&dropped), owned(&listed[valid.len()..]));
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
+10 -2
Переглянути файл
@@ -102,6 +102,14 @@ fn parse<T>(r: serde_json::Result<T>) -> IdeResult<T> {
r.map_err(|e| IdeError::new(ide_core::error::ErrorKind::Protocol, format!("unexpected gh output: {e}")))
}
/// The repository of `root`, free to execute: the host runs the `gh_*` commands only in a
/// trusted project.
fn trusted_repo(root: &Path) -> IdeResult<Repo> {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
repo.set_execution_allowed(true);
Ok(repo)
}
/// Whether gh is installed and signed in, and which GitHub repository `root` maps to.
pub fn availability(root: &Path) -> GhAvailability {
if gh_executable().is_none() {
@@ -233,7 +241,7 @@ pub fn create(
return Err(IdeError::invalid("choose a base branch"));
}
if req.push {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
let repo = trusted_repo(root)?;
repo.push(false, true, cancel, on_output)?;
}
let mut args = vec!["pr", "create", "--title", req.title.trim(), "--body", &req.body, "--base", req.base.trim()];
@@ -354,7 +362,7 @@ pub fn prepare_diff(
cancel: &CancellationToken,
on_output: &dyn Fn(String),
) -> IdeResult<PrDiffRevisions> {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
let repo = trusted_repo(root)?;
if !repo.has_commit(head_oid) || !repo.has_commit(base_oid) {
let remotes = repo.remotes()?;
let remote = remote_for(&remotes, name_with_owner)
+58 -8
Переглянути файл
@@ -22,7 +22,7 @@
mod templates;
use ide_extension::{Manifest, validate_plugin_id};
use ide_host::plugins::{PluginHost, plugin_icon, read_market_url, resolve_market_href};
use ide_host::plugins::{PluginHost, market_href_reachable_from, plugin_icon, read_market_url, resolve_market_href};
use ide_plugin::{Limits, PluginEngine};
use std::path::{Path, PathBuf};
use std::process::{Command, ExitCode};
@@ -591,7 +591,7 @@ fn cmd_index(args: &[String]) -> Result<(), String> {
let base = a.value("download-base").map(|b| b.trim_end_matches('/'));
// Revocations and other publishers of an existing index are kept.
let mut index = match std::fs::read_to_string(&out) {
Ok(text) => current_format(Index::parse(&text)?, &out.to_string_lossy())?,
Ok(text) => parse_index(&text, &out.to_string_lossy())?,
Err(_) => Index { format: package::INDEX_FORMAT, ..Index::default() },
};
let publisher = match (a.value("publisher"), a.value("key")) {
@@ -722,14 +722,22 @@ fn register_key(
Ok(publishers)
}
/// `index`, unless a newer `openidea-plugin` wrote it (rewriting it would drop what this one can't read).
fn current_format(index: Index, file: &str) -> Result<Index, String> {
/// The index in `text` (read from `file`), unless a newer `openidea-plugin` wrote it (rewriting it
/// would drop what this one can't read). Says which entries it does not keep: no IDE offers them.
fn parse_index(text: &str, file: &str) -> Result<Index, String> {
let (index, dropped) = Index::parse_dropping(text)?;
if index.is_newer() {
return Err(format!(
"{file}: index format {} is newer than this tool writes; update openidea-plugin",
index.format
));
}
for e in &dropped {
eprintln!(
"warning: {file}: dropped {:?} {:?}: its id or version is not one a manifest may carry",
e.id, e.version
);
}
Ok(index)
}
@@ -740,14 +748,15 @@ fn cmd_merge(args: &[String]) -> Result<(), String> {
};
let keep = a.value("keep").ok_or("--keep is required")?;
let out = a.value("out").unwrap_or(base);
let mut index =
current_format(Index::parse(&std::fs::read_to_string(base).map_err(|e| format!("{base}: {e}"))?)?, base)?;
let mut index = parse_index(&std::fs::read_to_string(base).map_err(|e| format!("{base}: {e}"))?, base)?;
let bytes = read_market_url(other, 16 << 20)?;
let theirs = Index::parse(&String::from_utf8(bytes).map_err(|_| format!("{other}: not UTF-8"))?)?;
let mut theirs = current_format(theirs, other)?;
let mut theirs = parse_index(&String::from_utf8(bytes).map_err(|_| format!("{other}: not UTF-8"))?, other)?;
// Their downloads are relative to their index, not to ours.
for e in &mut theirs.plugins {
e.download = resolve_market_href(other, &e.download);
if !market_href_reachable_from(other, &e.download) {
return Err(format!("{other}: the download of {} {} is not on the network", e.id, e.version));
}
}
let count = theirs.plugins.len();
index.include(theirs, &[keep]).map_err(|e| format!("{other}: {e}"))?;
@@ -947,6 +956,47 @@ mod tests {
assert!(err.contains("unknown publisher"), "{err}");
}
/// An index read over the network is not merged when it names a download on this machine:
/// the merged index is a local one, which may.
#[test]
fn an_index_read_over_the_network_is_not_merged_with_local_downloads() {
use std::io::{BufRead, BufReader, Write};
let tmp = tempfile::tempdir().unwrap();
let base = tmp.path().join("index.json");
write_index(&Index { format: package::INDEX_FORMAT, ..Index::default() }, &base).unwrap();
let local = tmp.path().join("alice.dawn-1.0.0.oiplugin").to_string_lossy().into_owned();
let theirs = serde_json::json!({
"format": package::INDEX_FORMAT,
"publishers": [{ "id": "alice", "name": "Alice", "keys": [] }],
"plugins": [{
"id": "alice.dawn", "name": "Dawn", "version": "1.0.0", "publisher": "alice",
"download": local, "digest": "d", "engine": "^1.0",
}],
})
.to_string();
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let other = format!("http://127.0.0.1:{}/index.json", listener.local_addr().unwrap().port());
std::thread::spawn(move || {
for mut stream in listener.incoming().flatten() {
let mut reader = BufReader::new(stream.try_clone().unwrap());
let mut line = String::new();
while reader.read_line(&mut line).unwrap_or(0) > 0 && line != "\r\n" {
line.clear();
}
let _ = write!(
stream,
"HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{theirs}",
theirs.len()
);
}
});
let err = cmd_merge(&strings(&[base.to_str().unwrap(), &other, "--keep", "openidea"])).unwrap_err();
assert!(err.contains("is not on the network"), "{err}");
let merged = Index::parse(&std::fs::read_to_string(&base).unwrap()).unwrap();
assert!(merged.plugins.is_empty(), "{:?}", merged.plugins);
}
#[test]
fn the_index_carries_plugin_icons() {
let tmp = tempfile::tempdir().unwrap();
+320 -26
Переглянути файл
@@ -9,6 +9,14 @@ use std::path::{Path, PathBuf};
/// configuration and roaming data keep following the OS.
pub const LOCAL_DATA_ENV: &str = "OPENIDEA_LOCAL_DATA_DIR";
/// A custom local-data root chosen in Settings › System Settings › Caches,
/// stored in the configuration directory. The environment variable still wins.
pub const OVERRIDE_FILE: &str = "local-data-dir.txt";
/// A pending move of the local data to a newly chosen directory, carried out
/// at the next start (before any project opens), like File › Invalidate Caches.
const MIGRATION_FILE: &str = "local-data-migration.json";
/// `$OPENIDEA_LOCAL_DATA_DIR/<identifier>` when set, else the OS local data directory
/// (Tauri's `app_local_data_dir`). The identifier keeps release and per-worktree
/// debug builds apart under one root. A set but unusable path is an error, never a
@@ -17,6 +25,203 @@ pub fn local_data_dir(identifier: &str) -> Result<PathBuf, String> {
resolve_local(std::env::var_os(LOCAL_DATA_ENV), dirs::data_local_dir(), identifier)
}
/// The directory to actually start with: the environment override, else a valid
/// custom directory from Settings, else the OS default. A stored but unusable
/// custom directory is an error, never a silent fallback.
pub fn startup_local_data_dir(identifier: &str) -> Result<PathBuf, String> {
startup_resolve(std::env::var_os(LOCAL_DATA_ENV), stored_override(identifier), dirs::data_local_dir(), identifier)
}
fn startup_resolve(
var: Option<OsString>,
stored: Option<PathBuf>,
os: Option<PathBuf>,
identifier: &str,
) -> Result<PathBuf, String> {
if var.as_ref().is_some_and(|v| !v.is_empty()) {
return resolve_local(var, os, identifier);
}
let Some(root) = stored else { return resolve_local(None, os, identifier) };
if !root.is_absolute() {
return Err(format!("custom local data directory must be an absolute path: {}", root.display()));
}
resolve_root(root, identifier, "custom local data directory")
}
/// The file holding the custom local-data root, if the platform exposes its
/// configuration directory. Both startup and Settings use this one path.
pub fn override_path(identifier: &str) -> Option<PathBuf> {
dirs::config_dir().map(|dir| dir.join(identifier).join(OVERRIDE_FILE))
}
fn migration_path(identifier: &str) -> Option<PathBuf> {
dirs::config_dir().map(|dir| dir.join(identifier).join(MIGRATION_FILE))
}
/// The custom root stored by Settings, if any (validated when it is used).
pub fn stored_override(identifier: &str) -> Option<PathBuf> {
stored_override_in(&override_path(identifier)?)
}
fn stored_override_in(path: &Path) -> Option<PathBuf> {
let text = fs::read_to_string(path).ok()?;
let trimmed = text.trim();
(!trimmed.is_empty()).then(|| PathBuf::from(trimmed))
}
/// The OS default local data directory (what clearing the custom directory
/// returns to), ignoring any stored override. The environment still applies.
pub fn default_local_data_dir(identifier: &str) -> Result<PathBuf, String> {
local_data_dir(identifier)
}
/// Stores (`Some`) or clears (`None`) the custom local-data root, returning the
/// directory that takes effect after a restart. A pending custom migration to
/// anywhere else is dropped: only the caller schedules the matching one.
pub fn set_override(identifier: &str, root: Option<&Path>) -> Result<PathBuf, String> {
let Some(path) = override_path(identifier) else {
return Err("cannot find the configuration directory".into());
};
set_override_in(&path, &migration_path(identifier), root, identifier)
}
fn set_override_in(
path: &Path,
migration: &Option<PathBuf>,
root: Option<&Path>,
identifier: &str,
) -> Result<PathBuf, String> {
if let Some(migration) = migration {
let _ = fs::remove_file(migration);
}
let Some(root) = root else {
if let Err(error) = fs::remove_file(path)
&& error.kind() != io::ErrorKind::NotFound
{
return Err(format!("cannot clear the custom local data directory: {error}"));
}
return local_data_dir(identifier);
};
if !root.is_absolute() {
return Err(format!("custom local data directory must be an absolute path: {}", root.display()));
}
let dir = resolve_root(root.to_path_buf(), identifier, "custom local data directory")?;
if let Some(parent) = path.parent()
&& let Err(error) = fs::create_dir_all(parent)
{
return Err(format!("cannot store the custom local data directory: {error}"));
}
if let Err(error) = fs::write(path, format!("{}\n", root.display())) {
return Err(format!("cannot store the custom local data directory: {error}"));
}
Ok(dir)
}
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
struct CustomMigration {
from: PathBuf,
to: PathBuf,
}
/// Schedules a move of the local data from `from` to `to` at the next start.
/// Returns false when there is nothing to move. The caller restarts the IDE.
pub fn schedule_custom_migration(identifier: &str, from: &Path, to: &Path) -> Result<bool, String> {
let Some(path) = migration_path(identifier) else {
return Err("cannot find the configuration directory".into());
};
schedule_custom_migration_in(&path, from, to)
}
fn schedule_custom_migration_in(path: &Path, from: &Path, to: &Path) -> Result<bool, String> {
if from == to {
return Ok(false);
}
if let Some(parent) = path.parent()
&& let Err(error) = fs::create_dir_all(parent)
{
return Err(format!("cannot schedule the data move: {error}"));
}
let json = serde_json::to_vec(&CustomMigration { from: from.into(), to: to.into() })
.map_err(|error| format!("cannot schedule the data move: {error}"))?;
if let Err(error) = fs::write(path, json) {
return Err(format!("cannot schedule the data move: {error}"));
}
Ok(true)
}
/// Carries out a scheduled custom migration aimed at `local` (call at start-up,
/// before projects open). Entries that appear in the meantime are never
/// overwritten; a partially moved tree is retried at the next start.
pub fn apply_custom_migration(identifier: &str, local: &Path) {
let Some(path) = migration_path(identifier) else { return };
apply_custom_migration_in(&path, local);
}
fn apply_custom_migration_in(path: &Path, local: &Path) {
let Ok(bytes) = fs::read(path) else { return };
let Ok(job) = serde_json::from_slice::<CustomMigration>(&bytes) else {
let _ = fs::remove_file(path);
return;
};
if job.to != local || job.from == local {
let _ = fs::remove_file(path);
return;
}
move_entries(&job.from, local, &custom_entries(&job.from));
let leftover = custom_entries(&job.from).into_iter().any(|entry| job.from.join(entry).exists());
if !leftover {
let _ = fs::remove_file(path);
}
}
/// Entries of a local-data tree that move to a newly chosen directory.
/// Non-relocatable tool installs (virtual environments with absolute interpreter
/// paths) stay behind and are reinstalled on demand, as in `migrate_entries`.
fn custom_entries(root: &Path) -> Vec<PathBuf> {
let mut entries = local_entries(root);
for extra in ["logs", "agents/tools"] {
let entry = PathBuf::from(extra);
if !entries.contains(&entry) {
entries.push(entry);
}
}
entries
}
/// Moves the local-data tree from `from` to `to`, entry by entry. Same-volume
/// entries rename at once; cross-volume copies run in the background. Targets
/// created in the meantime are never merged into or overwritten.
fn move_entries(from: &Path, to: &Path, entries: &[PathBuf]) {
if from == to {
return;
}
let mut copies = Vec::new();
for entry in entries {
let source = from.join(entry);
let target = to.join(entry);
match try_rename(&source, &target) {
Ok(true) => copies.push((source, target)),
Ok(false) => {}
Err(error) => {
tracing::warn!(?source, ?target, %error, "data move failed; keeping old data, using new location")
}
}
}
if !copies.is_empty() {
let result = std::thread::Builder::new().name("openidea-data-migration".into()).spawn(move || {
for (source, target) in copies {
tracing::info!(?source, ?target, "copying data to the new location in background");
if let Err(error) = copy_then_remove(&source, &target) {
tracing::warn!(?source, ?target, %error, "data copy failed; keeping old data");
}
}
});
if let Err(error) = result {
tracing::warn!(%error, "cannot start data migration worker; keeping old data");
}
}
}
fn resolve_local(var: Option<OsString>, os: Option<PathBuf>, identifier: &str) -> Result<PathBuf, String> {
let Some(root) = var.filter(|v| !v.is_empty()).map(PathBuf::from) else {
return os.map(|dir| dir.join(identifier)).ok_or_else(|| "cannot find the local data directory".into());
@@ -25,10 +230,14 @@ fn resolve_local(var: Option<OsString>, os: Option<PathBuf>, identifier: &str) -
if !root.is_absolute() {
return Err(format!("{LOCAL_DATA_ENV} must be an absolute path: {}", root.display()));
}
resolve_root(root, identifier, LOCAL_DATA_ENV)
}
fn resolve_root(root: PathBuf, identifier: &str, what: &str) -> Result<PathBuf, String> {
let dir = root.join(identifier);
fs::create_dir_all(&dir)
.and_then(|()| tempfile::NamedTempFile::new_in(&dir).map(drop))
.map_err(|error| format!("{LOCAL_DATA_ENV}: cannot write to {}: {error}", dir.display()))?;
.map_err(|error| format!("{what}: cannot write to {}: {error}", dir.display()))?;
Ok(dir)
}
@@ -77,6 +286,11 @@ pub fn migrate(data: &Path, local: &Path) {
}
fn migrate_entries(data: &Path, local: &Path) {
move_entries(data, local, &local_entries(data));
}
/// The rebuildable entries of the legacy roaming tree that move to Local.
fn local_entries(data: &Path) -> Vec<PathBuf> {
let mut entries: Vec<PathBuf> = [
"language-server-runtimes",
"index",
@@ -118,31 +332,7 @@ fn migrate_entries(data: &Path, local: &Path) {
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
Err(error) => tracing::warn!(%error, "cannot enumerate legacy tools; keeping old data, using Local"),
}
let mut copies = Vec::new();
for entry in entries {
let source = data.join(&entry);
let target = data_path(data, local, &entry);
match try_rename(&source, &target) {
Ok(true) => copies.push((source, target)),
Ok(false) => {}
Err(error) => {
tracing::warn!(?source, ?target, %error, "data migration failed; keeping old data, using Local")
}
}
}
if !copies.is_empty() {
let result = std::thread::Builder::new().name("openidea-data-migration".into()).spawn(move || {
for (source, target) in copies {
tracing::info!(?source, ?target, "copying legacy data in background");
if let Err(error) = copy_then_remove(&source, &target) {
tracing::warn!(?source, ?target, %error, "data copy failed; keeping old data, using Local");
}
}
});
if let Err(error) = result {
tracing::warn!(%error, "cannot start data migration worker; keeping old data, using Local");
}
}
entries
}
/// Returns true when a cross-volume copy is needed.
@@ -597,4 +787,108 @@ mod tests {
.any(|e| e.file_name().to_string_lossy().starts_with(".migration-"))
);
}
#[test]
fn custom_override_round_trips_and_validates() {
let tmp = tempfile::tempdir().unwrap();
let file = tmp.path().join("local-data-dir.txt");
let migration = tmp.path().join("local-data-migration.json");
let id = "ai.test.openidea";
assert_eq!(stored_override_in(&file), None);
let root = tmp.path().join("custom-root");
let dir = set_override_in(&file, &Some(migration.clone()), Some(&root), id).unwrap();
assert_eq!(dir, root.join(id));
assert!(dir.is_dir());
assert_eq!(stored_override_in(&file), Some(root.clone()));
assert_eq!(startup_resolve(None, stored_override_in(&file), Some(tmp.path().join("os")), id).unwrap(), dir);
// A relative root is rejected before anything is written.
let before = fs::read_to_string(&file).unwrap();
let error = set_override_in(&file, &Some(migration.clone()), Some(Path::new("relative")), id).unwrap_err();
assert!(error.contains("absolute"), "{error}");
assert_eq!(fs::read_to_string(&file).unwrap(), before);
// An unwritable root (an existing file) is rejected too.
let blocker = tmp.path().join("blocker");
fs::write(&blocker, "").unwrap();
let error = set_override_in(&file, &Some(migration.clone()), Some(&blocker), id).unwrap_err();
assert!(error.contains("cannot write"), "{error}");
// Clearing removes the file and returns the default.
let dir = set_override_in(&file, &Some(migration), None, id).unwrap();
assert!(!file.exists());
assert_eq!(dir, local_data_dir(id).unwrap());
assert_eq!(stored_override_in(&file), None);
}
#[test]
fn startup_prefers_the_environment_then_the_override() {
let tmp = tempfile::tempdir().unwrap();
let os = tmp.path().join("os");
let custom = tmp.path().join("custom");
let env = tmp.path().join("env");
let id = "ai.test.openidea";
let error = startup_resolve(None, Some(PathBuf::from("relative")), Some(os.clone()), id).unwrap_err();
assert!(error.contains("absolute"), "{error}");
// The environment wins over a stored override.
let dir = startup_resolve(Some(env.clone().into()), Some(custom.clone()), Some(os.clone()), id).unwrap();
assert_eq!(dir, env.join(id));
// Without the environment the override wins over the OS default.
let dir = startup_resolve(None, Some(custom.clone()), Some(os.clone()), id).unwrap();
assert_eq!(dir, custom.join(id));
// And without either the OS default applies.
let dir = startup_resolve(None, None, Some(os.clone()), id).unwrap();
assert_eq!(dir, os.join(id));
}
#[test]
fn custom_migration_moves_the_tree_and_clears_itself() {
let tmp = tempfile::tempdir().unwrap();
let marker = tmp.path().join("local-data-migration.json");
let from = tmp.path().join("old-local");
let to = tmp.path().join("new-local");
fs::create_dir_all(from.join("index/p1")).unwrap();
fs::write(from.join("index/p1/symbols"), "index").unwrap();
fs::create_dir_all(from.join("cache/p1")).unwrap();
fs::write(from.join("cache/p1/build"), "cache").unwrap();
fs::create_dir_all(from.join("tools/demo/bin")).unwrap();
fs::write(from.join("tools/demo/bin/demo.cmd"), "payload").unwrap();
fs::write(from.join("history"), "roaming stays").unwrap();
// Nothing to move: no marker is written.
assert!(!schedule_custom_migration_in(&marker, &from, &from).unwrap());
assert!(!marker.exists());
assert!(schedule_custom_migration_in(&marker, &from, &to).unwrap());
apply_custom_migration_in(&marker, &to);
assert!(!marker.exists());
assert_eq!(fs::read_to_string(to.join("index/p1/symbols")).unwrap(), "index");
assert_eq!(fs::read_to_string(to.join("cache/p1/build")).unwrap(), "cache");
assert!(to.join("tools/demo/bin/demo.cmd").exists());
assert!(!from.join("index").exists());
// Roaming content is not local data and stays behind.
assert_eq!(fs::read_to_string(from.join("history")).unwrap(), "roaming stays");
// A migration aimed elsewhere is dropped without touching anything.
fs::create_dir_all(from.join("lsp")).unwrap();
assert!(schedule_custom_migration_in(&marker, &from, &tmp.path().join("elsewhere")).unwrap());
apply_custom_migration_in(&marker, &to);
assert!(!marker.exists());
assert!(from.join("lsp").exists());
assert!(!to.join("lsp").exists());
}
#[test]
fn custom_entries_cover_logs_and_agent_tools() {
let tmp = tempfile::tempdir().unwrap();
let entries = custom_entries(tmp.path());
for expected in ["logs", "agents/tools", "index", "cache", "lsp", "agents/registry.json"] {
assert!(entries.contains(&PathBuf::from(expected)), "{entries:?}");
}
}
}
+311 -28
Переглянути файл
@@ -45,26 +45,12 @@ fn innermost_repo<T>(repos: Vec<(PathBuf, T)>, path: &Path) -> Option<(PathBuf,
// Read tools must never invoke repository-configured executables, including in Restricted Mode.
fn read_command(git: &str, root: &Path) -> std::process::Command {
let mut command = std::process::Command::new(git);
ide_vcs::restrict(&mut command);
command
.current_dir(root)
.args([
"-c",
"core.fsmonitor=false",
"-c",
"core.hooksPath=",
"-c",
"diff.external=",
"-c",
"core.attributesFile=",
"-c",
"submodule.recurse=false",
"-c",
"color.ui=false",
])
.env("GIT_NO_LAZY_FETCH", "1")
.args(["-c", "color.ui=false"])
.env("GIT_TERMINAL_PROMPT", "0")
.env("GIT_OPTIONAL_LOCKS", "0")
.env_remove("GIT_EXTERNAL_DIFF")
.env_remove("GIT_DIR")
.env_remove("GIT_WORK_TREE")
.env_remove("GIT_INDEX_FILE");
@@ -72,6 +58,32 @@ fn read_command(git: &str, root: &Path) -> std::process::Command {
command
}
/// `-c` arguments that empty the clean and process filters the repository's own config names:
/// git runs them to compare a changed file with the index. Filters of the user's config stay
/// (Git LFS), as they are not the repository's choice.
fn filter_overrides(git: &str, root: &Path) -> IdeResult<Vec<String>> {
let mut cmd = read_command(git, root);
cmd.args(["config", "--null", "--list", "--show-scope"]);
let out = cmd.output().map_err(|e| IdeError::unavailable(format!("cannot run git: {e}")))?;
if !out.status.success() {
return Err(IdeError::internal(String::from_utf8_lossy(&out.stderr).trim().to_owned()));
}
repository_filter_overrides(&out.stdout)
}
/// [`filter_overrides`] out of `listing`, the output of `git config --null --list --show-scope`.
fn repository_filter_overrides(listing: &[u8]) -> IdeResult<Vec<String>> {
let mut fields = listing.split(|b| *b == 0);
let mut own = Vec::new();
while let (Some(scope), Some(record)) = (fields.next(), fields.next()) {
if matches!(scope, b"local" | b"worktree") {
own.extend_from_slice(record);
own.push(0);
}
}
ide_vcs::filter_overrides_from(&own)
}
/// The branch HEAD is on (unborn too); `None` when HEAD is detached or git fails.
fn read_branch(git: &str, root: &Path) -> Option<String> {
let mut cmd = read_command(git, root);
@@ -167,7 +179,10 @@ fn git_executable(call: &Call) -> String {
/// `git status` of the repository at `root`, paths relative to it.
fn read_status(git: &str, root: &Path, untracked: bool, ignored: bool) -> IdeResult<Vec<StatusEntry>> {
let mut cmd = read_command(git, root);
cmd.args(filter_overrides(git, root)?);
cmd.args(["-c", "core.quotepath=false", "status", "--porcelain", "-z", "--find-renames"]);
// git asks a submodule with changes for its own status, which runs under the submodule's config.
cmd.arg("--ignore-submodules=dirty");
// git refuses `--ignored=matching` with `--untracked-files=no`: untracked files are dropped below.
cmd.arg(if untracked || ignored { "--untracked-files=all" } else { "--untracked-files=no" });
if ignored {
@@ -183,6 +198,39 @@ fn read_status(git: &str, root: &Path, untracked: bool, ignored: bool) -> IdeRes
Ok(entries)
}
/// `git diff` of the repository at `root` (of `file` alone, when given): the index against HEAD
/// when `staged`, else disk against the index.
fn read_diff(
git: &str,
root: &Path,
file: Option<&Path>,
staged: bool,
cancel: &ide_core::CancellationToken,
) -> IdeResult<String> {
let mut cmd = read_command(git, root);
cmd.args(filter_overrides(git, root)?);
cmd.args(["--no-pager", "diff", "--no-ext-diff", "--no-textconv"]);
// git asks a submodule with changes for its own status, and for its own diff when the
// repository's `diff.submodule` says so: both run under the submodule's config.
cmd.args(["--ignore-submodules=dirty", "--submodule=short"]);
if staged {
cmd.arg("--cached");
}
cmd.arg("--");
if let Some(file) = file {
cmd.arg(file.strip_prefix(root).map_err(|e| IdeError::invalid(e.to_string()))?);
}
ide_core::process::prepare(&mut cmd);
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
let out = cmd.output()?;
if !out.status.success() {
return Err(IdeError::unavailable(String::from_utf8_lossy(&out.stderr).trim()));
}
Ok(String::from_utf8_lossy(&out.stdout).into_owned())
}
/// A git repository in the project, for paths outside the repositories IDEA knows: the
/// closest folder with `.git` from `start` up to the project root.
fn find_git_root(root: &Path, start: &Path) -> Option<PathBuf> {
@@ -307,18 +355,7 @@ fn git_diff() -> Tool {
let staged = call.flag("staged", false)?;
let git = git_executable(&call);
let cancel = call.cancel.clone();
let text = call.host.blocking(move || {
let mut cmd = read_command(&git, &root);
cmd.args(["--no-pager", "diff", "--no-ext-diff", "--no-textconv"]);
if staged { cmd.arg("--cached"); }
cmd.arg("--");
if let Some(file) = file { cmd.arg(file.strip_prefix(&root).map_err(|e| IdeError::invalid(e.to_string()))?); }
ide_core::process::prepare(&mut cmd);
if cancel.is_cancelled() { return Err(IdeError::cancelled()); }
let out = cmd.output()?;
if !out.status.success() { return Err(IdeError::unavailable(String::from_utf8_lossy(&out.stderr).trim())); }
Ok(String::from_utf8_lossy(&out.stdout).into_owned())
}).await?;
let text = call.host.blocking(move || read_diff(&git, &root, file.as_deref(), staged, &cancel)).await?;
Ok(Output::text(text))
})
.param(Param::optional("repositoryPathRelativeToProject", ParamType::String, "Select the containing repository"))
@@ -342,6 +379,252 @@ mod tests {
assert!(e[3].conflicted() && !e[3].staged() && !e[3].unstaged());
}
/// Security audit lead 2 (variant): reading status or a diff must not run the repository's filters.
#[test]
fn status_and_diff_do_not_run_repository_filters() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap();
let git = |args: &[&str]| {
let out = std::process::Command::new("git").current_dir(&root).args(args).output().unwrap();
assert!(out.status.success(), "git {args:?}: {}", String::from_utf8_lossy(&out.stderr));
};
git(&["init", "-q"]);
git(&["config", "user.email", "[email protected]"]);
git(&["config", "user.name", "t"]);
git(&["config", "commit.gpgsign", "false"]);
std::fs::write(root.join(".gitattributes"), "*.txt filter=marker\n").unwrap();
std::fs::write(root.join("a.txt"), "one\n").unwrap();
git(&["add", "."]);
git(&["commit", "-qm", "init"]);
git(&["config", "filter.marker.clean", "echo x > executed-marker; cat"]);
git(&["config", "filter.marker.process", "echo x > executed-marker; cat"]);
git(&["config", "filter.marker.required", "true"]);
let marker = root.join("executed-marker");
let mut ran = Vec::new();
// The size it has in the index, so git has to read the file to tell whether it changed.
std::fs::write(root.join("a.txt"), "two\n").unwrap();
let status = read_status("git", &root, false, false);
if marker.exists() {
ran.push("status");
}
let _ = std::fs::remove_file(&marker);
std::fs::write(root.join("a.txt"), "one\ntwo\nthree\n").unwrap();
let diff = read_diff("git", &root, None, false, &ide_core::CancellationToken::new());
if marker.exists() {
ran.push("diff");
}
// Control: the fixture's filter really runs when git is used directly.
let _ = std::fs::remove_file(&marker);
std::fs::write(root.join("a.txt"), "six\n").unwrap();
let _ = std::process::Command::new("git").current_dir(&root).args(["status", "--porcelain"]).output();
assert!(marker.exists(), "control: plain `git status` did not run the filter, fixture is not live");
assert!(ran.is_empty(), "a read tool ran the repository's filter via: {ran:?}");
assert_eq!(status.unwrap().len(), 1);
let diff = diff.unwrap();
assert!(diff.contains("+three"), "{diff}");
}
#[test]
fn only_filters_of_the_repository_s_own_config_are_emptied() {
let listing = b"system\0filter.lfs.clean\ngit-lfs clean -- %f\0\
global\0filter.lfs.process\ngit-lfs filter-process\0\
global\0filter.lfs.required\ntrue\0\
local\0core.bare\nfalse\0\
local\0filter.a.clean\nx\0\
worktree\0filter.b.process\ny\0\
local\0filter.a.required\ntrue\0\
command\0filter.c.clean\nz\0";
assert_eq!(
repository_filter_overrides(listing).unwrap(),
["-c", "filter.a.clean=", "-c", "filter.b.process=", "-c", "filter.a.required=false"]
);
}
fn git(dir: &Path, args: &[&str]) {
let out = std::process::Command::new("git").current_dir(dir).args(args).output().unwrap();
assert!(out.status.success(), "git {args:?}: {}", String::from_utf8_lossy(&out.stderr));
}
/// A repository at `dir` with `a.txt` committed.
fn repository(dir: &Path, attributes: &[u8]) {
std::fs::create_dir_all(dir).unwrap();
git(dir, &["init", "-q"]);
git(dir, &["config", "user.email", "[email protected]"]);
git(dir, &["config", "user.name", "t"]);
git(dir, &["config", "commit.gpgsign", "false"]);
std::fs::write(dir.join(".gitattributes"), attributes).unwrap();
std::fs::write(dir.join("a.txt"), "one\n").unwrap();
git(dir, &["add", "."]);
git(dir, &["commit", "-qm", "init"]);
}
/// A shell command that leaves `marker` behind and passes its input on.
fn leaves(marker: &Path) -> String {
format!("echo x > '{}'; cat", marker.to_string_lossy().replace('\\', "/"))
}
/// Whether a program of the fixture left `marker` behind; removes it.
fn left(marker: &Path) -> bool {
let was = marker.exists();
let _ = std::fs::remove_file(marker);
was
}
/// Reads the status and then the diff of `root` after rewriting `file`, then has plain git
/// read the status as a control. Each write keeps the size the file has in the index, so git
/// has to read it to tell whether it changed.
fn read_both(root: &Path, file: &Path, marker: &Path) -> (IdeResult<Vec<StatusEntry>>, IdeResult<String>) {
let mut ran = Vec::new();
std::fs::write(file, "two\n").unwrap();
let status = read_status("git", root, false, false);
if left(marker) {
ran.push("status");
}
std::fs::write(file, "six\n").unwrap();
let diff = read_diff("git", root, None, false, &ide_core::CancellationToken::new());
if left(marker) {
ran.push("diff");
}
std::fs::write(file, "ten\n").unwrap();
let _ = std::process::Command::new("git").current_dir(root).args(["status", "--porcelain"]).output();
assert!(left(marker), "control: plain `git status` ran no program of the fixture, it is not live");
assert!(ran.is_empty(), "a read tool ran a program of the repository via: {ran:?}");
(status, diff)
}
#[test]
fn status_and_diff_do_not_run_filters_of_any_repository_config() {
type Configure = fn(&Path, &str);
let cases: [(&str, Configure); 4] = [
("clean", |root, command| git(root, &["config", "filter.marker.clean", command])),
("process", |root, command| git(root, &["config", "filter.marker.process", command])),
("included file", |root, command| {
let included = format!("[filter \"marker\"]\n\tclean = \"{command}\"\n");
std::fs::write(root.join("extra.gitconfig"), included).unwrap();
git(root, &["config", "include.path", "../extra.gitconfig"]);
}),
("worktree config", |root, command| {
git(root, &["config", "extensions.worktreeConfig", "true"]);
git(root, &["config", "--worktree", "filter.marker.clean", command]);
}),
];
for (case, configure) in cases {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap().join("repo");
let marker = root.with_file_name("marker");
repository(&root, b"*.txt filter=marker\n");
configure(&root, &leaves(&marker));
let (status, diff) = read_both(&root, &root.join("a.txt"), &marker);
assert_eq!(status.unwrap().len(), 1, "{case}");
let diff = diff.unwrap();
assert!(diff.contains("+six"), "{case}: {diff}");
}
}
/// git splits `-c name=value` at the first `=` and takes no name that is not UTF-8 on every
/// platform, so such a filter cannot be emptied: the tools read nothing instead.
#[test]
fn status_and_diff_refuse_a_repository_filter_git_cannot_be_told_to_skip() {
for name in [&b"x=y"[..], &[0xE9]] {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap().join("repo");
let marker = root.with_file_name("marker");
repository(&root, &[b"*.txt filter=", name, b"\n"].concat());
let section =
[b"[filter \"", name, format!("\"]\n\tclean = \"{}\"\n", leaves(&marker)).as_bytes()].concat();
let config = root.join(".git").join("config");
std::fs::write(&config, [std::fs::read(&config).unwrap(), section].concat()).unwrap();
let (status, diff) = read_both(&root, &root.join("a.txt"), &marker);
assert!(status.is_err() && diff.is_err(), "{name:?}");
}
}
/// git asks a submodule for its own status to tell whether it is dirty, under the
/// submodule's config.
#[test]
fn status_and_diff_do_not_run_filters_of_a_submodule() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap().join("repo");
let (sub, marker) = (root.join("sub"), root.with_file_name("marker"));
repository(&root, b"");
repository(&sub, b"*.txt filter=marker\n");
git(&root, &["add", "sub"]);
git(&root, &["commit", "-qm", "submodule"]);
git(&sub, &["config", "filter.marker.clean", &leaves(&marker)]);
git(&root, &["config", "diff.ignoreSubmodules", "none"]);
git(&root, &["config", "submodule.sub.ignore", "none"]);
let (status, diff) = read_both(&root, &sub.join("a.txt"), &marker);
status.unwrap();
diff.unwrap();
}
/// `git diff` writes the index back when it finds a file whose stat changed and whose content
/// did not, and git runs the hooks a config names for `post-index-change` on that write.
#[test]
fn status_and_diff_do_not_run_hooks_of_the_repository_s_config() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap().join("repo");
let marker = root.with_file_name("marker");
repository(&root, b"");
git(&root, &["config", "hook.marker.event", "post-index-change"]);
let command = format!("echo x > '{}'", marker.to_string_lossy().replace('\\', "/"));
git(&root, &["config", "hook.marker.command", &command]);
let listed = std::process::Command::new("git")
.current_dir(&root)
.args(["hook", "list", "post-index-change"])
.output()
.unwrap();
if !String::from_utf8_lossy(&listed.stdout).contains("marker") {
return; // This git has no hooks named by a config.
}
// The content it has in the index, with a time of its own that is not the index's.
let touch = |seconds: u64| {
let file = std::fs::File::options().write(true).open(root.join("a.txt")).unwrap();
file.set_modified(std::time::SystemTime::now() - std::time::Duration::from_secs(seconds)).unwrap();
};
let mut ran = Vec::new();
touch(60);
let status = read_status("git", &root, false, false);
if left(&marker) {
ran.push("status");
}
touch(120);
let diff = read_diff("git", &root, None, false, &ide_core::CancellationToken::new());
if left(&marker) {
ran.push("diff");
}
touch(180);
let _ = std::process::Command::new("git").current_dir(&root).args(["--no-pager", "diff"]).output();
assert!(left(&marker), "control: plain `git diff` did not run the hook, fixture is not live");
assert!(ran.is_empty(), "a read tool ran a hook of the repository's config via: {ran:?}");
assert!(status.unwrap().is_empty());
assert_eq!(diff.unwrap(), "");
}
/// `diff.submodule=diff` has git run `git diff` in the submodule, under the submodule's config.
#[test]
fn diff_does_not_run_a_diff_driver_of_a_submodule() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap().join("repo");
let (sub, marker) = (root.join("sub"), root.with_file_name("marker"));
repository(&root, b"");
repository(&sub, b"*.txt diff=marker\n");
git(&root, &["add", "sub"]);
git(&root, &["commit", "-qm", "submodule"]);
std::fs::write(sub.join("a.txt"), "one\ntwo\n").unwrap();
git(&sub, &["commit", "-qam", "second"]);
git(&sub, &["config", "diff.marker.command", &leaves(&marker)]);
git(&root, &["config", "diff.submodule", "diff"]);
let diff = read_diff("git", &root, None, false, &ide_core::CancellationToken::new());
let ran = left(&marker);
let _ = std::process::Command::new("git").current_dir(&root).args(["--no-pager", "diff"]).output();
assert!(left(&marker), "control: plain `git diff` did not run the diff driver, fixture is not live");
assert!(!ran, "git_diff ran the submodule's diff driver");
let diff = diff.unwrap();
assert!(diff.contains("Subproject commit"), "{diff}");
assert_eq!(read_status("git", &root, false, false).unwrap().len(), 1);
}
#[cfg(unix)]
#[test]
fn paths_through_a_symlinked_root_compare_canonically() {
+36 -2
Переглянути файл
@@ -5,8 +5,9 @@ use crate::Host;
use crate::rpc::Dispatcher;
use ide_agent::registry::{AgentRegistry, RegistryPaths};
use ide_agent::{
AgentChange, AgentCommand, AgentEntry, AgentEvent, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask,
ExternalSession, McpScope, McpServer, McpServerInput, NewAgentTask,
AcpImportOutcome, AgentAcceptOutcome, AgentChange, AgentCommand, AgentEntry, AgentEvent, AgentProbe, AgentPrompt,
AgentProvider, AgentQueued, AgentTask, CustomAgentInput, ExternalSession, McpScope, McpServer, McpServerInput,
NewAgentTask,
};
use ide_core::error::ErrorKind;
use ide_core::{IdeError, IdeEvent, IdeResult, IpcPath, RequestId, SharedSink, TaskId};
@@ -44,6 +45,24 @@ pub fn set_env(
host.project(window)?.agents.set_builtin_env(&id, env)
}
/// Saves a new custom ACP agent through its agent file.
pub fn save_custom_agent(host: &Host, window: &str, input: CustomAgentInput) -> IdeResult<()> {
host.require_trust(window)?;
host.project(window)?.agents.save_custom_agent(input)
}
/// Removes a custom agent's file (registry installs go through `registry_remove`).
pub fn remove_custom_agent(host: &Host, window: &str, id: String) -> IdeResult<()> {
host.require_trust(window)?;
host.project(window)?.agents.remove_custom_agent(&id)
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom agents.
pub fn import_acp_agents(host: &Host, window: &str, text: String) -> IdeResult<Vec<AcpImportOutcome>> {
host.require_trust(window)?;
host.project(window)?.agents.import_acp_agents(&text)
}
/// Where ACP Registry agents are saved (`<config>/agents`, like the user's own) and installed.
pub(crate) fn registry_paths(host: &Host) -> RegistryPaths {
RegistryPaths::new(&host.config_dir, &host.local_data_dir)
@@ -297,6 +316,18 @@ pub async fn rollback_changes(host: &Host, window: &str, task: String, paths: Op
host.project(window)?.agents.rollback_changes(&task, paths).await
}
/// Accept: copies a worktree task's changed files among `paths` (all without) into the
/// local checkout, uncommitted; files the checkout changed itself are left alone and reported.
pub async fn accept_changes(
host: &Host,
window: &str,
task: String,
paths: Option<Vec<String>>,
) -> IdeResult<AgentAcceptOutcome> {
host.require_trust(window)?;
host.project(window)?.agents.accept_changes(&task, paths).await
}
type McpServerMap = std::collections::BTreeMap<String, ide_core::settings::AgentMcpServer>;
fn mcp_maps(host: &Host, window: &str) -> (McpServerMap, McpServerMap, Option<std::sync::Arc<crate::Project>>) {
@@ -449,6 +480,9 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("agent_rollback_changes", |c| async move {
rollback_changes(&c.host, &c.window, c.arg("task")?, c.arg("paths")?).await
});
d.add("agent_accept_changes", |c| async move {
accept_changes(&c.host, &c.window, c.arg("task")?, c.arg("paths")?).await
});
}
fn validate_project_root(root: &std::path::Path, requested: Option<&std::path::Path>) -> IdeResult<std::path::PathBuf> {
+5 -3
Переглянути файл
@@ -17,9 +17,11 @@ pub async fn fs_read_range(_host: &Host, _window: &str, path: PathBuf, offset: u
blocking(move || ide_vfs::read_range(&path, offset, len)).await
}
/// An image file as a `data:` URL.
pub async fn fs_read_image(_host: &Host, _window: &str, path: PathBuf) -> IdeResult<String> {
blocking(move || ide_vfs::read_data_url(&path)).await
/// An image file as a `data:` URL. A network path is read only from a share an open project is
/// on: the path may come from Markdown, and opening a share signs in to its server.
pub async fn fs_read_image(host: &Host, _window: &str, path: PathBuf) -> IdeResult<String> {
let opened: Vec<PathBuf> = host.projects().iter().map(|project| project.root.clone()).collect();
blocking(move || ide_vfs::read_data_url(&path, &opened)).await
}
/// Commands of this module the remote server answers.
+4
Переглянути файл
@@ -1,3 +1,6 @@
mod kotlin;
#[cfg(test)]
mod kotlin_tests;
mod output;
use crate::Host;
@@ -41,6 +44,7 @@ pub async fn debug_start(host: &Host, window: &str, config: LaunchConfig) -> Ide
})
.await?;
}
config = kotlin::choose_kotlin_debugger(host, window, config).await?;
if config.adapter == ide_dap::kotlin::ADAPTER {
let data_dir = host.local_data_dir.clone();
let runtime_root = config.cwd.clone().map(PathBuf::from).unwrap_or_default();
+415
Переглянути файл
@@ -0,0 +1,415 @@
//! Kotlin debugging: kotlin-lsp's debugger when the project's kotlin-lsp can serve it, else
//! kotlin-debug-adapter with a notice saying why.
use crate::Host;
use ide_core::event::NotifyLevel;
use ide_core::task::CancelHandle;
use ide_core::{CancellationToken, IdeEvent, IdeResult, SharedSink, TaskId};
use ide_dap::LaunchConfig;
use ide_lsp::{KotlinDebugServer, KotlinDebugUnavailable};
use parking_lot::Mutex;
use std::collections::HashSet;
use std::future::Future;
use std::path::PathBuf;
use std::time::Duration;
pub(super) const KOTLIN_IMPORT_WAIT: Duration = Duration::from_secs(120);
/// A ready kotlin-lsp answers well within this; only a longer wait shows a task.
const TASK_DELAY: Duration = Duration::from_millis(500);
// Catalogued as written (`{detail}` replaced at runtime): the text catalog test counts these
// literals as the use of its entries.
const TASK_TITLE: &str = "Waiting for kotlin-lsp to import the project";
const NOTICE_TITLE: &str = "Debugging Kotlin with kotlin-debug-adapter";
const NOT_CONFIGURED: &str = "kotlin-lsp is not the Kotlin language server of this project.";
const NOT_INSTALLED: &str = "kotlin-lsp is not installed.";
const NOT_STARTED: &str = "kotlin-lsp did not start: {detail}";
const STILL_IMPORTING: &str =
"kotlin-lsp is still importing the project. Debug again once the import has finished to use its debugger.";
const IMPORT_FAILED: &str = "kotlin-lsp could not import the project.";
const FAILED: &str = "kotlin-lsp could not start its debugger: {detail}";
/// `config` for kotlin-lsp's debug server on `port`; the rest of the launch stays as it is.
pub(super) fn use_kotlin_lsp(mut config: LaunchConfig, port: u16) -> IdeResult<LaunchConfig> {
let mut extra = extra_object(&config)?;
extra.insert("debugServer".into(), port.into());
config.adapter = ide_dap::kotlin_lsp::ADAPTER.into();
config.extra_json = Some(serde_json::Value::Object(extra).to_string());
Ok(config)
}
/// The dedupe key of `reason` and the notice body (English, translated by the UI's catalog).
pub(super) fn fallback_notice(reason: &KotlinDebugUnavailable) -> (&'static str, String) {
match reason {
KotlinDebugUnavailable::NotConfigured => ("notConfigured", NOT_CONFIGURED.into()),
KotlinDebugUnavailable::NotInstalled => ("notInstalled", NOT_INSTALLED.into()),
KotlinDebugUnavailable::NotStarted(detail) => ("notStarted", NOT_STARTED.replace("{detail}", detail)),
KotlinDebugUnavailable::StillImporting => ("stillImporting", STILL_IMPORTING.into()),
KotlinDebugUnavailable::ImportFailed => ("importFailed", IMPORT_FAILED.into()),
KotlinDebugUnavailable::Failed(detail) => ("failed", FAILED.replace("{detail}", detail)),
}
}
/// A kotlin-debug-adapter launch becomes a kotlin-lsp one when the project's kotlin-lsp has
/// imported the module and started its debug server; otherwise it stays on
/// kotlin-debug-adapter and the window is told why (once per reason).
pub(super) async fn choose_kotlin_debugger(host: &Host, window: &str, config: LaunchConfig) -> IdeResult<LaunchConfig> {
if config.adapter == ide_dap::kotlin_lsp::ADAPTER {
require_debug_server(&config)?;
return Ok(config);
}
if config.adapter != ide_dap::kotlin::ADAPTER {
return Ok(config);
}
let Ok(project) = host.project(window) else {
tracing::info!(
"debugging Kotlin with kotlin-debug-adapter: {}",
fallback_notice(&KotlinDebugUnavailable::NotConfigured).1
);
return Ok(config);
};
let path = debug_path(&config).unwrap_or_else(|| project.root.clone());
let lsp = project.lsp.clone();
let chosen = wait_for_debug_server(&project.sink, |cancel| async move {
lsp.kotlin_debug_server(&path, KOTLIN_IMPORT_WAIT, &cancel).await
})
.await;
match chosen {
Ok(server) => use_kotlin_lsp(config, server.port),
Err(reason) => {
report_fallback(&project.kotlin_debug_notices, &project.sink, &reason);
Ok(config)
}
}
}
/// kotlin-lsp is only reached through the port its language server opened; without one
/// ide-dap would try to spawn a `kotlin-lsp` adapter that does not exist.
fn require_debug_server(config: &LaunchConfig) -> IdeResult<()> {
match extra_object(config)?.get("debugServer") {
None | Some(serde_json::Value::Null) => Err(ide_core::IdeError::invalid(
"A kotlin-lsp debug configuration needs \"debugServer\": the port of kotlin-lsp's debug server. \
Use kotlin-debug-adapter to let the IDE start it.",
)),
Some(_) => Ok(()),
}
}
fn extra_object(config: &LaunchConfig) -> IdeResult<serde_json::Map<String, serde_json::Value>> {
match config.extra_json.as_deref().map(serde_json::from_str).transpose() {
Ok(None) => Ok(serde_json::Map::new()),
Ok(Some(serde_json::Value::Object(extra))) => Ok(extra),
Ok(Some(other)) => Err(ide_core::IdeError::invalid(format!("launch arguments are not an object: {other}"))),
Err(e) => Err(ide_core::IdeError::invalid(e.to_string())),
}
}
/// The module to debug: the launch's `projectRoot`, else its working directory.
fn debug_path(config: &LaunchConfig) -> Option<PathBuf> {
let extra: Option<serde_json::Value> = config.extra_json.as_deref().and_then(|e| serde_json::from_str(e).ok());
extra
.as_ref()
.and_then(|e| e.get("projectRoot")?.as_str())
.filter(|root| !root.is_empty())
.map(PathBuf::from)
.or_else(|| config.cwd.as_deref().map(PathBuf::from))
}
/// Runs `debug_server` with a token the user can cancel; a wait longer than [`TASK_DELAY`]
/// shows as a cancellable task, which ends however the wait does (also when dropped).
async fn wait_for_debug_server<F, Fut>(
sink: &SharedSink,
debug_server: F,
) -> Result<KotlinDebugServer, KotlinDebugUnavailable>
where
F: FnOnce(CancellationToken) -> Fut,
Fut: Future<Output = Result<KotlinDebugServer, KotlinDebugUnavailable>>,
{
let cancel = CancellationToken::new();
let id = TaskId::next();
let handle = ide_core::task::on_cancel(id, {
let cancel = cancel.clone();
move || cancel.cancel()
});
let wait = debug_server(cancel);
tokio::pin!(wait);
if let Ok(answer) = tokio::time::timeout(TASK_DELAY, &mut wait).await {
return answer;
}
let _task = WaitTask::start(sink.clone(), id, handle);
wait.await
}
/// The visible import wait; dropping it ends the task.
struct WaitTask {
sink: SharedSink,
id: TaskId,
_cancel: CancelHandle,
}
impl WaitTask {
fn start(sink: SharedSink, id: TaskId, cancel: CancelHandle) -> Self {
let task = Self { sink, id, _cancel: cancel };
task.emit(false);
task
}
fn emit(&self, done: bool) {
self.sink.emit(IdeEvent::Task {
id: self.id,
title: TASK_TITLE.into(),
message: None,
fraction: None,
done,
cancellable: !done,
});
}
}
impl Drop for WaitTask {
fn drop(&mut self) {
self.emit(true);
}
}
/// Logs every fallback; notifies the window once per reason.
fn report_fallback(notices: &Mutex<HashSet<&'static str>>, sink: &SharedSink, reason: &KotlinDebugUnavailable) {
let (key, body) = fallback_notice(reason);
tracing::info!("debugging Kotlin with kotlin-debug-adapter: {body}");
if notices.lock().insert(key) {
sink.emit(IdeEvent::Notify { level: NotifyLevel::Info, title: NOTICE_TITLE.into(), body: Some(body) });
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
use std::sync::Arc;
fn launch(adapter: &str, extra: serde_json::Value) -> LaunchConfig {
LaunchConfig {
name: "AppKt".into(),
adapter: adapter.into(),
request: "launch".into(),
program: None,
args: vec![],
cwd: Some("/cwd".into()),
env: Default::default(),
extra_json: Some(extra.to_string()),
}
}
fn extra(config: &LaunchConfig) -> serde_json::Value {
serde_json::from_str(config.extra_json.as_deref().unwrap()).unwrap()
}
fn reasons() -> Vec<KotlinDebugUnavailable> {
vec![
KotlinDebugUnavailable::NotConfigured,
KotlinDebugUnavailable::NotInstalled,
KotlinDebugUnavailable::NotStarted("needs JDK 21".into()),
KotlinDebugUnavailable::StillImporting,
KotlinDebugUnavailable::ImportFailed,
KotlinDebugUnavailable::Failed("boom".into()),
]
}
fn recording_sink() -> (SharedSink, Arc<Mutex<Vec<IdeEvent>>>) {
let events = Arc::new(Mutex::new(Vec::new()));
let recorded = events.clone();
(Arc::new(move |event| recorded.lock().push(event)), events)
}
/// `(done, cancellable)` of each task event, with the task id.
fn tasks(events: &Mutex<Vec<IdeEvent>>) -> Vec<(ide_core::TaskId, String, bool, bool)> {
events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Task { id, title, done, cancellable, .. } => Some((*id, title.clone(), *done, *cancellable)),
_ => None,
})
.collect()
}
fn server() -> KotlinDebugServer {
KotlinDebugServer { port: 4711, root: PathBuf::from("/p") }
}
#[test]
fn kotlin_lsp_takes_the_debug_server_port() {
let config = launch(ide_dap::kotlin::ADAPTER, serde_json::json!({"mainClass": "a.AppKt", "projectRoot": "/p"}));
let c = use_kotlin_lsp(config, 4711).unwrap();
assert_eq!(c.adapter, ide_dap::kotlin_lsp::ADAPTER);
let args = extra(&c);
assert_eq!(args["debugServer"], 4711);
assert_eq!(args["mainClass"], "a.AppKt");
assert_eq!(args["projectRoot"], "/p");
assert!(args.get("__adapterJavaHome").is_none());
assert_eq!(
(c.name.as_str(), c.request.as_str(), c.cwd.as_deref()),
("AppKt", "launch", Some(Path::new("/cwd")))
);
let mut bare = launch(ide_dap::kotlin::ADAPTER, serde_json::json!({}));
bare.extra_json = None;
assert_eq!(extra(&use_kotlin_lsp(bare, 4711).unwrap())["debugServer"], 4711);
}
#[test]
fn every_fallback_reason_has_a_catalogued_text() {
let catalogs: [(&str, serde_json::Value); 2] = [
("en", serde_json::from_str(include_str!("../../../../../src/platform/locales/text/en.json")).unwrap()),
("zh", serde_json::from_str(include_str!("../../../../../src/platform/locales/text/zh.json")).unwrap()),
];
let templates = [NOT_CONFIGURED, NOT_INSTALLED, NOT_STARTED, STILL_IMPORTING, IMPORT_FAILED, FAILED];
for reason in reasons() {
let (_, body) = fallback_notice(&reason);
let detail = match &reason {
KotlinDebugUnavailable::NotStarted(d) | KotlinDebugUnavailable::Failed(d) => d.as_str(),
_ => "",
};
assert!(
templates.iter().any(|t| t.replace("{detail}", detail) == body),
"{reason:?}: {body:?} is not built from a catalogued template"
);
}
for text in templates.iter().chain([&NOTICE_TITLE, &TASK_TITLE]) {
for (locale, catalog) in &catalogs {
let translated = catalog[*text].as_str().unwrap_or_else(|| panic!("{locale}.json lacks {text:?}"));
if *locale == "zh" {
assert_ne!(translated, *text, "zh.json leaves {text:?} untranslated");
}
}
}
}
#[test]
fn fallback_reasons_have_distinct_keys() {
let keys: HashSet<_> = reasons().iter().map(|r| fallback_notice(r).0).collect();
assert_eq!(keys.len(), reasons().len());
// The detail does not make another reason.
assert_eq!(
fallback_notice(&KotlinDebugUnavailable::Failed("a".into())).0,
fallback_notice(&KotlinDebugUnavailable::Failed("b".into())).0
);
}
#[test]
fn a_kotlin_lsp_config_without_a_debug_server_is_refused() {
let refused = require_debug_server(&launch(ide_dap::kotlin_lsp::ADAPTER, serde_json::json!({}))).unwrap_err();
assert_eq!(refused.kind, ide_core::error::ErrorKind::InvalidInput);
assert!(refused.message.contains("debugServer"), "{}", refused.message);
let mut bare = launch(ide_dap::kotlin_lsp::ADAPTER, serde_json::json!({}));
bare.extra_json = None;
assert!(require_debug_server(&bare).is_err());
assert!(
require_debug_server(&launch(ide_dap::kotlin_lsp::ADAPTER, serde_json::json!({"debugServer": null})))
.is_err()
);
for port in [serde_json::json!(4711), serde_json::json!("10.0.0.5:4711")] {
require_debug_server(&launch(ide_dap::kotlin_lsp::ADAPTER, serde_json::json!({"debugServer": port})))
.unwrap();
}
}
#[test]
fn the_module_is_the_project_root_else_the_working_directory() {
let with_root = launch(ide_dap::kotlin::ADAPTER, serde_json::json!({"projectRoot": "/p/app"}));
assert_eq!(debug_path(&with_root), Some(PathBuf::from("/p/app")));
assert_eq!(debug_path(&launch(ide_dap::kotlin::ADAPTER, serde_json::json!({}))), Some(PathBuf::from("/cwd")));
let mut neither = launch(ide_dap::kotlin::ADAPTER, serde_json::json!({}));
neither.cwd = None;
assert_eq!(debug_path(&neither), None);
}
#[tokio::test(start_paused = true)]
async fn a_quick_answer_shows_no_task() {
let (sink, events) = recording_sink();
assert_eq!(wait_for_debug_server(&sink, |_| async { Ok(server()) }).await, Ok(server()));
assert!(events.lock().is_empty());
}
#[tokio::test(start_paused = true)]
async fn a_longer_wait_shows_a_task_that_ends_on_every_outcome() {
for outcome in [Ok(server()), Err(KotlinDebugUnavailable::ImportFailed)] {
let (sink, events) = recording_sink();
let answer = outcome.clone();
let result = wait_for_debug_server(&sink, |_| async move {
tokio::time::sleep(Duration::from_secs(5)).await;
answer
})
.await;
assert_eq!(result, outcome);
let tasks = tasks(&events);
assert_eq!(tasks.len(), 2, "{tasks:?}");
assert_eq!(tasks[0].0, tasks[1].0);
assert_eq!(tasks[0].1, TASK_TITLE);
assert_eq!((tasks[0].2, tasks[0].3), (false, true));
assert_eq!((tasks[1].2, tasks[1].3), (true, false));
}
}
#[tokio::test(start_paused = true)]
async fn cancelling_the_task_stops_the_wait() {
let (sink, events) = recording_sink();
let wait = tokio::spawn(async move {
wait_for_debug_server(&sink, |cancel| async move {
cancel.cancelled().await;
Err(KotlinDebugUnavailable::StillImporting)
})
.await
});
tokio::time::sleep(Duration::from_secs(1)).await;
let id = tasks(&events)[0].0;
assert!(ide_core::task::cancel(id));
assert_eq!(wait.await.unwrap(), Err(KotlinDebugUnavailable::StillImporting));
assert_eq!(tasks(&events).last().map(|t| t.2), Some(true));
assert!(!ide_core::task::cancel(id), "an ended wait is no longer cancellable");
}
#[tokio::test(start_paused = true)]
async fn a_dropped_wait_ends_its_task() {
let (sink, events) = recording_sink();
let wait = tokio::spawn(async move {
wait_for_debug_server(&sink, |_| std::future::pending::<Result<KotlinDebugServer, _>>()).await
});
tokio::time::sleep(Duration::from_secs(1)).await;
wait.abort();
let _ = wait.await;
assert_eq!(tasks(&events).iter().map(|t| t.2).collect::<Vec<_>>(), [false, true]);
}
#[test]
fn the_notice_shows_once_per_reason() {
let (sink, events) = recording_sink();
let notices = Mutex::new(HashSet::new());
for reason in [
KotlinDebugUnavailable::StillImporting,
KotlinDebugUnavailable::StillImporting,
KotlinDebugUnavailable::NotStarted("needs JDK 21".into()),
KotlinDebugUnavailable::NotStarted("crashed".into()),
] {
report_fallback(&notices, &sink, &reason);
}
let notes: Vec<_> = events
.lock()
.iter()
.map(|e| match e {
IdeEvent::Notify { level, title, body } => (*level, title.clone(), body.clone()),
other => panic!("{other:?}"),
})
.collect();
assert_eq!(
notes,
[
(NotifyLevel::Info, NOTICE_TITLE.to_string(), Some(STILL_IMPORTING.to_string())),
(
NotifyLevel::Info,
NOTICE_TITLE.to_string(),
Some("kotlin-lsp did not start: needs JDK 21".to_string())
),
]
);
}
}
+123
Переглянути файл
@@ -0,0 +1,123 @@
//! Kotlin's debugger choice through a real `Host` and project: the fallback notice and the
//! refusal of a kotlin-lsp configuration without its debug server.
use super::*;
use crate::HostConfig;
use ide_core::IdeEvent;
use ide_core::event::NotifyLevel;
use ide_lsp::KotlinDebugUnavailable;
use ide_workspace::SettingsScope;
use parking_lot::Mutex;
use std::sync::Arc;
const NOTICE_TITLE: &str = "Debugging Kotlin with kotlin-debug-adapter";
struct Fixture {
host: Host,
root: PathBuf,
events: Arc<Mutex<Vec<IdeEvent>>>,
_temp: tempfile::TempDir,
}
impl Fixture {
/// A trusted Gradle project opened with these global settings; kotlin-debug-adapter is a
/// missing executable, so a session fails right after the debugger is chosen.
async fn new(settings: &[(&str, serde_json::Value)]) -> Self {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let root = base.join("project");
std::fs::create_dir_all(&root).unwrap();
std::fs::write(root.join("settings.gradle.kts"), "rootProject.name = \"app\"\n").unwrap();
let events = Arc::new(Mutex::new(Vec::new()));
let captured = events.clone();
let host = Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(move |_| {
let events = captured.clone();
Arc::new(move |event| events.lock().push(event))
}),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
let missing = base.join("missing").join("kotlin-debug-adapter");
host.global_settings
.set(SettingsScope::Global, "tools", serde_json::json!({"kotlin-debug-adapter": {"path": missing}}))
.unwrap();
for (key, value) in settings {
host.global_settings.set(SettingsScope::Global, key, value.clone()).unwrap();
}
host.open_project("main", &root).unwrap();
host.set_project_trust(&root, true, false).await.unwrap();
Self { host, root, events, _temp: temp }
}
fn launch(&self, adapter: &str, extra: serde_json::Value) -> LaunchConfig {
LaunchConfig {
name: "AppKt".into(),
adapter: adapter.into(),
request: "launch".into(),
program: None,
args: vec![],
cwd: Some(self.root.clone().into()),
env: Default::default(),
extra_json: Some(extra.to_string()),
}
}
fn kotlin_launch(&self) -> LaunchConfig {
let extra = serde_json::json!({"mainClass": "com.example.AppKt", "projectRoot": self.root});
self.launch(ide_dap::kotlin::ADAPTER, extra)
}
/// `(level, body)` of each fallback notice.
fn notices(&self) -> Vec<(NotifyLevel, Option<String>)> {
self.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Notify { level, title, body } if title == NOTICE_TITLE => Some((*level, body.clone())),
_ => None,
})
.collect()
}
}
/// Two Debugs, each failing only at kotlin-debug-adapter's missing executable.
async fn debug_twice(f: &Fixture) {
for _ in 0..2 {
let error = debug_start(&f.host, "main", f.kotlin_launch()).await.unwrap_err();
assert!(!error.message.contains("Restricted Mode"), "{}", error.message);
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn kotlin_lsp_turned_off_gives_one_notice_over_two_debugs() {
let f = Fixture::new(&[("languageServers", serde_json::json!({"kotlin-lsp": {"enabled": false}}))]).await;
debug_twice(&f).await;
let (_, body) = kotlin::fallback_notice(&KotlinDebugUnavailable::NotConfigured);
assert_eq!(f.notices(), [(NotifyLevel::Info, Some(body))]);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn a_missing_kotlin_lsp_gives_one_notice_over_two_debugs() {
let missing = serde_json::json!({"kotlin-lsp": {"command": "/nonexistent/kotlin-lsp"}});
let f = Fixture::new(&[("languageServers", missing)]).await;
debug_twice(&f).await;
let (_, body) = kotlin::fallback_notice(&KotlinDebugUnavailable::NotInstalled);
assert_eq!(f.notices(), [(NotifyLevel::Info, Some(body))]);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn a_kotlin_lsp_config_without_its_debug_server_is_refused() {
let f = Fixture::new(&[]).await;
let config = f.launch(ide_dap::kotlin_lsp::ADAPTER, serde_json::json!({"mainClass": "com.example.AppKt"}));
let refused = debug_start(&f.host, "main", config).await.unwrap_err();
assert_eq!(refused.kind, ide_core::error::ErrorKind::InvalidInput);
assert!(refused.message.contains("debugServer"), "{}", refused.message);
assert!(f.notices().is_empty());
}
+30
Переглянути файл
@@ -169,6 +169,25 @@ pub async fn remote_rename(state: &Host, connection: FsConnection, from: String,
state.remote.remote_rename(&connection, &from, &to).await
}
/// Remote Host › Download to…: remote files and folders into a local folder, each under its
/// own name.
pub async fn remote_download_into(
window: &str,
state: &Host,
connection: FsConnection,
paths: Vec<String>,
target: IpcPath,
request: RequestId,
) -> IdeResult<TransferReport> {
let project = state.project(window)?;
let token = state.begin_request(request);
let (mut report, done) = progress(project.sink.clone(), "Downloading", token.clone());
let r = state.remote.download_into(&connection, &paths, &target, &token, &mut report).await;
done();
state.end_request(request);
r
}
pub async fn deploy_test_connection(state: &Host, connection: FsConnection, path: String) -> IdeResult<()> {
state.remote.test_connection(&connection, &path).await
}
@@ -197,6 +216,17 @@ pub(crate) fn register(d: &mut crate::rpc::Dispatcher) {
d.add("remote_of", |c| async move { Ok(remote_of(&c.host, c.arg("path")?)) });
d.add("remote_delete", |c| async move { remote_delete(&c.host, c.arg("connection")?, c.arg("path")?).await });
d.add("remote_mkdir", |c| async move { remote_mkdir(&c.host, c.arg("connection")?, c.arg("path")?).await });
d.add("remote_download_into", |c| async move {
remote_download_into(
&c.window,
&c.host,
c.arg("connection")?,
c.arg("paths")?,
c.arg("target")?,
c.arg("request")?,
)
.await
});
d.add("remote_rename", |c| async move {
remote_rename(&c.host, c.arg("connection")?, c.arg("from")?, c.arg("to")?).await
});
+21 -1
Переглянути файл
@@ -19,7 +19,17 @@ pub async fn blocking<T: Send + 'static>(f: impl FnOnce() -> IdeResult<T> + Send
pub async fn doc_open_virtual(host: &Host, window: &str, uri: String) -> IdeResult<DocumentInfo> {
let project = host.project(window)?;
let vd = crate::ops::lsp::lsp_virtual_document(host, window, uri).await?;
blocking(move || project.documents.open_virtual(&vd.uri, vd.language_id, &vd.text)).await
blocking(move || {
project.documents.open_virtual(
&vd.uri,
vd.language_id,
&vd.text,
&vd.encoding,
vd.has_bom,
vd.can_reload_encoding,
)
})
.await
}
/// The CodeMirror mode `path` gets when opened (`DocumentInfo::ui_mode`),
@@ -30,6 +40,8 @@ pub async fn doc_ui_mode(host: &Host, window: &str, path: PathBuf) -> IdeResult<
}
/// "Reload with Encoding" (`reload`) or "Convert to Encoding"; returns the new encoding and BOM state.
/// Virtual documents re-read their origin (archive, language server, Git blob)
/// instead of a disk file; converting them would write to it, so it stays refused.
pub async fn doc_set_encoding(
host: &Host,
window: &str,
@@ -38,6 +50,14 @@ pub async fn doc_set_encoding(
reload: bool,
) -> IdeResult<DocumentEncoding> {
let project = host.project(window)?;
if reload && let Some(uri) = project.documents.virtual_uri(doc)? {
let vd = if uri.starts_with("git-revision:") {
crate::ops::vcs::revision_virtual_document(host, window, &uri, Some(&encoding)).await?
} else {
project.lsp.reload_virtual(&uri, &encoding).await?
};
return host.blocking(move || project.documents.replace_virtual(doc, &vd.text, &vd.encoding, vd.has_bom)).await;
}
host.blocking(move || project.documents.set_encoding(doc, &encoding, reload)).await
}
+5 -2
Переглянути файл
@@ -176,8 +176,11 @@ async fn binary_and_virtual_documents_are_skipped() {
std::fs::write(&path, b"\0\0binary payload").unwrap();
let binary = f.open(&path).await;
assert!(binary.read_only);
let virtual_doc =
f.project.documents.open_virtual("test://virtual", ide_core::LanguageId::plain_text(), "virtual").unwrap();
let virtual_doc = f
.project
.documents
.open_virtual("test://virtual", ide_core::LanguageId::plain_text(), "virtual", "utf-8", false, false)
.unwrap();
std::fs::write(&path, "now ordinary text").unwrap();
f.events.lock().clear();
f.sync().await;
+201 -17
Переглянути файл
@@ -544,7 +544,7 @@ pub async fn lsp_goto_implementation(
/// classes and disk class files. Cached per project.
pub async fn lsp_virtual_document(host: &Host, window: &str, uri: String) -> IdeResult<VirtualDocument> {
if uri.starts_with("git-revision:") {
return crate::ops::vcs::revision_virtual_document(host, window, &uri).await;
return crate::ops::vcs::revision_virtual_document(host, window, &uri, None).await;
}
host.project(window)?.lsp.virtual_document(&uri).await
}
@@ -830,7 +830,7 @@ pub async fn lsp_code_actions(
};
let plugin_actions: Vec<CodeAction> = plugin_actions
.into_iter()
.map(|(plugin, a)| plugin_code_action(&plugin, &project.root, a))
.map(|(plugin, a)| plugin_code_action_in(&plugin, &plugin_edit_roots(&project), a))
.filter(|a| {
only.is_empty()
|| a.kind.as_ref().is_some_and(|k| only.iter().any(|o| k == o || k.starts_with(&format!("{o}."))))
@@ -846,10 +846,21 @@ pub async fn lsp_code_actions(
prepare_actions(project, doc, version, actions).await
}
/// Where a plugin's code action may change or open files: the project root and its content
/// roots (a group's members, extra Project Structure folders), never git's own files.
fn plugin_edit_roots(project: &crate::Project) -> Vec<std::path::PathBuf> {
std::iter::once(project.root.clone()).chain(project.content_roots()).collect()
}
#[cfg(test)]
fn plugin_code_action(plugin: &str, root: &Path, a: ide_plugin::CodeAction) -> CodeAction {
plugin_code_action_in(plugin, &[root.to_path_buf()], a)
}
fn plugin_code_action_in(plugin: &str, roots: &[std::path::PathBuf], a: ide_plugin::CodeAction) -> CodeAction {
// With `data` and nothing to apply, the plugin computes the edits when the action is picked.
let lazy = a.edits.is_empty() && a.operations.is_empty() && a.data.is_some();
let (files, open, disabled_reason) = match plugin_file_edits(root, &a) {
let (files, open, disabled_reason) = match plugin_file_edits(roots, &a) {
Ok((files, open)) => (files, open, None),
Err(reason) => (Vec::new(), None, Some(reason)),
};
@@ -876,17 +887,16 @@ fn plugin_code_action(plugin: &str, root: &Path, a: ide_plugin::CodeAction) -> C
}
/// The action's file operations, then its text edits, as one workspace edit's files, and the
/// file to open. An action with file operations or an `open` target stays inside the project
/// root as a whole: a path outside it disables the action.
/// file to open. Every path stays inside `roots` (the project root first) and off git's own
/// files: one that does not disables the action.
fn plugin_file_edits(
root: &Path,
roots: &[std::path::PathBuf],
a: &ide_plugin::CodeAction,
) -> Result<(Vec<ide_core::FileEdit>, Option<std::path::PathBuf>), String> {
use ide_core::FileEdit;
use ide_plugin::FileOperation;
let confine = !a.operations.is_empty() || a.open.is_some();
let confined = |path: &Path| ide_plugin::resolve_new_in_root(root, &path.to_string_lossy());
let writable = |path: &Path| ide_plugin::resolve_writable_in_root(root, &path.to_string_lossy());
let confined = |path: &Path| ide_plugin::resolve_new_in_roots(roots, &path.to_string_lossy());
let writable = |path: &Path| ide_plugin::resolve_writable_in_roots(roots, &path.to_string_lossy());
let mut files = Vec::new();
let mut created_with_text = Vec::new();
for operation in &a.operations {
@@ -913,13 +923,14 @@ fn plugin_file_edits(
}
}
for (path, edits) in &a.edits {
let path = if confine { writable(path)? } else { path.clone() };
let path = writable(path)?;
if created_with_text.contains(&path) {
return Err("put the text of a created file in its `create`, not in an edit".into());
}
files.push(FileEdit::Edit { path: path.into(), edits: edits.clone(), version: None });
}
let open = a.open.as_deref().map(confined).transpose()?;
// Opening a file hands its text to the plugin, so a link is judged by the file it leads to.
let open = a.open.as_deref().map(writable).transpose()?;
Ok((files, open))
}
@@ -947,8 +958,9 @@ async fn resolve_action(host: &Host, window: &str, doc: DocId, action: CodeActio
let plugin = r.plugin.clone();
let invocation = ide_plugin::Invocation { client: Some(window.to_owned()), ..Default::default() };
let resolved = blocking(move || plugins.resolve_code_action(&plugin, &d, pending, invocation)).await?;
let root = host.project(window)?.root.clone();
Ok(plugin_code_action(&r.plugin, &root, ide_plugin::CodeAction { data: None, ..resolved }))
let project = host.project(window)?;
let roots = plugin_edit_roots(&project);
Ok(plugin_code_action_in(&r.plugin, &roots, ide_plugin::CodeAction { data: None, ..resolved }))
}
/// Cancellable code actions with the kinds the servers advertise (Refactor This, refactoring availability).
@@ -2246,18 +2258,18 @@ mod tests {
};
for name in ["outside", "dangling"] {
let create = FileOperation::Create { path: root.join(name), text: "text".into(), overwrite: true };
assert!(plugin_file_edits(&root, &action(vec![create], Vec::new())).is_err());
assert!(plugin_file_edits(std::slice::from_ref(&root), &action(vec![create], Vec::new())).is_err());
let edits = vec![(root.join(name), vec![TextEdit { range: Range::default(), new_text: "text".into() }])];
assert!(plugin_file_edits(&root, &action(Vec::new(), edits)).is_err());
assert!(plugin_file_edits(std::slice::from_ref(&root), &action(Vec::new(), edits)).is_err());
}
let create = FileOperation::Create { path: root.join("inside"), text: "text".into(), overwrite: true };
let (files, _) = plugin_file_edits(&root, &action(vec![create], Vec::new())).unwrap();
let (files, _) = plugin_file_edits(std::slice::from_ref(&root), &action(vec![create], Vec::new())).unwrap();
assert!(matches!(&files[0], FileEdit::Create { path, .. } if **path == root.join("target.txt")));
let operations = vec![
FileOperation::Rename { from: root.join("outside"), to: root.join("dangling") },
FileOperation::Delete { path: root.join("outside") },
];
let (files, _) = plugin_file_edits(&root, &action(operations, Vec::new())).unwrap();
let (files, _) = plugin_file_edits(std::slice::from_ref(&root), &action(operations, Vec::new())).unwrap();
assert!(matches!(&files[0], FileEdit::Rename { from, to }
if **from == root.join("outside") && **to == root.join("dangling")));
assert!(matches!(&files[1], FileEdit::Delete { path, .. } if **path == root.join("outside")));
@@ -2341,6 +2353,178 @@ mod tests {
assert_eq!((r.data.as_deref(), r.file), (Some("ref"), true));
}
/// Advisory #4b: an action carrying only text edits (no file operation, no `open`) must still
/// be confined to the project root and kept off git's own files. A plugin needs no grant to
/// offer a quick fix, so this is decided here.
#[test]
fn a_plugin_edits_only_action_stays_in_the_project_and_off_git() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("proj");
std::fs::create_dir_all(root.join(".git")).unwrap();
std::fs::write(root.join(".git/config"), "[core]\n").unwrap();
std::fs::write(root.join("a.txt"), "a").unwrap();
let root = dunce::canonicalize(&root).unwrap();
let edits_only = |path: std::path::PathBuf| ide_plugin::CodeAction {
title: "Add import".into(),
kind: Some("quickfix".into()),
preferred: false,
edits: vec![(path, vec![TextEdit { range: Range::default(), new_text: "x".into() }])],
operations: Vec::new(),
open: None,
data: None,
file_action: false,
};
// In the project root: allowed.
assert_eq!(plugin_code_action("acme.x", &root, edits_only(root.join("a.txt"))).disabled_reason, None);
// Outside the root: disabled (previously an edits-only action was not confined).
let outside = plugin_code_action("acme.x", &root, edits_only(tmp.path().join("evil.txt")));
assert!(
outside.disabled_reason.as_deref().is_some_and(|r| r.contains("outside the project root")),
"{outside:?}"
);
assert!(outside.edit.unwrap().files.is_empty());
// Into .git: disabled.
let git = plugin_code_action("acme.x", &root, edits_only(root.join(".git/config")));
assert!(git.disabled_reason.as_deref().is_some_and(|r| r.contains("git's own files")), "{git:?}");
}
/// A project window group's files live in its members, outside the container root: a quick
/// fix reaches them, and their `.git` stays out of reach.
#[test]
fn a_plugin_action_reaches_the_content_roots_but_not_their_git() {
use ide_plugin::FileOperation::Create;
let tmp = tempfile::tempdir().unwrap();
for dir in ["group/.openidea", "member/.git", "member/src", "elsewhere"] {
std::fs::create_dir_all(tmp.path().join(dir)).unwrap();
}
let base = dunce::canonicalize(tmp.path()).unwrap();
std::fs::write(base.join("member/src/a.txt"), "a").unwrap();
std::fs::write(base.join("member/.git/config"), "[core]\n").unwrap();
let roots = [base.join("group"), base.join("member")];
let action =
|edits: Vec<std::path::PathBuf>, operations: Vec<ide_plugin::FileOperation>| ide_plugin::CodeAction {
title: "Fix".into(),
kind: Some("quickfix".into()),
preferred: false,
edits: edits
.into_iter()
.map(|path| (path, vec![TextEdit { range: Range::default(), new_text: "x".into() }]))
.collect(),
operations,
open: None,
data: None,
file_action: false,
};
let offer = |a| plugin_code_action_in("acme.x", &roots, a);
assert_eq!(offer(action(vec![base.join("member/src/a.txt")], Vec::new())).disabled_reason, None);
let created = Create { path: base.join("member/src/b.txt"), text: String::new(), overwrite: false };
assert_eq!(offer(action(Vec::new(), vec![created])).disabled_reason, None);
for refused in [base.join("member/.git/config"), base.join("elsewhere/c.txt")] {
let offered = offer(action(vec![refused.clone()], Vec::new()));
assert!(offered.disabled_reason.is_some(), "{refused:?}: {offered:?}");
assert!(offered.edit.unwrap().files.is_empty());
}
let hook = Create { path: base.join("member/.git/hooks/pre-commit"), text: "x".into(), overwrite: false };
assert!(offer(action(Vec::new(), vec![hook])).disabled_reason.is_some());
}
/// Advisory #4b: a quick fix's file operations and its `open` are held to the same rule as its
/// edits, under every name that reaches git's own files.
#[test]
fn a_plugin_action_cannot_change_or_open_git_files() {
use ide_plugin::FileOperation::{Create, Delete, Rename};
let tmp = tempfile::tempdir().unwrap();
for dir in ["proj/.git/hooks", "worktree"] {
std::fs::create_dir_all(tmp.path().join(dir)).unwrap();
}
let root = dunce::canonicalize(tmp.path().join("proj")).unwrap();
let worktree = dunce::canonicalize(tmp.path().join("worktree")).unwrap();
std::fs::write(
root.join(".git/config"),
"[core]
",
)
.unwrap();
std::fs::write(root.join("a.txt"), "a").unwrap();
std::fs::write(
worktree.join(".git"),
"gitdir: elsewhere
",
)
.unwrap();
std::fs::write(tmp.path().join("profile.txt"), "outside").unwrap();
let action =
|edits: Vec<std::path::PathBuf>, operations: Vec<ide_plugin::FileOperation>| ide_plugin::CodeAction {
title: "Add import".into(),
kind: Some("quickfix".into()),
preferred: false,
edits: edits
.into_iter()
.map(|path| (path, vec![TextEdit { range: Range::default(), new_text: "x".into() }]))
.collect(),
operations,
open: None,
data: None,
file_action: false,
};
let refused = |root: &Path, action: ide_plugin::CodeAction, why: &str| {
let offered = plugin_code_action("acme.x", root, action);
assert!(offered.disabled_reason.as_deref().is_some_and(|r| r.contains(why)), "{offered:?}");
assert!(offered.edit.unwrap().files.is_empty());
};
for git in [root.join(".git/config"), root.join(".GIT/config"), ".git/hooks/pre-commit".into()] {
refused(&root, action(vec![git], Vec::new()), "git's own files");
}
for operation in [
Create {
path: root.join(".git/hooks/pre-commit"),
text: "#!/bin/sh
"
.into(),
overwrite: false,
},
Rename { from: root.join("a.txt"), to: root.join(".git/config") },
Rename { from: root.join(".git/config"), to: root.join("b.txt") },
Delete { path: root.join(".git/config") },
] {
refused(&root, action(Vec::new(), vec![operation]), "git's own files");
}
// Opening a file hands its text to the plugin's providers.
let opens =
|path: std::path::PathBuf| ide_plugin::CodeAction { open: Some(path), ..action(Vec::new(), Vec::new()) };
refused(&root, opens(root.join(".git/config")), "git's own files");
// An edit lands in the file a link leads to, and `open` shows that file: both are judged by it.
#[cfg(unix)]
{
std::os::unix::fs::symlink(tmp.path().join("profile.txt"), root.join("link.txt")).unwrap();
std::os::unix::fs::symlink(root.join(".git/config"), root.join("alias")).unwrap();
refused(&root, action(vec![root.join("link.txt")], Vec::new()), "outside the project root");
refused(&root, action(vec![root.join("alias")], Vec::new()), "git's own files");
refused(&root, opens(root.join("link.txt")), "outside the project root");
refused(&root, opens(root.join("alias")), "git's own files");
}
// A linked worktree's `.git` file, under the short name Windows opens it by as well.
if cfg!(windows) && worktree.join("GIT~1").exists() {
for operation in [
Delete { path: worktree.join("GIT~1") },
Create {
path: worktree.join("GIT~1"),
text: "gitdir: other
"
.into(),
overwrite: true,
},
] {
refused(&worktree, action(Vec::new(), vec![operation]), "git's own files");
}
refused(&worktree, action(vec![worktree.join("GIT~1")], Vec::new()), "git's own files");
refused(&worktree, opens(worktree.join("GIT~1")), "git's own files");
} else {
eprintln!("short name not tried: this volume keeps none");
}
}
#[test]
fn every_completion_kind_the_plugin_api_documents_is_understood() {
let wit = include_str!("../../../../wit/providers.wit");
+10 -2
Переглянути файл
@@ -231,8 +231,16 @@ pub async fn run_tomcat_build(host: &Host, window: &str, config: RunConfig) -> I
.ok_or_else(|| IdeError::invalid("Choose a WAR artifact"))?;
let base = ide_run::tomcat::effective_base_dir(&project.root, &project.cache_dir, &config)?;
// "Build before launch" is skipped when the built WAR is already newer than the
// module's sources, so a repeated run does not rebuild (IDEA's up-to-date check).
if ide_run::tomcat::artifact_is_current(&project.root, &base, &config) {
// sources of every module in the build, so a repeated run does not rebuild (IDEA's
// up-to-date check) but a change in a dependency module does.
let modules: Vec<std::path::PathBuf> = model
.maven
.iter()
.flat_map(|m| m.projects.iter().map(|p| p.dir.as_str()))
.chain(model.gradle.iter().flat_map(|g| g.projects.iter().map(|p| p.dir.as_str())))
.map(std::path::PathBuf::from)
.collect();
if ide_run::tomcat::artifact_is_current(&project.root, &base, &config, &modules) {
return Ok(None);
}
let mut plan = ide_build::web::build(&project.root, &model, id, &base)?;
+59 -1
Переглянути файл
@@ -20,10 +20,13 @@ struct LogAccess {
}
/// Immutable Git objects exposed through the editor's ordinary read-only VFS.
/// `encoding` re-decodes the blob with it ("Reload with Encoding") instead of
/// the project's detection.
pub async fn revision_virtual_document(
host: &Host,
window: &str,
uri: &str,
encoding: Option<&str>,
) -> IdeResult<ide_lsp::types::VirtualDocument> {
#[derive(Deserialize)]
struct Target {
@@ -47,7 +50,7 @@ pub async fn revision_virtual_document(
.collect::<IdeResult<Vec<_>>>()?;
let target: Target = serde_json::from_slice(&bytes).map_err(|e| ide_core::IdeError::invalid(e.to_string()))?;
let path = target.repo.join(&target.path);
let text = vcs_file_at(host, window, path, Some(target.revision.clone()))
let (text, charset, has_bom) = revision_text(host, window, &path, &target.revision, encoding)
.await?
.ok_or_else(|| ide_core::IdeError::not_found("file is absent at this revision"))?;
Ok(ide_lsp::types::VirtualDocument {
@@ -57,8 +60,63 @@ pub async fn revision_virtual_document(
title: target.path.file_name().unwrap_or_default().to_string_lossy().into_owned(),
description: Some(format!("{} · {}", target.repo.display(), target.revision)),
decompiled: false,
encoding: charset.name().to_ascii_lowercase(),
has_bom,
can_reload_encoding: true,
})
}
/// Revision text with the charset it was decoded with and whether a matching
/// BOM was stripped; `None` when the file is absent at the revision.
async fn revision_text(
host: &Host,
window: &str,
path: &Path,
rev: &str,
encoding: Option<&str>,
) -> IdeResult<Option<(String, &'static encoding_rs::Encoding, bool)>> {
let project = host.project(window).ok();
let external = host.external_logs.read().get(host.owner_window(window)).and_then(|rs| {
rs.iter().filter(|r| path.starts_with(r.root())).max_by_key(|r| r.root().components().count()).cloned()
});
let (path, rev, encoding) = (path.to_path_buf(), rev.to_owned(), encoding.map(str::to_owned));
blocking(move || {
fn with_repo<T>(
project: &Option<std::sync::Arc<Project>>,
external: &Option<std::sync::Arc<ide_vcs::Repo>>,
path: &Path,
f: impl FnOnce(&ide_vcs::Repo) -> IdeResult<T>,
) -> IdeResult<T> {
match project {
Some(project) => project.with_log_repo_of(path, f),
None => f(external
.as_ref()
.ok_or_else(|| ide_core::IdeError::unavailable("external Git log is not open"))?),
}
}
let (text_encoding, charset) = match &encoding {
Some(label) => {
let charset = encoding_rs::Encoding::for_label(label.trim().as_bytes())
.ok_or_else(|| ide_core::IdeError::invalid(format!("unknown encoding {label:?}")))?;
(ide_vcs::TextEncoding::from(charset), charset)
}
None => match &project {
Some(project) => {
let text_encoding = project.vcs_encoding(&path, Some(&rev))?;
(text_encoding, text_encoding.charset)
}
None => (ide_vcs::TextEncoding::from(encoding_rs::UTF_8), encoding_rs::UTF_8),
},
};
let bytes = with_repo(&project, &external, &path, |r| r.file_bytes_at(&path, Some(&rev)))?;
let Some(bytes) = bytes else { return Ok(None) };
let has_bom = encoding_rs::Encoding::for_bom(&bytes).is_some_and(|(enc, _)| enc == charset);
let text =
with_repo(&project, &external, &path, |r| r.file_at(&path, Some(&rev), text_encoding))?.unwrap_or_default();
Ok(Some((text, charset, has_bom)))
})
.await
}
impl LogAccess {
fn new(host: &Host, window: &str, at: Option<&Path>) -> Self {
let project = host.project(window).ok();
+51 -2
Переглянути файл
@@ -896,6 +896,13 @@ impl Shared {
self.prompts_blocked.lock().remove(plugin);
}
/// A window other than `client` where `plugin` waits for the user's answer (a prompt or a
/// Commit request). The plugin runs one call at a time, so anything else asked of it waits
/// behind that answer.
pub fn waiting_elsewhere(&self, plugin: &str, client: &str) -> Option<String> {
self.prompts.lock().values().find(|p| p.plugin == plugin && p.client != client).map(|p| p.client.clone())
}
/// The user answered (`Some`) or dismissed (`None`) a prompt.
pub fn answer_prompt(&self, id: u64, answer: Option<Vec<String>>) -> bool {
let mut prompts = self.prompts.lock();
@@ -1263,6 +1270,8 @@ pub(super) fn commit_draft(
let Some(r) = ide_vcs::Repo::discover(&repo.root.0).map_err(failed)? else {
return Err(HostError::NotFound(format!("{} is not in a git repository", repo.root.0.display())));
};
// Both callers have checked that the window's project is trusted.
r.set_execution_allowed(true);
let pending = r.pending_changes(&repo.options, budget, &|path| Ok(workspace.vcs_encoding(path))).map_err(
|e| match e.kind {
ide_core::error::ErrorKind::InvalidInput => HostError::Invalid(e.message),
@@ -2093,7 +2102,7 @@ impl HostApi for PluginApi {
}
fn vcs_status(&self, root: &Path) -> Result<Option<RepositoryStatus>, HostError> {
let Some(repo) = ide_vcs::Repo::discover(root).map_err(failed)? else { return Ok(None) };
let Some(repo) = self.repo(root)? else { return Ok(None) };
let info = repo.branch_info().map_err(failed)?;
let changed = repo.status().map_err(failed)?.len();
Ok(Some(RepositoryStatus {
@@ -2119,7 +2128,7 @@ impl HostApi for PluginApi {
if !workspace.is_some_and(|w| w.trusted(root)) {
return Err(HostError::Failed("the project is in Restricted Mode: git does not run there".into()));
}
let Some(repo) = ide_vcs::Repo::discover(root).map_err(failed)? else {
let Some(repo) = self.repo(root)? else {
return Err(HostError::NotFound("the project is not in a git repository".into()));
};
match repo.fetch(cancel, &|_| {}) {
@@ -3936,6 +3945,16 @@ fn valid_range(document: &ide_text::Document, range: ide_core::Range) -> bool {
}
impl PluginApi {
/// The repository of `root`, which may execute only when `root` is a trusted project.
/// Status needs it there: a restricted repository is read without its config, so it knows
/// no upstream. The other reads use [`Self::read_repo`] and never execute.
fn repo(&self, root: &Path) -> Result<Option<ide_vcs::Repo>, HostError> {
let repo = ide_vcs::Repo::discover(root).map_err(failed)?;
if let Some(repo) = &repo {
repo.set_execution_allowed(self.shared.workspace().is_some_and(|w| w.trusted(root)));
}
Ok(repo)
}
fn read_repo(&self, root: &Path) -> Result<Option<ide_vcs::Repo>, HostError> {
if !self.shared.workspace().is_some_and(|w| w.trusted(root)) {
return Err(HostError::Failed("the project is in Restricted Mode: git does not run there".into()));
@@ -4200,6 +4219,36 @@ mod workbench_tests {
std::fs::write(&path, "x".repeat(5 << 20)).unwrap();
assert!(api.vcs_diff(&root, None, None, &cancel).is_err());
}
/// Security audit lead 2: status has no trust check to stop it, so in Restricted Mode the
/// repository it opens must not run what its config names.
#[test]
fn vcs_status_runs_no_repository_program_in_restricted_mode() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap();
git(&root, &["init", "-q"]);
git(&root, &["config", "user.name", "Test"]);
git(&root, &["config", "user.email", "[email protected]"]);
git(&root, &["config", "commit.gpgsign", "false"]);
std::fs::write(root.join("a.txt"), "base\n").unwrap();
git(&root, &["add", "."]);
git(&root, &["commit", "-qm", "base"]);
let marker = dir.path().join("FILTER-RAN");
let clean = format!("echo x > '{}'; cat", marker.to_str().unwrap().replace('\\', "/"));
git(&root, &["config", "filter.marker.clean", &clean]);
std::fs::write(root.join(".gitattributes"), "*.txt filter=marker\n").unwrap();
// Same length as the committed text: only reading the file tells them apart.
std::fs::write(root.join("a.txt"), "BASE\n").unwrap();
let restricted = api(&root, false, None);
let status = restricted.vcs_status(&root);
assert!(!marker.exists(), "status ran the repository's clean filter in Restricted Mode");
assert_eq!(status.unwrap().unwrap().changed_files, 2);
// Control: the same call runs the fixture's filter in a trusted project.
api(&root, true, None).vcs_status(&root).unwrap();
assert!(marker.exists(), "control: a trusted status did not run the clean filter, fixture is not live");
}
#[cfg(unix)]
#[test]
fn vcs_reads_never_run_repository_configured_helpers() {
+115 -16
Переглянути файл
@@ -14,6 +14,7 @@ use ide_extension::{MANIFEST_FILE, Manifest};
use semver::Version;
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use std::time::Duration;
/// The official marketplace: the index the release workflow publishes with
/// the app, next to the signed packages of the official plugins.
@@ -93,22 +94,29 @@ fn safe_suggestions(paths: Vec<String>) -> Vec<String> {
paths.into_iter().filter(ok).take(MAX_SUGGESTIONS).collect()
}
/// Whether `url` is read over the network: `http(s)://…`, not `file://…` or a local path.
fn remote(url: &str) -> bool {
url.starts_with("https://") || url.starts_with("http://")
}
/// Whether an index read from `base` may send the IDE to `href` (resolved). One read over
/// the network names nothing on this machine: the IDE would read the file for it, or sign
/// in to the share it names.
pub fn reachable_from(base: &str, href: &str) -> bool {
!remote(base) || remote(href)
}
/// The slowest transfer a read over the network waits for, in bytes per second: with
/// `READ_GRACE`, the largest index is given up on after about five minutes, the largest
/// package after a little over an hour.
const SLOWEST_READ: usize = 64 << 10;
/// What a read over the network may take before its first byte.
const READ_GRACE: Duration = Duration::from_secs(30);
/// Reads `url`: `http(s)://…`, `file://…` or a local path.
pub fn read_url(url: &str, max: usize) -> Result<Vec<u8>, String> {
if url.starts_with("https://") || url.starts_with("http://") {
let client = ide_http::client::shared().map_err(|e| e.message)?;
let url = url.to_owned();
return super::api::net_runtime().block_on(async move {
let response = client.get(&url).send().await.map_err(|e| format!("{url}: {e}"))?;
if !response.status().is_success() {
return Err(format!("{url}: HTTP {}", response.status()));
}
let bytes = response.bytes().await.map_err(|e| format!("{url}: {e}"))?;
if bytes.len() > max {
return Err(format!("{url}: larger than {} MiB", max >> 20));
}
Ok(bytes.to_vec())
});
if remote(url) {
return read_remote(url, max, READ_GRACE + Duration::from_secs((max / SLOWEST_READ) as u64));
}
let path = url.strip_prefix("file://").unwrap_or(url);
let bytes = std::fs::read(path).map_err(|e| format!("{path}: {e}"))?;
@@ -118,6 +126,35 @@ pub fn read_url(url: &str, max: usize) -> Result<Vec<u8>, String> {
Ok(bytes)
}
/// Reads `url` over http(s): no more than `max` bytes, whatever the server announces or
/// sends, and within `deadline` from the request to the last byte. The marketplace is read
/// with nobody asking, so its server decides neither how much is kept nor for how long.
fn read_remote(url: &str, max: usize, deadline: Duration) -> Result<Vec<u8>, String> {
let client = ide_http::client::shared().map_err(|e| e.message)?;
super::api::net_runtime().block_on(async move {
let failed = |e: reqwest::Error| match e.is_timeout() {
true => format!("{url}: no answer in time"),
false => format!("{url}: {e}"),
};
let too_large = || format!("{url}: larger than {} MiB", max >> 20);
let mut response = client.get(url).timeout(deadline).send().await.map_err(failed)?;
if !response.status().is_success() {
return Err(format!("{url}: HTTP {}", response.status()));
}
if response.content_length().is_some_and(|announced| announced > max as u64) {
return Err(too_large());
}
let mut bytes = Vec::new();
while let Some(chunk) = response.chunk().await.map_err(failed)? {
if bytes.len() + chunk.len() > max {
return Err(too_large());
}
bytes.extend_from_slice(&chunk);
}
Ok(bytes)
})
}
/// The index at `url`, following `moved`, with downloads made absolute against
/// where it was read from.
fn read_index(url: &str) -> IdeResult<Index> {
@@ -132,12 +169,25 @@ fn read_index(url: &str) -> IdeResult<Index> {
if url.starts_with("https://") && !next.starts_with("https://") {
return Err(IdeError::invalid(format!("the marketplace moved to {next}, which isn't https")));
}
if !reachable_from(&url, &next) {
return Err(IdeError::invalid("the marketplace moved to an address that is not on the network"));
}
url = next;
}
None => {
for e in &mut index.plugins {
e.download = resolve(&url, &e.download);
}
// Judged here, against where this index was read: the address in use may be
// an index on this machine that moved to it.
let listed = index.plugins.len();
index.plugins.retain(|e| reachable_from(&url, &e.download));
if index.plugins.len() < listed {
tracing::warn!(
"plugin marketplace: {} downloads are not on the network; they are not offered",
listed - index.plugins.len()
);
}
return Ok(index);
}
}
@@ -295,13 +345,23 @@ impl PluginHost {
entry.name, entry.version, entry.engine
)));
}
let bytes = read_url(&resolve(&url, &entry.download), MAX_PACKAGE).map_err(IdeError::unavailable)?;
let download = resolve(&url, &entry.download);
// An index read by this version holds only downloads it may reach. The one kept by an
// earlier version does not say where it was read: it is judged against the marketplace in use.
if !reachable_from(&url, &download) {
return Err(IdeError::invalid(format!(
"the marketplace's download of {} {} is not on the network",
entry.id, entry.version
)));
}
let bytes = read_url(&download, MAX_PACKAGE).map_err(IdeError::unavailable)?;
let downloads = self.plugins_dir.join(DOWNLOADS);
std::fs::create_dir_all(&downloads)?;
let archive = downloads.join(format!("{}-{}.{}", entry.id, entry.version, package::PACKAGE_EXTENSION));
std::fs::write(&archive, bytes)?;
let dir = self.unpack_package(&archive)?;
let dir = self.unpack_package(&archive);
let _ = std::fs::remove_file(&archive);
let dir = dir?;
let manifest = Manifest::load(&dir).map_err(IdeError::invalid)?;
let files = self.package_files_in(&dir)?;
let digest = package::package_digest(&dir, &files)?;
@@ -508,4 +568,43 @@ mod tests {
assert_eq!(safe_suggestions(paths.iter().map(|p| p.to_string()).collect()), ["pom.xml", "config/routes.rb"]);
assert_eq!(safe_suggestions(vec!["a".into(); 20]).len(), MAX_SUGGESTIONS);
}
#[test]
fn an_index_read_over_the_network_reaches_only_the_network() {
let base = "https://x.dev/market/index.json";
let reaches = |href: &str| reachable_from(base, &resolve(base, href));
for href in ["p/a.oiplugin", "../a.oiplugin", "https://cdn.dev/a"] {
assert!(reaches(href), "{href}");
}
for href in ["/etc/passwd", "//attacker/share/a", "file:///etc/passwd", "file://attacker/share/a"] {
assert!(!reaches(href), "{href}");
}
#[cfg(windows)]
for href in [r"C:\Users\me\.ssh\id_ed25519", r"\\attacker\share\a", r"\\?\UNC\attacker\share\a"] {
assert!(!reaches(href), "{href}");
}
// A local index names what whoever put it there chose.
assert!(reachable_from("/srv/market/index.json", "/srv/market/a.oiplugin"));
}
#[test]
fn a_server_that_stops_answering_does_not_hold_the_read() {
use std::io::{BufRead, BufReader, Write};
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let url = format!("http://127.0.0.1:{}/index.json", listener.local_addr().unwrap().port());
std::thread::spawn(move || {
for mut stream in listener.incoming().flatten() {
let mut reader = BufReader::new(stream.try_clone().unwrap());
let mut line = String::new();
while reader.read_line(&mut line).unwrap_or(0) > 0 && line != "\r\n" {
line.clear();
}
// The headers, then nothing for longer than the read waits.
let _ = stream.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\n");
std::thread::sleep(Duration::from_secs(5));
}
});
let err = read_remote(&url, MAX_INDEX, Duration::from_millis(300)).unwrap_err();
assert!(err.contains("no answer in time"), "{err}");
}
}
+12 -1
Переглянути файл
@@ -63,7 +63,7 @@ pub use icons::{
pub(crate) use inlay_hints::merge as merge_inlay_hints;
pub use market::{
INDEX_FILE as MARKET_INDEX_FILE, MarketPlugin, OFFICIAL_INDEX as OFFICIAL_MARKET_INDEX, OFFICIAL_PUBLISHER,
read_url as read_market_url, resolve as resolve_market_href,
reachable_from as market_href_reachable_from, read_url as read_market_url, resolve as resolve_market_href,
};
pub use points::{COMMANDS, CommandDecl, register_builtin_points};
pub use providers::{ProviderKind, set_editor_providers};
@@ -1381,6 +1381,17 @@ impl PluginHost {
invocation: Invocation,
cancel: Option<&ide_core::CancellationToken>,
) -> IdeResult<Option<String>> {
// Queued behind a prompt open in another window, the command would sit there with no
// sign of life until that prompt is answered: point the user at it instead.
if let Some(client) = invocation.client.as_deref()
&& let Some(waiting) = self.shared.waiting_elsewhere(id, client)
{
self.sink.emit(Some(&waiting), PluginEvent::PromptWaiting { plugin_id: id.to_owned() });
let name = self.get(id).map(|p| p.manifest().name.clone()).unwrap_or_else(|_| id.to_owned());
return Err(IdeError::invalid(format!(
"{name} is waiting for your answer in another window. Answer or close it there, then try again."
)));
}
let (cmd, arg) = (command.to_owned(), arg.map(str::to_owned));
self.code_call_cancellable(id, invocation, "running the command", cancel, move |rt, inv| {
rt.run_command(&cmd, arg.as_deref(), inv)
+36 -5
Переглянути файл
@@ -121,9 +121,14 @@ pub(crate) struct NavigationIndex {
fn contains(range: Range, pos: Pos) -> bool {
(range.start.line, range.start.col) <= (pos.line, pos.col) && (pos.line, pos.col) < (range.end.line, range.end.col)
}
/// Inside the document, start not after end. An empty range marks a position (a template file's
/// definition at its start, say): navigation lands there, but it has no text to rename.
fn valid_range(lines: &[usize], range: Range) -> bool {
let pos = |p: Pos| lines.get(p.line as usize).is_some_and(|&length| p.col as usize <= length);
pos(range.start) && pos(range.end) && (range.start.line, range.start.col) < (range.end.line, range.end.col)
pos(range.start) && pos(range.end) && (range.start.line, range.start.col) <= (range.end.line, range.end.col)
}
fn is_empty(range: Range) -> bool {
(range.start.line, range.start.col) == (range.end.line, range.end.col)
}
impl PluginHost {
@@ -280,12 +285,20 @@ impl PluginHost {
.chain(&edges.references)
.any(|a| a.key.is_empty() || a.key.len() > 1024 || !valid_range(&lines, a.range))
{
return Err(IdeError::invalid("invalid navigation anchors"));
return Err(IdeError::invalid(format!(
"{}: invalid navigation anchors in {}",
plugin.id(),
path.display()
)));
}
if edges.markers.len() > MAX_EDGES
|| edges.markers.iter().any(|a| a.key.is_empty() || a.key.len() > 1024 || !valid_range(&lines, a.range))
{
return Err(IdeError::invalid("invalid navigation markers"));
return Err(IdeError::invalid(format!(
"{}: invalid navigation markers in {}",
plugin.id(),
path.display()
)));
}
let mut index = self.navigation.lock();
if index.epochs.get(root).copied() != generation {
@@ -406,7 +419,7 @@ impl PluginHost {
let answer = result.get_or_insert_with(|| IndexedSymbol { range: primary.range, files: vec![] });
for (path, file) in &graph.files {
let mut ranges = std::collections::BTreeSet::new();
for candidate in &file.edges.definitions {
for candidate in file.edges.definitions.iter().filter(|a| !is_empty(a.range)) {
if targets.contains(&Location::file(path.clone(), candidate.range))
|| (targets.is_empty() && file.scope == source.scope && keys.contains(candidate.key.as_str()))
{
@@ -414,7 +427,7 @@ impl PluginHost {
}
}
let mut matches = HashMap::new();
for candidate in &file.edges.references {
for candidate in file.edges.references.iter().filter(|a| !is_empty(a.range)) {
let same = *matches.entry(candidate.key.as_str()).or_insert_with(|| {
let resolved = graph.scoped_targets(&candidate.key, &file.scope);
resolved.iter().any(|target| targets.contains(target))
@@ -580,3 +593,21 @@ impl ExtensionPoint for IndexersPoint {
Ok(Prepared::new(d.extensions.join(", "), || Ok(Disposable::new(|| {}))).capability(Capability::ProjectRead))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn empty_ranges_inside_the_document_are_valid_positions() {
let lines = [3, 0];
let range = |a: (u32, u32), b: (u32, u32)| Range::new(Pos::new(a.0, a.1), Pos::new(b.0, b.1));
assert!(valid_range(&lines, range((0, 0), (0, 0))));
assert!(valid_range(&lines, range((1, 0), (1, 0))));
assert!(valid_range(&lines, range((0, 1), (0, 3))));
assert!(!valid_range(&lines, range((0, 2), (0, 1))), "reversed");
assert!(!valid_range(&lines, range((0, 4), (0, 4))), "past the line end");
assert!(!valid_range(&lines, range((2, 0), (2, 0))), "past the last line");
assert!(is_empty(range((0, 0), (0, 0))) && !is_empty(range((0, 0), (0, 1))));
}
}
+1
Переглянути файл
@@ -165,6 +165,7 @@ pub fn forwarded(event: &PluginEvent) -> bool {
| PluginEvent::OpenProject { .. }
| PluginEvent::PromptRequested { .. }
| PluginEvent::PromptClosed { .. }
| PluginEvent::PromptWaiting { .. }
| PluginEvent::RevealView { .. }
| PluginEvent::CommitRequested { .. }
| PluginEvent::EndpointsChanged { .. }
+4
Переглянути файл
@@ -321,6 +321,10 @@ pub enum PluginEvent {
/// The call that asked gave up (cancelled, or its plugin stopped): close the prompt.
#[serde(rename_all = "camelCase")]
PromptClosed { prompt_id: String },
/// A command of this window's prompting plugin was run from another window and refused
/// while the plugin waits for the answer here: bring this window to the front.
#[serde(rename_all = "camelCase")]
PromptWaiting { plugin_id: String },
/// Bring the tool window of the plugin's view `view` to the front.
#[serde(rename_all = "camelCase")]
RevealView { plugin_id: String, view: String },
+12 -2
Переглянути файл
@@ -13,7 +13,7 @@ use ide_vfs::Watcher;
use ide_workspace::documents::DocumentObserver;
use ide_workspace::{DocumentInfo, Documents, FileMoves, LocalHistory, ProjectRoots, SettingsStore, SourceFolder};
use parking_lot::{Mutex, RwLock};
use std::collections::{BTreeMap, HashMap};
use std::collections::{BTreeMap, HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -50,6 +50,8 @@ pub struct Project {
pub cache_dir: PathBuf,
/// Superseded launch snapshots retained until this project session ends.
pub(crate) java_launch_snapshots: Mutex<crate::ops::java::cache::Session>,
/// Reasons this window was already told why Kotlin is debugged with kotlin-debug-adapter.
pub(crate) kotlin_debug_notices: Mutex<HashSet<&'static str>>,
/// Database tool window: data sources and their connections.
pub db: Arc<ide_db::DbManager>,
/// HTTP Client responses (IDEA's `.idea/httpRequests`), in the app data dir.
@@ -563,6 +565,7 @@ impl Host {
java_launch_snapshots: Mutex::new(crate::ops::java::cache::Session::new(
self.local_data_dir.join("cache").join(&project_key).join("build"),
)),
kotlin_debug_notices: Mutex::default(),
db,
http_history,
agents,
@@ -596,6 +599,12 @@ impl Host {
weak.upgrade().map(|p| p.exclude_globs()).unwrap_or_default()
}));
}
{
let weak = Arc::downgrade(&project);
project.agents.set_security_hook(Arc::new(move || {
weak.upgrade().map(|p| p.settings.effective().agent_security).unwrap_or_default()
}));
}
self.insert_project(window, project.clone());
project.lsp.prewarm_java_configuration();
if let Some(bridge) = &project.kotlin_bridge {
@@ -718,6 +727,7 @@ impl Host {
java_launch_snapshots: Mutex::new(crate::ops::java::cache::Session::new(
self.local_data_dir.join("cache").join(&project_key).join("build"),
)),
kotlin_debug_notices: Mutex::default(),
db,
http_history,
agents,
@@ -1221,7 +1231,7 @@ fn content_roots(settings: &ide_core::settings::Settings, root: &Path, build_cac
/// The roots a file may navigate into: its own root (the workspace member containing it, or
/// the project root) plus that root's configured dependencies. A workspace member depends
/// only on the roots picked per member in Settings › Project Structure › Dependencies —
/// only on the roots picked per member in Project Structure › Dependencies —
/// being attached to the workspace never makes its code navigable by itself.
/// Cached navigation roots per (project, file), with the instant they were computed.
type NavigationRootsCache = HashMap<(PathBuf, PathBuf), (Instant, Vec<PathBuf>)>;
+218
Переглянути файл
@@ -209,6 +209,49 @@ fn write_index(
path.to_string_lossy().into_owned()
}
/// Security audit lead 5: an index entry's id and version name the download file,
/// so an entry whose id or version would place it outside the downloads folder
/// is not offered, and nothing is written for it.
#[test]
fn market_entry_ids_cannot_place_the_download_outside_the_downloads_folder() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, hello) = signed_hello(&market, "1.0.0", &key, &f.host);
// Never read: an entry that is not offered is not downloaded.
std::fs::write(market.join("junk.bin"), b"attacker bytes").unwrap();
// The id leaves the folder on every platform (`.downloads` exists by then); the version
// only where `..` is resolved before the path is opened (Windows).
for (id, version) in [("../../escape", "1.0.0"), ("acme.hello", "1.0.0/../../../escape")] {
let entry = IndexEntry { id: id.into(), version: version.into(), download: "junk.bin".into(), ..hello.clone() };
let url = write_index(&market, &key, vec![entry], vec![], false);
f.host.set_market_url(Some(url));
f.host.refresh_market().unwrap();
assert!(f.host.market_plugins().is_empty(), "{id} {version} is offered");
let err = f.host.download_market_plugin(id).unwrap_err();
assert!(err.message.contains("is not in the marketplace"), "{id} {version}: {}", err.message);
}
let downloads = f.plugins_dir.join(".downloads");
let mut escaped = Vec::new();
let mut pending = vec![f._tmp.path().to_path_buf()];
while let Some(dir) = pending.pop() {
for e in std::fs::read_dir(&dir).unwrap().flatten() {
let path = e.path();
if path.is_dir() {
pending.push(path);
} else if path.extension().is_some_and(|x| x == "oiplugin")
&& !path.starts_with(&downloads)
&& !path.starts_with(&market)
{
escaped.push(path);
}
}
}
assert!(escaped.is_empty(), "the download was written outside {}: {escaped:?}", downloads.display());
}
/// First run: nothing cached yet, so listing the marketplace reads its index
/// instead of showing an empty one; later listings use the index read.
#[test]
@@ -366,6 +409,181 @@ fn a_moved_marketplace_is_read_where_it_moved_to() {
assert!(err.message.contains("moved more than"), "{}", err.message);
}
/// A loopback HTTP server whose answer to every request `respond` writes; the URL of its index.
fn serve_with(respond: impl Fn(&mut std::net::TcpStream) + Send + 'static) -> String {
use std::io::{BufRead, BufReader};
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let url = format!("http://127.0.0.1:{}/index.json", listener.local_addr().unwrap().port());
std::thread::spawn(move || {
for mut stream in listener.incoming().flatten() {
let mut reader = BufReader::new(stream.try_clone().unwrap());
let mut line = String::new();
while reader.read_line(&mut line).unwrap_or(0) > 0 && line != "\r\n" {
line.clear();
}
respond(&mut stream);
}
});
url
}
/// A loopback HTTP server answering every request with `body`; the URL of its index.
fn serve(body: String) -> String {
use std::io::Write;
serve_with(move |stream| {
let _ = write!(
stream,
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
);
})
}
/// An index is read up to the size an index may have, whatever its server says or sends:
/// the marketplace is read again every day, with nobody asking.
#[test]
fn an_index_larger_than_an_index_may_be_is_not_read_to_its_end() {
use std::io::Write;
use std::sync::atomic::{AtomicUsize, Ordering};
let f = fixture();
// Announced: refused on its headers, before any of it is read.
f.host.set_market_url(Some(serve_with(|stream| {
let _ = stream.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 1073741824\r\nConnection: close\r\n\r\n{");
})));
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("larger than 16 MiB"), "{}", err.message);
// Not announced: the read stops at the limit, and the server is left with the rest.
const BODY: usize = 96 << 20;
let sent = Arc::new(AtomicUsize::new(0));
f.host.set_market_url(Some(serve_with({
let sent = sent.clone();
move |stream| {
let _ = stream.write_all(b"HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n");
let chunk = [b' '; 64 << 10];
let mut written = 0;
while written < BODY && stream.write_all(&chunk).is_ok() {
written += chunk.len();
}
sent.store(written.max(1), Ordering::SeqCst);
}
})));
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("larger than 16 MiB"), "{}", err.message);
let waiting = std::time::Instant::now();
while sent.load(Ordering::SeqCst) == 0 && waiting.elapsed() < std::time::Duration::from_secs(30) {
std::thread::sleep(std::time::Duration::from_millis(20));
}
let sent = sent.load(Ordering::SeqCst);
assert!(sent > 0 && sent < BODY, "the server sent {sent} of {BODY} bytes");
}
/// An index read over the network names downloads on the network: not a file of this
/// machine, nor a share the machine would sign in to.
#[test]
fn an_index_read_over_the_network_cannot_name_a_local_download() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (package, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
let path = package.to_string_lossy().into_owned();
// A package that installs, so only where the index was read from can refuse it.
for download in [path.clone(), format!("file://{path}")] {
let entry = IndexEntry { download: download.clone(), ..entry.clone() };
let local = write_index(&market, &key, vec![entry], vec![], false);
f.host.set_market_url(Some(local.clone()));
f.host.refresh_market().unwrap();
let dir = f.host.download_market_plugin("acme.hello").unwrap();
assert_eq!(f.host.preview_install(&dir).unwrap().version, "1.0.0", "{download}");
f.host.set_market_url(Some(serve(std::fs::read_to_string(&local).unwrap())));
assert!(f.host.refresh_market().unwrap().is_empty(), "{download} is offered");
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("is not in the marketplace"), "{download}: {}", err.message);
}
}
/// Nor when an index on this machine is what sent the IDE to the one on the network.
#[test]
fn an_index_reached_through_a_local_one_cannot_name_a_local_download() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (package, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
let entry = IndexEntry { download: package.to_string_lossy().into_owned(), ..entry };
let served = serve(std::fs::read_to_string(write_index(&market, &key, vec![entry], vec![], false)).unwrap());
let old = f.work.join("old");
std::fs::create_dir_all(&old).unwrap();
let moved = serde_json::json!({ "format": package::INDEX_FORMAT, "moved": served }).to_string();
std::fs::write(old.join("index.json"), moved).unwrap();
f.host.set_market_url(Some(old.join("index.json").to_string_lossy().into_owned()));
assert!(f.host.refresh_market().unwrap().is_empty(), "the local download is offered");
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("is not in the marketplace"), "{}", err.message);
}
/// The index kept from the last read does not say where it was read from: what an earlier
/// version kept is judged against the marketplace in use.
#[test]
fn a_kept_index_cannot_name_a_local_download_for_a_marketplace_on_the_network() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (package, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
let entry = IndexEntry { download: package.to_string_lossy().into_owned(), ..entry };
let kept = write_index(&market, &key, vec![entry], vec![], false);
std::fs::copy(kept, f.plugins_dir.join("market-index.json")).unwrap();
let again = PluginHost::new(f.plugins_dir.clone(), Arc::new(|_: Option<&str>, _| {}), Limits::default()).unwrap();
// Never read: the check comes before any request.
again.set_market_url(Some("https://market.invalid/index.json".into()));
assert_eq!(again.market_plugins()[0].entry.id, "acme.hello");
let err = again.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("is not on the network"), "{}", err.message);
}
/// Nor may it send the IDE to an index on this machine.
#[test]
fn an_index_read_over_the_network_cannot_move_to_a_local_one() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
let local = write_index(&market, &key, vec![entry], vec![], false);
let moved = serde_json::json!({ "format": package::INDEX_FORMAT, "moved": local }).to_string();
f.host.set_market_url(Some(serve(moved)));
let err = f.host.refresh_market().unwrap_err();
assert!(err.message.contains("is not on the network"), "{}", err.message);
assert!(f.host.market_plugins().is_empty(), "the local index was read");
}
/// A download that turns out not to be a package is not kept.
#[test]
fn a_download_that_is_not_a_package_leaves_nothing_behind() {
let f = fixture();
let market = f.work.join("market");
std::fs::create_dir_all(&market).unwrap();
let key = PublisherKey::from_seed([3; 32]);
let (_, entry) = signed_hello(&market, "1.0.0", &key, &f.host);
std::fs::write(market.join("junk.bin"), b"not a package").unwrap();
let entry = IndexEntry { download: "junk.bin".into(), ..entry };
f.host.set_market_url(Some(write_index(&market, &key, vec![entry], vec![], false)));
f.host.refresh_market().unwrap();
let err = f.host.download_market_plugin("acme.hello").unwrap_err();
assert!(err.message.contains("not a plugin package"), "{}", err.message);
let left: Vec<_> = std::fs::read_dir(f.plugins_dir.join(".downloads"))
.map(|dir| dir.flatten().map(|e| e.file_name()).collect())
.unwrap_or_default();
assert!(left.is_empty(), "{left:?}");
}
/// `dir/index.json` listing `publishers` and `plugins`; its path.
fn index_of(dir: &Path, publishers: Vec<Publisher>, plugins: Vec<IndexEntry>) -> String {
let index = Index { format: package::INDEX_FORMAT, publishers, plugins, ..Index::default() };
+8 -2
Переглянути файл
@@ -93,9 +93,15 @@ async fn external_history_is_readable_and_never_becomes_a_write_target() {
let payload =
serde_json::to_vec(&serde_json::json!({ "repo": external, "revision": first, "path": "deleted.txt" })).unwrap();
let hex = payload.iter().map(|b| format!("{b:02x}")).collect::<String>();
let document =
vcs::revision_virtual_document(&host, "standalone", &format!("git-revision:{hex}/deleted.txt")).await.unwrap();
let uri = format!("git-revision:{hex}/deleted.txt");
let document = vcs::revision_virtual_document(&host, "standalone", &uri, None).await.unwrap();
assert_eq!(document.text, "historical content\n");
assert_eq!(document.encoding, "utf-8");
assert!(document.can_reload_encoding);
// Reload with Encoding re-decodes the blob with the chosen charset.
let reloaded = vcs::revision_virtual_document(&host, "standalone", &uri, Some("utf-16le")).await.unwrap();
assert_eq!(reloaded.encoding, "utf-16le");
assert_ne!(reloaded.text, document.text);
host.trust.decide(&root, true, false).unwrap();
let project = host.open_project("main", &root).unwrap();
assert!(vcs::vcs_log(&host, "main", Some(external.clone()), Default::default()).await.is_err());
+70
Переглянути файл
@@ -0,0 +1,70 @@
//! An image path may come from Markdown someone else wrote, and on Windows opening a network
//! share signs in to its server with the user's credentials: the host reads an image from a
//! share only when a project is open on it. The share here is this machine's own `C$` over
//! loopback.
#![cfg(windows)]
use ide_core::NullSink;
use ide_host::ops::bytes::fs_read_image;
use ide_host::{Host, HostConfig};
use std::path::{Path, PathBuf};
use std::sync::Arc;
/// `path` as seen through the admin share of `server`, a name of this machine.
fn over_smb(server: &str, path: &Path) -> PathBuf {
let s = path.to_str().unwrap();
PathBuf::from(format!(r"\\{server}\{}${}", &s[..1], &s[2..]))
}
#[test]
fn images_on_a_share_are_read_only_where_a_project_is_open() {
let tmp = tempfile::tempdir().unwrap();
let local = dunce::canonicalize(tmp.path()).unwrap();
let root = local.join("project");
std::fs::create_dir_all(root.join("docs")).unwrap();
for image in [root.join("docs/a.png"), local.join("outside.png")] {
std::fs::write(image, b"\x89PNG dummy").unwrap();
}
let runtime = tokio::runtime::Builder::new_multi_thread().worker_threads(2).enable_all().build().unwrap();
let _guard = runtime.enter();
let host = Host::new(HostConfig {
config_dir: tmp.path().join("config"),
data_dir: tmp.path().join("data"),
local_data_dir: tmp.path().join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |w| w,
runtime: runtime.handle().clone(),
project_environment: false,
});
let read = |path: PathBuf| runtime.block_on(fs_read_image(&host, "main", path));
let in_project = over_smb("127.0.0.1", &root.join("docs/a.png"));
let beside_project = over_smb("127.0.0.1", &local.join("outside.png"));
let other_server = over_smb("localhost", &root.join("docs/a.png"));
for path in [&in_project, &beside_project, &other_server] {
assert!(std::fs::read(path).is_ok(), "control: {} is not reachable over SMB", path.display());
}
let wanted = read(root.join("docs/a.png")).unwrap();
// No project is open on any share.
for path in [&in_project, &beside_project, &other_server] {
assert!(read(path.clone()).is_err(), "read {} with no project on its share", path.display());
}
let project_root = over_smb("127.0.0.1", &root);
host.trust.decide(&project_root, true, false).unwrap();
host.open_project("main", &project_root).unwrap();
assert_eq!(read(in_project.clone()).unwrap(), wanted);
// The whole share is one sign-in: a file outside the project folder is on it too.
assert_eq!(read(beside_project.clone()).unwrap(), wanted);
assert_eq!(read(PathBuf::from(in_project.to_str().unwrap().replace('\\', "/"))).unwrap(), wanted);
// The path as the webview holds it: the project root it was given, joined with `/`.
let root_in_webview = ide_core::ipc_path::to_ipc(&host.project("main").unwrap().root);
assert_eq!(read(PathBuf::from(format!("{root_in_webview}/docs/a.png"))).unwrap(), wanted);
assert!(read(other_server.clone()).is_err(), "read {} from a server no project is on", other_server.display());
host.close_project("main");
assert!(read(in_project.clone()).is_err(), "read {} after its project closed", in_project.display());
}
+99
Переглянути файл
@@ -4157,6 +4157,35 @@ fn code_action_resolution_prompts_in_the_editor_window_under_the_command_budget(
assert!(f.take_events().iter().all(|(_, e)| !matches!(e, PluginEvent::PromptRequested { .. })));
}
#[test]
fn a_command_from_another_window_is_refused_while_the_plugin_waits_for_an_answer() {
let (f, doc) = interactive_actions_fixture();
let action = f
.host
.provide_code_actions(&doc, ide_core::Range::default())
.into_iter()
.find(|(_, action)| action.data.as_deref() == Some("pick"))
.unwrap()
.1;
std::thread::scope(|scope| {
let running = scope.spawn(|| f.host.resolve_code_action("misbehave", &doc, action, action_invocation()));
let (id, _) = action_prompt(&f);
// Queued behind the open prompt it would not run until the prompt is answered.
let other = Invocation { client: Some("other-window".into()), ..Default::default() };
let err = f.host.run_command("misbehave", "anything", None, other).unwrap_err();
assert_eq!(err.kind, ErrorKind::InvalidInput);
assert!(err.message.contains("waiting for your answer in another window"), "{}", err.message);
let events = f.take_events();
assert!(
events.iter().any(|(client, e)| client.as_deref() == Some("editor-window")
&& matches!(e, PluginEvent::PromptWaiting { plugin_id } if plugin_id == "misbehave")),
"the prompting window is brought forward: {events:?}"
);
assert!(f.host.answer_prompt(&id, Some(vec!["b".into()])));
running.join().unwrap().unwrap();
});
}
#[test]
fn code_action_prompts_dismiss_cancel_and_close_without_edits_or_hanging() {
for (how, kind) in [
@@ -6574,3 +6603,73 @@ fn document_indexer_covers_workspace_members() {
.unwrap();
assert!(f.plugins.navigation_at(&f.root, &outside, 1, ide_core::Pos::new(0, 4), Some("main"), |_| None).is_none());
}
/// Advisory #4: `project.write` (Medium) must not reach VCS metadata such as `.git/hooks` or
/// `.git/config`, which git runs as code; the `.git` is inside the project root, so root
/// confinement alone does not stop it.
#[test]
fn project_write_cannot_reach_git_metadata() {
let f = fixture(Limits::default());
let init = std::process::Command::new("git").current_dir(&f.project).args(["init", "-q"]).output().unwrap();
assert!(init.status.success(), "git init");
f.install(&misbehave_with(f._tmp.path(), "misbehave-project-writer", "project.write"));
// The grant works for an ordinary path, so each refusal below is the `.git` rule and nothing else.
assert_eq!(f.run("misbehave", "projwrite", Some("src/fresh.txt")).unwrap(), None);
for rel in [".git/hooks/pre-commit", ".git/config", ".GIT/config", "src/../.git/config"] {
let before = std::fs::read(f.project.join(rel)).ok();
let err = f.run("misbehave", "projwrite", Some(rel)).unwrap_err();
assert!(err.contains("git's own files"), "{rel}: {err}");
assert_eq!(std::fs::read(f.project.join(rel)).ok(), before, "{rel}");
}
}
/// In a linked worktree or a submodule `.git` is a file naming the real git folder. Replacing it,
/// under any name that opens it, would point git at a folder of the plugin's choosing or leave the
/// checkout without a repository.
#[test]
fn project_write_cannot_replace_the_git_file_of_a_linked_worktree() {
let f = fixture(Limits::default());
let pointer = f.project.join(".git");
let gitdir = "gitdir: /elsewhere/.git/worktrees/project\n";
std::fs::write(&pointer, gitdir).unwrap();
f.install(&misbehave_with(f._tmp.path(), "misbehave-project-writer", "project.write"));
assert_eq!(f.run("misbehave", "projwrite", Some("src/fresh.txt")).unwrap(), None);
let mut spellings = vec![".git", ".GIT", ".git.", ".git ", ".git::$DATA"];
// Windows opens a file by its short name too, on a volume that keeps them.
if cfg!(windows) && f.project.join("GIT~1").exists() {
spellings.extend(["GIT~1", "git~1", "GIT~1.", "GIT~1 ", "GIT~1::$DATA"]);
} else {
eprintln!("short name not tried: this volume keeps none");
}
for rel in spellings {
let err = f.run("misbehave", "projwrite", Some(rel)).unwrap_err();
assert!(err.contains("git's own files"), "{rel}: {err}");
assert_eq!(std::fs::read_to_string(&pointer).unwrap(), gitdir, "{rel}");
}
}
/// A link already in the project is written through, so it is judged by where it points.
#[cfg(unix)]
#[test]
fn project_write_judges_a_link_by_its_target() {
let f = fixture(Limits::default());
let config = f.project.join(".git").join("config");
std::fs::create_dir(f.project.join(".git")).unwrap();
std::fs::write(&config, "[core]\n").unwrap();
let secret = f._tmp.path().join("secret.txt");
std::os::unix::fs::symlink(&config, f.project.join("alias")).unwrap();
std::os::unix::fs::symlink(&secret, f.project.join("escape")).unwrap();
f.install(&misbehave_with(f._tmp.path(), "misbehave-project-writer", "project.write"));
// A folder that is not there, in front of the link, does not keep it from being followed.
for rel in ["alias", "missing/../alias"] {
let err = f.run("misbehave", "projwrite", Some(rel)).unwrap_err();
assert!(err.contains("git's own files"), "{rel}: {err}");
assert_eq!(std::fs::read_to_string(&config).unwrap(), "[core]\n", "{rel}");
}
for rel in ["escape", "missing/../escape"] {
let err = f.run("misbehave", "projwrite", Some(rel)).unwrap_err();
assert!(err.contains("outside the project root"), "{rel}: {err}");
assert_eq!(std::fs::read_to_string(&secret).unwrap(), "top secret", "{rel}");
}
}
+1
Переглянути файл
@@ -330,6 +330,7 @@ async fn local_deployment_runs_on_the_project_host() {
"remote_delete",
"remote_mkdir",
"remote_rename",
"remote_download_into",
"db_sources",
"db_save_sources",
"db_set_password",
+1
Переглянути файл
@@ -94,6 +94,7 @@ async fn application_commands_work_in_restricted_projects_and_without_a_project(
"remote_delete",
"remote_mkdir",
"remote_rename",
"remote_download_into",
"remote_import_local",
"remote_bootstrap_exec",
"deploy_upload",
+308
Переглянути файл
@@ -0,0 +1,308 @@
//! Security audit lead 2: every `vcs_*` command Restricted Mode allows, run against a
//! repository whose config names a program for a signed commit, an index read, an external
//! diff, a clean filter and a missing object. None may run.
use ide_core::{NullSink, RequestId};
use ide_host::{Host, HostConfig, ops, ops::vcs};
use std::collections::BTreeSet;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::pin::Pin;
use std::process::{Command, Stdio};
use std::sync::Arc;
/// Allowed in Restricted Mode and not swept: none of them starts git for the project.
const NOT_THE_PROJECT_REPOSITORY: [&str; 7] = [
"vcs_clone", // clones a URL the user gives into a new folder
"vcs_diff_hunks", // compares two texts
"vcs_detected_git", // looks for the git executable
"vcs_shelves", // the shelf and the changelists are files of the IDE
"vcs_shelf_patch",
"vcs_changelists_load",
"vcs_write_patch", // writes a patch the UI already holds
];
fn git(root: &Path, args: &[&str]) -> String {
let output = Command::new("git").current_dir(root).args(args).output().unwrap();
assert!(output.status.success(), "{args:?}: {}", String::from_utf8_lossy(&output.stderr));
String::from_utf8(output.stdout).unwrap().trim().to_owned()
}
fn spelled(path: &Path) -> String {
path.to_str().unwrap().replace('\\', "/")
}
struct Armed {
root: PathBuf,
first: String,
head: String,
/// A commit whose tree is missing: git asks the promisor for it.
promised: String,
}
/// A repository that starts a program of its own for a signed commit (`gpg`), an index read
/// (`fsmonitor`), a file diff (`extdiff`), a touched file (`filter`) and a missing object
/// (`promisor`). Each program leaves a file named after itself in `markers`. Other ways in
/// (textconv, process filters, merge drivers) are not armed here.
fn arm(root: &Path, markers: &Path) -> Armed {
std::fs::create_dir_all(root.join("src")).unwrap();
git(root, &["init", "-q", "-b", "main"]);
for (key, value) in [("user.name", "Test"), ("user.email", "[email protected]"), ("commit.gpgsign", "false")] {
git(root, &["config", key, value]);
}
std::fs::write(root.join("src/a.txt"), "one\ntwo\n").unwrap();
git(root, &["add", "."]);
git(root, &["commit", "-qm", "first"]);
let first = git(root, &["rev-parse", "HEAD"]);
std::fs::write(root.join("src/a.txt"), "one\ntwo\nthree\n").unwrap();
git(root, &["commit", "-qam", "second"]);
// HEAD again, carrying a signature header: `git log` then starts the verifier.
let body = git(root, &["cat-file", "commit", "HEAD"]);
let (headers, message) = body.split_once("\n\n").unwrap();
let signed = format!(
"{headers}\ngpgsig -----BEGIN PGP SIGNATURE-----\n \n iQEzBAABCAAdFiEEfake\n -----END PGP SIGNATURE-----\n\n{message}\n"
);
let mut child = Command::new("git")
.current_dir(root)
.args(["hash-object", "-t", "commit", "-w", "--stdin"])
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.unwrap();
std::io::Write::write_all(&mut child.stdin.take().unwrap(), signed.as_bytes()).unwrap();
let head = String::from_utf8(child.wait_with_output().unwrap().stdout).unwrap().trim().to_owned();
git(root, &["update-ref", "HEAD", &head]);
// A branch whose tip has a tree of its own, which is then dropped from the object database.
git(root, &["checkout", "-q", "-b", "promised", &first]);
std::fs::write(root.join("only-here.txt"), "promised\n").unwrap();
git(root, &["add", "."]);
git(root, &["commit", "-qm", "promised"]);
let promised = git(root, &["rev-parse", "HEAD"]);
let tree = git(root, &["rev-parse", "HEAD^{tree}"]);
git(root, &["checkout", "-q", "main"]);
std::fs::remove_file(root.join(".git/objects").join(&tree[..2]).join(&tree[2..])).unwrap();
let marker = |name: &str| spelled(&markers.join(name));
let script = |name: &str| {
let path = root.join(".git").join(format!("{name}.sh"));
std::fs::write(&path, format!("#!/bin/sh\necho x > '{}'\nexit 1\n", marker(name))).unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap();
}
spelled(&path)
};
let settings = [
("log.showSignature", "true".to_owned()),
("gpg.program", script("gpg")),
("core.fsmonitor", script("fsmonitor")),
("diff.external", script("extdiff")),
("filter.marker.clean", format!("echo x > '{}'; cat", marker("filter"))),
("extensions.partialClone", "origin".to_owned()),
("remote.origin.url", spelled(root)),
("remote.origin.promisor", "true".to_owned()),
("remote.origin.partialclonefilter", "blob:none".to_owned()),
("remote.origin.uploadpack", format!("echo x > '{}'; git-upload-pack", marker("promisor"))),
];
std::fs::write(root.join(".gitattributes"), "*.txt filter=marker\n").unwrap();
// Same length as the committed text: only reading the file tells them apart.
std::fs::write(root.join("src/a.txt"), "ONE\ntwo\nthree\n").unwrap();
for (key, value) in &settings {
git(root, &["config", key, value]);
}
Armed { root: root.to_path_buf(), first, head, promised }
}
fn ran(markers: &Path) -> Vec<String> {
let mut names: Vec<String> = std::fs::read_dir(markers)
.unwrap()
.flatten()
.map(|entry| entry.file_name().to_string_lossy().into_owned())
.collect();
names.sort();
names
}
fn clear(markers: &Path) {
let _ = std::fs::remove_dir_all(markers);
std::fs::create_dir_all(markers).unwrap();
}
type Call<'a> = (&'static str, &'static str, Pin<Box<dyn Future<Output = bool> + 'a>>);
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn restricted_vcs_commands_run_no_program_of_the_repository() {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let markers = base.join("markers");
clear(&markers);
let project = arm(&base.join("project"), &markers);
let external = arm(&base.join("external"), &markers);
let host = Host::new(HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("data"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(NullSink)),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
host.trust.decide(&project.root, false, false).unwrap();
clear(&markers);
assert!(!host.open_project("main", &project.root).unwrap().is_trusted());
let host = &host;
let w = "main";
let file = project.root.join("src/a.txt");
let root = || Some(project.root.clone());
let log = || Some(external.root.clone());
macro_rules! call {
($command:literal, $what:literal, $call:expr) => {
($command, $what, Box::pin(async { $call.await.is_ok() }) as Pin<Box<dyn Future<Output = bool> + '_>>)
};
}
macro_rules! sync {
($command:literal, $what:literal, $call:expr) => {
($command, $what, Box::pin(async { $call.is_ok() }) as Pin<Box<dyn Future<Output = bool> + '_>>)
};
}
let calls: Vec<Call> = vec![
call!("vcs_status", "", ops::vcs::status(host, w)),
sync!("vcs_roots", "", vcs::vcs_roots(host, w)),
call!("vcs_branch_info", "", vcs::vcs_branch_info(host, w, root())),
call!("vcs_line_hunks", "", vcs::vcs_line_hunks(host, w, file.clone())),
call!("vcs_file_at", "", vcs::vcs_file_at(host, w, file.clone(), None)),
call!("vcs_blame", "", vcs::vcs_blame(host, w, file.clone(), None, None)),
call!("vcs_block_history", "", vcs::vcs_block_history(host, w, RequestId(1), file.clone(), None, (0, 1))),
call!("vcs_log", "", vcs::vcs_log(host, w, root(), Default::default())),
call!("vcs_log_refs", "", vcs::vcs_log_refs(host, w, root())),
call!("vcs_log_users", "", vcs::vcs_log_users(host, w, root())),
call!("vcs_log_details", "", vcs::vcs_log_details(host, w, root(), project.head.clone())),
call!(
"vcs_log_details_batch",
"",
vcs::vcs_log_details_batch(
host,
w,
root(),
vec![project.head.clone(), project.first.clone()],
RequestId(2)
)
),
call!(
"vcs_revision_children",
"",
vcs::vcs_revision_children(host, w, root(), project.head.clone(), "src".into())
),
call!(
"vcs_revision_children",
"missing tree",
vcs::vcs_revision_children(host, w, root(), project.promised.clone(), String::new())
),
call!("vcs_revision_files", "", vcs::vcs_revision_files(host, w, root(), project.head.clone())),
call!("vcs_revision_files", "missing tree", vcs::vcs_revision_files(host, w, root(), project.promised.clone())),
call!("vcs_commit_files", "", vcs::vcs_commit_files(host, w, root(), project.head.clone())),
call!("vcs_commits_changes", "", vcs::vcs_commits_changes(host, w, root(), vec![project.head.clone()])),
call!("vcs_branches", "", vcs::vcs_branches(host, w, root())),
call!("vcs_stashes", "", vcs::vcs_stashes(host, w, root())),
call!("vcs_conflict_versions", "", vcs::vcs_conflict_versions(host, w, file.clone())),
call!("vcs_diff_text", "", vcs::vcs_diff_text(host, w, file.clone(), false)),
call!("vcs_config_get", "", vcs::vcs_config_get(host, w, root(), "user.name".into())),
call!("vcs_tags", "", vcs::vcs_tags(host, w, root())),
call!("vcs_remotes", "", vcs::vcs_remotes(host, w, root())),
call!("vcs_worktrees", "", vcs::vcs_worktrees(host, w, root())),
call!("vcs_diff_with_worktree", "", vcs::vcs_diff_with_worktree(host, w, root(), "HEAD".into())),
call!("vcs_compare_scope", "", vcs::vcs_compare_scope(host, w, vec![project.root.clone()])),
call!(
"vcs_compare_paths",
"",
vcs::vcs_compare_paths(host, w, vec![project.root.clone()], "HEAD".into(), None, RequestId(3))
),
call!(
"vcs_diff_revisions",
"",
vcs::vcs_diff_revisions(host, w, root(), project.first.clone(), project.head.clone())
),
call!(
"vcs_diff_numstat",
"",
vcs::vcs_diff_numstat(host, w, root(), project.first.clone(), project.head.clone(), Vec::new())
),
call!("vcs_merge_base", "", vcs::vcs_merge_base(host, w, root(), project.first.clone(), project.head.clone())),
call!("vcs_has_hook", "", vcs::vcs_has_hook(host, w, root(), "pre-push".into())),
// A worktree outside the project, opened for its history only.
call!("vcs_open_external_log", "", vcs::vcs_open_external_log(host, w, external.root.clone())),
call!("vcs_log", "external", vcs::vcs_log(host, w, log(), Default::default())),
call!("vcs_log_users", "external", vcs::vcs_log_users(host, w, log())),
call!("vcs_log_details", "external", vcs::vcs_log_details(host, w, log(), external.head.clone())),
call!(
"vcs_revision_children",
"external, missing tree",
vcs::vcs_revision_children(host, w, log(), external.promised.clone(), String::new())
),
call!("vcs_revision_files", "external", vcs::vcs_revision_files(host, w, log(), external.head.clone())),
call!("vcs_commit_files", "external", vcs::vcs_commit_files(host, w, log(), external.head.clone())),
// Last: it rewrites nothing of an existing repository, but it does touch it.
call!("vcs_init", "", vcs::vcs_init(host, w, project.root.clone())),
];
// The sweep covers what `trust.rs` allows: a newly allowed `vcs_*` command fails here until
// it is swept or listed as not touching the project's repository.
let source = include_str!("../src/trust.rs");
let start = source.find("pub fn restricted_command_allowed").unwrap();
let end = source.find("pub fn projectless_command_allowed").unwrap();
let allowed: BTreeSet<&str> = source[start..end].split('"').filter(|s| s.starts_with("vcs_")).collect();
let swept: BTreeSet<&str> = calls.iter().map(|(command, _, _)| *command).collect();
let known: BTreeSet<&str> = swept.iter().copied().chain(NOT_THE_PROJECT_REPOSITORY).collect();
assert!(allowed.len() > 30, "trust.rs was not read: {allowed:?}");
assert_eq!(allowed, known, "the sweep and Restricted Mode's allowed vcs commands differ");
for command in &known {
assert!(ide_host::trust::restricted_command_allowed(command), "{command} is not allowed in Restricted Mode");
}
let mut executed = Vec::new();
let mut failed = Vec::new();
let opening = ran(&markers);
if !opening.is_empty() {
executed.push(format!("open_project: {}", opening.join(", ")));
}
for (command, what, call) in calls {
clear(&markers);
let ok = call.await;
let name = if what.is_empty() { command.to_owned() } else { format!("{command} ({what})") };
let programs = ran(&markers);
if !programs.is_empty() {
executed.push(format!("{name}: {}", programs.join(", ")));
}
if !ok {
failed.push(name);
}
}
// Control: plain git, in the same repository, starts every one of the fixture's programs.
clear(&markers);
for args in [&["log", "-1", "--format=%H"][..], &["status", "--porcelain"], &["ls-tree", &project.promised]] {
let _ = Command::new("git").current_dir(&project.root).args(args).output().unwrap();
}
let _ = Command::new("git").current_dir(&project.root).args(["diff"]).output().unwrap();
assert_eq!(
ran(&markers),
["extdiff", "filter", "fsmonitor", "gpg", "promisor"],
"control: the fixture is not live"
);
assert!(executed.is_empty(), "Restricted Mode ran a program of the repository:\n{}", executed.join("\n"));
// Only the listings of the commit whose tree is gone have nothing to return.
assert_eq!(
failed,
[
"vcs_revision_children (missing tree)",
"vcs_revision_files (missing tree)",
"vcs_revision_children (external, missing tree)"
]
);
}
+121
Переглянути файл
@@ -105,6 +105,89 @@ pub fn environment_file(file: &Path, root: Option<&Path>, private: bool) -> IdeR
Ok(path)
}
/// Where `{{name}}` is defined in the environment files that apply to `file`: the key `name`
/// in each environment (and `$shared`) of the nearest private file, then of the nearest public
/// file — the lookup order. Positions are 0-based lines and UTF-16 columns of the key, quotes
/// included.
pub fn variable_definitions(file: &Path, root: Option<&Path>, name: &str) -> Vec<ide_core::Location> {
let dirs = search_dirs(Some(file), root);
let mut out = Vec::new();
for file_name in [PRIVATE_FILE, PUBLIC_FILE] {
let Some(path) = dirs.iter().map(|d| d.join(file_name)).find(|p| p.is_file()) else { continue };
let Ok(text) = ide_vfs::load_text(&path) else { continue };
for range in environment_keys(&text.text, name) {
out.push(ide_core::Location { path: path.clone().into(), range, uri: None });
}
}
out
}
/// The ranges of keys equal to `name` one level inside the top-level object (an
/// environment's variables), found by scanning the JSON text without building a value.
fn environment_keys(text: &str, name: &str) -> Vec<ide_core::Range> {
use ide_core::{Pos, Range};
let mut found = Vec::new();
let (mut line, mut col) = (0u32, 0u32);
let mut depth = 0usize;
let mut chars = text.char_indices().peekable();
// Whether the last string closed at depth 2 may be a key: set after `{` or `,`.
let mut expect_key = false;
while let Some((_, c)) = chars.next() {
match c {
'"' => {
let start = Pos::new(line, col);
col += 1;
let mut value = String::new();
let mut escaped = false;
for (_, c) in chars.by_ref() {
col += c.len_utf16() as u32;
if escaped {
value.push(match c {
'n' => '\n',
't' => '\t',
other => other,
});
escaped = false;
} else if c == '\\' {
escaped = true;
} else if c == '"' {
break;
} else {
value.push(c);
}
}
let end = Pos::new(line, col);
if depth == 2 && expect_key && value == name {
let mut rest = chars.clone();
while rest.peek().is_some_and(|(_, c)| c.is_whitespace()) {
rest.next();
}
if rest.peek().is_some_and(|(_, c)| *c == ':') {
found.push(Range { start, end });
}
}
expect_key = false;
continue;
}
'{' | '[' => {
depth += 1;
expect_key = c == '{';
}
'}' | ']' => depth = depth.saturating_sub(1),
',' => expect_key = true,
':' => expect_key = false,
'\n' => {
line += 1;
col = 0;
continue;
}
_ => {}
}
col += c.len_utf16() as u32;
}
found
}
impl Environments {
pub fn load(file: Option<&Path>, root: Option<&Path>) -> IdeResult<Self> {
let dirs = search_dirs(file, root);
@@ -221,6 +304,44 @@ impl Resolver<'_> {
mod tests {
use super::*;
#[test]
fn finds_variable_keys_in_every_environment_private_first() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
std::fs::create_dir_all(root.join("api")).unwrap();
std::fs::write(
root.join(PUBLIC_FILE),
"{\n \"$shared\": { \"host\": \"shared\" },\n \"dev\": {\n \"name\": \"host\",\n \"host\": \"localhost\",\n \"ssl\": { \"host\": \"nested\" }\n },\n \"prod\": { \"hostname\": \"x\", \"host\": \"example.com\" }\n}\n",
)
.unwrap();
std::fs::write(root.join("api").join(PRIVATE_FILE), "{ \"dev\": { \"😀\": 1, \"host\": \"secret\" } }")
.unwrap();
let file = root.join("api").join("requests.http");
let found = variable_definitions(&file, Some(root), "host");
let at: Vec<_> = found
.iter()
.map(|l| {
(
l.path.0.file_name().unwrap().to_string_lossy().into_owned(),
l.range.start.line,
l.range.start.col,
l.range.end.col,
)
})
.collect();
assert_eq!(
at,
vec![
(PRIVATE_FILE.to_owned(), 0, 20, 26),
(PUBLIC_FILE.to_owned(), 1, 15, 21),
(PUBLIC_FILE.to_owned(), 4, 4, 10),
(PUBLIC_FILE.to_owned(), 7, 29, 35),
],
"values and nested objects are not definitions"
);
assert!(variable_definitions(&file, Some(root), "missing").is_empty());
}
#[test]
fn merges_private_public_and_shared() {
let envs = Environments::from_json(
+16
Переглянути файл
@@ -55,6 +55,10 @@ pub struct LanguageConfig {
/// the first one whose root marker is found applies.
#[serde(default)]
pub framework_ui_modes: Vec<FrameworkUiMode>,
/// Files of this language in a framework's projects that their extension alone would
/// give to another language (Django templates are `.html` files in `templates`).
#[serde(default)]
pub framework_files: Vec<FrameworkFiles>,
#[serde(default)]
pub tree_sitter: Option<String>,
#[serde(default)]
@@ -90,6 +94,18 @@ pub struct FrameworkUiMode {
pub root_markers: Vec<String>,
}
/// Files in a framework's folder: a file with one of `extensions` below a folder named
/// `folder`, which has a root marker in a directory above it.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub struct FrameworkFiles {
pub folder: String,
/// Without the dot, compared ignoring case.
pub extensions: Vec<String>,
/// Matched like language servers' `root-markers`, e.g. `manage.py`.
pub root_markers: Vec<String>,
}
#[derive(Clone, Copy, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all(serialize = "camelCase", deserialize = "kebab-case"))]
pub struct IndentConfig {
+54 -7
Переглянути файл
@@ -22,6 +22,7 @@ const BUILTIN: &[(&str, &str)] = &[
("d", include_str!("../../../languages/d.toml")),
("dart", include_str!("../../../languages/dart.toml")),
("diff", include_str!("../../../languages/diff.toml")),
("django-html", include_str!("../../../languages/django-html.toml")),
("dockerfile", include_str!("../../../languages/dockerfile.toml")),
("dotenv", include_str!("../../../languages/dotenv.toml")),
("elixir", include_str!("../../../languages/elixir.toml")),
@@ -371,14 +372,13 @@ impl LanguageRegistry {
.map(|f| f.ui_mode.clone())
}
/// Creating or removing `path` can change a file's `framework_ui_mode`:
/// it is a framework's root marker.
/// Creating or removing `path` can change a file's `framework_ui_mode` or its
/// language (`framework-files`): it is a framework's root marker.
pub fn is_framework_marker(&self, path: &Path) -> bool {
self.state()
.languages
.iter()
.flat_map(|l| &l.framework_ui_modes)
.any(|f| f.root_markers.iter().any(|m| crate::markers::is_marker_path(path, m)))
let state = self.state();
let modes = state.languages.iter().flat_map(|l| &l.framework_ui_modes).map(|f| &f.root_markers);
let files = state.languages.iter().flat_map(|l| &l.framework_files).map(|f| &f.root_markers);
modes.chain(files).flatten().any(|m| crate::markers::is_marker_path(path, m))
}
/// Detects by Override File Type, exact file name, then file-name pattern
@@ -400,6 +400,9 @@ impl LanguageRegistry {
if let Some((_, i)) = t.name_globs.iter().find(|(p, _)| wildcard_match(p, file_name)) {
return languages[*i].id.clone();
}
if let Some(l) = languages.iter().find(|l| l.framework_files.iter().any(|f| is_framework_file(f, path))) {
return l.id.clone();
}
let lower = file_name.to_ascii_lowercase();
let mut rest = lower.as_str();
while let Some(dot) = rest.find('.') {
@@ -420,6 +423,19 @@ impl LanguageRegistry {
}
}
/// `path` is one of `files`: its extension is listed, and the nearest folder above it named
/// `files.folder` has a root marker in a directory above that folder.
fn is_framework_file(files: &crate::FrameworkFiles, path: &Path) -> bool {
let Some(ext) = path.extension().and_then(|e| e.to_str()) else { return false };
if !files.extensions.iter().any(|e| e.eq_ignore_ascii_case(ext)) {
return false;
}
path.ancestors()
.skip(1)
.find(|dir| dir.file_name().is_some_and(|name| name == files.folder.as_str()))
.is_some_and(|folder| crate::marked_root(None, folder, &files.root_markers).is_some())
}
/// `*` matches any (possibly empty) run of characters, `?` exactly one; everything else is literal.
fn wildcard_match(pattern: &str, name: &str) -> bool {
let p: Vec<char> = pattern.chars().collect();
@@ -497,6 +513,7 @@ mod tests {
"Brainfuck",
"Cobol",
"C#",
"Django Template",
"Clojure",
"ClojureScript",
"Closure Stylesheets (GSS)",
@@ -801,6 +818,36 @@ mod tests {
assert!(!reg.is_framework_marker(&root.join("apps/admin/package.json")));
}
#[test]
fn django_projects_make_html_in_templates_folders_templates() {
let reg = reg();
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let detect = |file: &str| reg.detect(&root.join(file), None).as_str().to_owned();
std::fs::create_dir_all(root.join("src/orders/templates/orders/email")).unwrap();
std::fs::create_dir_all(root.join("site/templates")).unwrap();
std::fs::write(root.join("src/manage.py"), "").unwrap();
assert_eq!(detect("src/orders/templates/orders/detail.html"), "django-html");
assert_eq!(detect("src/orders/templates/orders/email/body.TXT"), "django-html");
assert_eq!(detect("src/templates/base.html"), "django-html");
assert_eq!(detect("src/orders/templates/orders/data.json"), "json");
assert_eq!(detect("src/orders/static/orders/page.html"), "html");
// No `manage.py` above this `templates` folder.
assert_eq!(detect("site/templates/index.html"), "html");
assert_eq!(detect("page.djhtml"), "django-html");
assert!(reg.is_framework_marker(&root.join("src/manage.py")));
}
#[test]
fn the_django_fixture_has_templates_and_a_page() {
let reg = reg();
let fixture = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../fixtures/django-app");
let detect = |file: &str| reg.detect(&fixture.join(file), None).as_str().to_owned();
assert_eq!(detect("shop/templates/shop/product_list.html"), "django-html");
assert_eq!(detect("shop/templates/shop/email/order.txt"), "django-html");
assert_eq!(detect("pages/promo.html"), "html");
}
#[test]
fn markers_outside_the_project_do_not_make_angular_templates() {
let reg = reg();
+1
Переглянути файл
@@ -29,4 +29,5 @@ zip.workspace = true
[dev-dependencies]
tempfile.workspace = true
encoding_rs.workspace = true
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "test-util"] }
+223
Переглянути файл
@@ -0,0 +1,223 @@
//! go.mod as GoLand shows it: available upgrades, vulnerabilities and deprecated modules are
//! reported as soon as the file is opened. gopls reports upgrades only from its "Check for
//! upgrades" code lens, so that runs once per go.mod and session; its results (and the
//! vulnerabilities from the `vulncheck` setting) come back as gopls diagnostics with quick fixes.
//! gopls never reports a deprecated module, so `go list -m -u` does, on open and on save.
use crate::LspManager;
use crate::server::Server;
use ide_core::{DocId, Pos, Range};
use serde::Deserialize;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
pub(crate) const CHECK_UPGRADES: &str = "gopls.check_upgrades";
/// gopls answers code lenses once it has loaded the module.
const ATTEMPTS: u32 = 10;
const RETRY: Duration = Duration::from_secs(2);
/// Producer of the deprecation diagnostics (one full set per go.mod).
const DEPRECATIONS: &str = "go-mod-deprecations";
fn is_go_mod(path: &Path) -> bool {
path.file_name().is_some_and(|n| n == "go.mod")
}
impl LspManager {
pub(crate) async fn check_go_mod(&self, doc: DocId, path: &Path, server: Arc<Server>) {
if server.name != "gopls" || !is_go_mod(path) {
return;
}
self.report_deprecated_modules(doc, &server).await;
if !self.go_mod_checked.lock().insert(path.to_owned()) {
return;
}
for attempt in 0..ATTEMPTS {
if attempt > 0 {
tokio::time::sleep(RETRY).await;
}
if !self.docs.lock().contains_key(&doc) {
break;
}
let Ok(lenses) = self.code_lens(doc).await else { continue };
let Some(command) = lenses.into_iter().filter_map(|l| l.command).find(|c| c.command == CHECK_UPGRADES)
else {
continue;
};
if let Err(e) = self.execute_command(doc, command).await {
tracing::debug!("gopls: {CHECK_UPGRADES} for {}: {}", path.display(), e.message);
}
return;
}
// Not checked after all (closed, or gopls never offered the lens): a later open retries.
self.go_mod_checked.lock().remove(path);
}
/// After a save of a go.mod: its requirements may have changed.
pub(crate) fn go_mod_saved(&self, doc: DocId) {
let server = {
let docs = self.docs.lock();
let Some(d) = docs.get(&doc).filter(|d| is_go_mod(&d.path)) else { return };
let Some(a) = d.servers.iter().find(|a| a.server.name == "gopls") else { return };
a.server.clone()
};
let Some(me) = self.me.upgrade() else { return };
self.runtime.spawn(async move { me.report_deprecated_modules(doc, &server).await });
}
async fn report_deprecated_modules(&self, doc: DocId, server: &Server) {
let Some((path, text)) = self.docs.lock().get(&doc).map(|d| (d.path.clone(), d.text.as_str().to_owned()))
else {
return;
};
let required = required_modules(&text);
if required.is_empty() {
return;
}
let config = server.effective_config();
let search = crate::launch::server_search_path(&config);
let Some(dir) = path.parent().map(Path::to_path_buf) else { return };
let modules: Vec<String> = required.iter().map(|r| r.path.clone()).collect();
let listed = tokio::task::spawn_blocking(move || list_deprecations(&dir, &search, &modules)).await;
let Ok(Some(deprecated)) = listed else { return };
// The document changed meanwhile: its own save runs again.
if self.docs.lock().get(&doc).is_none_or(|d| d.text.as_str() != text) {
return;
}
let items = required
.into_iter()
.filter_map(|r| {
let message = deprecated.iter().find(|(p, _)| *p == r.path)?.1.trim();
Some(ide_core::Diagnostic {
range: r.range,
severity: ide_core::Severity::Warning,
message: format!("{} is deprecated: {message}", r.path),
source: Some("go".into()),
code: None,
unnecessary: false,
deprecated: true,
server: Some("gopls".into()),
})
})
.collect();
self.sink().emit(ide_core::IdeEvent::Diagnostics { path: path.into(), source: DEPRECATIONS.into(), items });
}
}
#[derive(Debug, PartialEq)]
struct Required {
path: String,
/// The module path in go.mod (UTF-16 columns).
range: Range,
}
/// Modules of go.mod's `require` directives, single-line or in a block.
fn required_modules(text: &str) -> Vec<Required> {
let mut out = Vec::new();
let mut in_block = false;
for (line, raw) in text.lines().enumerate() {
let code = raw.split("//").next().unwrap_or("");
let trimmed = code.trim_start();
let entry = if in_block {
if trimmed.starts_with(')') {
in_block = false;
continue;
}
trimmed
} else if let Some(rest) = trimmed.strip_prefix("require").filter(|r| r.starts_with(char::is_whitespace)) {
let rest = rest.trim_start();
if rest.starts_with('(') {
in_block = true;
continue;
}
rest
} else {
continue;
};
let mut words = entry.split_whitespace();
let (Some(module), Some(_version)) = (words.next(), words.next()) else { continue };
let module = module.trim_matches('"');
let Some(start) = code.find(module) else { continue };
let col = |byte: usize| code[..byte].encode_utf16().count() as u32;
let line = line as u32;
out.push(Required {
path: module.to_owned(),
range: Range { start: Pos { line, col: col(start) }, end: Pos { line, col: col(start + module.len()) } },
});
}
out
}
#[derive(Deserialize)]
struct ListedModule {
#[serde(rename = "Path")]
path: String,
#[serde(rename = "Deprecated", default)]
deprecated: Option<String>,
}
/// `(module, message)` of each deprecated module in `modules`; `None` when `go` failed.
fn list_deprecations(dir: &Path, search: &std::ffi::OsStr, modules: &[String]) -> Option<Vec<(String, String)>> {
let go: PathBuf = ide_core::tool_path::find_command("go", search, dir)?;
let output = ide_core::process::prepare(&mut std::process::Command::new(go))
.args(["list", "-m", "-u", "-e", "-json"])
.args(modules)
.current_dir(dir)
.env("PATH", search)
.stdin(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.output()
.ok()?;
if output.stdout.is_empty() {
tracing::debug!("go list -m -u in {}: {}", dir.display(), output.status);
return None;
}
Some(parse_deprecations(&output.stdout))
}
fn parse_deprecations(json: &[u8]) -> Vec<(String, String)> {
serde_json::Deserializer::from_slice(json)
.into_iter::<ListedModule>()
.map_while(Result::ok)
.filter_map(|m| Some((m.path, m.deprecated.filter(|d| !d.trim().is_empty())?)))
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
fn at(line: u32, from: u32, to: u32) -> Range {
Range { start: Pos { line, col: from }, end: Pos { line, col: to } }
}
#[test]
fn finds_required_modules_in_blocks_and_single_lines() {
let text = "module example.com/m\n\ngo 1.24\n\nrequire github.com/a/b v1.0.0 // indirect\n\
require (\n\tgithub.com/c/d v1.2.0\n\t\"github.com/e/é\" v0.1.0 // indirect\n)\n\
// require github.com/x/y v1.0.0\nreplace github.com/c/d => ../d\ntool golang.org/x/tools/cmd/stringer\n";
assert_eq!(
required_modules(text),
vec![
Required { path: "github.com/a/b".into(), range: at(4, 8, 22) },
Required { path: "github.com/c/d".into(), range: at(6, 1, 15) },
Required { path: "github.com/e/é".into(), range: at(7, 2, 16) },
]
);
}
#[test]
fn reads_deprecations_from_the_go_list_stream() {
let json = br#"{"Path": "github.com/golang/protobuf", "Version": "v1.5.4",
"Deprecated": "Use the \"google.golang.org/protobuf\" module instead."}
{"Path": "github.com/sergi/go-diff", "Version": "v1.3.1"}
{"Path": "example.com/blank", "Deprecated": " "}"#;
assert_eq!(
parse_deprecations(json),
vec![(
"github.com/golang/protobuf".to_owned(),
"Use the \"google.golang.org/protobuf\" module instead.".to_owned()
)]
);
}
}
+22 -7
Переглянути файл
@@ -231,14 +231,9 @@ impl ClientHandler {
};
let Some(path) = uri_to_path(params.uri.as_str()) else { return };
if self.server.name == "jdtls"
&& let Some(body) = crate::jdtls::build_plan_failure(&path, &self.server.root, &params.diagnostics)
&& mgr.first_message(&self.server.name, &body)
&& let Some(failure) = crate::jdtls::build_plan_failure(&path, &self.server.root, &params.diagnostics)
{
mgr.sink().emit(ide_core::IdeEvent::Notify {
level: ide_core::event::NotifyLevel::Error,
title: "Maven project could not be configured".into(),
body: Some(body),
});
self.build_plan_failure(mgr, failure);
}
if !mgr.accepts_injected_diagnostics(&path, params.version) {
return;
@@ -254,6 +249,17 @@ impl ClientHandler {
mgr.emit_diagnostics(&self.server, path, shown, self.encoding());
}
/// Once per unresolved plugin, whether m2e marked a `pom.xml` or only logged it.
fn build_plan_failure(&self, mgr: &LspManager, (plugin, body): (String, String)) {
if mgr.first_message(&self.server.name, &format!("build plan: {plugin}")) {
mgr.sink().emit(ide_core::IdeEvent::Notify {
level: ide_core::event::NotifyLevel::Error,
title: "Maven project could not be configured".into(),
body: Some(body),
});
}
}
fn show_message(&self, mgr: &LspManager, params: &Value) {
let text = params.get("message").and_then(Value::as_str).unwrap_or_default().to_owned();
// rust-analyzer built from source reports its own slow main-loop turns as errors, meant for
@@ -375,6 +381,12 @@ impl Handler for ClientHandler {
{
self.show_message(&mgr, &json!({ "type": 1, "message": text.lines().next().unwrap_or(text) }));
}
if self.server.name == "jdtls"
&& params.get("type").and_then(Value::as_u64) == Some(1)
&& let Some(failure) = crate::jdtls::logged_build_plan_failure(text)
{
self.build_plan_failure(&mgr, failure);
}
match params.get("type").and_then(Value::as_u64) {
Some(1) if self.server.name == "jdtls" && text.contains(crate::jdtls::reserved_names::IMPORT_FAILURE) => {
tracing::warn!("[{}] {text}", self.server.name);
@@ -386,6 +398,9 @@ impl Handler for ClientHandler {
}
}
"tsserver/request" => mgr.forward_tsserver_requests(self.server.clone(), self.generation, params),
"intellij/workspaceImportState" if self.server.name == crate::kotlin_lsp::KOTLIN_LSP => {
crate::kotlin_lsp::record_import_state(&self.server, self.generation, &params)
}
"language/status" if self.server.name == "jdtls" && params["type"] == "ServiceReady" => {
mgr.java_import_ready(self.server.clone(), self.generation);
}
+71 -18
Переглянути файл
@@ -16,7 +16,7 @@ use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant};
const SERVER: &str = "jdtls";
pub(crate) const SERVER: &str = "jdtls";
pub(crate) mod configuration;
mod formatter;
@@ -1091,27 +1091,52 @@ impl LspManager {
}
}
/// m2e's error on a `pom.xml` whose build plan can't be computed: a build plugin (or one of its
/// dependencies) doesn't resolve, e.g. a version a company mirror lacks. The module is then left
/// without its Java configuration and navigation finds nothing, with only this marker to tell.
pub(crate) fn build_plan_failure(path: &Path, root: &Path, diagnostics: &[lsp_types::Diagnostic]) -> Option<String> {
if path.file_name()? != "pom.xml" {
return None;
}
let message = diagnostics
.iter()
.filter(|d| d.severity == Some(lsp_types::DiagnosticSeverity::ERROR))
.find_map(|d| d.message.split_once("Could not calculate build plan:").map(|(_, m)| m))?;
/// m2e couldn't compute a build plan because a build plugin (or one of its dependencies) doesn't
/// resolve, e.g. a default version a company mirror lacks: "Plugin g:a:v or one of its dependencies
/// could not be resolved", without the repository details that follow.
fn unresolved_build_plugin(message: &str) -> Option<&str> {
let (_, message) = message.split_once("Could not calculate build plan:")?;
const UNRESOLVED: &str = "could not be resolved";
let reason = match message.find(UNRESOLVED) {
Some(end) => &message[..end + UNRESOLVED.len()],
None => message.lines().next().unwrap_or(message),
};
Some(format!(
"{}: {}. Java navigation and completion don't work in this module until Maven resolves it: make it available in your Maven repository or mirror, or set a version the repository has in pom.xml.",
Some(reason.trim().trim_end_matches(':'))
}
const BUILD_PLAN_FIX: &str =
"make it available in your Maven repository or mirror, or set a version the repository has in pom.xml.";
/// The error m2e puts on a `pom.xml` whose build plan can't be computed, as (plugin, notification
/// body). A plugin the Java configuration needs (`maven-compiler-plugin`) leaves the module without
/// it, so navigation finds nothing, with only this marker to tell.
pub(crate) fn build_plan_failure(
path: &Path,
root: &Path,
diagnostics: &[lsp_types::Diagnostic],
) -> Option<(String, String)> {
if path.file_name()? != "pom.xml" {
return None;
}
let reason = diagnostics
.iter()
.filter(|d| d.severity == Some(lsp_types::DiagnosticSeverity::ERROR))
.find_map(|d| unresolved_build_plugin(&d.message))?;
let body = format!(
"{}: {reason}. Java navigation and completion may not work in this module until Maven resolves it: {BUILD_PLAN_FIX}",
path.strip_prefix(root).unwrap_or(path).display(),
reason.trim().trim_end_matches(':')
))
);
Some((reason.to_owned(), body))
}
/// The same failure when m2e only logs it: an unresolved plugin of the build itself (such as
/// `maven-resources-plugin`) fails the import's last step and every build, but marks no file.
pub(crate) fn logged_build_plan_failure(message: &str) -> Option<(String, String)> {
let reason = unresolved_build_plugin(message.lines().take(2).collect::<Vec<_>>().join("\n").as_str())?.to_owned();
let body = format!(
"{reason}. Maven can't build this project in the IDE, and Java navigation and completion may be incomplete until it resolves: {BUILD_PLAN_FIX}"
);
Some((reason, body))
}
impl LspManager {
@@ -1158,13 +1183,41 @@ mod tests {
"Plugin execution not covered by lifecycle configuration",
);
let (root, pom) = (Path::new("/w"), Path::new("/w/app/pom.xml"));
let body = build_plan_failure(pom, root, &[info.clone(), error.clone()]).unwrap();
assert!(body.starts_with("app/pom.xml: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.13.0 or one of its dependencies could not be resolved. Java navigation"), "{body}");
let (plugin, body) = build_plan_failure(pom, root, &[info.clone(), error.clone()]).unwrap();
assert_eq!(
plugin,
"Plugin org.apache.maven.plugins:maven-compiler-plugin:3.13.0 or one of its dependencies could not be resolved"
);
assert!(body.starts_with(&format!("app/pom.xml: {plugin}. Java navigation")), "{body}");
assert!(!body.contains("nexus.corp"));
assert_eq!(build_plan_failure(pom, root, &[info]), None);
assert_eq!(build_plan_failure(Path::new("/w/app/src/Main.java"), root, &[error]), None);
}
#[test]
fn a_build_plan_failure_only_logged_is_explained() {
// m2e's log when `maven-resources-plugin` is missing: no marker on pom.xml.
let log = "Nov 11, 2026, 9:29:04 PM Failed to configure some Maven project(s)\n\
org.eclipse.core.runtime.CoreException: Could not calculate build plan: Plugin org.apache.maven.plugins:maven-resources-plugin:3.4.0 or one of its dependencies could not be resolved:\n\
\tThe following artifacts could not be resolved: org.apache.maven.plugins:maven-resources-plugin:pom:3.4.0 (absent): Could not transfer artifact from/to central (https://nexus.corp/repository/public)\n\
\tat org.eclipse.m2e.core.internal.project.registry.MavenProjectFacade.lambda$5(MavenProjectFacade.java:550)\n\
Caused by: org.apache.maven.plugin.PluginResolutionException: Could not calculate build plan: elsewhere";
let (plugin, body) = logged_build_plan_failure(log).unwrap();
assert_eq!(
plugin,
"Plugin org.apache.maven.plugins:maven-resources-plugin:3.4.0 or one of its dependencies could not be resolved"
);
assert!(body.starts_with(&format!("{plugin}. Maven can't build this project")), "{body}");
assert!(!body.contains("nexus.corp"));
// Only the message itself counts, not a cause deep in a stack trace.
assert_eq!(
logged_build_plan_failure(
"Errors occurred during the build.\n\tat x\n\tat y\nCaused by: Could not calculate build plan: z"
),
None
);
}
#[test]
fn source_settings_preserve_java_defaults_and_library_resolution() {
let mut cfg = java_config();
+195
Переглянути файл
@@ -0,0 +1,195 @@
//! kotlin-lsp's built-in debug adapter: `start_debug_server` once the server has imported
//! the project (breakpoints set before that are "verified" but never hit).
use crate::LspManager;
use crate::server::Server;
use ide_core::event::ServerStatus;
use ide_core::{CancellationToken, LanguageId};
use serde_json::{Value, json};
use std::path::{Path, PathBuf};
use std::time::Duration;
/// The server's name in `languages/kotlin.toml`.
pub const KOTLIN_LSP: &str = "kotlin-lsp";
/// A cold start imports the Gradle model before answering `initialize`.
const READY_TIMEOUT: Duration = Duration::from_secs(240);
const COMMAND_TIMEOUT: Duration = Duration::from_secs(120);
/// The last `intellij/workspaceImportState` of the current instance.
#[derive(Clone, Debug, PartialEq)]
pub(crate) struct ImportState {
pub finished: bool,
/// Workspace folders whose import did not succeed.
pub failed: Vec<PathBuf>,
}
#[derive(Clone, Debug, PartialEq)]
pub struct KotlinDebugServer {
/// Local TCP port of the DAP server.
pub port: u16,
/// Root of the kotlin-lsp instance that serves it.
pub root: PathBuf,
}
/// Why kotlin-lsp cannot debug the module, for the fallback notice.
#[derive(Clone, Debug, PartialEq)]
pub enum KotlinDebugUnavailable {
/// No `kotlin-lsp` among the module's servers (disabled in settings, another server chosen).
NotConfigured,
NotInstalled,
/// The status message, else the status.
NotStarted(String),
/// The import did not finish in time, or the wait was cancelled.
StillImporting,
ImportFailed,
Failed(String),
}
/// Records `intellij/workspaceImportState` from instance `generation`. Completion is this
/// notification, not "no `$/progress` left": kotlin-lsp reports "Indexing" every few seconds.
pub(crate) fn record_import_state(server: &Server, generation: u64, params: &Value) {
let failed = params["folders"]
.as_array()
.into_iter()
.flatten()
.filter(|folder| folder["status"] != "SUCCESS")
.filter_map(|folder| folder["folderUri"].as_str().and_then(crate::uri::uri_to_path))
.collect();
server.set_import_state(generation, ImportState { finished: params["phase"] == "FINISHED", failed });
}
fn importing(server: &Server) -> bool {
server.progress.lock().values().any(|p| p.title.contains("Importing"))
}
impl LspManager {
/// Starts kotlin-lsp's debug server for `path` (a module directory or source file) once
/// the project is imported, starting kotlin-lsp if needed. `import_wait` bounds the wait
/// for the import after the server is up; cancelling `cancel` ends it as
/// [`KotlinDebugUnavailable::StillImporting`].
pub async fn kotlin_debug_server(
&self,
path: &Path,
import_wait: Duration,
cancel: &CancellationToken,
) -> Result<KotlinDebugServer, KotlinDebugUnavailable> {
let server = self
.servers_for(&LanguageId::new("kotlin"), path)
.into_iter()
.find(|s| s.name == KOTLIN_LSP)
.ok_or(KotlinDebugUnavailable::NotConfigured)?;
self.ensure_started(&server);
let ready = tokio::select! {
biased;
() = cancel.cancelled() => return Err(KotlinDebugUnavailable::StillImporting),
ready = server.wait_ready(READY_TIMEOUT) => ready,
};
if ready.is_none() {
let status = server.status();
return Err(match status {
ServerStatus::Missing => KotlinDebugUnavailable::NotInstalled,
ServerStatus::Misconfigured | ServerStatus::Crashed => {
KotlinDebugUnavailable::NotStarted(server.status_message().unwrap_or_else(|| format!("{status:?}")))
}
_ => KotlinDebugUnavailable::NotStarted(format!("{status:?}")),
});
}
tokio::select! {
biased;
() = cancel.cancelled() => return Err(KotlinDebugUnavailable::StillImporting),
imported = tokio::time::timeout(import_wait, wait_for_import(&server, path)) => {
imported.unwrap_or(Err(KotlinDebugUnavailable::StillImporting))?
}
}
// The instance that finished importing, also when it replaced the one waited for.
let Some(run) = server.running() else {
return Err(KotlinDebugUnavailable::NotStarted(format!("{:?}", server.status())));
};
let params =
json!({ "command": "start_debug_server", "arguments": [crate::uri::path_to_uri_string(&server.root)] });
let request = run.peer.request::<Value, Value>("workspace/executeCommand", params, Some(cancel.clone()));
let port = match tokio::time::timeout(COMMAND_TIMEOUT, request).await {
Ok(Ok(port)) => port,
Ok(Err(e)) => return Err(KotlinDebugUnavailable::Failed(e.to_string())),
Err(_) => {
return Err(KotlinDebugUnavailable::Failed(format!(
"start_debug_server timed out after {} s",
COMMAND_TIMEOUT.as_secs()
)));
}
};
match port.as_u64().and_then(|p| u16::try_from(p).ok()).filter(|p| *p != 0) {
Some(port) => Ok(KotlinDebugServer { port, root: server.root.clone() }),
None => Err(KotlinDebugUnavailable::Failed(format!("unexpected debug server port: {port}"))),
}
}
}
/// Waits until the current instance reports its import finished and its "Importing" task
/// ended, which arrive in either order.
async fn wait_for_import(server: &Server, path: &Path) -> Result<(), KotlinDebugUnavailable> {
let mut state = server.import_state.subscribe();
loop {
// Created before the check so a progress end in between still wakes us.
let progress_ended = server.phase_notified();
let current = state.borrow_and_update().clone();
if let Some(current) = current
&& current.finished
&& !importing(server)
{
if current.failed.iter().any(|dir| canonical(path).starts_with(canonical(dir))) {
return Err(KotlinDebugUnavailable::ImportFailed);
}
return Ok(());
}
tokio::select! {
_ = state.changed() => {}
() = progress_ended => {}
}
}
}
/// `path` with its deepest existing ancestor resolved, so a symlinked root (or `/tmp` vs
/// `/private/tmp` on macOS) and the folder URIs kotlin-lsp reports name one directory alike.
fn canonical(path: &Path) -> PathBuf {
for ancestor in path.ancestors() {
if let Ok(real) = dunce::canonicalize(ancestor) {
let rest = path.strip_prefix(ancestor).unwrap_or(Path::new(""));
return if rest.as_os_str().is_empty() { real } else { real.join(rest) };
}
}
path.to_path_buf()
}
#[cfg(test)]
mod tests {
use super::*;
use ide_core::{IdeEvent, SharedSink};
use std::sync::Arc;
#[test]
fn an_instance_that_was_replaced_does_not_record_its_import() {
let cfg = toml::from_str("name = \"kotlin-lsp\"\ncommand = \"kotlin-lsp\"").unwrap();
let server = Server::new(cfg, PathBuf::from("/p"), LanguageId::new("kotlin"));
let finished = json!({ "phase": "FINISHED", "folders": [
{ "folderUri": "file:///p/", "status": "SUCCESS" },
{ "folderUri": "file:///p/lib/", "status": "FAILED" },
]});
let old = server.begin_start().unwrap();
record_import_state(&server, old, &json!({ "phase": "IN_PROGRESS", "folders": [] }));
assert_eq!(*server.import_state.borrow(), Some(ImportState { finished: false, failed: vec![] }));
record_import_state(&server, old, &finished);
let done = Some(ImportState { finished: true, failed: vec![PathBuf::from("/p/lib")] });
assert_eq!(*server.import_state.borrow(), done);
let sink: SharedSink = Arc::new(|_: IdeEvent| {});
server.begin_stop(&sink, false);
let new = server.begin_start().unwrap();
assert_eq!(*server.import_state.borrow(), None);
// The dying instance's FINISHED arrives after the restart.
record_import_state(&server, old, &finished);
assert_eq!(*server.import_state.borrow(), None);
record_import_state(&server, new, &finished);
assert_eq!(*server.import_state.borrow(), done);
}
}
+1 -1
Переглянути файл
@@ -146,7 +146,7 @@ fn required_env_set(config: &LanguageServerConfig) -> bool {
}
/// A configured `PATH` (e.g. a selected toolchain prepended) also resolves the binary.
fn server_search_path(config: &LanguageServerConfig) -> OsString {
pub(crate) fn server_search_path(config: &LanguageServerConfig) -> OsString {
config.env.get("PATH").map(OsString::from).unwrap_or_else(search_path)
}
+259 -28
Переглянути файл
@@ -15,11 +15,13 @@ mod convert;
mod external_project;
mod file_ops;
mod file_watch;
mod go_mod;
mod go_scope;
mod handler;
mod injections;
mod inspect;
pub mod jdtls;
mod kotlin_lsp;
mod launch;
mod library_source;
mod loading;
@@ -38,6 +40,7 @@ mod vue_navigation;
pub use injections::{InjectedRange, InjectionProvider, MAX_INJECTED_DOCUMENT_BYTES};
pub use inspect::{FileInspection, InspectHost, InspectOptions};
pub use jdtls::jvm_inspect::JdtJvmInspector;
pub use kotlin_lsp::{KotlinDebugServer, KotlinDebugUnavailable};
pub use launch::{LaunchError, Launcher, ProcessLauncher};
pub use scope::{CandidateFiles, CandidateFn, CandidateQuery, MAX_CANDIDATES};
pub use uri::{is_virtual_uri, path_to_uri_string, uri_target, uri_to_path};
@@ -219,6 +222,8 @@ pub struct LspManager {
configuration_cache: Arc<Mutex<jdtls::configuration::ConfigurationCache>>,
/// `window/showMessage` texts already surfaced; servers repeat them on every restart/reconfigure.
shown_messages: Mutex<HashSet<(String, String)>>,
/// go.mod files whose upgrades gopls has been asked to check this session.
go_mod_checked: Mutex<HashSet<PathBuf>>,
/// Virtual (non-`file:`) URIs a server returned, by the server that owns them.
virtual_origins: Mutex<HashMap<String, ide_core::ServerId>>,
/// Contents fetched for virtual URIs.
@@ -317,9 +322,48 @@ impl<T> Outcome<T> {
}
}
/// How often a rename the server withdrew (`ContentModified`) is asked, and the pause between.
const RENAME_ATTEMPTS: u32 = 5;
const RENAME_RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(300);
/// Implementation requests of one line-marker pass in flight at once.
const IMPLEMENTATION_CONCURRENCY: usize = 4;
/// How a virtual document's content was decoded, and whether it can be
/// re-decoded ("Reload with Encoding").
struct Decoding {
encoding: String,
has_bom: bool,
can_reload: bool,
}
impl Decoding {
/// Text the server decoded itself: no raw bytes here to re-read.
fn server_text() -> Self {
Decoding { encoding: "utf-8".into(), has_bom: false, can_reload: false }
}
}
/// jdtls' source attachment of a `jdt:` class (`java.project.resolveSourceAttachment`).
struct SourceAttachment {
path: Option<String>,
encoding: Option<String>,
can_edit_encoding: bool,
}
/// Reads a `jar:` document's source: an entry as is, a class from the sources
/// jar next to its binary. `encoding` decodes with it instead of detecting.
fn read_jar_source(uri: &str, encoding: Option<&str>) -> Option<(String, ide_vfs::LoadedText)> {
match library_source::parse(uri) {
Some(e) if !e.entry.ends_with(".class") => {
library_source::read_entry(&e.jar, &e.entry, encoding).map(|loaded| (e.entry, loaded))
}
Some(_) => library_source::class_source(uri, encoding)
.and_then(|(source, loaded)| Some((library_source::parse(&source)?.entry, loaded))),
None => None,
}
}
/// Library classes in `jar:` targets (the official Kotlin LSP) go to their source
/// in the artifact's sources jar when it is cached, at the declaration of the
/// name navigated from; [`LspManager::virtual_document`] serves it.
@@ -334,8 +378,9 @@ async fn library_sources(snap: &DocSnap, pos: Pos, locs: Vec<Location>) -> Vec<L
let Some(uri) = loc.uri.as_deref().filter(|u| u.starts_with("jar:") && u.ends_with(".class")) else {
return loc;
};
let Some((source, text)) = library_source::class_source(uri) else { return loc };
let range = name.as_deref().map(|n| library_source::declaration_range(&text, n)).unwrap_or_default();
let Some((source, loaded)) = library_source::class_source(uri, None) else { return loc };
let range =
name.as_deref().map(|n| library_source::declaration_range(&loaded.text, n)).unwrap_or_default();
Location { path: source.clone().into(), range, uri: Some(source) }
})
.collect()
@@ -427,6 +472,7 @@ impl LspManager {
debug_servers: Mutex::new(HashMap::new()),
configuration_cache: Arc::new(Mutex::new(Default::default())),
shown_messages: Mutex::new(HashSet::new()),
go_mod_checked: Mutex::new(HashSet::new()),
virtual_origins: Mutex::new(HashMap::new()),
virtual_docs: Mutex::new(HashMap::new()),
injections: injections::State::default(),
@@ -1026,6 +1072,7 @@ impl LspManager {
servers: Vec<Arc<Server>>,
) {
let canonical = dunce::canonicalize(&path).unwrap_or_else(|_| path.clone());
let canonical_path = canonical.clone();
let compose_detected = language.as_str() == "yaml" && compose::is_content_detected_compose_file(&path, text);
let mut state = DocState {
path,
@@ -1056,6 +1103,13 @@ impl LspManager {
self.schedule_pull(doc, None, Duration::ZERO);
for s in &servers {
self.ensure_started(s);
// Waits for a server still starting, so it is not tied to `running()` below.
if s.name == "gopls"
&& let Some(me) = self.me.upgrade()
{
let (path, server) = (canonical_path.clone(), s.clone());
self.runtime.spawn(async move { me.check_go_mod(doc, &path, server).await });
}
if s.running().is_some()
&& let Some(me) = self.me.upgrade()
{
@@ -1197,6 +1251,7 @@ impl LspManager {
drop(docs);
self.schedule_pull(doc, None, Duration::ZERO);
self.pull_dependents(doc);
self.go_mod_saved(doc);
}
pub fn did_close(&self, doc: DocId) {
@@ -2374,8 +2429,25 @@ impl LspManager {
let query = || async {
let mut params = snap.pos_params(t, pos);
params["newName"] = json!(new_name);
let r: Option<lsp::WorkspaceEdit> =
self.call(t, "textDocument/rename", params, Some(cancel), LONG_TIMEOUT).await?;
// A server still analysing an edit withdraws the rename (`ContentModified`, which
// arrives as a cancellation). The user asked for it: ask again shortly instead of
// dropping it without a word, and say why if it keeps being withdrawn.
let mut attempt = 0;
let r: Option<lsp::WorkspaceEdit> = loop {
match self.call(t, "textDocument/rename", params.clone(), Some(cancel), LONG_TIMEOUT).await {
Err(e) if e.kind == ide_core::error::ErrorKind::Cancelled && !cancel.is_cancelled() => {
attempt += 1;
if attempt >= RENAME_ATTEMPTS {
return Err(IdeError::invalid(format!(
"{} was still analysing a change and withdrew the rename; try again",
t.server.name
)));
}
tokio::time::sleep(RENAME_RETRY_DELAY).await;
}
r => break r?,
}
};
Ok(r.map(|e| convert::workspace_edit(&self.conv(t.enc()), e)).unwrap_or_default())
};
// The Angular server renames in the file's own project only, whatever is open.
@@ -2997,8 +3069,9 @@ impl LspManager {
let original = uri.to_owned();
let archive = sources.clone();
let found = tokio::task::spawn_blocking(move || {
library_source::source_in_archive(&original, &archive).map(|(uri, text)| {
let range = name.as_deref().map(|n| library_source::declaration_range(&text, n)).unwrap_or_default();
library_source::source_in_archive(&original, &archive, None).map(|(uri, loaded)| {
let range =
name.as_deref().map(|n| library_source::declaration_range(&loaded.text, n)).unwrap_or_default();
Location { path: uri.clone().into(), uri: Some(uri), range }
})
})
@@ -3040,28 +3113,10 @@ impl LspManager {
if scheme == "jar" {
let found = {
let uri = uri.to_owned();
tokio::task::spawn_blocking(move || match library_source::parse(&uri) {
Some(e) if !e.entry.ends_with(".class") => {
library_source::read_entry(&e.jar, &e.entry).map(|text| (e.entry, text))
}
Some(_) => library_source::class_source(&uri)
.and_then(|(source, text)| Some((library_source::parse(&source)?.entry, text))),
None => None,
})
.await
.ok()
.flatten()
tokio::task::spawn_blocking(move || read_jar_source(&uri, None)).await.ok().flatten()
};
if let Some((entry, text)) = found {
let title = entry.rsplit('/').next().unwrap_or(&entry).to_owned();
return Ok(self.store_virtual(
uri,
text,
Some(title),
None,
&[],
self.origin_server(uri).map(|s| s.id),
));
if let Some((entry, loaded)) = found {
return Ok(self.store_jar_entry(uri, entry, loaded));
}
}
let class_file = scheme == "jdt" || (scheme == "file" && uri.ends_with(".class"));
@@ -3132,7 +3187,21 @@ impl LspManager {
continue;
}
};
return Ok(self.store_virtual(uri, text, None, description, &languages, Some(t.server.id)));
// jdtls decodes attached sources itself: its attachment encoding is
// the document's real charset, and editable ones can be re-decoded.
let decoding = if scheme == "jdt" && t.server.name == jdtls::SERVER && !virtual_doc::is_decompiled(&text) {
match self.jdt_source_attachment(uri).await {
Some(att) if att.path.is_some() => Decoding {
encoding: att.encoding.unwrap_or_else(|| "utf-8".into()),
has_bom: false,
can_reload: att.can_edit_encoding,
},
_ => Decoding::server_text(),
}
} else {
Decoding::server_text()
};
return Ok(self.store_virtual(uri, text, None, description, &languages, Some(t.server.id), decoding));
}
if jar_class && error.is_none() {
return Err(IdeError::not_found(format!(
@@ -3144,6 +3213,7 @@ impl LspManager {
}
/// Caches `text` as the virtual document `uri`, titled `title` (else after the URI).
#[allow(clippy::too_many_arguments)]
fn store_virtual(
&self,
uri: &str,
@@ -3152,6 +3222,7 @@ impl LspManager {
description: Option<String>,
languages: &[LanguageId],
server: Option<ide_core::ServerId>,
decoding: Decoding,
) -> VirtualDocument {
let decompiled = if virtual_doc::scheme(uri) == "csharp" {
virtual_doc::is_csharp_decompiled(uri)
@@ -3172,6 +3243,9 @@ impl LspManager {
title,
description: description.or_else(|| virtual_doc::description(uri)),
decompiled,
encoding: decoding.encoding,
has_bom: decoding.has_bom,
can_reload_encoding: decoding.can_reload,
};
if let Some(server) = server {
self.virtual_origins.lock().entry(uri.to_owned()).or_insert(server);
@@ -3184,6 +3258,163 @@ impl LspManager {
doc
}
/// Caches a decoded `jar:` entry under the document's original URI, titled
/// after the entry; the backend owns these bytes, so they can be re-decoded.
fn store_jar_entry(&self, uri: &str, entry: String, loaded: ide_vfs::LoadedText) -> VirtualDocument {
let title = entry.rsplit('/').next().unwrap_or(&entry).to_owned();
self.store_virtual(
uri,
loaded.text,
Some(title),
None,
&[],
self.origin_server(uri).map(|s| s.id),
Decoding { encoding: loaded.encoding, has_bom: loaded.has_bom, can_reload: true },
)
}
/// The source attachment jdtls resolves for a `jdt:` class; `None` when the
/// command is unavailable or the class's container cannot be queried.
async fn jdt_source_attachment(&self, uri: &str) -> Option<SourceAttachment> {
let result = self
.jdtls_command(
&self.project_root,
"java.project.resolveSourceAttachment",
vec![json!({ "classFileUri": uri })],
)
.await
.ok()?;
if result.get("errorMessage").and_then(Value::as_str).is_some_and(|s| !s.is_empty()) {
return None;
}
let attrs = result.get("attributes")?;
let non_empty = |key: &str| attrs.get(key).and_then(Value::as_str).filter(|s| !s.is_empty()).map(str::to_owned);
Some(SourceAttachment {
path: non_empty("sourceAttachmentPath"),
encoding: non_empty("sourceAttachmentEncoding"),
can_edit_encoding: attrs.get("canEditEncoding").and_then(Value::as_bool).unwrap_or(true),
})
}
/// Updates a `jdt:` class's source attachment encoding. The command drops
/// the attachment when the path is omitted, so the resolved path is always
/// sent along with the encoding.
async fn update_jdt_source_encoding(&self, uri: &str, path: &str, encoding: &str) -> IdeResult<()> {
let result = self
.jdtls_command(
&self.project_root,
"java.project.updateSourceAttachment",
vec![json!({
"classFileUri": uri,
"attributes": { "sourceAttachmentPath": path, "sourceAttachmentEncoding": encoding },
})],
)
.await?;
if let Some(error) = result.get("errorMessage").and_then(Value::as_str).filter(|s| !s.is_empty()) {
return Err(IdeError::unavailable(error));
}
Ok(())
}
/// Waits until the resolved attachment reports `encoding`; `false` once
/// `until` passes. Maven applies attachment updates in a background job,
/// so the new encoding is not visible immediately after the update.
async fn wait_for_jdt_source_encoding(&self, uri: &str, encoding: &str, until: tokio::time::Instant) -> bool {
loop {
match self.jdt_source_attachment(uri).await {
Some(att) if att.encoding.as_deref() == Some(encoding) => return true,
_ if tokio::time::Instant::now() < until => {
tokio::time::sleep(std::time::Duration::from_millis(150)).await;
}
_ => return false,
}
}
}
/// "Reload with Encoding" for a virtual document: re-reads its origin with
/// `encoding` and replaces the cached content. `jar:` entries are re-read
/// here; `jdt:` sources are decoded by jdtls, so its source attachment
/// encoding is updated and the contents fetched again (keeping the server's
/// text and positions in sync). Server-generated content cannot be re-decoded.
pub async fn reload_virtual(&self, uri: &str, encoding: &str) -> IdeResult<VirtualDocument> {
ide_vfs::encode("", encoding, false)?;
match virtual_doc::scheme(uri) {
"jar" => {
let (owned_uri, owned_encoding) = (uri.to_owned(), encoding.to_owned());
let found = tokio::task::spawn_blocking(move || read_jar_source(&owned_uri, Some(&owned_encoding)))
.await
.map_err(|e| IdeError::internal(e.to_string()))?;
let Some((entry, loaded)) = found else {
return Err(IdeError::not_found(format!(
"no source is available for {}",
virtual_doc::title(uri, false)
)));
};
Ok(self.store_jar_entry(uri, entry, loaded))
}
"jdt" if self.origin_server(uri).is_none_or(|s| s.name == jdtls::SERVER) => {
let Some(att) = self.jdt_source_attachment(uri).await else {
return Err(IdeError::unavailable("jdtls cannot resolve the source attachment of this class"));
};
let Some(path) = att.path else {
return Err(IdeError::invalid("this class has no attached sources to re-decode"));
};
if !att.can_edit_encoding {
return Err(IdeError::invalid("the source attachment encoding of this library cannot be changed"));
}
self.update_jdt_source_encoding(uri, &path, encoding).await?;
// Maven containers apply the update in a background job (persist
// attached sources, then refresh the classpath): wait until the
// resolved attachment reports the new encoding, else the refetch
// below would still decode with the old one.
let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(10);
let first_try = tokio::time::Instant::now() + std::time::Duration::from_secs(3);
if !self.wait_for_jdt_source_encoding(uri, encoding, first_try).await {
// jdtls matches the existing encoding attribute by reference
// (`==`), so an entry restored from the saved container state
// gains a duplicate attribute and keeps the old value. That
// update still refreshes the container, after which a second
// update replaces the value correctly.
self.update_jdt_source_encoding(uri, &path, encoding).await?;
if !self.wait_for_jdt_source_encoding(uri, encoding, deadline).await {
return Err(IdeError::unavailable(
"jdtls did not apply the source attachment encoding; try reloading again",
));
}
}
// JDT keeps the class file's decoded buffer cached, so
// classFileContents would still return the old decoding: re-read
// the attached source entry here, like `jar:` documents. The
// server's open document is kept in sync by the regular
// document change flow.
let (owned_uri, owned_path, owned_encoding) = (uri.to_owned(), path.clone(), encoding.to_owned());
let found = tokio::task::spawn_blocking(move || {
library_source::source_in_archive(&owned_uri, Path::new(&owned_path), Some(&owned_encoding))
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?;
if let Some((_source_uri, loaded)) = found {
return Ok(self.store_virtual(
uri,
loaded.text,
None,
None,
&[],
self.origin_server(uri).map(|s| s.id),
Decoding { encoding: loaded.encoding, has_bom: loaded.has_bom, can_reload: true },
));
}
// Attachments that are not readable archives (e.g. folders):
// fall back to the server's contents.
self.virtual_docs.lock().remove(uri);
self.virtual_document(uri).await
}
_ => Err(IdeError::invalid(
"this document's content is generated by a language server and cannot be re-decoded with another encoding",
)),
}
}
/// Called when a server sends `workspace/applyEdit`; installed by `src-tauri`
/// so edits flow through `ide-workspace::Documents`.
pub fn set_config_hook(&self, hook: Box<ConfigFn>) {
+119 -33
Переглянути файл
@@ -7,8 +7,8 @@
//! it in the Maven or Gradle cache.
use ide_core::{Pos, Range};
use ide_vfs::LoadedText;
use percent_encoding::{AsciiSet, CONTROLS, percent_decode_str, utf8_percent_encode};
use std::io::Read;
use std::path::{Path, PathBuf};
/// An entry of an archive named by a `jar:` URI.
@@ -57,29 +57,32 @@ impl JarEntry {
}
}
/// The text of `entry` in the archive `jar`, when it is a UTF-8 file.
pub(crate) fn read_entry(jar: &Path, entry: &str) -> Option<String> {
let mut zip = zip::ZipArchive::new(std::fs::File::open(jar).ok()?).ok()?;
let mut file = zip.by_name(entry).ok()?;
let mut text = String::new();
file.read_to_string(&mut text).ok()?;
Some(text)
/// The text of `entry` in the archive `jar`, decoded like a file opened in the
/// editor; `encoding` decodes with it instead of detecting ("Reload with
/// Encoding"). `None` for binary or unreadable entries.
pub(crate) fn read_entry(jar: &Path, entry: &str, encoding: Option<&str>) -> Option<LoadedText> {
let path = ide_vfs::archive_entry_path(jar, entry);
let loaded = match encoding {
Some(enc) => ide_vfs::load_text_as(&path, enc).ok()?,
None => ide_vfs::load_text(&path).ok()?,
};
(!loaded.binary).then_some(loaded)
}
/// The library source for a class entry: its `.kt` / `.java` file in the
/// artifact's sources jar, as `(uri, text)`.
pub(crate) fn class_source(uri: &str) -> Option<(String, String)> {
pub(crate) fn class_source(uri: &str, encoding: Option<&str>) -> Option<(String, LoadedText)> {
let jar = parse(uri)?;
let class = jar.entry.strip_suffix(".class")?;
for sources in sources_jars(&jar.jar) {
if let Some((entry, text)) = source_entry(&sources, class) {
if let Some((entry, text)) = source_entry(&sources, class, encoding) {
return Some((jar.uri_of(&sources, &entry), text));
}
}
None
}
pub(crate) fn source_in_archive(uri: &str, sources: &Path) -> Option<(String, String)> {
pub(crate) fn source_in_archive(uri: &str, sources: &Path, encoding: Option<&str>) -> Option<(String, LoadedText)> {
let jar = if uri.starts_with("jdt://contents/") {
let path = uri.split('?').next()?.strip_prefix("jdt://contents/")?;
let parts: Vec<_> = path.split('/').collect();
@@ -96,7 +99,7 @@ pub(crate) fn source_in_archive(uri: &str, sources: &Path) -> Option<(String, St
parse(uri)?
};
let class = jar.entry.strip_suffix(".class")?;
let (entry, text) = source_entry(sources, class)?;
let (entry, text) = source_entry(sources, class, encoding)?;
Some((jar.uri_of(sources, &entry), text))
}
@@ -117,8 +120,8 @@ fn sources_jars(jar: &Path) -> Vec<PathBuf> {
}
/// The source file of `class` (`pkg/Name`, `pkg/Outer$Inner`, `pkg/FileKt`) in `sources`.
fn source_entry(sources: &Path, class: &str) -> Option<(String, String)> {
let mut zip = zip::ZipArchive::new(std::fs::File::open(sources).ok()?).ok()?;
fn source_entry(sources: &Path, class: &str, encoding: Option<&str>) -> Option<(String, LoadedText)> {
let zip = zip::ZipArchive::new(std::fs::File::open(sources).ok()?).ok()?;
let (package, name) = class.rsplit_once('/').unwrap_or(("", class));
let outer = name.split('$').next().unwrap_or(name);
let dir = if package.is_empty() { String::new() } else { format!("{package}/") };
@@ -127,13 +130,9 @@ fn source_entry(sources: &Path, class: &str) -> Option<(String, String)> {
if let Some(file) = outer.strip_suffix("Kt").filter(|f| !f.is_empty()) {
candidates.push(format!("{dir}{file}.kt"));
}
let read = |zip: &mut zip::ZipArchive<std::fs::File>, entry: &str| {
let mut text = String::new();
zip.by_name(entry).ok()?.read_to_string(&mut text).ok()?;
Some(text)
};
let read = |entry: &str| read_entry(sources, entry, encoding);
for entry in &candidates {
if let Some(text) = read(&mut zip, entry) {
if let Some(text) = read(entry) {
return Some((entry.clone(), text));
}
}
@@ -145,8 +144,8 @@ fn source_entry(sources: &Path, class: &str) -> Option<(String, String)> {
.collect();
let jvm_name = format!("@file:JvmName(\"{outer}\")");
for entry in kotlin_files {
if let Some(text) = read(&mut zip, &entry)
&& text.contains(&jvm_name)
if let Some(text) = read(&entry)
&& text.text.contains(&jvm_name)
{
return Some((entry, text));
}
@@ -233,11 +232,15 @@ mod tests {
use std::io::Write;
fn jar(path: &Path, entries: &[(&str, &str)]) {
jar_bytes(path, &entries.iter().map(|(n, t)| (*n, t.as_bytes())).collect::<Vec<_>>());
}
fn jar_bytes(path: &Path, entries: &[(&str, &[u8])]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let mut zip = zip::ZipWriter::new(std::fs::File::create(path).unwrap());
for (name, text) in entries {
for (name, bytes) in entries {
zip.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(text.as_bytes()).unwrap();
zip.write_all(bytes).unwrap();
}
zip.finish().unwrap();
}
@@ -274,12 +277,12 @@ mod tests {
"jdt://contents/lib-1.jar/p/Lib.java?=project/escaped-jar-path",
"jar:file:///m2/g/lib/1/lib-1.jar!/p/Lib$Inner.class",
] {
let (uri, text) = source_in_archive(uri, &sources).unwrap();
let (uri, text) = source_in_archive(uri, &sources, None).unwrap();
assert!(uri.ends_with("library%20sources.zip!/p/Lib.java"), "{uri}");
assert!(text.contains("class Lib"));
assert!(text.text.contains("class Lib"));
}
assert!(source_in_archive("jdt://contents/lib.jar/p/Missing.class", &sources).is_none());
assert!(source_in_archive("csharp:/p/App.csproj/decompiled/Lib.cs", &sources).is_none());
assert!(source_in_archive("jdt://contents/lib.jar/p/Missing.class", &sources, None).is_none());
assert!(source_in_archive("csharp:/p/App.csproj/decompiled/Lib.cs", &sources, None).is_none());
}
#[test]
@@ -312,20 +315,103 @@ mod tests {
("org/springframework/boot/Builder.class", "Builder.kt"),
("Root.class", "Root.java"),
] {
let (source_uri, text) = class_source(&uri(class)).unwrap_or_else(|| panic!("{class} in {binary:?}"));
let (source_uri, text) =
class_source(&uri(class), None).unwrap_or_else(|| panic!("{class} in {binary:?}"));
let dir = class.rsplit_once('/').map_or(String::new(), |(p, _)| format!("{p}/"));
assert!(source_uri.ends_with(&format!("-sources.jar!/{dir}{file}")), "{source_uri}");
assert_eq!(
parse(&source_uri).and_then(|e| read_entry(&e.jar, &e.entry)).as_deref(),
Some(text.as_str())
parse(&source_uri).and_then(|e| read_entry(&e.jar, &e.entry, None)).map(|l| l.text),
Some(text.text.clone())
);
}
assert_eq!(class_source(&uri("org/springframework/boot/Missing.class")), None);
assert!(class_source(&uri("org/springframework/boot/Missing.class"), None).is_none());
}
// No sources jar: nothing to show from here.
let lone = tmp.path().join("lone/x-1.jar");
jar(&lone, &[]);
assert_eq!(class_source(&format!("jar://{}!/X.class", lone.to_string_lossy().replace('\\', "/"))), None);
assert!(class_source(&format!("jar://{}!/X.class", lone.to_string_lossy().replace('\\', "/")), None).is_none());
}
/// The reported case: a library source file saved as GBK bytes, not UTF-8.
const GBK_SAMPLE: &str = "package example.encoding;\n\n/** 中文编码测试。 */\npublic final class EncodingSample {\n public static String message() {\n return \"中文编码测试:是否启用\";\n }\n}\n";
#[test]
fn archive_entries_decode_like_files_opened_in_the_editor() {
let tmp = tempfile::tempdir().unwrap();
let gbk = encoding_rs::GBK.encode(GBK_SAMPLE).0.into_owned();
// encoding_rs' encode() falls back to UTF-8 for UTF-16 (WHATWG has no encoder for it).
let mut utf16 = vec![0xFF, 0xFE];
utf16.extend(GBK_SAMPLE.encode_utf16().flat_map(u16::to_le_bytes));
let cases: Vec<(&str, Vec<u8>, &str, &str, bool)> = vec![
("utf8.java", GBK_SAMPLE.as_bytes().to_vec(), GBK_SAMPLE, "utf-8", false),
("bom.java", [b"\xEF\xBB\xBF", GBK_SAMPLE.as_bytes()].concat(), GBK_SAMPLE, "utf-8", true),
("gbk.java", gbk, GBK_SAMPLE, "gbk", false),
("utf16.java", utf16, GBK_SAMPLE, "utf-16le", true),
];
for (name, bytes, text, encoding, has_bom) in cases {
let jar_path = tmp.path().join(format!("lib-{name}.jar"));
jar_bytes(&jar_path, &[(&format!("p/{name}"), &bytes)]);
let loaded = read_entry(&jar_path, &format!("p/{name}"), None).unwrap_or_else(|| panic!("{name}"));
assert_eq!(loaded.text, text, "{name}");
assert_eq!(loaded.encoding, encoding, "{name}");
assert_eq!(loaded.has_bom, has_bom, "{name}");
assert!(!loaded.binary, "{name}");
}
// Binary entries are not shown as text.
let jar_path = tmp.path().join("bin.jar");
jar_bytes(&jar_path, &[("p/img.png", &[0x89, b'P', b'N', b'G', 0, 0, 0, 0])]);
assert!(read_entry(&jar_path, "p/img.png", None).is_none());
assert!(read_entry(&jar_path, "p/missing.java", None).is_none());
}
#[test]
fn reload_decodes_archive_entries_with_the_chosen_encoding() {
let tmp = tempfile::tempdir().unwrap();
let jar_path = tmp.path().join("lib.jar");
let gbk = encoding_rs::GBK.encode(GBK_SAMPLE).0.into_owned();
jar_bytes(&jar_path, &[("p/EncodingSample.java", &gbk)]);
let loaded = read_entry(&jar_path, "p/EncodingSample.java", Some("gbk")).unwrap();
assert_eq!((loaded.text.as_str(), loaded.encoding.as_str()), (GBK_SAMPLE, "gbk"));
// An explicit encoding wins over detection, even when it decodes to replacement characters.
let forced = read_entry(&jar_path, "p/EncodingSample.java", Some("utf-8")).unwrap();
assert!(forced.text.contains('\u{FFFD}'));
assert_eq!(forced.encoding, "utf-8");
assert!(read_entry(&jar_path, "p/EncodingSample.java", Some("not-an-encoding")).is_none());
}
#[test]
fn class_sources_in_maven_caches_decode_non_utf8() {
let tmp = tempfile::tempdir().unwrap();
let m2 = tmp.path().join("m2/com/example/encoding-sample/1.0.0");
let gbk = encoding_rs::GBK.encode(GBK_SAMPLE).0.into_owned();
let facade = "@file:JvmName(\"SampleUtils\")\npackage example.encoding\n\n/** 中文注释。 */\nfun message() = \"中文编码测试:是否启用\"\n";
let facade_gbk = encoding_rs::GBK.encode(facade).0.into_owned();
jar(&m2.join("encoding-sample-1.0.0.jar"), &[]);
jar_bytes(
&m2.join("encoding-sample-1.0.0-sources.jar"),
&[("example/encoding/EncodingSample.java", &gbk), ("example/encoding/Renamed.kt", &facade_gbk)],
);
let uri = |class: &str| {
format!("jar://{}!/{class}", m2.join("encoding-sample-1.0.0.jar").to_string_lossy().replace('\\', "/"))
};
for (class, text) in
[("example/encoding/EncodingSample.class", GBK_SAMPLE), ("example/encoding/SampleUtils.class", facade)]
{
let (source_uri, loaded) = class_source(&uri(class), None).unwrap_or_else(|| panic!("{class}"));
assert!(
source_uri.ends_with("-sources.jar!/example/encoding/EncodingSample.java")
|| source_uri.ends_with("-sources.jar!/example/encoding/Renamed.kt")
);
assert_eq!(loaded.text, text, "{class}");
assert_eq!(loaded.encoding, "gbk", "{class}");
// The manually attached archive path decodes the same way.
let jdt = "jdt://contents/encoding-sample-1.0.0.jar/example.encoding/EncodingSample.class?=project/x";
let (_, attached) = source_in_archive(jdt, &m2.join("encoding-sample-1.0.0-sources.jar"), None).unwrap();
assert_eq!((attached.text.as_str(), attached.encoding.as_str()), (GBK_SAMPLE, "gbk"));
}
// Reload with an explicit encoding re-reads the same source.
let (_, reloaded) = class_source(&uri("example/encoding/EncodingSample.class"), Some("gbk")).unwrap();
assert_eq!((reloaded.text.as_str(), reloaded.encoding.as_str()), (GBK_SAMPLE, "gbk"));
}
#[test]
+13
Переглянути файл
@@ -148,6 +148,8 @@ pub struct Server {
/// The instance whose just registered external project is loading, until an open file's
/// diagnostics arrive (0: none).
pub external_loading: std::sync::atomic::AtomicU64,
/// kotlin-lsp's project import as the current instance last reported it.
pub(crate) import_state: watch::Sender<Option<crate::kotlin_lsp::ImportState>>,
}
impl Server {
@@ -193,6 +195,7 @@ impl Server {
external_diagnostics: std::sync::atomic::AtomicBool::new(false),
external_refresh_queued: std::sync::atomic::AtomicBool::new(false),
external_loading: std::sync::atomic::AtomicU64::new(0),
import_state: watch::channel(None).0,
}
}
@@ -249,10 +252,20 @@ impl Server {
*self.started.lock() = Some(Instant::now());
// A crash report quotes only this instance's output, not its predecessors'.
self.stderr.lock().clear();
self.import_state.send_replace(None);
self.phase.send_replace(Phase::Starting);
Some(st.generation)
}
/// Publishes instance `generation`'s import state unless a newer start replaced it; checked
/// under the state lock so it cannot land after [`Self::begin_start`]'s reset.
pub(crate) fn set_import_state(&self, generation: u64, state: crate::kotlin_lsp::ImportState) {
let st = self.state.lock();
if st.generation == generation {
self.import_state.send_replace(Some(state));
}
}
/// Starting → Missing/Failed, publishing the status the failed start reports
/// before waiters wake: a request that wakes on `Phase::Down` reads the status
/// to say why it has no server, and must not still see `Starting`.
+8
Переглянути файл
@@ -357,6 +357,14 @@ pub struct VirtualDocument {
pub description: Option<String>,
/// Generated Java from a decompiler, rather than real source or a disassembler stub.
pub decompiled: bool,
/// Charset the content was decoded with (a WHATWG label, e.g. `gbk`);
/// `utf-8` for text the server decoded itself.
pub encoding: String,
pub has_bom: bool,
/// "Reload with Encoding" can re-decode the origin's bytes (archive
/// entries, JDT sources with an editable attachment encoding); false for
/// server-generated content such as decompiled code.
pub can_reload_encoding: bool,
}
/// Reference updates for a file / folder move (`workspace/willRenameFiles`).
+64 -10
Переглянути файл
@@ -23,9 +23,20 @@ const PATH: &AsciiSet = &CONTROLS
.add(b'}');
pub fn path_to_uri_string(path: &Path) -> String {
let s = path.to_string_lossy();
let s = if cfg!(windows) { s.replace('\\', "/") } else { s.into_owned() };
let s = if cfg!(windows) {
dunce::simplified(path).to_string_lossy().replace('\\', "/")
} else {
path.to_string_lossy().into_owned()
};
let mut out = String::from("file://");
// `\\server\share\p` → `file://server/share/p` (RFC 8089 Appendix E.3).
let unc = s.strip_prefix("//?/UNC/").or_else(|| s.strip_prefix("//")).filter(|_| cfg!(windows));
if let Some(unc) = unc {
let (host, rest) = unc.split_at(unc.find('/').unwrap_or(unc.len()));
out.extend(utf8_percent_encode(host, PATH));
out.extend(utf8_percent_encode(rest, PATH));
return out;
}
if !s.starts_with('/') {
out.push('/');
}
@@ -41,21 +52,30 @@ pub fn path_to_uri(path: &Path) -> lsp_types::Uri {
})
}
/// Converts a `file:` URI to a path; `None` for other schemes.
/// Converts a `file:` URI to a path; `None` for other schemes. On Windows a
/// host names a share (`file://server/share/p` → `\\server\share\p`) and the
/// drive letter is upper-cased, as the rest of the IDE spells it.
pub fn uri_to_path(uri: &str) -> Option<PathBuf> {
let rest = uri.strip_prefix("file://").or_else(|| uri.strip_prefix("FILE://"))?;
// Skip an authority (`file://localhost/...`).
let path = match rest.find('/') {
Some(0) => rest,
Some(i) if rest[..i].eq_ignore_ascii_case("localhost") => &rest[i..],
_ => return None,
};
let (host, path) = rest.split_at(rest.find('/')?);
let local = host.is_empty() || host.eq_ignore_ascii_case("localhost");
let path = path.split(['?', '#']).next().unwrap_or(path);
let decoded = percent_decode_str(path).decode_utf8().ok()?;
if cfg!(windows) {
if !local {
let host = percent_decode_str(host).decode_utf8().ok()?;
return Some(PathBuf::from(format!(r"\\{host}{}", decoded.replace('/', "\\"))));
}
// `/C:/x` → `C:\x`
let trimmed = decoded.strip_prefix('/').filter(|s| s.as_bytes().get(1) == Some(&b':')).unwrap_or(&decoded);
return Some(PathBuf::from(trimmed.replace('/', "\\")));
let mut path = trimmed.replace('/', "\\");
if path.as_bytes().get(1) == Some(&b':') {
path[..1].make_ascii_uppercase();
}
return Some(PathBuf::from(path));
}
if !local {
return None;
}
Some(PathBuf::from(decoded.into_owned()))
}
@@ -89,6 +109,40 @@ mod tests {
assert_eq!(uri_to_path("jdt://contents/x"), None);
}
#[cfg(unix)]
#[test]
fn remote_hosts_are_not_local_paths() {
assert_eq!(uri_to_path("file://server/share/a.ts"), None);
}
#[cfg(windows)]
#[test]
fn windows_uris() {
for (uri, path) in [
("file:///C:/a%20b/c.rs", r"C:\a b\c.rs"),
("file:///c:/x/y.ts", r"C:\x\y.ts"),
("file:///c%3A/x/y.ts", r"C:\x\y.ts"),
("file:///C%3a/x", r"C:\x"),
("file://localhost/d:/x", r"D:\x"),
("file://server/share/a.ts", r"\\server\share\a.ts"),
("file://server/share/a%20b/c.ts", r"\\server\share\a b\c.ts"),
("file:////server/share/a.ts", r"\\server\share\a.ts"),
] {
let got = uri_to_path(uri).unwrap();
assert_eq!(got.as_os_str(), path, "{uri}");
}
assert_eq!(uri_to_path("file://server"), None);
for (path, uri, back) in [
(r"C:\a b\c.rs", "file:///C:/a%20b/c.rs", r"C:\a b\c.rs"),
(r"\\?\C:\x\y.ts", "file:///C:/x/y.ts", r"C:\x\y.ts"),
(r"\\server\share\a b.ts", "file://server/share/a%20b.ts", r"\\server\share\a b.ts"),
(r"\\?\UNC\server\share\a.ts", "file://server/share/a.ts", r"\\server\share\a.ts"),
] {
assert_eq!(path_to_uri(Path::new(path)).as_str(), uri, "{path}");
assert_eq!(uri_to_path(uri).unwrap().as_os_str(), back, "{path}");
}
}
#[test]
fn virtual_targets() {
let jdt = "jdt://contents/rt.jar/java.lang/String.class?=p/%5C/x.jar%3Cjava.lang(String.class";
+631 -1
Переглянути файл
@@ -7,7 +7,7 @@ use ide_core::{
use ide_jsonrpc::{Handler, NoopHandler, Peer};
use ide_language::{LanguageConfig, LanguageRegistry, LanguageServerConfig};
use ide_lsp::types::*;
use ide_lsp::{LaunchError, Launcher, LspManager};
use ide_lsp::{KotlinDebugServer, KotlinDebugUnavailable, LaunchError, Launcher, LspManager};
use ide_text::AppliedChange;
use parking_lot::Mutex;
use serde_json::{Value, json};
@@ -306,6 +306,7 @@ fn harness_configured(
indent: None,
ui_mode: None,
framework_ui_modes: Vec::new(),
framework_files: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
@@ -3211,6 +3212,33 @@ async fn rename_merges_the_servers_that_can_rename_there() {
assert_eq!(edits_in(&edit, &html), [(rng(0, 3, 0, 8), "heading".into())]);
}
#[tokio::test(flavor = "multi_thread")]
async fn a_rename_the_server_withdraws_while_analysing_is_asked_again_then_explained() {
let h = harness(&[("dart", incremental_caps())]);
let (doc, path) = h.open("a.mk", "count = 1\n");
h.wait_status("dart", ServerStatus::Ready).await;
let dart = h.launcher.state("dart");
let r = json!({"start":{"line":0,"character":0},"end":{"line":0,"character":5}});
dart.respond("textDocument/rename", json!({ "changes": { uri_of(&path): [{ "range": r, "newText": "total" }] } }));
// Withdrawn while the server analyses an edit, then answered.
dart.errors.lock().insert("textDocument/rename".into(), (-32801, "Document was modified".into()));
let renaming = {
let mgr = h.mgr.clone();
tokio::spawn(async move { mgr.rename(doc, Pos::new(0, 1), "total".into(), Default::default()).await })
};
tokio::time::sleep(std::time::Duration::from_millis(400)).await;
dart.errors.lock().clear();
let edit = renaming.await.unwrap().unwrap().edit;
assert_eq!(edits_in(&edit, &path), [(rng(0, 0, 0, 5), "total".into())]);
assert!(dart.requested("textDocument/rename").len() >= 2);
// Withdrawn every time: an error that says so, not a silent cancellation.
dart.errors.lock().insert("textDocument/rename".into(), (-32801, "Document was modified".into()));
let err = h.mgr.rename(doc, Pos::new(0, 1), "total".into(), Default::default()).await.unwrap_err();
assert_ne!(err.kind, ide_core::error::ErrorKind::Cancelled);
assert!(err.message.contains("try again"), "{}", err.message);
}
#[tokio::test(flavor = "multi_thread")]
async fn angular_server_is_asked_about_typescript_only_in_component_templates() {
let servers = [("vtsls", incremental_caps()), ("angular-language-server", incremental_caps())];
@@ -3641,6 +3669,340 @@ async fn jdt_class_file_contents() {
assert_eq!(source.title, "WithSources.java");
}
#[tokio::test(flavor = "multi_thread")]
async fn jdt_library_sources_reload_with_the_attachment_encoding() {
let h = harness_for("java", "java", &[("jdtls", incremental_caps())], &[]);
h.open("App.java", "import example.encoding.EncodingSample;\n");
h.wait_status("jdtls", ServerStatus::Ready).await;
let s = h.launcher.state("jdtls");
let jdt = "jdt://contents/encoding-sample-1.0.jar/example.encoding/EncodingSample.class?=demo/%5C/m2%5C/encoding-sample-1.0.jar%3Cexample.encoding%28EncodingSample.class";
// The mock jdtls decodes the GBK attachment with the configured charset:
// until the encoding is updated, the text arrives garbled (U+FFFD).
let source = "package example.encoding;\n\n/** 中文编码测试。 */\npublic final class EncodingSample {}";
let garbled =
"package example.encoding;\n\n/** \u{FFFD}\u{FFFD}\u{FFFD}\u{FFFD} */\npublic final class EncodingSample {}";
let attachment_encoding: Arc<Mutex<Option<String>>> = Arc::default();
*s.responder.lock() = Some(Arc::new({
let encoding = attachment_encoding.clone();
move |_, method, params| match method {
"java/classFileContents" => {
let text = if encoding.lock().as_deref() == Some("gbk") { source } else { garbled };
Some(json!(text))
}
"workspace/executeCommand" => match params["command"].as_str()? {
"java.project.resolveSourceAttachment" => Some(json!({
"errorMessage": null,
"attributes": {
"jarPath": "/m2/encoding-sample-1.0.jar",
"sourceAttachmentPath": "/m2/encoding-sample-1.0-sources.jar",
"sourceAttachmentEncoding": encoding.lock().clone(),
"canEditEncoding": true,
}
})),
"java.project.updateSourceAttachment" => {
let attrs = &params["arguments"][0]["attributes"];
*encoding.lock() = attrs["sourceAttachmentEncoding"].as_str().map(str::to_owned);
Some(json!({ "errorMessage": null, "attributes": null }))
}
_ => None,
},
_ => None,
}
}));
let vd = h.mgr.virtual_document(jdt).await.unwrap();
assert_eq!(vd.text, garbled);
assert_eq!(vd.encoding, "utf-8", "no attachment encoding configured yet");
assert!(vd.can_reload_encoding);
let reloaded = h.mgr.reload_virtual(jdt, "gbk").await.unwrap();
assert_eq!(reloaded.text, source);
assert_eq!(reloaded.encoding, "gbk");
assert!(reloaded.can_reload_encoding);
// The update kept the attachment path (omitting it would drop the sources)
// and set the encoding; the contents were fetched again.
let updates: Vec<Value> = s
.requested("workspace/executeCommand")
.into_iter()
.filter(|p| p["command"] == "java.project.updateSourceAttachment")
.collect();
assert_eq!(updates.len(), 1);
assert_eq!(updates[0]["arguments"][0]["classFileUri"], jdt);
assert_eq!(updates[0]["arguments"][0]["attributes"]["sourceAttachmentPath"], "/m2/encoding-sample-1.0-sources.jar");
assert_eq!(updates[0]["arguments"][0]["attributes"]["sourceAttachmentEncoding"], "gbk");
assert!(s.requested("java/classFileContents").len() >= 2);
// The cache serves the re-decoded text without another round trip.
assert_eq!(h.mgr.virtual_document(jdt).await.unwrap().text, source);
// Round trip back to UTF-8.
let back = h.mgr.reload_virtual(jdt, "utf-8").await.unwrap();
assert_eq!(back.text, garbled);
assert_eq!(back.encoding, "utf-8");
}
#[tokio::test(flavor = "multi_thread")]
async fn jdt_reload_waits_for_the_maven_container_refresh() {
let h = harness_for("java", "java", &[("jdtls", incremental_caps())], &[]);
h.open("App.java", "class App {}\n");
h.wait_status("jdtls", ServerStatus::Ready).await;
let s = h.launcher.state("jdtls");
let jdt = "jdt://contents/lib.jar/p/Lib.class?=demo/%5C/m2%5C/lib.jar%3Cp%28Lib.class";
// m2e persists and refreshes the container in a background job: resolve
// keeps reporting the old encoding for a few calls after the update.
let state: Arc<Mutex<(Option<String>, usize)>> = Arc::default();
*s.responder.lock() = Some(Arc::new({
let state = state.clone();
move |_, method, params| match method {
"java/classFileContents" => {
let enc = state.lock().0.clone();
Some(json!(if enc.as_deref() == Some("gbk") {
"package p; /** 中文 */ class Lib {}"
} else {
"package p; /** \u{FFFD} */ class Lib {}"
}))
}
"workspace/executeCommand" => match params["command"].as_str()? {
"java.project.resolveSourceAttachment" => {
let mut st = state.lock();
st.1 += 1;
let reported = if st.1 > 2 { st.0.clone() } else { None };
Some(json!({
"attributes": {
"sourceAttachmentPath": "/m2/lib-sources.jar",
"sourceAttachmentEncoding": reported,
"canEditEncoding": true,
}
}))
}
"java.project.updateSourceAttachment" => {
let mut st = state.lock();
st.0 = params["arguments"][0]["attributes"]["sourceAttachmentEncoding"].as_str().map(str::to_owned);
st.1 = 0;
Some(json!({ "errorMessage": null }))
}
_ => None,
},
_ => None,
}
}));
let vd = h.mgr.virtual_document(jdt).await.unwrap();
assert!(vd.text.contains('\u{FFFD}'));
let reloaded = h.mgr.reload_virtual(jdt, "gbk").await.unwrap();
assert_eq!(reloaded.text, "package p; /** 中文 */ class Lib {}");
assert_eq!(reloaded.encoding, "gbk");
assert!(state.lock().1 >= 3, "polled until the container reported the new encoding");
}
#[tokio::test(flavor = "multi_thread")]
async fn jdt_reload_retries_when_a_restored_container_ignores_the_update() {
let h = harness_for("java", "java", &[("jdtls", incremental_caps())], &[]);
h.open("App.java", "class App {}\n");
h.wait_status("jdtls", ServerStatus::Ready).await;
let s = h.launcher.state("jdtls");
let jdt = "jdt://contents/lib.jar/p/Lib.class?=demo/%5C/m2%5C/lib.jar%3Cp%28Lib.class";
// jdtls replaces the encoding attribute by reference equality: an entry
// restored from the saved container state keeps the old value on the first
// update (a duplicate attribute is appended); only a second update, after
// the container was refreshed, applies the new one.
let state: Arc<Mutex<(Option<String>, usize)>> = Arc::default();
*s.responder.lock() = Some(Arc::new({
let state = state.clone();
move |_, method, params| match method {
"java/classFileContents" => {
let enc = state.lock().0.clone();
Some(json!(if enc.as_deref() == Some("gbk") {
"package p; /** 中文 */ class Lib {}"
} else {
"package p; /** \u{FFFD} */ class Lib {}"
}))
}
"workspace/executeCommand" => match params["command"].as_str()? {
"java.project.resolveSourceAttachment" => Some(json!({
"attributes": {
"sourceAttachmentPath": "/m2/lib-sources.jar",
"sourceAttachmentEncoding": state.lock().0.clone(),
"canEditEncoding": true,
}
})),
"java.project.updateSourceAttachment" => {
let mut st = state.lock();
st.1 += 1;
if st.1 > 1 {
st.0 = params["arguments"][0]["attributes"]["sourceAttachmentEncoding"]
.as_str()
.map(str::to_owned);
}
Some(json!({ "errorMessage": null }))
}
_ => None,
},
_ => None,
}
}));
let vd = h.mgr.virtual_document(jdt).await.unwrap();
assert!(vd.text.contains('\u{FFFD}'));
let reloaded = h.mgr.reload_virtual(jdt, "gbk").await.unwrap();
assert_eq!(reloaded.text, "package p; /** 中文 */ class Lib {}");
assert_eq!(reloaded.encoding, "gbk");
let updates: Vec<Value> = s
.requested("workspace/executeCommand")
.into_iter()
.filter(|p| p["command"] == "java.project.updateSourceAttachment")
.collect();
assert_eq!(updates.len(), 2, "the ignored first update was retried once");
assert!(updates.iter().all(|u| u["arguments"][0]["attributes"]["sourceAttachmentPath"] == "/m2/lib-sources.jar"));
}
#[tokio::test(flavor = "multi_thread")]
async fn jdt_reload_reads_the_attachment_when_the_server_buffer_is_stale() {
use std::io::Write;
let tmp = tempfile::tempdir().unwrap();
let sources = tmp.path().join("lib-sources.jar");
let source = "package p; /** 中文 */ class Lib {}";
let gbk = encoding_rs::GBK.encode(source).0.into_owned();
let mut zip = zip::ZipWriter::new(std::fs::File::create(&sources).unwrap());
zip.start_file("p/Lib.java", zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(&gbk).unwrap();
zip.finish().unwrap();
let h = harness_for("java", "java", &[("jdtls", incremental_caps())], &[]);
h.open("App.java", "class App {}\n");
h.wait_status("jdtls", ServerStatus::Ready).await;
let s = h.launcher.state("jdtls");
let jdt = "jdt://contents/lib.jar/p/Lib.class?=demo/%5C/m2%5C/lib.jar%3Cp%28Lib.class";
// JDT caches the decoded buffer of a class file: even after the attachment
// encoding is updated, classFileContents keeps returning the old decoding.
let encoding: Arc<Mutex<Option<String>>> = Arc::default();
*s.responder.lock() = Some(Arc::new({
let encoding = encoding.clone();
let sources = sources.clone();
move |_, method, params| match method {
"java/classFileContents" => Some(json!("package p; /** \u{FFFD} */ class Lib {}")),
"workspace/executeCommand" => match params["command"].as_str()? {
"java.project.resolveSourceAttachment" => Some(json!({
"attributes": {
"sourceAttachmentPath": sources.to_string_lossy(),
"sourceAttachmentEncoding": encoding.lock().clone(),
"canEditEncoding": true,
}
})),
"java.project.updateSourceAttachment" => {
*encoding.lock() =
params["arguments"][0]["attributes"]["sourceAttachmentEncoding"].as_str().map(str::to_owned);
Some(json!({ "errorMessage": null }))
}
_ => None,
},
_ => None,
}
}));
let vd = h.mgr.virtual_document(jdt).await.unwrap();
assert!(vd.text.contains('\u{FFFD}'));
let reloaded = h.mgr.reload_virtual(jdt, "gbk").await.unwrap();
assert_eq!(reloaded.text, source, "read from the attachment, not the stale server buffer");
assert_eq!(reloaded.encoding, "gbk");
assert!(reloaded.can_reload_encoding);
}
#[tokio::test(flavor = "multi_thread")]
async fn jdt_reload_is_refused_when_the_sources_cannot_be_re_decoded() {
let h = harness_for("java", "java", &[("jdtls", incremental_caps())], &[]);
h.open("App.java", "class App {}\n");
h.wait_status("jdtls", ServerStatus::Ready).await;
let s = h.launcher.state("jdtls");
let class =
|name: &str| format!("jdt://contents/lib.jar/p/{name}.class?=demo/%5C/m2%5C/lib.jar%3Cp%28{name}.class");
s.respond("java/classFileContents", json!("package p;\npublic class Lib {}"));
// A container that does not allow editing the attachment encoding.
*s.responder.lock() = Some(Arc::new(|_, method, params| match method {
"workspace/executeCommand" if params["command"] == "java.project.resolveSourceAttachment" => Some(json!({
"attributes": { "sourceAttachmentPath": "/m2/lib-sources.jar", "canEditEncoding": false }
})),
_ => None,
}));
let vd = h.mgr.virtual_document(&class("ReadOnly")).await.unwrap();
assert!(!vd.can_reload_encoding);
let err = h.mgr.reload_virtual(&class("ReadOnly"), "gbk").await.unwrap_err();
assert!(err.message.contains("cannot be changed"), "{err}");
// Decompiled content has no attached sources to re-decode.
*s.responder.lock() = Some(Arc::new(|_, method, params| match method {
"workspace/executeCommand" if params["command"] == "java.project.resolveSourceAttachment" => Some(json!({
"attributes": { "sourceAttachmentPath": null, "canEditEncoding": true }
})),
_ => None,
}));
s.respond(
"java/classFileContents",
json!("// Source code is decompiled from a .class file using FernFlower decompiler (from Intellij IDEA).\npackage p;\npublic class Lib {}"),
);
let decompiled = h.mgr.virtual_document(&class("Decompiled")).await.unwrap();
assert!(decompiled.decompiled && !decompiled.can_reload_encoding);
let err = h.mgr.reload_virtual(&class("Decompiled"), "gbk").await.unwrap_err();
assert!(err.message.contains("no attached sources"), "{err}");
// The resolve command fails (unqueryable container): no reload offered.
s.errors.lock().insert("workspace/executeCommand".into(), (-32603, "unsupported".into()));
let unresolved = h.mgr.virtual_document(&class("Unresolved")).await.unwrap();
assert_eq!(unresolved.encoding, "utf-8");
assert!(!unresolved.can_reload_encoding);
let err = h.mgr.reload_virtual(&class("Unresolved"), "gbk").await.unwrap_err();
assert!(err.message.contains("cannot resolve"), "{err}");
// Server-generated content of other schemes cannot be re-decoded at all.
let err = h.mgr.reload_virtual("csharp:/p/App.csproj/decompiled/Lib.cs", "gbk").await.unwrap_err();
assert!(err.message.contains("generated by a language server"), "{err}");
// Unknown encodings are refused before any server round trip.
assert!(h.mgr.reload_virtual(&class("ReadOnly"), "not-an-encoding").await.is_err());
}
#[tokio::test(flavor = "multi_thread")]
async fn jar_library_sources_reload_with_another_encoding() {
use std::io::Write;
let repo = tempfile::tempdir().unwrap();
let dir = repo.path().join("com/example/encoding-sample/1.0.0");
std::fs::create_dir_all(&dir).unwrap();
let source = "package example.encoding\n\n/** 中文编码测试。 */\nfun message() = \"中文编码测试:是否启用\"\n";
let gbk = encoding_rs::GBK.encode(source).0.into_owned();
let jar_bytes = |path: &Path, entries: &[(&str, &[u8])]| {
let mut zip = zip::ZipWriter::new(std::fs::File::create(path).unwrap());
for (name, bytes) in entries {
zip.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
zip.write_all(bytes).unwrap();
}
zip.finish().unwrap();
};
jar_bytes(&dir.join("encoding-sample-1.0.jar"), &[("example/encoding/EncodingSampleKt.class", b"\0")]);
jar_bytes(&dir.join("encoding-sample-1.0-sources.jar"), &[("example/encoding/EncodingSample.kt", &gbk)]);
let jar_uri = |jar: &Path, entry: &str| format!("jar://{}!/{entry}", uri_of(jar).trim_start_matches("file://"));
let h = harness_for("kotlin", "kt", &[("kotlin-lsp", incremental_caps())], &[]);
h.open("Main.kt", "fun main() {}\n");
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
let class = jar_uri(&dir.join("encoding-sample-1.0.jar"), "example/encoding/EncodingSampleKt.class");
let entry = jar_uri(&dir.join("encoding-sample-1.0-sources.jar"), "example/encoding/EncodingSample.kt");
let vd = h.mgr.virtual_document(&class).await.unwrap();
assert_eq!(vd.text, source);
assert_eq!(vd.encoding, "gbk");
assert!(vd.can_reload_encoding);
// An explicit encoding wins over detection, even when it garbles the text.
let forced = h.mgr.reload_virtual(&class, "utf-8").await.unwrap();
assert!(forced.text.contains('\u{FFFD}'));
assert_eq!(forced.encoding, "utf-8");
let back = h.mgr.reload_virtual(&class, "gbk").await.unwrap();
assert_eq!((back.text.as_str(), back.encoding.as_str()), (source, "gbk"));
// Non-class entries reload the same way.
let entry_vd = h.mgr.virtual_document(&entry).await.unwrap();
assert_eq!((entry_vd.text.as_str(), entry_vd.encoding.as_str()), (source, "gbk"));
let entry_reload = h.mgr.reload_virtual(&entry, "gbk").await.unwrap();
assert_eq!(entry_reload.text, source);
// Missing sources fail cleanly.
let missing = jar_uri(&dir.join("encoding-sample-1.0.jar"), "example/encoding/Missing.class");
assert!(h.mgr.reload_virtual(&missing, "gbk").await.is_err());
}
#[tokio::test(flavor = "multi_thread")]
async fn class_file_opening_routes_sources_and_binary_documents() {
let mut caps = incremental_caps();
@@ -5963,6 +6325,208 @@ async fn hierarchy_refactorings_name_the_servers_without_them() {
assert_eq!(e.message, "plain doesn't support Pull Members Up");
}
// ------------------------------------------------------------------ kotlin-lsp debug server
type KotlinDebug = Result<KotlinDebugServer, KotlinDebugUnavailable>;
fn kotlin_lsp_harness() -> (Harness, Arc<MockState>) {
let h = harness_for("kotlin", "kt", &[("kotlin-lsp", incremental_caps())], &[]);
let s = h.launcher.state("kotlin-lsp");
s.respond("workspace/executeCommand", json!(4711));
(h, s)
}
fn spawn_kotlin_debug_server(
h: &Harness,
path: &Path,
import_wait: Duration,
cancel: &CancellationToken,
) -> tokio::task::JoinHandle<KotlinDebug> {
let (mgr, path, cancel) = (h.mgr.clone(), path.to_owned(), cancel.clone());
tokio::spawn(async move { mgr.kotlin_debug_server(&path, import_wait, &cancel).await })
}
async fn joined(debug: tokio::task::JoinHandle<KotlinDebug>) -> KotlinDebug {
tokio::time::timeout(Duration::from_secs(10), debug).await.expect("kotlin_debug_server returned").unwrap()
}
fn import_progress(s: &MockState, value: Value) {
s.peer().notify("$/progress", json!({ "token": "import", "value": value })).unwrap();
}
fn import_finished(s: &MockState, folders: &[(&Path, &str)]) {
let folders: Vec<Value> = folders
.iter()
.map(|(dir, status)| json!({ "folderUri": format!("{}/", uri_of(dir)), "tool": "gradle", "status": status }))
.collect();
s.peer().notify("intellij/workspaceImportState", json!({ "phase": "FINISHED", "folders": folders })).unwrap();
}
fn debug_server_requests(s: &MockState) -> Vec<Value> {
s.requested("workspace/executeCommand")
}
#[tokio::test(flavor = "multi_thread")]
async fn waits_for_the_import_before_starting_the_debug_server() {
// kotlin-lsp sends the progress end and FINISHED within the same millisecond, in either order.
for finished_first in [false, true] {
let (h, s) = kotlin_lsp_harness();
let main = h.root.join("Main.kt");
std::fs::write(&main, "fun main() {}").unwrap();
let debug = spawn_kotlin_debug_server(&h, &main, Duration::from_secs(10), &CancellationToken::new());
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
// Repeats every few seconds while the project stays usable: never waited for.
s.peer()
.notify("$/progress", json!({ "token": "index", "value": { "kind": "begin", "title": "Indexing" } }))
.unwrap();
import_progress(&s, json!({ "kind": "begin", "title": "Importing" }));
h.wait_status("kotlin-lsp", ServerStatus::Busy).await;
tokio::time::sleep(Duration::from_millis(300)).await;
assert!(debug_server_requests(&s).is_empty(), "started while importing");
if finished_first {
import_finished(&s, &[(&h.root, "SUCCESS")]);
tokio::time::sleep(Duration::from_millis(200)).await;
assert!(debug_server_requests(&s).is_empty(), "started while the Importing task runs");
import_progress(&s, json!({ "kind": "end" }));
} else {
import_progress(&s, json!({ "kind": "end" }));
tokio::time::sleep(Duration::from_millis(200)).await;
assert!(debug_server_requests(&s).is_empty(), "started before FINISHED");
import_finished(&s, &[(&h.root, "SUCCESS")]);
}
assert_eq!(joined(debug).await, Ok(KotlinDebugServer { port: 4711, root: h.root.clone() }));
assert_eq!(
debug_server_requests(&s),
vec![json!({ "command": "start_debug_server", "arguments": [uri_of(&h.root)] })],
"finished_first: {finished_first}"
);
}
}
#[tokio::test(flavor = "multi_thread")]
async fn still_importing_after_the_wait_is_reported() {
let (h, s) = kotlin_lsp_harness();
let r = h.mgr.kotlin_debug_server(&h.root, Duration::from_millis(200), &CancellationToken::new()).await;
assert_eq!(r, Err(KotlinDebugUnavailable::StillImporting));
assert!(debug_server_requests(&s).is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn cancelling_the_import_wait_reports_still_importing() {
let (h, s) = kotlin_lsp_harness();
let cancel = CancellationToken::new();
let debug = spawn_kotlin_debug_server(&h, &h.root, Duration::from_secs(60), &cancel);
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(!debug.is_finished());
cancel.cancel();
assert_eq!(joined(debug).await, Err(KotlinDebugUnavailable::StillImporting));
assert!(debug_server_requests(&s).is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn a_failed_import_of_the_module_is_reported() {
// Another module that failed does not keep this one from debugging.
for other_module_failed in [false, true] {
let (h, s) = kotlin_lsp_harness();
let main = h.root.join("app/Main.kt");
let debug = spawn_kotlin_debug_server(&h, &main, Duration::from_secs(10), &CancellationToken::new());
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
let lib = h.root.join("lib");
let (folders, expected) = if other_module_failed {
(
vec![(h.root.as_path(), "SUCCESS"), (lib.as_path(), "FAILED")],
Ok(KotlinDebugServer { port: 4711, root: h.root.clone() }),
)
} else {
(vec![(h.root.as_path(), "FAILED")], Err(KotlinDebugUnavailable::ImportFailed))
};
import_finished(&s, &folders);
assert_eq!(joined(debug).await, expected, "other_module_failed: {other_module_failed}");
}
}
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread")]
async fn a_failed_import_is_reported_through_a_symlinked_root() {
// The debugged path names the root through a symlink, the folder URI names its target,
// and the file does not exist yet.
let (h, s) = kotlin_lsp_harness();
let links = tempfile::tempdir().unwrap();
let link = links.path().join("project");
std::os::unix::fs::symlink(&h.root, &link).unwrap();
let debug =
spawn_kotlin_debug_server(&h, &link.join("app/Main.kt"), Duration::from_secs(10), &CancellationToken::new());
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
import_finished(&s, &[(&h.root, "FAILED")]);
assert_eq!(joined(debug).await, Err(KotlinDebugUnavailable::ImportFailed));
}
#[tokio::test(flavor = "multi_thread")]
async fn an_older_finished_import_does_not_count_after_a_restart() {
let (h, s) = kotlin_lsp_harness();
let debug = spawn_kotlin_debug_server(&h, &h.root, Duration::from_secs(10), &CancellationToken::new());
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
import_finished(&s, &[(&h.root, "SUCCESS")]);
assert!(joined(debug).await.is_ok());
let id = h.mgr.servers()[0].id;
h.mgr.restart(id).await.unwrap();
let r = h.mgr.kotlin_debug_server(&h.root, Duration::from_millis(300), &CancellationToken::new()).await;
assert_eq!(r, Err(KotlinDebugUnavailable::StillImporting));
assert_eq!(debug_server_requests(&s).len(), 1);
let debug = spawn_kotlin_debug_server(&h, &h.root, Duration::from_secs(10), &CancellationToken::new());
import_finished(&s, &[(&h.root, "SUCCESS")]);
assert_eq!(joined(debug).await, Ok(KotlinDebugServer { port: 4711, root: h.root.clone() }));
assert_eq!(debug_server_requests(&s).len(), 2);
}
#[tokio::test(flavor = "multi_thread")]
async fn without_kotlin_lsp_the_reason_is_not_configured() {
let wait = Duration::from_secs(1);
let cancel = CancellationToken::new();
let h = harness_for("kotlin", "kt", &[("kotlin-language-server", incremental_caps())], &[]);
assert_eq!(h.mgr.kotlin_debug_server(&h.root, wait, &cancel).await, Err(KotlinDebugUnavailable::NotConfigured));
let (h, s) = kotlin_lsp_harness();
s.missing.store(true, Ordering::SeqCst);
assert_eq!(h.mgr.kotlin_debug_server(&h.root, wait, &cancel).await, Err(KotlinDebugUnavailable::NotInstalled));
let (h, _) = kotlin_lsp_harness();
h.mgr.set_config_hook(Box::new(|cfg, _| cfg.launch_error = Some("needs JDK 21".into())));
assert_eq!(
h.mgr.kotlin_debug_server(&h.root, wait, &cancel).await,
Err(KotlinDebugUnavailable::NotStarted("needs JDK 21".into()))
);
}
#[tokio::test(flavor = "multi_thread")]
async fn a_debug_server_error_is_reported() {
let (h, s) = kotlin_lsp_harness();
s.errors.lock().insert("workspace/executeCommand".into(), (-32603, "boom".into()));
let debug = spawn_kotlin_debug_server(&h, &h.root, Duration::from_secs(10), &CancellationToken::new());
h.wait_status("kotlin-lsp", ServerStatus::Ready).await;
import_finished(&s, &[(&h.root, "SUCCESS")]);
match joined(debug).await {
Err(KotlinDebugUnavailable::Failed(m)) => assert!(m.contains("boom"), "{m}"),
other => panic!("{other:?}"),
}
s.errors.lock().clear();
for port in [json!(0), json!(65536), json!("4711"), Value::Null] {
s.respond("workspace/executeCommand", port.clone());
match h.mgr.kotlin_debug_server(&h.root, Duration::from_secs(1), &CancellationToken::new()).await {
Err(KotlinDebugUnavailable::Failed(m)) => {
assert_eq!(m, format!("unexpected debug server port: {port}"))
}
other => panic!("{port}: {other:?}"),
}
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn injected_documents_route_and_resolve_without_exposing_virtual_files() {
for encoding in ["utf-16", "utf-8"] {
@@ -6151,3 +6715,69 @@ async fn missing_injected_server_uses_the_normal_single_install_notification() {
h.mgr.did_close(host);
h.mgr.shutdown_all().await;
}
/// #2248: `go.mod` files are served by gopls with the language id it
/// understands, and its pushed diagnostics reach the Problems view.
#[tokio::test(flavor = "multi_thread")]
async fn gomod_files_are_served_by_gopls() {
let dir = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(dir.path()).unwrap();
let registry = Arc::new(LanguageRegistry::load(None).unwrap());
// The bundled configs route Go files of every kind to gopls.
for lang in ["go", "go.mod", "go.work"] {
let cfg = registry.get(&LanguageId::new(lang)).unwrap_or_else(|| panic!("{lang} has no language definition"));
assert!(cfg.language_servers.iter().any(|s| s.name == "gopls"), "{lang} is not served by gopls");
}
let launcher = Arc::new(MockLauncher::default());
*launcher.state("gopls").caps.lock() = incremental_caps();
let events = Arc::new(Mutex::new(Vec::new()));
let sink = {
let events = events.clone();
Arc::new(move |e: IdeEvent| events.lock().push(e))
};
let mgr = LspManager::with_launcher(root.clone(), registry, sink, launcher.clone());
let text = "module example.com/app\n\ngo 1.22\n";
let path = root.join("go.mod");
std::fs::write(&path, text).unwrap();
assert!(mgr.serves_path(&path));
mgr.did_open(DocId::next(), &path, &LanguageId::new("go.mod"), 1, text);
let gopls = launcher.state("gopls");
eventually(|| !gopls.notified("textDocument/didOpen").is_empty()).await;
// gopls only treats the file as a module file for the `go.mod` id.
let opened = &gopls.notified("textDocument/didOpen")[0]["textDocument"];
assert_eq!(opened["languageId"], json!("go.mod"));
// The module root (the go.mod itself is the marker) scopes the server.
let init = &gopls.requested("initialize")[0];
assert_eq!(init["rootUri"], json!(uri_of(&root)));
// Same startup options as when a `.go` file launches the server first.
assert_eq!(init["initializationOptions"]["semanticTokens"], json!(true));
// Pushed go.mod diagnostics surface with gopls as their source.
gopls
.peer()
.notify(
"textDocument/publishDiagnostics",
json!({ "uri": uri_of(&path), "diagnostics": [{
"range": {"start": {"line": 2, "character": 3}, "end": {"line": 2, "character": 7}},
"severity": 1, "message": "missing requirement", "source": "gopls" }] }),
)
.unwrap();
eventually(|| {
events
.lock()
.iter()
.any(|e| matches!(e, IdeEvent::Diagnostics { path: p, source, .. } if p.0 == path && source == "gopls"))
})
.await;
let items = events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Diagnostics { path: p, items, .. } if p.0 == path => Some(items.clone()),
_ => None,
})
.next_back()
.unwrap();
assert_eq!(items.len(), 1);
assert_eq!(items[0].message, "missing requirement");
mgr.shutdown_all().await;
}
+1
Переглянути файл
@@ -55,6 +55,7 @@ fn lang(id: &str, ext: &str, server: LanguageServerConfig) -> LanguageConfig {
indent: None,
ui_mode: None,
framework_ui_modes: Vec::new(),
framework_files: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
+1 -1
Переглянути файл
@@ -22,7 +22,7 @@ pub use runtime::{
};
pub use sandbox::{
MAX_READ_BYTES, inside_root, inside_roots, read_text, resolve_in_root, resolve_in_roots, resolve_new_in_root,
resolve_real, resolve_writable_in_root, resolve_writable_in_roots,
resolve_new_in_roots, resolve_real, resolve_writable_in_root, resolve_writable_in_roots,
};
pub use types::*;
pub use wasmtime::component::Component;
+92 -14
Переглянути файл
@@ -118,6 +118,24 @@ pub fn inside_roots(roots: &[PathBuf], path: &str) -> Result<bool, String> {
/// be created) or that names a symlink itself (to delete or rename it): the
/// directory part is resolved through symlinks, the final name is kept as is,
/// and the result must stay inside `root`.
/// Whether a path (relative to the project root) goes through git's own `.git` folder or file.
/// git runs what it finds in `.git` (hooks, `gpg.program`, filters), so a content write must
/// never reach it. Names are normalized for trailing dots and spaces and NTFS stream suffixes;
/// the caller also checks the short-name-resolved path via [`resolve_real`].
pub fn names_git_dir(rel: &Path) -> bool {
rel.components().any(|part| {
let name = part.as_os_str().to_string_lossy();
let name = name.split(':').next().unwrap_or_default().trim_end_matches(['.', ' ']);
name.eq_ignore_ascii_case(".git")
})
}
/// A resolved write destination that reaches git's own `.git` metadata, judged by the name
/// given and by the name the file system resolves it to.
fn reaches_git_dir(root: &Path, real: &Path) -> bool {
real.strip_prefix(root).is_ok_and(names_git_dir) || resolve_real(real).strip_prefix(root).is_ok_and(names_git_dir)
}
pub fn resolve_new_in_root(root: &Path, path: &str) -> Result<PathBuf, String> {
let outside = || format!("{path}: access outside the project root is not allowed");
if path.is_empty() {
@@ -154,6 +172,9 @@ pub fn resolve_new_in_root(root: &Path, path: &str) -> Result<PathBuf, String> {
if !real.starts_with(&root) {
return Err(outside());
}
if reaches_git_dir(&root, &real) {
return Err(format!("{path}: writing git's own files is not allowed"));
}
Ok(real)
}
@@ -165,26 +186,34 @@ pub fn resolve_writable_in_root(root: &Path, path: &str) -> Result<PathBuf, Stri
if !real.starts_with(&root) {
return Err(format!("{path}: access outside the project root is not allowed"));
}
if reaches_git_dir(&root, &real) {
return Err(format!("{path}: writing git's own files is not allowed"));
}
Ok(real)
}
/// [`resolve_writable_in_root`] over several roots, as [`resolve_in_roots`] does.
pub fn resolve_writable_in_roots(roots: &[PathBuf], path: &str) -> Result<PathBuf, String> {
let (project_root, members) = match roots.split_first() {
Some((project_root, members)) => (project_root, members),
None => return Err(format!("{path}: access outside the project root is not allowed")),
first_root_resolving(roots, path, resolve_writable_in_root)
}
/// [`resolve_new_in_root`] over several roots, as [`resolve_in_roots`] does.
pub fn resolve_new_in_roots(roots: &[PathBuf], path: &str) -> Result<PathBuf, String> {
first_root_resolving(roots, path, resolve_new_in_root)
}
/// `resolve` against the project root, then each member; the project root's refusal when none
/// accepts `path`.
fn first_root_resolving(
roots: &[PathBuf],
path: &str,
resolve: fn(&Path, &str) -> Result<PathBuf, String>,
) -> Result<PathBuf, String> {
let Some((project_root, members)) = roots.split_first() else {
return Err(format!("{path}: access outside the project root is not allowed"));
};
match resolve_writable_in_root(project_root, path) {
Ok(real) => Ok(real),
Err(project_root_error) => {
for root in members {
if let Ok(real) = resolve_writable_in_root(root, path) {
return Ok(real);
}
}
Err(project_root_error)
}
}
resolve(project_root, path)
.or_else(|refusal| members.iter().find_map(|root| resolve(root, path).ok()).ok_or(refusal))
}
fn resolve_write_target(path: &Path, links_left: usize) -> Result<PathBuf, std::io::Error> {
@@ -559,6 +588,55 @@ mod tests {
assert!(resolve_new_in_root(&root, "").is_err());
}
/// Advisory #4: a content write may not reach git's own `.git` metadata, which git runs as
/// code (hooks, `gpg.program`, filters). The `.git` folder is inside the project root, so the
/// root confinement alone does not stop it.
#[test]
fn writes_cannot_reach_git_metadata() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("proj");
std::fs::create_dir_all(root.join(".git/hooks")).unwrap();
std::fs::create_dir_all(root.join("src")).unwrap();
std::fs::write(root.join(".git/config"), "[core]\n").unwrap();
// An ordinary write is fine; each refusal below is the `.git` rule.
assert!(resolve_new_in_root(&root, "src/ok.txt").is_ok());
assert!(resolve_writable_in_root(&root, "src/ok.txt").is_ok());
for rel in [
".git/config",
".git/hooks/pre-commit",
".GIT/config",
"src/../.git/config",
".git./config",
".git /config",
] {
assert!(resolve_new_in_root(&root, rel).unwrap_err().contains("git's own files"), "new: {rel}");
assert!(resolve_writable_in_root(&root, rel).unwrap_err().contains("git's own files"), "writable: {rel}");
}
assert!(names_git_dir(Path::new("a/.git/b")));
assert!(!names_git_dir(Path::new("a/git/b")));
assert!(!names_git_dir(Path::new(".github/workflows/ci.yml")));
}
#[test]
fn git_folder_is_recognised_under_every_spelling_that_opens_it() {
for path in [
"proj/.git/hooks/pre-commit",
"proj/.git/config",
"proj/sub/.git",
"proj/.GIT/config",
"proj/.git./config",
"proj/.git /config",
"proj/.git::$INDEX_ALLOCATION/config",
] {
assert!(names_git_dir(Path::new(path)), "{path}");
}
for path in ["proj/.gitignore", "proj/.github/workflows/ci.yml", "proj/src/git/.gitkeep", "proj/a.git/x"] {
assert!(!names_git_dir(Path::new(path)), "{path}");
}
// A project opened from inside a git folder is git's own files all the way down.
assert!(names_git_dir(Path::new("repo/.git/modules/lib/hooks/pre-commit")));
}
#[cfg(unix)]
#[test]
fn a_symlink_is_named_not_followed() {
+1
Переглянути файл
@@ -7,6 +7,7 @@ license.workspace = true
[dependencies]
ide-core.workspace = true
ide-jvm-attach.workspace = true
ide-run.workspace = true
serde.workspace = true
serde_json.workspace = true
+58 -10
Переглянути файл
@@ -1,6 +1,7 @@
//! Profile formats: collapsed ("folded") stacks, V8 `.cpuprofile` JSON and
//! `perf script` text.
use crate::ProfileFrameKind;
use crate::tree::{FrameKey, TreeBuilder};
use serde_json::Value;
use std::collections::HashMap;
@@ -19,7 +20,12 @@ fn folded_frame(raw: &str, base: &Path) -> FrameKey {
{
let p = Path::new(file);
let file = if p.is_absolute() { p.to_path_buf() } else { base.join(p) };
return FrameKey { name: raw[..open].to_owned(), file: Some(file), line: Some(line.saturating_sub(1)) };
return FrameKey {
name: raw[..open].to_owned(),
file: Some(file),
line: Some(line.saturating_sub(1)),
kind: ProfileFrameKind::Code,
};
}
}
// async-profiler frame type suffixes (`_[j]` JIT, `_[i]` interpreted, `_[k]` kernel…).
@@ -30,8 +36,16 @@ fn folded_frame(raw: &str, base: &Path) -> FrameKey {
FrameKey::named(name)
}
/// Adds collapsed stacks (`root;caller;callee <count>` per line).
pub(crate) fn add_folded(b: &mut TreeBuilder, text: &str, base: &Path) {
/// The allocated type of an async-profiler allocation leaf (`byte[]_[i]`, `java.lang.String_[k]`).
fn allocated_type(raw: &str) -> Option<&str> {
let raw = raw.trim();
let name = raw.strip_suffix("_[i]").or_else(|| raw.strip_suffix("_[k]"))?;
(!name.is_empty()).then_some(name)
}
/// Adds collapsed stacks (`root;caller;callee <count>` per line). With `allocation_leaf`
/// the last frame of a stack is the allocated type.
pub(crate) fn add_folded(b: &mut TreeBuilder, text: &str, base: &Path, allocation_leaf: bool) {
for line in text.lines() {
let line = line.trim_end();
let Some((stack, count)) = line.rsplit_once(' ') else { continue };
@@ -39,7 +53,13 @@ pub(crate) fn add_folded(b: &mut TreeBuilder, text: &str, base: &Path) {
if stack.is_empty() || count <= 0.0 {
continue;
}
b.add_stack(stack.split(';').map(|f| folded_frame(f, base)), count);
let mut frames: Vec<&str> = stack.split(';').collect();
let leaf = if allocation_leaf { frames.last().and_then(|f| allocated_type(f)) } else { None };
if leaf.is_some() {
frames.pop();
}
let stack = frames.into_iter().map(|f| folded_frame(f, base)).chain(leaf.map(FrameKey::allocated));
b.add_stack(stack, count);
}
}
@@ -137,7 +157,7 @@ pub(crate) fn add_cpuprofile(b: &mut TreeBuilder, json: &str) -> Result<(), Stri
let name = if name.is_empty() { "(anonymous)".to_owned() } else { name.to_owned() };
let line =
file.as_ref().and(cf.get("lineNumber").and_then(Value::as_i64)).filter(|l| *l >= 0).map(|l| l as u32);
Some(FrameKey { name, file, line })
Some(FrameKey { name, file, line, kind: ProfileFrameKind::Code })
};
for (id, weight) in self_time {
if weight <= 0.0 {
@@ -202,6 +222,34 @@ pub(crate) fn add_perf_script(b: &mut TreeBuilder, text: &str) {
#[cfg(test)]
mod tests {
use super::*;
use crate::ProfileMode;
#[test]
fn allocation_leaves_become_types() {
let mut b = TreeBuilder::new();
let text = "Alloc.main;Alloc.garbage;byte[]_[i] 541064184\n\
Alloc.main;Alloc.strings;java/lang/StringBuilder.toString;java.lang.String_[i] 6815731\n\
Alloc.main;Alloc.retained;byte[]_[k] 4112\n\
java.lang.String_[i] 524287\n";
add_folded(&mut b, text, Path::new("/p"), true);
let p = b.finish("async-profiler", "bytes", ProfileMode::Allocations);
assert_eq!(p.total, 541064184.0 + 6815731.0 + 4112.0 + 524287.0);
let kind = |n: &str| p.frames.iter().find(|f| f.name == n).unwrap().kind;
assert_eq!(kind("byte[]"), ProfileFrameKind::AllocatedType);
assert_eq!(kind("java.lang.String"), ProfileFrameKind::AllocatedType);
assert_eq!(kind("Alloc.garbage"), ProfileFrameKind::Code);
assert_eq!(kind("java/lang/StringBuilder.toString"), ProfileFrameKind::Code);
assert_eq!(p.frames.iter().filter(|f| f.name == "byte[]").count(), 1, "_[i] and _[k] are one type");
}
#[test]
fn cpu_mode_keeps_kernel_leaf_as_code() {
let mut b = TreeBuilder::new();
add_folded(&mut b, "java/lang/Thread.run;__psynch_cvwait_[k] 3\n", Path::new("/p"), false);
let p = b.finish("async-profiler", "samples", ProfileMode::Cpu);
assert!(p.frames.iter().all(|f| f.kind == ProfileFrameKind::Code));
assert!(p.frames.iter().any(|f| f.name == "__psynch_cvwait"));
}
#[test]
fn folded_stacks_with_py_spy_and_async_profiler_frames() {
@@ -210,8 +258,8 @@ mod tests {
<module> (main.py:10) 1\n\
java/lang/Thread.run_[j];com/x/App.work_[i] 2\n\
garbage\n";
add_folded(&mut b, text, Path::new("/proj"));
let p = b.finish("t", "samples");
add_folded(&mut b, text, Path::new("/proj"), false);
let p = b.finish("t", "samples", ProfileMode::Cpu);
assert_eq!(p.total, 6.0);
let compute = p.frames.iter().find(|f| f.name == "compute").unwrap();
assert_eq!(compute.file.as_deref(), Some(Path::new("/abs/lib.py")));
@@ -243,7 +291,7 @@ mod tests {
"startTime":0,"endTime":10,"samples":[3,3,2,4],"timeDeltas":[0,1000,2000,500]}"#;
let mut b = TreeBuilder::new();
add_cpuprofile(&mut b, json).unwrap();
let p = b.finish("node", "ms");
let p = b.finish("node", "ms", ProfileMode::Cpu);
// Sample i gets delta i+1: 3→1ms, 3→2ms, 2→0.5ms, 4→0.
assert!((p.total - 3.5).abs() < 1e-9);
let main = p.frames.iter().find(|f| f.name == "main").unwrap();
@@ -259,7 +307,7 @@ mod tests {
{"id":2,"callFrame":{"functionName":"f","url":"node:internal/x"},"hitCount":4}]}"#;
let mut b = TreeBuilder::new();
add_cpuprofile(&mut b, json).unwrap();
let p = b.finish("node", "samples");
let p = b.finish("node", "samples", ProfileMode::Cpu);
assert_eq!(p.total, 4.0);
assert_eq!(p.frames.iter().find(|f| f.name == "f").unwrap().file, None);
}
@@ -275,7 +323,7 @@ mod tests {
\t ffffff [unknown] ([kernel.kallsyms])\n";
let mut b = TreeBuilder::new();
add_perf_script(&mut b, text);
let p = b.finish("perf", "samples");
let p = b.finish("perf", "samples", ProfileMode::Cpu);
assert_eq!(p.total, 2.0);
let root = &p.nodes[0];
let names: Vec<&str> =
+47 -3
Переглянути файл
@@ -1,6 +1,6 @@
//! JVM frames (`com/x/App$Inner.work`, `com.x.App.work`) → source files of the project.
use crate::Profile;
use crate::{Profile, ProfileFrameKind};
use std::collections::HashMap;
use std::path::{Path, PathBuf};
@@ -72,6 +72,19 @@ fn class_key(frame: &str) -> Option<String> {
Some(class.replace('.', "/"))
}
/// `com.x.App$Inner[]` → `com/x/App`; primitive arrays (`byte[]` → `byte`) match no source.
fn type_key(name: &str) -> Option<String> {
let mut class = name;
while let Some(inner) = class.strip_suffix("[]") {
class = inner;
}
let class = class.split('$').next().unwrap_or(class);
if class.is_empty() || class.contains(['(', ' ', ':']) {
return None;
}
Some(class.replace('.', "/"))
}
pub(crate) fn resolve_sources(profile: &mut Profile, root: &Path) {
if profile.frames.iter().all(|f| f.file.is_some()) {
return;
@@ -81,7 +94,11 @@ pub(crate) fn resolve_sources(profile: &mut Profile, root: &Path) {
return;
}
for f in profile.frames.iter_mut().filter(|f| f.file.is_none()) {
if let Some(path) = class_key(&f.name).and_then(|k| index.get(&k)) {
let key = match f.kind {
ProfileFrameKind::Code => class_key(&f.name),
ProfileFrameKind::AllocatedType => type_key(&f.name),
};
if let Some(path) = key.and_then(|k| index.get(&k)) {
f.file = Some(path.clone().into());
}
}
@@ -90,6 +107,7 @@ pub(crate) fn resolve_sources(profile: &mut Profile, root: &Path) {
#[cfg(test)]
mod tests {
use super::*;
use crate::ProfileMode;
use crate::tree::{FrameKey, TreeBuilder};
#[test]
@@ -101,11 +119,37 @@ mod tests {
let mut b = TreeBuilder::new();
b.add_stack([FrameKey::named("java/lang/Thread.run"), FrameKey::named("com/x/App$Worker.work")], 1.0);
b.add_stack([FrameKey::named("com.x.App.main")], 1.0);
let mut p = b.finish("async-profiler", "samples");
let mut p = b.finish("async-profiler", "samples", ProfileMode::Cpu);
resolve_sources(&mut p, dir.path());
let file = |name: &str| p.frames.iter().find(|f| f.name == name).unwrap().file.clone();
assert_eq!(file("com/x/App$Worker.work").as_deref(), Some(src.join("App.java").as_path()));
assert_eq!(file("com.x.App.main").as_deref(), Some(src.join("App.java").as_path()));
assert_eq!(file("java/lang/Thread.run"), None);
}
#[test]
fn allocated_types_map_to_outer_class_source() {
let dir = tempfile::tempdir().unwrap();
let pkg = dir.path().join("app/src/main/java/com/x");
std::fs::create_dir_all(&pkg).unwrap();
std::fs::write(pkg.join("App.java"), "").unwrap();
let default_pkg = dir.path().join("app/src/main/java");
std::fs::write(default_pkg.join("Alloc.java"), "").unwrap();
let mut b = TreeBuilder::new();
b.add_stack([FrameKey::named("com.x.App.work"), FrameKey::allocated("com.x.App$Inner[]")], 1.0);
b.add_stack([FrameKey::named("com.x.App.work"), FrameKey::allocated("com.x.App")], 1.0);
b.add_stack([FrameKey::allocated("Alloc")], 1.0);
b.add_stack([FrameKey::allocated("byte[]")], 1.0);
b.add_stack([FrameKey::allocated("java.lang.String")], 1.0);
let mut p = b.finish("async-profiler", "bytes", ProfileMode::Allocations);
resolve_sources(&mut p, dir.path());
let file = |name: &str| p.frames.iter().find(|f| f.name == name).unwrap().file.clone();
let app = pkg.join("App.java");
assert_eq!(file("com.x.App$Inner[]").as_deref(), Some(app.as_path()));
assert_eq!(file("com.x.App").as_deref(), Some(app.as_path()));
assert_eq!(file("Alloc").as_deref(), Some(default_pkg.join("Alloc.java").as_path()));
assert_eq!(file("byte[]"), None);
assert_eq!(file("java.lang.String"), None);
assert_eq!(file("com.x.App.work").as_deref(), Some(app.as_path()), "code frames map as before");
}
}
+86 -9
Переглянути файл
@@ -13,9 +13,11 @@
mod formats;
mod jvm;
mod live;
mod plan;
mod tree;
pub use live::gc_before_stop;
pub use plan::prepare;
use ide_core::{IdeError, IdeResult, IpcPath};
@@ -27,6 +29,29 @@ use std::path::{Path, PathBuf};
use std::process::Command;
use tree::TreeBuilder;
/// What a profile measures.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum ProfileMode {
/// Sampled CPU time.
#[default]
Cpu,
/// Bytes allocated, by allocation stack.
Allocations,
/// Objects still alive when the run stopped, by allocation stack.
LiveObjects,
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum ProfileFrameKind {
/// A function; `file` / `line` locate its source.
#[default]
Code,
/// The leaf of an allocation stack: the type that was allocated, not a source line.
AllocatedType,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ProfileFrame {
@@ -34,6 +59,7 @@ pub struct ProfileFrame {
pub file: Option<IpcPath>,
/// 0-based.
pub line: Option<u32>,
pub kind: ProfileFrameKind,
}
/// A call-tree node; node 0 is the synthetic root (`all`).
@@ -64,8 +90,9 @@ pub struct MethodStat {
pub struct Profile {
/// `node --cpu-prof`, `py-spy`, `async-profiler`, `perf`.
pub profiler: String,
/// `ms` (time-weighted) or `samples`.
/// `ms` (time-weighted), `samples` (sample or live-object counts) or `bytes` (allocated).
pub unit: String,
pub mode: ProfileMode,
pub total: f64,
pub frames: Vec<ProfileFrame>,
/// Depth-first; `nodes[0]` is the root.
@@ -83,8 +110,9 @@ pub enum ProfileSpec {
/// out when another one did.
CpuProfiles { dir: IpcPath, root: IpcPath },
/// Collapsed-stack files with extension `ext` in `dir`; relative frame files resolve
/// against `base`. `jvm_root`: map JVM class frames to sources below it.
Folded { profiler: String, dir: IpcPath, ext: String, base: IpcPath, jvm_root: Option<IpcPath> },
/// against `base`. `jvm_root`: map JVM class frames to sources below it. For the
/// allocation modes the last frame of a JVM stack is the allocated type.
Folded { profiler: String, dir: IpcPath, ext: String, base: IpcPath, jvm_root: Option<IpcPath>, mode: ProfileMode },
/// `perf script -i <data>` output.
Perf { perf: String, data: IpcPath, cwd: IpcPath, env: BTreeMap<String, String> },
}
@@ -142,16 +170,21 @@ pub fn collect(spec: &ProfileSpec) -> IdeResult<Profile> {
if b.is_empty() {
return Err(nothing_recorded("no .cpuprofile was written"));
}
Ok(b.finish("node --cpu-prof", "ms"))
Ok(b.finish("node --cpu-prof", "ms", ProfileMode::Cpu))
}
ProfileSpec::Folded { profiler, dir, ext, base, jvm_root } => {
ProfileSpec::Folded { profiler, dir, ext, base, jvm_root, mode } => {
for f in files_with_ext(dir, ext) {
formats::add_folded(&mut b, &std::fs::read_to_string(&f)?, base);
formats::add_folded(&mut b, &std::fs::read_to_string(&f)?, base, *mode != ProfileMode::Cpu);
}
if b.is_empty() {
return Err(nothing_recorded(&format!("{profiler} recorded no samples")));
let mut err = nothing_recorded(&format!("{profiler} recorded no samples"));
if *mode == ProfileMode::LiveObjects {
err.message.push_str(" Profiling live objects needs JDK 11 or newer.");
}
return Err(err);
}
let mut profile = b.finish(profiler, "samples");
let unit = if *mode == ProfileMode::Allocations { "bytes" } else { "samples" };
let mut profile = b.finish(profiler, unit, *mode);
if let Some(root) = jvm_root {
jvm::resolve_sources(&mut profile, root);
}
@@ -178,7 +211,51 @@ pub fn collect(spec: &ProfileSpec) -> IdeResult<Profile> {
if b.is_empty() {
return Err(nothing_recorded("perf recorded no samples"));
}
Ok(b.finish("perf", "samples"))
Ok(b.finish("perf", "samples", ProfileMode::Cpu))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn folded(dir: &Path, mode: ProfileMode) -> ProfileSpec {
ProfileSpec::Folded {
profiler: "async-profiler".into(),
dir: dir.to_path_buf().into(),
ext: "collapsed".into(),
base: dir.to_path_buf().into(),
jvm_root: None,
mode,
}
}
#[test]
fn live_without_samples_mentions_jdk11() {
let dir = tempfile::tempdir().unwrap();
let err = collect(&folded(dir.path(), ProfileMode::LiveObjects)).unwrap_err();
assert!(err.message.contains("JDK 11"), "{}", err.message);
let err = collect(&folded(dir.path(), ProfileMode::Allocations)).unwrap_err();
assert!(!err.message.contains("JDK 11"), "{}", err.message);
}
#[test]
fn allocation_profile_is_in_bytes() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("profile-1.collapsed"), "A.main;byte[]_[i] 1024\n").unwrap();
let p = collect(&folded(dir.path(), ProfileMode::Allocations)).unwrap();
assert_eq!(p.unit, "bytes");
assert_eq!(p.mode, ProfileMode::Allocations);
assert_eq!(p.total, 1024.0);
assert!(p.frames.iter().any(|f| f.name == "byte[]" && f.kind == ProfileFrameKind::AllocatedType));
}
#[test]
fn cpu_profile_stays_in_samples() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("profile-1.collapsed"), "A.main;work_[j] 3\n").unwrap();
let p = collect(&folded(dir.path(), ProfileMode::Cpu)).unwrap();
assert_eq!((p.unit.as_str(), p.mode), ("samples", ProfileMode::Cpu));
}
}
+79
Переглянути файл
@@ -0,0 +1,79 @@
//! Live-object profiles hold what is alive when the agent writes them at exit; a full
//! collection right before the run stops drops the garbage that has not been swept yet.
use ide_jvm_attach::AttachError;
use std::time::Duration;
pub const GC_TIMEOUT: Duration = Duration::from_secs(5);
/// Runs `GC.run` in every JVM of `pid_tree` with an attach listener, so the live-object
/// profile the agent writes at exit holds only reachable objects. Returns how many JVMs collected.
pub fn gc_before_stop(pid_tree: &[i32]) -> u32 {
gc_each(pid_tree, ide_jvm_attach::has_listener, |pid| ide_jvm_attach::jcmd(pid, "GC.run", GC_TIMEOUT))
}
fn gc_each(pids: &[i32], has_listener: impl Fn(u32) -> bool, gc: impl Fn(u32) -> Result<String, AttachError>) -> u32 {
let mut collected = 0;
for &pid in pids {
let Ok(pid) = u32::try_from(pid) else { continue };
if pid == 0 || !has_listener(pid) {
continue;
}
match gc(pid) {
Ok(_) => collected += 1,
Err(e) => tracing::debug!("GC.run in JVM {pid} failed: {e}"),
}
}
collected
}
#[cfg(test)]
mod tests {
use super::*;
use std::cell::RefCell;
#[test]
fn gc_each_skips_pids_without_listener_and_counts_successes() {
let called = RefCell::new(Vec::new());
let n = gc_each(
&[10, 11, 12],
|pid| pid != 10,
|pid| {
called.borrow_mut().push(pid);
Ok(String::new())
},
);
assert_eq!(n, 2);
assert_eq!(*called.borrow(), vec![11, 12]);
}
#[test]
fn gc_each_continues_after_failure() {
let called = RefCell::new(Vec::new());
let n = gc_each(
&[11, 12],
|_| true,
|pid| {
called.borrow_mut().push(pid);
if pid == 11 { Err(AttachError::Timeout) } else { Ok(String::new()) }
},
);
assert_eq!(n, 1);
assert_eq!(*called.borrow(), vec![11, 12]);
}
#[test]
fn gc_each_ignores_negative_pids() {
let called = RefCell::new(Vec::new());
let n = gc_each(
&[-1, 5],
|_| true,
|pid| {
called.borrow_mut().push(pid);
Ok(String::new())
},
);
assert_eq!(n, 1);
assert_eq!(*called.borrow(), vec![5]);
}
}
+206 -22
Переглянути файл
@@ -1,6 +1,6 @@
//! Which profiler samples a configuration, and how its command line changes.
use crate::{ProfilePrepared, ProfileSpec, ProfilerInstall};
use crate::{ProfileMode, ProfilePrepared, ProfileSpec, ProfilerInstall};
use ide_run::RunConfig;
use ide_run::wrap::{self, program_name};
use std::path::{Path, PathBuf};
@@ -24,21 +24,30 @@ fn ready(profiler: &str, launch: RunConfig, spec: ProfileSpec) -> ProfilePrepare
ProfilePrepared::Ready { profiler: profiler.to_owned(), launch: Box::new(launch), spec }
}
/// What `key` holds for `config`: its own environment first, then the IDE's.
fn env_base(config: &RunConfig, key: &str) -> String {
config.env.get(key).cloned().or_else(|| std::env::var(key).ok()).unwrap_or_default()
}
/// Appends `value` to the space-separated option variable `key` of `config`'s environment.
fn append_env(config: &mut RunConfig, key: &str, value: &str) {
let base = config.env.get(key).cloned().or_else(|| std::env::var(key).ok()).unwrap_or_default();
let base = env_base(config, key);
let joined = if base.trim().is_empty() { value.to_owned() } else { format!("{base} {value}") };
config.env.insert(key.to_owned(), joined);
}
/// How to run `config` under a sampling profiler. `out` is an existing empty
/// directory for the profile. Probes tools: call from a blocking thread.
pub fn prepare(root: &Path, config: &RunConfig, out: &Path) -> ProfilePrepared {
/// Allocation modes exist for JVM programs only.
pub fn prepare(root: &Path, config: &RunConfig, out: &Path, mode: ProfileMode) -> ProfilePrepared {
let dir = wrap::run_dir(root, config);
let program = program_name(&config.command);
if mode != ProfileMode::Cpu && program != "java" {
return unsupported("Allocation profiling is available for JVM applications.");
}
match program.as_str() {
"node" | "npx" | "tsx" | "ts-node" | "npm" | "pnpm" | "yarn" => node(root, config, out),
"java" => jvm(root, config, &dir, out),
"java" => jvm(root, config, &dir, out, mode),
"cargo" => cargo(config, &dir, out),
"bun" | "deno" => unsupported(format!("Profiling {program} programs is not supported.")),
_ if config.kind == "native" => native(config, &dir, out),
@@ -105,6 +114,7 @@ fn python(root: &Path, config: &RunConfig, dir: &Path, out: &Path, py: wrap::Pyt
ext: s("txt"),
base: dir.to_path_buf().into(),
jvm_root: None,
mode: ProfileMode::Cpu,
};
ready("py-spy", launch, spec)
}
@@ -127,10 +137,29 @@ fn async_profiler_lib(config: &RunConfig) -> Option<PathBuf> {
homes.into_iter().map(|h| h.join("lib").join(lib)).find(|p| p.is_file())
}
fn jvm(root: &Path, config: &RunConfig, dir: &Path, out: &Path) -> ProfilePrepared {
/// async-profiler's `live` option needs JDK 11+. An unreadable version does not stop the run.
/// Only an absolute launcher is checked: a bare `java` is resolved to the project's JDK at
/// launch, which need not be the first `java` on the search path.
fn live_objects_jdk_too_old(config: &RunConfig) -> Option<ProfilePrepared> {
let command = Path::new(&config.command);
if !command.is_absolute() {
return None;
}
let version = ide_run::java_cmdline::java_feature_version(command)?;
(version < 11).then(|| {
unsupported(format!("Profiling live objects needs JDK 11 or newer; '{}' runs JDK {version}.", config.name))
})
}
fn jvm(root: &Path, config: &RunConfig, dir: &Path, out: &Path, mode: ProfileMode) -> ProfilePrepared {
if cfg!(windows) {
return unsupported("async-profiler, used to profile JVM applications, does not run on Windows.");
}
if mode == ProfileMode::LiveObjects
&& let Some(refusal) = live_objects_jdk_too_old(config)
{
return refusal;
}
let Some(lib) = async_profiler_lib(config) else {
return ProfilePrepared::MissingTool {
tool: s("async-profiler"),
@@ -143,7 +172,20 @@ fn jvm(root: &Path, config: &RunConfig, dir: &Path, out: &Path) -> ProfilePrepar
};
let mut launch = config.clone();
let file = out.join("profile-%p.collapsed");
let agent = format!("-agentpath:{}=start,event=cpu,collapsed,file={}", path(&lib), path(&file));
// One event per run: async-profiler cannot combine `cpu` with `alloc` in collapsed output.
let event = match mode {
ProfileMode::Cpu => "cpu",
ProfileMode::Allocations => "alloc,total",
ProfileMode::LiveObjects => "alloc,live",
};
let agent = format!("-agentpath:{}=start,event={event},collapsed,file={}", path(&lib), path(&file));
if mode == ProfileMode::LiveObjects {
// The IDE reaches the running JVM through its attach listener, never by SIGQUIT.
let jvm_args = format!("{} {}", env_base(config, "JAVA_TOOL_OPTIONS"), config.args.join(" "));
if ide_jvm_attach::wants_listener_flag(&jvm_args) {
append_env(&mut launch, "JAVA_TOOL_OPTIONS", ide_jvm_attach::START_LISTENER_FLAG);
}
}
append_env(&mut launch, "JAVA_TOOL_OPTIONS", &quote_opt(&agent));
let spec = ProfileSpec::Folded {
profiler: s("async-profiler"),
@@ -151,6 +193,7 @@ fn jvm(root: &Path, config: &RunConfig, dir: &Path, out: &Path) -> ProfilePrepar
ext: s("collapsed"),
base: dir.to_path_buf().into(),
jvm_root: Some(root.to_path_buf().into()),
mode,
};
ready("async-profiler", launch, spec)
}
@@ -242,7 +285,7 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let mut c = cfg("node", "node", &["index.js"]);
c.env.insert("NODE_OPTIONS".into(), "--max-old-space-size=4096".into());
match prepare(dir.path(), &c, dir.path()) {
match prepare(dir.path(), &c, dir.path(), ProfileMode::Cpu) {
ProfilePrepared::Ready { launch, spec, .. } => {
let opts = &launch.env["NODE_OPTIONS"];
assert!(opts.starts_with("--max-old-space-size=4096 --cpu-prof --cpu-prof-dir="));
@@ -262,7 +305,7 @@ mod tests {
std::fs::write(&python, "").unwrap();
let mut c = cfg("python", &python.to_string_lossy(), &["main.py"]);
c.env.insert("PATH".into(), dir.path().join("nothing").to_string_lossy().into_owned());
match prepare(dir.path(), &c, dir.path()) {
match prepare(dir.path(), &c, dir.path(), ProfileMode::Cpu) {
ProfilePrepared::MissingTool { tool, install: Some(i), .. } => {
assert_eq!(tool, "py-spy");
assert_eq!(i.args, ["-m", "pip", "install", "py-spy"]);
@@ -271,7 +314,7 @@ mod tests {
}
let spy = bin.join(if cfg!(windows) { "py-spy.exe" } else { "py-spy" });
std::fs::write(&spy, "").unwrap();
match prepare(dir.path(), &c, dir.path()) {
match prepare(dir.path(), &c, dir.path(), ProfileMode::Cpu) {
ProfilePrepared::Ready { launch, .. } => {
assert_eq!(launch.command, spy.to_string_lossy());
assert_eq!(&launch.args[..4], ["record", "--format", "raw", "--function"]);
@@ -291,35 +334,176 @@ mod tests {
let mut c = cfg("application", "java", &["-cp", "${classpath}", "com.x.App"]);
c.env.insert("PATH".into(), dir.path().to_string_lossy().into_owned());
c.env.insert("ASYNC_PROFILER_HOME".into(), dir.path().join("none").to_string_lossy().into_owned());
assert!(matches!(prepare(dir.path(), &c, dir.path()), ProfilePrepared::MissingTool { install: None, .. }));
let lib_dir = dir.path().join("ap/lib");
std::fs::create_dir_all(&lib_dir).unwrap();
let lib =
lib_dir.join(if cfg!(target_os = "macos") { "libasyncProfiler.dylib" } else { "libasyncProfiler.so" });
std::fs::write(&lib, "").unwrap();
c.env.insert("ASYNC_PROFILER_HOME".into(), dir.path().join("ap").to_string_lossy().into_owned());
match prepare(dir.path(), &c, dir.path()) {
ProfilePrepared::Ready { launch, .. } => {
assert!(launch.env["JAVA_TOOL_OPTIONS"].contains("collapsed,file="));
assert!(matches!(
prepare(dir.path(), &c, dir.path(), ProfileMode::Cpu),
ProfilePrepared::MissingTool { install: None, .. }
));
let home = fake_async_profiler(dir.path());
c.env.insert("ASYNC_PROFILER_HOME".into(), home.to_string_lossy().into_owned());
match prepare(dir.path(), &c, dir.path(), ProfileMode::Cpu) {
ProfilePrepared::Ready { launch, spec, .. } => {
let opts = &launch.env["JAVA_TOOL_OPTIONS"];
assert!(opts.contains("=start,event=cpu,collapsed,file="));
assert!(!opts.contains("StartAttachListener"));
assert!(matches!(spec, ProfileSpec::Folded { mode: ProfileMode::Cpu, .. }));
}
other => panic!("{other:?}"),
}
}
/// A JVM configuration whose async-profiler and `PATH` are both under `dir`.
fn jvm_cfg(dir: &Path, command: &str, args: &[&str]) -> RunConfig {
let mut c = cfg("application", command, args);
c.env.insert("PATH".into(), dir.join("nothing").to_string_lossy().into_owned());
c.env.insert("ASYNC_PROFILER_HOME".into(), fake_async_profiler(dir).to_string_lossy().into_owned());
c.env.insert("JAVA_TOOL_OPTIONS".into(), "-Xmx2g".into());
c
}
/// Creates `<dir>/ap/lib/libasyncProfiler.*` and returns the home directory.
fn fake_async_profiler(dir: &Path) -> PathBuf {
let home = dir.join("ap");
let lib_dir = home.join("lib");
std::fs::create_dir_all(&lib_dir).unwrap();
let lib =
lib_dir.join(if cfg!(target_os = "macos") { "libasyncProfiler.dylib" } else { "libasyncProfiler.so" });
std::fs::write(lib, "").unwrap();
home
}
fn ready_jvm(c: &RunConfig, dir: &Path, mode: ProfileMode) -> (RunConfig, ProfileSpec) {
match prepare(dir, c, dir, mode) {
ProfilePrepared::Ready { launch, spec, .. } => (*launch, spec),
other => panic!("{other:?}"),
}
}
#[test]
fn jvm_allocations_records_bytes() {
if cfg!(windows) {
return;
}
let dir = tempfile::tempdir().unwrap();
let c = jvm_cfg(dir.path(), "java", &["-cp", "${classpath}", "com.x.App"]);
let (launch, spec) = ready_jvm(&c, dir.path(), ProfileMode::Allocations);
let opts = &launch.env["JAVA_TOOL_OPTIONS"];
assert!(opts.contains("=start,event=alloc,total,collapsed,file="), "{opts}");
assert!(!opts.contains("StartAttachListener"), "{opts}");
assert!(!opts.contains("alloc="), "{opts}");
assert!(
matches!(spec, ProfileSpec::Folded { mode: ProfileMode::Allocations, ref ext, .. } if ext == "collapsed")
);
}
#[test]
fn jvm_live_objects_starts_the_attach_listener() {
if cfg!(windows) {
return;
}
let dir = tempfile::tempdir().unwrap();
let c = jvm_cfg(dir.path(), "java", &["-cp", "${classpath}", "com.x.App"]);
let (launch, spec) = ready_jvm(&c, dir.path(), ProfileMode::LiveObjects);
let opts = &launch.env["JAVA_TOOL_OPTIONS"];
assert!(opts.starts_with("-Xmx2g -XX:+StartAttachListener -agentpath:"), "{opts}");
assert!(opts.contains("event=alloc,live,collapsed"), "{opts}");
assert!(!opts.contains("total"), "{opts}");
assert!(
matches!(spec, ProfileSpec::Folded { mode: ProfileMode::LiveObjects, ref ext, .. } if ext == "collapsed")
);
}
#[test]
fn live_objects_keeps_user_attach_opt_out() {
if cfg!(windows) {
return;
}
let dir = tempfile::tempdir().unwrap();
let c = jvm_cfg(dir.path(), "java", &["-XX:+DisableAttachMechanism", "com.x.App"]);
let (launch, _) = ready_jvm(&c, dir.path(), ProfileMode::LiveObjects);
assert!(!launch.env["JAVA_TOOL_OPTIONS"].contains("StartAttachListener"));
let mut c = jvm_cfg(dir.path(), "java", &["com.x.App"]);
c.env.insert("JAVA_TOOL_OPTIONS".into(), "-XX:-StartAttachListener".into());
let (launch, _) = ready_jvm(&c, dir.path(), ProfileMode::LiveObjects);
let opts = &launch.env["JAVA_TOOL_OPTIONS"];
assert!(opts.starts_with("-XX:-StartAttachListener -agentpath:"), "{opts}");
assert!(!opts.contains("+StartAttachListener"), "{opts}");
}
#[test]
fn live_objects_refuses_jdk8() {
if cfg!(windows) {
return;
}
let dir = tempfile::tempdir().unwrap();
let java = dir.path().join("jdk8/bin/java");
std::fs::create_dir_all(java.parent().unwrap()).unwrap();
std::fs::write(&java, "").unwrap();
std::fs::write(dir.path().join("jdk8/release"), "JAVA_VERSION=\"1.8.0_512\"\n").unwrap();
// No async-profiler is installed: the refusal does not depend on it.
let mut c = cfg("application", &java.to_string_lossy(), &["com.x.App"]);
c.env.insert("PATH".into(), dir.path().join("nothing").to_string_lossy().into_owned());
c.env.insert("ASYNC_PROFILER_HOME".into(), dir.path().join("none").to_string_lossy().into_owned());
match prepare(dir.path(), &c, dir.path(), ProfileMode::LiveObjects) {
ProfilePrepared::Unsupported { reason } => assert!(reason.contains("JDK 11"), "{reason}"),
other => panic!("{other:?}"),
}
// Allocations do not need the attach listener, so JDK 8 is fine.
assert!(matches!(
prepare(dir.path(), &c, dir.path(), ProfileMode::Allocations),
ProfilePrepared::MissingTool { .. }
));
std::fs::write(dir.path().join("jdk8/release"), "JAVA_VERSION=\"11.0.29\"\n").unwrap();
c.env.insert("ASYNC_PROFILER_HOME".into(), fake_async_profiler(dir.path()).to_string_lossy().into_owned());
assert!(matches!(prepare(dir.path(), &c, dir.path(), ProfileMode::LiveObjects), ProfilePrepared::Ready { .. }));
}
#[test]
fn live_objects_runs_a_bare_java_command_whatever_jdk_is_on_the_path() {
if cfg!(windows) {
return;
}
let dir = tempfile::tempdir().unwrap();
let bin = dir.path().join("jdk8/bin");
std::fs::create_dir_all(&bin).unwrap();
std::fs::write(bin.join("java"), "").unwrap();
std::fs::write(dir.path().join("jdk8/release"), "JAVA_VERSION=\"1.8.0_512\"\n").unwrap();
// The launcher picks the project's JDK, not the first `java` on the search path.
let mut c = cfg("application", "java", &["com.x.App"]);
c.env.insert("PATH".into(), bin.to_string_lossy().into_owned());
c.env.insert("ASYNC_PROFILER_HOME".into(), fake_async_profiler(dir.path()).to_string_lossy().into_owned());
let prepared = prepare(dir.path(), &c, dir.path(), ProfileMode::LiveObjects);
assert!(matches!(prepared, ProfilePrepared::Ready { .. }), "{prepared:?}");
}
#[test]
fn allocation_modes_need_a_jvm() {
let dir = tempfile::tempdir().unwrap();
let c = cfg("node", "node", &["index.js"]);
for mode in [ProfileMode::Allocations, ProfileMode::LiveObjects] {
match prepare(dir.path(), &c, dir.path(), mode) {
ProfilePrepared::Unsupported { reason } => assert!(reason.contains("JVM"), "{reason}"),
other => panic!("{other:?}"),
}
}
assert!(matches!(prepare(dir.path(), &c, dir.path(), ProfileMode::Cpu), ProfilePrepared::Ready { .. }));
}
#[test]
fn cargo_and_unknown() {
let dir = tempfile::tempdir().unwrap();
assert!(matches!(
prepare(dir.path(), &cfg("cargo", "cargo", &["test"]), dir.path()),
prepare(dir.path(), &cfg("cargo", "cargo", &["test"]), dir.path(), ProfileMode::Cpu),
ProfilePrepared::Unsupported { .. }
));
assert!(matches!(
prepare(dir.path(), &cfg("make", "make", &[]), dir.path()),
prepare(dir.path(), &cfg("make", "make", &[]), dir.path(), ProfileMode::Cpu),
ProfilePrepared::Unsupported { .. }
));
if !cfg!(target_os = "linux") {
assert!(matches!(
prepare(dir.path(), &cfg("cargo", "cargo", &["run"]), dir.path()),
prepare(dir.path(), &cfg("cargo", "cargo", &["run"]), dir.path(), ProfileMode::Cpu),
ProfilePrepared::Unsupported { .. }
));
}
+12 -6
Переглянути файл
@@ -1,19 +1,25 @@
//! Building the call tree from stacks.
use crate::{MethodStat, Profile, ProfileFrame, ProfileNode};
use crate::{MethodStat, Profile, ProfileFrame, ProfileFrameKind, ProfileMode, ProfileNode};
use std::collections::HashMap;
use std::path::PathBuf;
/// Identity of a frame; `kind` keeps an allocated type apart from a function of the same name.
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
pub(crate) struct FrameKey {
pub name: String,
pub file: Option<PathBuf>,
pub line: Option<u32>,
pub kind: ProfileFrameKind,
}
impl FrameKey {
pub fn named(name: impl Into<String>) -> Self {
Self { name: name.into(), file: None, line: None }
Self { name: name.into(), file: None, line: None, kind: ProfileFrameKind::Code }
}
pub fn allocated(name: impl Into<String>) -> Self {
Self { kind: ProfileFrameKind::AllocatedType, ..Self::named(name) }
}
}
@@ -77,7 +83,7 @@ impl TreeBuilder {
self.nodes.len() <= 1 && self.nodes[0].self_weight == 0.0
}
pub fn finish(self, profiler: &str, unit: &str) -> Profile {
pub fn finish(self, profiler: &str, unit: &str, mode: ProfileMode) -> Profile {
let n = self.nodes.len();
// Renumber in depth-first order, children by total weight (heaviest first).
// Children are always created after their parent, so a reverse sweep sums bottom-up.
@@ -112,10 +118,10 @@ impl TreeBuilder {
let frames: Vec<ProfileFrame> = self
.frames
.into_iter()
.map(|k| ProfileFrame { name: k.name, file: k.file.map(Into::into), line: k.line })
.map(|k| ProfileFrame { name: k.name, file: k.file.map(Into::into), line: k.line, kind: k.kind })
.collect();
let methods = methods(&nodes, frames.len());
Profile { profiler: profiler.to_owned(), unit: unit.to_owned(), total: totals[0], frames, nodes, methods }
Profile { profiler: profiler.to_owned(), unit: unit.to_owned(), mode, total: totals[0], frames, nodes, methods }
}
}
@@ -165,7 +171,7 @@ mod tests {
b.add_stack(stack(&["main", "b"]), 3.0);
b.add_stack(stack(&["main", "b", "b"]), 2.0);
b.add_stack(stack(&["main"]), 1.0);
let p = b.finish("t", "samples");
let p = b.finish("t", "samples", crate::ProfileMode::Cpu);
assert_eq!(p.total, 7.0);
let name = |n: &ProfileNode| p.frames[n.frame as usize].name.as_str();
let root = &p.nodes[0];
+212
Переглянути файл
@@ -0,0 +1,212 @@
//! Allocation and live-object profiles of a real JVM under async-profiler.
//!
//! Run with `OPENIDEA_TEST_JAVA=<path to java> ASYNC_PROFILER_HOME=<async-profiler home>
//! cargo test -p ide-profiler --test async_profiler_jvm -- --ignored`; without both the tests return early.
#![cfg(unix)]
use ide_profiler::{Profile, ProfileFrameKind, ProfileMode, ProfilePrepared, collect, gc_before_stop, prepare};
use ide_run::RunConfig;
use std::collections::BTreeMap;
use std::io::{BufRead, BufReader};
use std::path::{Path, PathBuf};
use std::process::{Child, Command, ExitStatus, Stdio};
use std::sync::mpsc::{self, Receiver};
use std::time::{Duration, Instant};
const MIB: f64 = 1024.0 * 1024.0;
/// 64 MiB stay reachable from a static field; 256 MiB are dropped at once. The volatile store
/// keeps the JIT from removing the garbage allocations.
const ALLOC_JAVA: &str = r#"
import java.util.ArrayList;
import java.util.List;
public class Alloc {
static final List<byte[]> keep = new ArrayList<>();
static volatile Object sink;
static void retained() {
for (int i = 0; i < 16384; i++) keep.add(new byte[4096]);
}
static void garbage() {
for (int i = 0; i < 262144; i++) sink = new byte[1024];
}
public static void main(String[] args) throws Exception {
retained();
garbage();
if (args.length > 0 && args[0].equals("wait")) {
System.out.println("WAITING");
System.out.flush();
while (true) Thread.sleep(1000);
}
}
}
"#;
struct Env {
java: PathBuf,
}
fn env() -> Option<Env> {
let java = std::env::var_os("OPENIDEA_TEST_JAVA")?;
std::env::var_os("ASYNC_PROFILER_HOME")?;
Some(Env { java: java.into() })
}
/// A started JVM that is always stopped when the test ends, even by a failed assertion.
struct Jvm {
child: Child,
lines: Receiver<String>,
}
impl Jvm {
fn start(launch: &RunConfig, cwd: &Path) -> Jvm {
let mut child = Command::new(&launch.command)
.args(&launch.args)
.envs(&launch.env)
.current_dir(cwd)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()
.expect("start java");
let stdout = child.stdout.take().unwrap();
let (tx, lines) = mpsc::channel();
std::thread::spawn(move || {
for line in BufReader::new(stdout).lines().map_while(Result::ok) {
if tx.send(line).is_err() {
break;
}
}
});
Jvm { child, lines }
}
fn pid(&self) -> i32 {
self.child.id() as i32
}
fn wait_for_line(&self, wanted: &str, timeout: Duration) {
let deadline = Instant::now() + timeout;
loop {
let left = deadline.saturating_duration_since(Instant::now());
match self.lines.recv_timeout(left) {
Ok(line) if line.trim() == wanted => return,
Ok(_) => {}
Err(e) => panic!("the JVM never printed {wanted}: {e}"),
}
}
}
fn interrupt(&self) {
Command::new("kill").args(["-INT", &self.pid().to_string()]).status().expect("run kill");
}
fn wait_exit(&mut self, timeout: Duration) -> ExitStatus {
self.try_wait_exit(timeout).unwrap_or_else(|| panic!("the JVM did not exit in {timeout:?}"))
}
fn try_wait_exit(&mut self, timeout: Duration) -> Option<ExitStatus> {
let deadline = Instant::now() + timeout;
loop {
if let Ok(Some(status)) = self.child.try_wait() {
return Some(status);
}
if Instant::now() >= deadline {
return None;
}
std::thread::sleep(Duration::from_millis(50));
}
}
}
impl Drop for Jvm {
fn drop(&mut self) {
if self.child.try_wait().ok().flatten().is_some() {
return;
}
self.interrupt();
if self.try_wait_exit(Duration::from_secs(10)).is_none() {
let _ = self.child.kill();
let _ = self.child.wait();
}
}
}
struct Run {
_dir: tempfile::TempDir,
cwd: PathBuf,
launch: RunConfig,
spec: ide_profiler::ProfileSpec,
}
fn prepared(env: &Env, mode: ProfileMode, program_mode: &str) -> Run {
let dir = tempfile::tempdir().unwrap();
let cwd = dir.path().to_path_buf();
let source = cwd.join("Alloc.java");
std::fs::write(&source, ALLOC_JAVA).unwrap();
let out = cwd.join("out");
std::fs::create_dir(&out).unwrap();
let config = RunConfig {
id: "alloc".into(),
name: "Alloc".into(),
kind: "application".into(),
command: env.java.to_string_lossy().into_owned(),
args: vec![source.to_string_lossy().into_owned(), program_mode.into()],
cwd: None,
env: BTreeMap::new(),
detected: false,
is_test: false,
debug_adapter: None,
allow_parallel: false,
extras: Default::default(),
};
match prepare(&cwd, &config, &out, mode) {
ProfilePrepared::Ready { launch, spec, .. } => Run { _dir: dir, cwd, launch: *launch, spec },
other => panic!("{other:?}"),
}
}
/// Weight of every node whose function is `name`.
fn weight_of(profile: &Profile, name: &str) -> f64 {
profile.nodes.iter().filter(|n| profile.frames[n.frame as usize].name == name).fold(0.0, |sum, n| sum + n.total)
}
#[test]
#[ignore = "needs a JDK (OPENIDEA_TEST_JAVA) and async-profiler (ASYNC_PROFILER_HOME)"]
fn allocations_profile_weighs_bytes() {
let Some(env) = env() else { return };
let run = prepared(&env, ProfileMode::Allocations, "exit");
let mut jvm = Jvm::start(&run.launch, &run.cwd);
assert!(jvm.wait_exit(Duration::from_secs(120)).success());
let profile = collect(&run.spec).unwrap();
assert_eq!(profile.unit, "bytes");
let garbage = weight_of(&profile, "Alloc.garbage");
eprintln!("allocations: Alloc.garbage = {:.1} MiB, total = {:.1} MiB", garbage / MIB, profile.total / MIB);
assert!((200.0 * MIB..=330.0 * MIB).contains(&garbage), "Alloc.garbage = {:.1} MiB", garbage / MIB);
assert!(profile.frames.iter().any(|f| f.name == "byte[]" && f.kind == ProfileFrameKind::AllocatedType));
}
#[test]
#[ignore = "needs a JDK (OPENIDEA_TEST_JAVA) and async-profiler (ASYNC_PROFILER_HOME)"]
fn live_objects_after_gc_keep_only_retained() {
let Some(env) = env() else { return };
let run = prepared(&env, ProfileMode::LiveObjects, "wait");
let mut jvm = Jvm::start(&run.launch, &run.cwd);
jvm.wait_for_line("WAITING", Duration::from_secs(120));
assert_eq!(gc_before_stop(&[jvm.pid()]), 1);
jvm.interrupt();
jvm.wait_exit(Duration::from_secs(60));
let profile = collect(&run.spec).unwrap();
assert_eq!(profile.unit, "samples");
let retained = weight_of(&profile, "Alloc.retained");
let garbage = weight_of(&profile, "Alloc.garbage");
let share = retained / (retained + garbage);
eprintln!("live objects: retained = {retained}, garbage = {garbage}, share = {:.1} %", share * 100.0);
assert!(retained > 0.0, "no live sample under Alloc.retained");
assert!(share >= 0.9, "retained share {:.1} %", share * 100.0);
}
+1 -1
Переглянути файл
@@ -69,7 +69,7 @@ pub struct TransferError {
}
impl TransferReport {
fn fail(&mut self, path: impl Into<String>, e: IdeError) {
pub(crate) fn fail(&mut self, path: impl Into<String>, e: IdeError) {
self.errors.push(TransferError { path: path.into(), message: e.message });
}
}
+131
Переглянути файл
@@ -286,6 +286,70 @@ impl RemoteService {
r
}
/// Remote Host › Download to…: `remotes` (folders recursively) into the local folder `dir`,
/// each under its own name, without the server path above it. Existing files are replaced.
pub async fn download_into(
&self,
connection: &FsConnection,
remotes: &[String],
dir: &Path,
cancel: &CancellationToken,
progress: &mut (dyn FnMut(&str, usize, usize) + Send),
) -> IdeResult<TransferReport> {
let fs = self.fs(connection).await?;
let mut report = TransferReport::default();
let mut folders = Vec::new();
let mut jobs = Vec::new();
for remote in remotes {
let Some(top) = download_target(dir, remote, remote) else {
report.skipped += 1;
continue;
};
match fs.stat(remote).await {
Ok(Some(e)) if e.is_dir => {
folders.push(top);
let mut entries = Vec::new();
if let Err(e) = remote_tree(&fs, remote, cancel, &mut entries).await {
report.fail(remote.clone(), e);
}
for entry in entries {
match download_target(dir, remote, &entry.path) {
Some(local) if entry.is_dir => folders.push(local),
Some(local) => jobs.push((entry.path, local)),
None => report.skipped += 1,
}
}
}
Ok(Some(_)) => jobs.push((remote.clone(), top)),
Ok(None) => report.fail(remote.clone(), IdeError::not_found(format!("{remote} does not exist"))),
Err(e) => report.fail(remote.clone(), e),
}
}
for folder in &folders {
if let Err(e) = tokio::fs::create_dir_all(folder).await {
report.fail(folder.display().to_string(), e.into());
}
}
for (i, (remote, local)) in jobs.iter().enumerate() {
if cancel.is_cancelled() {
break;
}
progress(remote, i, jobs.len());
if let Some(parent) = local.parent()
&& let Err(e) = tokio::fs::create_dir_all(parent).await
{
report.fail(remote.clone(), e.into());
continue;
}
match fs.download(remote, local).await {
Ok(()) => report.files += 1,
Err(e) => report.fail(remote.clone(), e),
}
}
fs.close().await;
Ok(report)
}
// --- save hook -----------------------------------------------------------------
/// After a document save: uploads a remote file's copy, or (automatic upload)
@@ -497,3 +561,70 @@ fn cache_key(connection: &FsConnection) -> String {
FsConnection::Sftp { ssh } => format!("sftp-{ssh}"),
}
}
/// Files and folders under remote folder `dir` (symlinked folders are not followed).
async fn remote_tree(
fs: &RemoteFs,
dir: &str,
cancel: &CancellationToken,
out: &mut Vec<RemoteEntry>,
) -> IdeResult<()> {
for e in fs.list(dir).await? {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
let descend = e.is_dir && !e.is_symlink;
let path = e.path.clone();
out.push(e);
if descend {
Box::pin(remote_tree(fs, &path, cancel, out)).await?;
}
}
Ok(())
}
/// Where remote `path`, inside the downloaded `selected` file or folder, lands under local `dir`:
/// `dir/<name of selected>/<rest>`. `None` for a name that would leave `dir`.
fn download_target(dir: &Path, selected: &str, path: &str) -> Option<PathBuf> {
let selected = selected.trim_end_matches('/');
let name = selected.rsplit('/').next()?;
let rest = path.trim_end_matches('/').strip_prefix(selected)?;
if !rest.is_empty() && !rest.starts_with('/') {
return None;
}
let mut local = dir.to_path_buf();
for part in std::iter::once(name).chain(rest.split('/').filter(|p| !p.is_empty())) {
if matches!(part, "" | "." | "..") || part.contains('\\') || (cfg!(windows) && part.contains(':')) {
return None;
}
local.push(part);
}
Some(local)
}
#[cfg(test)]
mod download_into_tests {
use super::*;
#[test]
fn downloads_keep_only_the_selected_name() {
let dir = Path::new("/p/project");
let selected = "/home/u/www/site/current/drush";
assert_eq!(download_target(dir, selected, selected), Some(dir.join("drush")));
assert_eq!(download_target(dir, "/home/u/www/site/current/drush/", selected), Some(dir.join("drush")));
assert_eq!(
download_target(dir, selected, "/home/u/www/site/current/drush/src/a.php"),
Some(dir.join("drush").join("src").join("a.php"))
);
assert_eq!(download_target(dir, "/srv/app.log", "/srv/app.log"), Some(dir.join("app.log")));
}
#[test]
fn names_that_would_leave_the_folder_are_refused() {
let dir = Path::new("/p/project");
assert_eq!(download_target(dir, "/srv/a", "/srv/a/../../etc/passwd"), None);
assert_eq!(download_target(dir, "/srv/a", "/srv/ab/x"), None, "a sibling with the same prefix");
assert_eq!(download_target(dir, "/", "/etc"), None, "the root has no name");
assert_eq!(download_target(dir, "/srv/..", "/srv/.."), None);
}
}
+34
Переглянути файл
@@ -319,3 +319,37 @@ async fn browser_mappings_resolve_symlinks_and_download_to_the_project() {
assert!(report.errors.is_empty());
assert_eq!(std::fs::read_to_string(local).unwrap(), "from the server");
}
#[tokio::test]
async fn download_into_copies_only_the_selected_items() {
let tmp = tempfile::tempdir().unwrap();
let server = tmp.path().join("srv/www/site/current");
let project = tmp.path().join("project");
std::fs::create_dir_all(server.join("drush/sub")).unwrap();
std::fs::create_dir_all(server.join("drush/empty")).unwrap();
std::fs::create_dir_all(&project).unwrap();
std::fs::write(server.join("drush/a.txt"), "a").unwrap();
std::fs::write(server.join("drush/sub/b.txt"), "b").unwrap();
std::fs::write(server.join("index.php"), "php").unwrap();
std::fs::write(project.join("index.php"), "old").unwrap();
let service = service(tmp.path());
let remotes = [server.join("drush"), server.join("index.php")].map(|p| p.to_str().unwrap().to_owned());
let report = service
.download_into(&FsConnection::Local, &remotes, &project, &CancellationToken::new(), &mut |_, _, _| {})
.await
.unwrap();
assert_eq!(report.files, 3, "{report:?}");
assert!(report.errors.is_empty(), "{report:?}");
assert_eq!(std::fs::read_to_string(project.join("drush/a.txt")).unwrap(), "a");
assert_eq!(std::fs::read_to_string(project.join("drush/sub/b.txt")).unwrap(), "b");
assert!(project.join("drush/empty").is_dir());
assert_eq!(std::fs::read_to_string(project.join("index.php")).unwrap(), "php");
assert!(!project.join("srv").exists() && !project.join("www").exists());
let missing = [server.join("gone").to_str().unwrap().to_owned()];
let report = service
.download_into(&FsConnection::Local, &missing, &project, &CancellationToken::new(), &mut |_, _, _| {})
.await
.unwrap();
assert_eq!((report.files, report.errors.len()), (0, 1), "{report:?}");
}
+179 -8
Переглянути файл
@@ -863,6 +863,48 @@ fn module_dir(root: &Path, module: Option<&Path>) -> PathBuf {
}
}
/// The module directory a Tomcat config's WAR belongs to. Created configs carry it as `cwd`
/// (set from the selected artifact); when it is missing, the selected artifact id still names
/// the module — `maven:<pom>` its POM, `gradle:…` its build — instead of falling back to the
/// project root, whose own `src` rarely exists and would make every launch look current.
fn artifact_module_dir(root: &Path, config: &RunConfig) -> PathBuf {
if config.cwd.is_some() {
return module_dir(root, config.cwd.as_ref().map(|c| c.0.as_path()));
}
let selected = config.extras.params.get("artifact").filter(|s| !s.is_empty());
if let Some(id) = selected {
if let Some(pom) = id.strip_prefix("maven:") {
// `maven:pom.xml` is the root module itself.
return match Path::new(pom).parent().filter(|p| !p.as_os_str().is_empty()) {
Some(dir) => root.join(dir),
None => root.to_path_buf(),
};
}
if let Some(rest) = id.strip_prefix("gradle:") {
// `gradle:<project path>` for the root build, else `gradle:["<build dir>", "<path>"]`.
if let Ok((dir, path)) = serde_json::from_str::<(String, String)>(rest) {
return root.join(gradle_module_dir(&dir, &path));
}
if !rest.trim_matches(':').is_empty() {
return root.join(gradle_module_dir("", rest));
}
}
}
root.to_path_buf()
}
/// Maps a Gradle project path (`:` for the build itself, `:web` for a subproject) below its
/// build directory. Custom `projectDir` mappings are invisible here, so this is the
/// conventional layout; still closer than the project root.
fn gradle_module_dir(dir: &str, path: &str) -> PathBuf {
let mut module = PathBuf::from(dir);
let rel = path.trim_start_matches(':').replace(':', "/");
if !rel.is_empty() {
module.push(rel);
}
module
}
/// The newest modification time of `path` or, for a directory, anything below it.
fn newest_mtime_under(path: &Path) -> Option<std::time::SystemTime> {
let meta = fs::metadata(path).ok()?;
@@ -902,14 +944,16 @@ fn find_built_war(root: &Path, module: Option<&Path>) -> Option<PathBuf> {
/// Whether the WAR a Tomcat config deploys is newer than the module's sources, so its
/// "Before launch › Build" task can be skipped (IDEA's up-to-date check). Conservative:
/// a missing WAR or a newer source/build file means it is not current. Only the module's
/// own sources and build files are inspected, not dependency modules.
pub fn artifact_is_current(root: &Path, base: &Path, config: &RunConfig) -> bool {
/// a missing WAR or a newer source/build file means it is not current. Besides the WAR
/// module's sources, web resources and build files, the sources and build files of every
/// module in `modules` (project-relative build modules) are inspected: a mapper XML or
/// class in a module the WAR depends on is packaged into it as a JAR.
pub fn artifact_is_current(root: &Path, base: &Path, config: &RunConfig, modules: &[PathBuf]) -> bool {
let p = &config.extras.params;
if p.get("deployment").map(String::as_str).unwrap_or("artifact") != "artifact" {
return true;
}
let module = module_dir(root, config.cwd.as_ref().map(|c| c.0.as_path()));
let module = artifact_module_dir(root, config);
let selected = p.get("artifact").filter(|s| !s.is_empty());
let recorded_id = fs::read_to_string(base.join("artifact-id.txt")).ok();
let id_matches = selected.is_none_or(|id| recorded_id.as_deref().map(str::trim) == Some(id.as_str()));
@@ -923,7 +967,24 @@ pub fn artifact_is_current(root: &Path, base: &Path, config: &RunConfig) -> bool
};
let Ok(war_time) = fs::metadata(&war).and_then(|m| m.modified()) else { return false };
let mut newest = newest_mtime_under(&module.join("src"));
for dir in [module.as_path(), root] {
// Web content packaged from outside `src` (a custom `warSourceDirectory`, `webResources`
// directories, a custom Gradle web app dir) also lands in the WAR — the running-session
// sync copies it from these same folders — so a save there must rebuild too (#2685).
for (dir, _) in web_sources(&module) {
if let Some(time) = newest_mtime_under(&dir)
&& newest.is_none_or(|cur| time > cur)
{
newest = Some(time);
}
}
let other_modules = modules.iter().map(|m| root.join(m));
for dir in [module.clone(), root.to_path_buf()].into_iter().chain(other_modules) {
if dir != module
&& let Some(time) = newest_mtime_under(&dir.join("src"))
&& newest.is_none_or(|cur| time > cur)
{
newest = Some(time);
}
for name in ["pom.xml", "build.gradle", "build.gradle.kts"] {
if let Ok(time) = fs::metadata(dir.join(name)).and_then(|m| m.modified())
&& newest.is_none_or(|cur| time > cur)
@@ -946,6 +1007,7 @@ pub fn deployment(root: &Path, base: &Path, config: &RunConfig, resources_only:
let recorded_id = fs::read_to_string(base.join("artifact-id.txt")).ok();
let id_matches = selected.is_none_or(|id| recorded_id.as_deref().map(str::trim) == Some(id.as_str()));
let recorded = fs::read_to_string(base.join("artifact-path.txt")).ok().map(|s| PathBuf::from(s.trim()));
let module = artifact_module_dir(root, config);
let war = match recorded.filter(|_| id_matches) {
Some(war) => {
if war.is_file() && is_war(&war) {
@@ -954,7 +1016,7 @@ pub fn deployment(root: &Path, base: &Path, config: &RunConfig, resources_only:
return Err(invalid(format!("The built WAR artifact does not exist: {}", war.display())));
}
}
None => find_built_war(root, config.cwd.as_ref().map(|c| c.0.as_path())).ok_or_else(|| {
None => find_built_war(root, Some(module.as_path())).ok_or_else(|| {
invalid("No built WAR artifact found - build the project first, or add 'Build' to Before launch")
})?,
};
@@ -1592,10 +1654,119 @@ mod tests {
c.cwd = Some(ide_core::IpcPath(PathBuf::from("web")));
c.extras.params.insert("artifact".into(), "maven:web/pom.xml".into());
// The WAR was packed after the source was written.
assert!(artifact_is_current(root, &base, &c));
assert!(artifact_is_current(root, &base, &c, &[]));
std::thread::sleep(std::time::Duration::from_millis(20));
fs::write(&source, "class App { int changed; }").unwrap();
assert!(!artifact_is_current(root, &base, &c));
assert!(!artifact_is_current(root, &base, &c, &[]));
}
#[test]
fn up_to_date_check_follows_the_selected_artifact_when_cwd_is_missing() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let module = root.join("web");
fs::create_dir_all(module.join("src/main/resources")).unwrap();
fs::create_dir_all(module.join("target")).unwrap();
fs::write(root.join("pom.xml"), "<project/>").unwrap();
fs::write(module.join("pom.xml"), "<project/>").unwrap();
let source = module.join("src/main/resources/applicationContext.xml");
fs::write(&source, "<beans/>").unwrap();
let war = module.join("target/app.war");
archive(&war, &[("WEB-INF/web.xml", "<web-app/>")]);
let base = root.join("cache");
fs::create_dir_all(&base).unwrap();
// A previous IDE build recorded its archive, as `ide_build::web::build` leaves behind.
fs::write(base.join("artifact-id.txt"), "maven:web/pom.xml").unwrap();
fs::write(base.join("artifact-path.txt"), war.to_string_lossy().as_bytes()).unwrap();
// Older configs may not carry the artifact's module as `cwd` (the UI sets it today).
let mut c = config();
c.extras.params.insert("artifact".into(), "maven:web/pom.xml".into());
assert!(artifact_is_current(root, &base, &c, &[]));
std::thread::sleep(std::time::Duration::from_millis(20));
fs::write(&source, "<beans><edited/></beans>").unwrap();
assert!(!artifact_is_current(root, &base, &c, &[]));
}
#[test]
fn up_to_date_check_watches_web_resources_outside_src() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let module = root.join("web");
fs::create_dir_all(module.join("src/main/java")).unwrap();
fs::create_dir_all(module.join("target")).unwrap();
fs::create_dir_all(module.join("WebContent/WEB-INF")).unwrap();
fs::write(
module.join("pom.xml"),
"<project><plugin><artifactId>maven-war-plugin</artifactId><configuration><webResources>\
<resource><directory>WebContent</directory></resource></webResources></configuration></plugin></project>",
)
.unwrap();
fs::write(module.join("src/main/java/App.java"), "class App {}").unwrap();
let web_xml = module.join("WebContent/WEB-INF/web.xml");
fs::write(&web_xml, "<web-app/>").unwrap();
let war = module.join("target/app.war");
archive(&war, &[("WEB-INF/web.xml", "<web-app/>"), ("index.html", "built")]);
let base = root.join("cache");
fs::create_dir_all(&base).unwrap();
let mut c = config();
c.cwd = Some(ide_core::IpcPath(PathBuf::from("web")));
c.extras.params.insert("artifact".into(), "maven:web/pom.xml".into());
assert!(artifact_is_current(root, &base, &c, &[]));
std::thread::sleep(std::time::Duration::from_millis(20));
fs::write(&web_xml, "<web-app><edited/></web-app>").unwrap();
assert!(!artifact_is_current(root, &base, &c, &[]));
}
#[test]
fn up_to_date_check_watches_the_modules_the_war_depends_on() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let web = root.join("web");
let dao = root.join("dao");
fs::create_dir_all(web.join("src/main/java")).unwrap();
fs::create_dir_all(web.join("target")).unwrap();
fs::create_dir_all(dao.join("src/main/resources/mapper")).unwrap();
fs::write(web.join("pom.xml"), "<project/>").unwrap();
fs::write(dao.join("pom.xml"), "<project/>").unwrap();
fs::write(web.join("src/main/java/App.java"), "class App {}").unwrap();
let mapper = dao.join("src/main/resources/mapper/UserMapper.xml");
fs::write(&mapper, "<mapper/>").unwrap();
archive(&web.join("target/app.war"), &[("WEB-INF/web.xml", "<web-app/>")]);
let base = root.join("cache");
fs::create_dir_all(&base).unwrap();
let mut c = config();
c.cwd = Some(ide_core::IpcPath(PathBuf::from("web")));
c.extras.params.insert("artifact".into(), "maven:web/pom.xml".into());
let modules = [PathBuf::from(""), PathBuf::from("web"), PathBuf::from("dao")];
assert!(artifact_is_current(root, &base, &c, &modules));
std::thread::sleep(std::time::Duration::from_millis(20));
fs::write(&mapper, "<mapper><select/></mapper>").unwrap();
assert!(!artifact_is_current(root, &base, &c, &modules), "a dependency module's XML changed");
assert!(artifact_is_current(root, &base, &c, &[]), "without the build's modules only the WAR module counts");
}
#[test]
fn artifact_module_prefers_cwd_then_selected_artifact() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let module_of = |cwd: Option<&str>, artifact: Option<&str>| {
let mut c = config();
c.cwd = cwd.map(|d| ide_core::IpcPath(PathBuf::from(d)));
if let Some(id) = artifact {
c.extras.params.insert("artifact".into(), id.into());
}
artifact_module_dir(root, &c)
};
// The UI keeps `cwd` on the artifact's module; it wins over a stale selection.
assert_eq!(module_of(Some("web"), Some("maven:other/pom.xml")), root.join("web"));
assert_eq!(module_of(None, Some("maven:web/pom.xml")), root.join("web"));
assert_eq!(module_of(None, Some("maven:pom.xml")), root);
assert_eq!(module_of(None, Some("gradle::web")), root.join("web"));
assert_eq!(module_of(None, Some("gradle::")), root);
assert_eq!(module_of(None, Some("gradle:[\"sub\",\":web\"]")), root.join("sub/web"));
assert_eq!(module_of(None, Some("other:whatever")), root);
assert_eq!(module_of(None, None), root);
assert_eq!(module_of(Some("web"), None), root.join("web"));
}
#[test]
+1 -1
Переглянути файл
@@ -1541,7 +1541,7 @@ pub fn usages_bundle(data_dir: &Path) -> Option<PathBuf> {
install_bundle(
data_dir,
"java-usages",
"1.12.0",
"1.13.0",
include_bytes!("../../../java/java-usages/openidea-java-usages.jar"),
)
}
+51 -1
Переглянути файл
@@ -441,7 +441,7 @@ fn jdk_groups(root: &Path, s: &Settings, cancel: &CancellationToken) -> Vec<Libr
Some(files) => files,
None => {
let java = [".classpath", ".project"].iter().any(|f| root.join(f).is_file())
|| root.join(".idea").is_dir()
|| idea_java_project(root)
|| !s.languages.java.jdk.trim().is_empty()
|| !s.project_structure.source_roots.is_empty();
if !java {
@@ -561,6 +561,27 @@ pub(crate) fn classpath_entries(dir: &Path, text: &str) -> Vec<PathBuf> {
}
/// The JDK home and its sources (`lib/src.zip`, `src.zip` in JDK 8) as library roots.
/// An IntelliJ project (`.idea`) that is a Java one: a Java SDK or language level in
/// `misc.xml`, or a Java module file. GoLand, PyCharm and WebStorm write `.idea` too, so the
/// folder alone doesn't make a project need the JDK.
fn idea_java_project(root: &Path) -> bool {
let idea = root.join(".idea");
if !idea.is_dir() {
return false;
}
let misc = std::fs::read_to_string(idea.join("misc.xml")).unwrap_or_default();
if misc.contains("JavaSDK") || misc.contains("languageLevel=\"JDK_") {
return true;
}
let java_module = |dir: &Path| {
std::fs::read_dir(dir).into_iter().flatten().flatten().any(|e| {
e.path().extension().is_some_and(|x| x == "iml")
&& std::fs::read_to_string(e.path()).is_ok_and(|t| t.contains("JAVA_MODULE"))
})
};
java_module(root) || java_module(&idea)
}
pub(crate) fn jdk_library(jdk: &crate::java::Jdk) -> Option<LibraryGroup> {
let name = jdk_name(&jdk.home).unwrap_or_else(|| format!("JDK {}", jdk.major));
let mut roots = named_roots(vec![(name.clone(), jdk.home.clone())]);
@@ -1263,3 +1284,32 @@ mod go_cache_tests {
assert_eq!(groups[1].roots.len(), 1);
}
}
#[cfg(test)]
mod idea_project_tests {
use super::*;
#[test]
fn an_idea_folder_marks_a_java_project_only_when_it_declares_java() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
assert!(!idea_java_project(root));
// GoLand / PyCharm write `.idea` without any Java in it.
std::fs::create_dir_all(root.join(".idea")).unwrap();
std::fs::write(root.join(".idea/misc.xml"), r#"<project version="4"><component name="GOROOT" /></project>"#)
.unwrap();
std::fs::write(root.join(".idea/demo.iml"), r#"<module type="WEB_MODULE" version="4" />"#).unwrap();
assert!(!idea_java_project(root));
// A Java module file, at the root or in `.idea`.
std::fs::write(root.join("app.iml"), r#"<module type="JAVA_MODULE" version="4" />"#).unwrap();
assert!(idea_java_project(root));
std::fs::remove_file(root.join("app.iml")).unwrap();
// The project JDK in misc.xml.
std::fs::write(
root.join(".idea/misc.xml"),
r#"<project version="4"><component name="ProjectRootManager" languageLevel="JDK_17" project-jdk-name="17" project-jdk-type="JavaSDK" /></project>"#,
)
.unwrap();
assert!(idea_java_project(root));
}
}
+95 -23
Переглянути файл
@@ -205,6 +205,49 @@ pub(crate) fn run_paths(root: &Path, args: &[&str], paths: &[String]) -> IdeResu
run_paths_env(root, args, paths, None, &[])
}
const RESTRICTED_CONFIG: [&str; 8] = [
"core.fsmonitor=false",
"core.hooksPath=",
// `git diff` writes the index back after refreshing it, and that write runs the hooks a
// config names for `post-index-change`, which `core.hooksPath` does not reach.
"diff.autoRefreshIndex=false",
"hook.post-index-change.enabled=false",
"diff.external=",
"core.attributesFile=",
"submodule.recurse=false",
"log.showSignature=false",
];
/// The keys that name a signature verifier; `gpg.openpgp.program` is an alias of `gpg.program`.
const VERIFIER_KEYS: [&str; 4] = ["gpg.program", "gpg.openpgp.program", "gpg.ssh.program", "gpg.x509.program"];
/// A program path git opens directly and cannot find, as nothing exists below a regular file.
/// An empty name is not enough: Git for Windows looks it up on `PATH` and runs a file named `.exe`.
fn no_program() -> &'static std::ffi::OsStr {
static PATH: OnceLock<std::ffi::OsString> = OnceLock::new();
PATH.get_or_init(|| match std::env::current_exe() {
Ok(exe) if exe.is_absolute() => exe.join("no-program").into_os_string(),
_ => Default::default(),
})
}
/// Keeps `cmd`, a git that reads a repository, from running what that repository's config names
/// for a read: the fsmonitor hook, hooks, the external diff, a signature verifier and a lazy
/// fetch. Call it before the subcommand is added. Clean and process filters have a key per
/// driver ([`filter_overrides_from`] empties those); signing, transport and credential helpers
/// are not covered.
pub fn restrict(cmd: &mut Command) {
for setting in RESTRICTED_CONFIG {
cmd.arg("-c").arg(setting);
}
for key in VERIFIER_KEYS {
let mut setting = std::ffi::OsString::from(format!("{key}="));
setting.push(no_program());
cmd.arg("-c").arg(setting);
}
cmd.env_remove("GIT_EXTERNAL_DIFF").env("GIT_NO_LAZY_FETCH", "1");
}
/// Builds a `git` command with the environment every invocation needs.
pub(crate) fn command(root: &Path) -> IdeResult<Command> {
let restricted = !crate::execution_allowed();
@@ -215,16 +258,7 @@ pub(crate) fn command(root: &Path) -> IdeResult<Command> {
};
let mut cmd = Command::new(binary);
if restricted {
for setting in [
"core.fsmonitor=false",
"core.hooksPath=",
"diff.external=",
"core.attributesFile=",
"submodule.recurse=false",
] {
cmd.arg("-c").arg(setting);
}
cmd.env_remove("GIT_EXTERNAL_DIFF").env("GIT_NO_LAZY_FETCH", "1");
restrict(&mut cmd);
}
cmd.current_dir(root)
.arg("-c")
@@ -385,6 +419,13 @@ fn run_cancellable_raw_stdin(
run_capture(root, args, stdin, cancel, usize::MAX)
}
/// Options for a `git diff` that reads the working tree in Restricted Mode: git asks a submodule
/// with changes for its own status, and for its own diff when `diff.submodule` says so, and both
/// run under the submodule's config.
pub(crate) fn submodule_guards() -> &'static [&'static str] {
if crate::execution_allowed() { &[] } else { &["--ignore-submodules=dirty", "--submodule=short"] }
}
/// Neutralizes repository clean/process filters for a read of working-tree bytes.
/// Git still applies them with --no-ext-diff and --no-textconv.
pub(crate) fn filter_overrides(root: &Path, cancel: &CancellationToken) -> IdeResult<Vec<String>> {
@@ -392,15 +433,27 @@ pub(crate) fn filter_overrides(root: &Path, cancel: &CancellationToken) -> IdeRe
return Ok(Vec::new());
}
let config = run_bounded(root, &["config", "--null", "--list"], cancel, 4 << 20)?;
filter_overrides_from(&config.stdout)
}
/// `-c` arguments that empty every clean and process filter in `config`, the output of
/// `git config --null --list`. Fails when one of them cannot be emptied this way: git splits
/// `-c name=value` at the first `=`, and a name that is not UTF-8 does not reach git as it is
/// on every platform.
pub fn filter_overrides_from(config: &[u8]) -> IdeResult<Vec<String>> {
let mut overrides = Vec::new();
for record in config.stdout_str().split('\0') {
let key = record.split('\n').next().unwrap_or_default();
if key.starts_with("filter.")
&& (key.ends_with(".clean") || key.ends_with(".process") || key.ends_with(".required"))
{
overrides.push("-c".into());
overrides.push(format!("{key}={}", if key.ends_with(".required") { "false" } else { "" }));
for record in config.split(|b| *b == 0) {
let key = record.split(|b| *b == b'\n').next().unwrap_or_default();
let required = key.ends_with(b".required");
if !key.starts_with(b"filter.") || !(key.ends_with(b".clean") || key.ends_with(b".process") || required) {
continue;
}
let Some(key) = std::str::from_utf8(key).ok().filter(|key| !key.contains('=')) else {
let key = String::from_utf8_lossy(key);
return Err(IdeError::invalid(format!("git cannot be told to skip the filter setting `{key}`")));
};
overrides.push("-c".into());
overrides.push(format!("{key}={}", if required { "false" } else { "" }));
}
Ok(overrides)
}
@@ -760,6 +813,25 @@ mod command_session_tests {
}
}
#[cfg(test)]
mod restrict_tests {
use super::*;
#[test]
fn restrict_names_a_verifier_git_opens_directly_and_cannot_find() {
let mut cmd = Command::new("git");
restrict(&mut cmd);
let args: Vec<_> = cmd.get_args().map(|arg| arg.to_string_lossy().into_owned()).collect();
let programs: Vec<_> = args.iter().filter_map(|arg| arg.split_once(".program=")).collect();
assert_eq!(programs.len(), 4, "{args:?}");
for (key, program) in programs {
let program = Path::new(program);
assert!(program.is_absolute(), "{key}.program is {program:?}: git would look that name up on PATH");
assert!(!program.exists(), "{key}.program is {program:?}, which exists");
}
}
}
#[cfg(test)]
mod batching_tests {
use super::{WINDOWS_LIMIT, base_command_line_len, batch_paths, command_line_limit};
@@ -836,12 +908,10 @@ mod index_lock_tests {
run(dir.path(), &["init", "-q", "-b", "main"], None).unwrap();
std::fs::write(dir.path().join("file"), "main\n").unwrap();
run(dir.path(), &["add", "file"], None).unwrap();
run(
dir.path(),
&["-c", "user.name=Test", "-c", "[email protected]", "commit", "-qm", "initial"],
None,
)
.unwrap();
// git started outside an execution scope is restricted and has no signing program, so the
// user's `commit.gpgsign` must not reach this commit.
let identity = ["-c", "user.name=Test", "-c", "[email protected]", "-c", "commit.gpgsign=false"];
run(dir.path(), &[&identity[..], &["commit", "-qm", "initial"]].concat(), None).unwrap();
run(dir.path(), &["branch", "other"], None).unwrap();
dir
}
@@ -891,6 +961,8 @@ mod index_lock_tests {
)
.unwrap();
std::fs::set_permissions(&hook, std::fs::Permissions::from_mode(0o755)).unwrap();
// Hooks run only in a trusted repository.
let _scope = crate::ExecutionScope::enter(true);
let out = run_raw(dir.path(), &["switch", "-q", "other"], None).unwrap();
assert_eq!(out.code, Some(1));
assert_eq!(std::fs::read_to_string(dir.path().join("hook-runs")).unwrap(), "ran\n");
+439 -15
Переглянути файл
@@ -151,18 +151,12 @@ impl Repos {
}
let resolved = resolved.stdout_str().trim().to_owned();
// Disable textconv/external diff; these are also safe in untrusted projects.
let names =
run(&["diff", "--name-status", "-z", "-M", "--no-ext-diff", "--no-textconv", &resolved, "--"])?;
let stats = run(&[
"diff",
"--numstat",
"-z",
"--no-renames",
"--no-ext-diff",
"--no-textconv",
&resolved,
"--",
])?;
let diff = |options: &[&str]| {
let fixed = &["diff", "--no-ext-diff", "--no-textconv"][..];
run(&[fixed, cli::submodule_guards(), options, &[resolved.as_str(), "--"]].concat())
};
let names = diff(&["--name-status", "-z", "-M"])?;
let stats = diff(&["--numstat", "-z", "--no-renames"])?;
let stats: std::collections::HashMap<_, _> = ops::parse_numstat(&stats.stdout_str())
.into_iter()
.map(|(path, added, removed)| (path, (added, removed)))
@@ -207,8 +201,7 @@ impl Repos {
});
}
if files.iter().any(|f| f.kind == ChangeKind::Renamed) {
let renamed =
run(&["diff", "--numstat", "-z", "-M", "--no-ext-diff", "--no-textconv", &resolved, "--"])?;
let renamed = diff(&["--numstat", "-z", "-M"])?;
let indexes: std::collections::HashMap<_, _> = files
.iter()
.enumerate()
@@ -289,7 +282,13 @@ mod tests {
use std::process::Command;
fn git(root: &Path, args: &[&str]) {
let output = Command::new("git").current_dir(root).args(args).output().unwrap();
let output = Command::new("git")
.current_dir(root)
.args(args)
.env("GIT_CONFIG_NOSYSTEM", "1")
.env("GIT_CONFIG_GLOBAL", if cfg!(windows) { "NUL" } else { "/dev/null" })
.output()
.unwrap();
assert!(output.status.success(), "{args:?}: {}", String::from_utf8_lossy(&output.stderr));
}
@@ -506,4 +505,429 @@ mod tests {
git(&root, &["ls-files"]);
assert!(marker.exists(), "control invocation did not run fsmonitor");
}
/// git splits `-c name=value` at the first `=` and takes no name that is not UTF-8 on every
/// platform, so such a filter cannot be emptied: Restricted Mode compares nothing instead.
#[test]
fn restricted_comparison_refuses_a_filter_git_cannot_be_told_to_skip() {
for name in [&b"x=y"[..], &[0xE9]] {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path()).join("repo");
let marker = root.with_file_name("marker");
init(&root, "main");
std::fs::write(root.join(".gitattributes"), [b"*.txt filter=", name, b"\n"].concat()).unwrap();
let command = format!("echo x > '{}'; cat", marker.to_str().unwrap().replace('\\', "/"));
let section = [b"[filter \"", name, format!("\"]\n\tclean = \"{command}\"\n").as_bytes()].concat();
let config = root.join(".git/config");
std::fs::write(&config, [std::fs::read(&config).unwrap(), section].concat()).unwrap();
// The size it has in the index, so git has to read the file to tell whether it changed.
write(&root, "src/move.txt", "more\n");
let repos = Repos::discover(&root, &|_| false).unwrap();
for repo in repos.all() {
repo.set_execution_allowed(false);
}
let result = repos.compare_paths(std::slice::from_ref(&root), "main", None, &CancellationToken::new());
let ran = marker.exists();
let _ = std::fs::remove_file(&marker);
write(&root, "src/move.txt", "less\n");
let _ = Command::new("git").current_dir(&root).args(["status", "--porcelain"]).output();
assert!(marker.exists(), "control: plain `git status` did not run the filter, fixture is not live");
assert!(!ran, "Restricted Mode ran the repository's filter {name:?}");
assert!(result.is_ok_and(|r| r.repositories[0].error.is_some()), "{name:?}");
}
}
/// A repository with a second one in `sub`, which has `attributes`, committed as a submodule;
/// returns the two roots and the path of the marker a program of the fixture leaves behind.
fn with_submodule(temp: &Path, attributes: &str) -> (PathBuf, PathBuf, PathBuf) {
let root = normalized(temp).join("repo");
let sub = root.join("sub");
init(&root, "main");
init(&sub, "main");
write(&sub, ".gitattributes", attributes);
git(&sub, &["add", "."]);
git(&sub, &["commit", "-qm", "attributes"]);
git(&root, &["add", "sub"]);
git(&root, &["commit", "-qm", "submodule"]);
(root.clone(), sub, root.with_file_name("marker"))
}
/// git asks a submodule with changes for its own status, which runs under the submodule's config.
#[test]
fn restricted_reads_do_not_run_filters_of_a_submodule() {
let temp = tempfile::tempdir().unwrap();
let (root, sub, marker) = with_submodule(temp.path(), "*.txt filter=marker\n");
let command = format!("echo x > '{}'; cat", marker.to_str().unwrap().replace('\\', "/"));
git(&sub, &["config", "filter.marker.clean", &command]);
let repos = Repos::discover(&root, &|_| false).unwrap();
for repo in repos.all() {
repo.set_execution_allowed(false);
}
let mut ran = Vec::new();
// The size it has in the index, so git has to read the file to tell whether it changed.
write(&sub, "src/move.txt", "more\n");
let compared = repos.compare_paths(std::slice::from_ref(&root), "main", None, &CancellationToken::new());
if marker.exists() {
ran.push("compare_paths");
}
let _ = std::fs::remove_file(&marker);
write(&sub, "src/move.txt", "less\n");
let patch = repos.at_root(&root).unwrap().working_tree_patch(None, None, 1 << 20, &CancellationToken::new());
if marker.exists() {
ran.push("working_tree_patch");
}
let _ = std::fs::remove_file(&marker);
write(&sub, "src/move.txt", "same\n");
let _ = Command::new("git").current_dir(&root).args(["status", "--porcelain"]).output();
assert!(marker.exists(), "control: plain `git status` did not run the filter, fixture is not live");
assert!(ran.is_empty(), "Restricted Mode ran the submodule's filter via: {ran:?}");
assert!(compared.unwrap().repositories.iter().all(|r| r.error.is_none()));
patch.unwrap();
}
/// git runs the hooks a config names for `post-index-change` when a command writes the index
/// back, as `git diff` does after refreshing it.
#[test]
fn restricted_reads_do_not_run_hooks_of_the_repository_s_config() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path()).join("repo");
let marker = root.with_file_name("marker");
init(&root, "main");
git(&root, &["config", "hook.marker.event", "post-index-change"]);
let command = format!("echo x > '{}'", marker.to_str().unwrap().replace('\\', "/"));
git(&root, &["config", "hook.marker.command", &command]);
let listed = Command::new("git").current_dir(&root).args(["hook", "list", "post-index-change"]).output();
if !String::from_utf8_lossy(&listed.unwrap().stdout).contains("marker") {
return; // This git has no hooks named by a config.
}
// The content it has in the index, with a time of its own that is not the index's.
let touch = |seconds: u64| {
let file = std::fs::File::options().write(true).open(root.join("src/move.txt")).unwrap();
file.set_modified(std::time::SystemTime::now() - std::time::Duration::from_secs(seconds)).unwrap();
};
let repos = Repos::discover(&root, &|_| false).unwrap();
let repo = repos.at_root(&root).unwrap();
repo.set_execution_allowed(false);
let mut ran = Vec::new();
touch(60);
let compared = repos.compare_paths(std::slice::from_ref(&root), "main", None, &CancellationToken::new());
if marker.exists() {
ran.push("compare_paths");
}
let _ = std::fs::remove_file(&marker);
touch(120);
let patch = repo.working_tree_patch(None, None, 1 << 20, &CancellationToken::new());
if marker.exists() {
ran.push("working_tree_patch");
}
let _ = std::fs::remove_file(&marker);
touch(180);
let _ = Command::new("git").current_dir(&root).args(["--no-pager", "diff"]).output();
assert!(marker.exists(), "control: plain `git diff` did not run the hook, fixture is not live");
assert!(ran.is_empty(), "Restricted Mode ran a hook of the repository's config via: {ran:?}");
assert!(compared.unwrap().repositories[0].error.is_none());
assert_eq!(patch.unwrap(), "");
}
/// `diff.submodule=diff` has git run `git diff` in the submodule, under the submodule's config.
#[test]
fn restricted_patch_does_not_run_a_diff_driver_of_a_submodule() {
let temp = tempfile::tempdir().unwrap();
let (root, sub, marker) = with_submodule(temp.path(), "*.txt diff=marker\n");
write(&sub, "src/move.txt", "move\nagain\n");
git(&sub, &["commit", "-qam", "second"]);
let command = format!("echo x > '{}'; cat", marker.to_str().unwrap().replace('\\', "/"));
git(&sub, &["config", "diff.marker.command", &command]);
git(&root, &["config", "diff.submodule", "diff"]);
let repos = Repos::discover(&root, &|_| false).unwrap();
for repo in repos.all() {
repo.set_execution_allowed(false);
}
let patch = repos.at_root(&root).unwrap().working_tree_patch(None, None, 1 << 20, &CancellationToken::new());
let ran = marker.exists();
let _ = std::fs::remove_file(&marker);
let _ = Command::new("git").current_dir(&root).args(["--no-pager", "diff", "HEAD"]).output();
assert!(marker.exists(), "control: plain `git diff` did not run the diff driver, fixture is not live");
assert!(!ran, "Restricted Mode ran the submodule's diff driver");
let patch = patch.unwrap();
assert!(patch.contains("Subproject commit"), "{patch}");
}
/// Replaces HEAD with a copy of itself carrying a (fake) gpgsig header. `armor` labels the
/// signature and so selects git's verifier: `PGP SIGNATURE`, `SSH SIGNATURE` or
/// `SIGNED MESSAGE` (X.509).
fn sign_head(root: &Path, armor: &str) {
use std::io::Write;
let out = Command::new("git").current_dir(root).args(["cat-file", "commit", "HEAD"]).output().unwrap();
let body = String::from_utf8(out.stdout).unwrap();
let (headers, message) = body.split_once("\n\n").unwrap();
// Drop the signature of an earlier call so only the new one remains.
let mut kept = Vec::new();
let mut in_sig = false;
for line in headers.lines() {
if line.starts_with("gpgsig") {
in_sig = true;
} else if !(in_sig && line.starts_with(' ')) {
in_sig = false;
kept.push(line);
}
}
let headers = kept.join("\n");
let signed = format!(
"{headers}\ngpgsig -----BEGIN {armor}-----\n \n iQEzBAABCAAdFiEEfake\n -----END {armor}-----\n\n{message}"
);
let mut child = Command::new("git")
.current_dir(root)
.args(["hash-object", "-t", "commit", "-w", "--stdin"])
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.spawn()
.unwrap();
child.stdin.take().unwrap().write_all(signed.as_bytes()).unwrap();
let id = String::from_utf8(child.wait_with_output().unwrap().stdout).unwrap();
git(root, &["update-ref", "HEAD", id.trim()]);
}
/// Writes a script into `.git` that records its own execution. Returns the marker file it
/// creates and its own path, spelled the way git config takes it.
fn verifier_script(root: &Path) -> (PathBuf, String) {
let marker = root.join("executed-marker");
let script = root.join(".git/verifier.sh");
std::fs::write(
&script,
format!("#!/bin/sh\necho x > '{}'\nexit 1\n", marker.to_str().unwrap().replace('\\', "/")),
)
.unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap();
}
(marker, script.to_str().unwrap().replace('\\', "/"))
}
/// Security audit lead 2: Restricted Mode `git log` must not run the repository's signature verifier.
#[test]
fn restricted_log_does_not_execute_signature_verifier() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path());
init(&root, "main");
write(&root, "src/move.txt", "move\nsigned change\n");
git(&root, &["commit", "-qam", "signed"]);
sign_head(&root, "PGP SIGNATURE");
let (marker, script) = verifier_script(&root);
git(&root, &["config", "log.showSignature", "true"]);
git(&root, &["config", "gpg.program", &script]);
let repos = Repos::discover(&root, &|_| false).unwrap();
let repo = repos.all().first().unwrap();
repo.set_execution_allowed(false);
let file = root.join("src/move.txt");
type Call<'a> = (&'static str, Box<dyn Fn() + 'a>);
let calls: [Call; 4] = [
("log_users", Box::new(|| assert_eq!(repo.log_users().unwrap(), ["Test <[email protected]>"]))),
// Restricted Mode does not ask git to verify here: it reports E without starting it.
(
"log_details",
Box::new(|| assert_eq!(repo.log_details("HEAD").unwrap().signature, "E", "signed, never verified")),
),
// Passes --no-show-signature itself, so it never ran the verifier.
(
"file history",
Box::new(|| {
let query = crate::LogQuery { path: Some(file.clone().into()), ..Default::default() };
assert!(!repo.log(&query).unwrap().is_empty(), "file history is empty");
}),
),
(
"block_history",
Box::new(|| {
let history =
repo.block_history(&file, None, (0, 1), encoding_rs::UTF_8, &CancellationToken::new()).unwrap();
assert!(!history.revisions.is_empty(), "block history is empty");
}),
),
];
let mut ran = Vec::new();
for (name, call) in &calls {
let _ = std::fs::remove_file(&marker);
call();
if marker.exists() {
ran.push(*name);
}
}
// Control: the fixture's verifier really runs when the same git is used directly.
let _ = std::fs::remove_file(&marker);
let git = cli::detected_git_executable().unwrap();
let _ = Command::new(git).current_dir(&root).args(["log", "-1", "--format=%H"]).output();
assert!(marker.exists(), "control: plain `git log` did not run gpg.program, fixture is not live");
assert!(ran.is_empty(), "Restricted Mode ran the repository's gpg.program via: {ran:?}");
}
/// Security audit lead 2: turning signature display off is not enough. A git that is asked to
/// verify (`%G?`, `--show-signature`, or a `format.pretty` the repository sets) runs the
/// verifier of the signature's format (OpenPGP, SSH and X.509 each have a program key), so
/// Restricted Mode must leave none of them pointing at the repository's program. And when
/// nothing asks, the repository's `log.showSignature` must not make git look for one.
#[test]
fn restricted_git_does_not_execute_signature_verifier_when_asked_to_verify() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path());
init(&root, "main");
let (marker, script) = verifier_script(&root);
for key in ["gpg.program", "gpg.ssh.program", "gpg.x509.program"] {
git(&root, &["config", key, &script]);
}
// git starts the SSH verifier only when an allowed-signers file exists.
let signers = root.join(".git/allowed-signers");
std::fs::write(&signers, "").unwrap();
git(&root, &["config", "gpg.ssh.allowedSignersFile", &signers.to_str().unwrap().replace('\\', "/")]);
git(&root, &["config", "format.pretty", "%H %G?"]);
git(&root, &["config", "log.showSignature", "true"]);
let unasked: &[&str] = &["log", "-1", "--format=%H"];
let requests: [&[&str]; 4] = [
&["log", "-1", "--format=%G?"],
&["log", "-1", "--show-signature", "--format=%H"],
&["log", "-1"],
unasked,
];
let mut ran = Vec::new();
for armor in ["PGP SIGNATURE", "SSH SIGNATURE", "SIGNED MESSAGE"] {
sign_head(&root, armor);
for args in requests {
for allowed in [false, true] {
let _ = std::fs::remove_file(&marker);
let out = {
let _scope = crate::ExecutionScope::enter(allowed);
cli::run(&root, args, None).unwrap()
};
assert!(!out.stdout.is_empty(), "{armor} {args:?}: git printed nothing");
if allowed {
// Control: the same request runs the fixture's verifier in a trusted project.
assert!(marker.exists(), "control: {armor} {args:?} did not run the verifier");
} else if marker.exists() {
ran.push(format!("{armor} {args:?}"));
} else if args == unasked {
// A git that looked for a verifier and found none says so here.
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(stderr.is_empty(), "{armor} {args:?}: git looked for a verifier: {stderr}");
}
}
}
}
assert!(ran.is_empty(), "Restricted Mode ran the repository's signature verifier via: {ran:?}");
}
/// A repository with a signed HEAD whose config has `git log` verify it with a program of
/// its own. Returns the marker that program creates.
fn signed_log_fixture(root: &Path) -> PathBuf {
init(root, "main");
git(root, &["config", "commit.gpgsign", "false"]);
write(root, "src/move.txt", "move\nsigned change\n");
git(root, &["commit", "-qam", "signed"]);
sign_head(root, "PGP SIGNATURE");
let (marker, script) = verifier_script(root);
git(root, &["config", "log.showSignature", "true"]);
git(root, &["config", "gpg.program", &script]);
marker
}
/// Security audit lead 2: a repository nobody has vouched for is restricted. Execution is
/// something a caller grants, not something it has to remember to take away.
#[test]
fn discovered_repository_runs_no_repository_program_until_execution_is_granted() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path());
let marker = signed_log_fixture(&root);
let repo = crate::Repo::discover(&root).unwrap().unwrap();
let users = repo.log_users().unwrap();
assert!(!marker.exists(), "a freshly discovered repository ran its gpg.program");
assert_eq!(users, ["Test <[email protected]>"]);
// Control: the same call runs the fixture's verifier once execution is granted.
repo.set_execution_allowed(true);
repo.log_users().unwrap();
assert!(marker.exists(), "control: granting execution did not run gpg.program, fixture is not live");
}
/// Security audit lead 2: status is read through gitoxide, which runs the clean filter of
/// the repository's config to compare a touched file with the index. A repository nobody
/// has vouched for is opened without that config.
#[test]
fn discovered_repository_status_runs_no_repository_filter_until_execution_is_granted() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path());
init(&root, "main");
let marker = root.join("filter-marker");
let clean = format!("echo x > '{}'; cat", marker.to_str().unwrap().replace('\\', "/"));
git(&root, &["config", "filter.marker.clean", &clean]);
write(&root, ".gitattributes", "*.txt filter=marker\n");
// Same length as the committed "move\n": only reading the content tells them apart.
write(&root, "src/move.txt", "MOVE\n");
let repo = crate::Repo::discover(&root).unwrap().unwrap();
let status = repo.status();
assert!(!marker.exists(), "a freshly discovered repository ran its clean filter");
assert!(status.unwrap().iter().any(|f| f.path.0.ends_with("move.txt")), "the touched file is not reported");
// Control: the same call runs the fixture's filter once execution is granted.
repo.set_execution_allowed(true);
repo.status().unwrap();
assert!(marker.exists(), "control: granting execution did not run the clean filter, fixture is not live");
}
/// Security audit lead 2: git started by code that declared no mode at all is restricted.
#[test]
fn git_started_outside_an_execution_scope_is_restricted() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path());
let marker = signed_log_fixture(&root);
assert!(!cli::run(&root, &["log", "-1", "--format=%H"], None).unwrap().stdout.is_empty());
assert!(!marker.exists(), "git outside an execution scope ran the repository's gpg.program");
// Control: the same request runs the fixture's verifier inside a scope that allows it.
{
let _scope = crate::ExecutionScope::enter(true);
cli::run(&root, &["log", "-1", "--format=%H"], None).unwrap();
}
assert!(marker.exists(), "control: an allowing scope did not run gpg.program, fixture is not live");
}
/// Security audit lead 2: git fetches an object it misses from a partial clone's promisor,
/// running the `uploadpack` program the repository names. Listing a revision's directory in
/// Restricted Mode must not.
#[test]
fn restricted_revision_listing_does_not_run_the_promisor_of_the_repository() {
let temp = tempfile::tempdir().unwrap();
let root = normalized(temp.path());
init(&root, "main");
let marker = root.join("lazy-fetch-marker");
let spelled = |p: &Path| p.to_str().unwrap().replace('\\', "/");
let out = Command::new("git").current_dir(&root).args(["rev-parse", "HEAD^{tree}"]).output().unwrap();
let tree = String::from_utf8(out.stdout).unwrap().trim().to_owned();
git(&root, &["config", "extensions.partialClone", "origin"]);
git(&root, &["config", "remote.origin.url", &spelled(&root)]);
git(&root, &["config", "remote.origin.promisor", "true"]);
git(&root, &["config", "remote.origin.partialclonefilter", "blob:none"]);
git(
&root,
&["config", "remote.origin.uploadpack", &format!("echo ran > '{}'; git-upload-pack", spelled(&marker))],
);
// The listing needs HEAD's tree; with it gone, git asks the promisor for it.
std::fs::remove_file(root.join(".git/objects").join(&tree[..2]).join(&tree[2..])).unwrap();
let repo = crate::Repo::discover(&root).unwrap().unwrap();
repo.set_execution_allowed(false);
assert!(repo.revision_children("HEAD", "").is_err(), "the tree is missing and must stay missing");
assert!(!marker.exists(), "Restricted Mode ran the repository's uploadpack");
// Control: in a trusted project the same listing asks the promisor.
repo.set_execution_allowed(true);
let _ = repo.revision_children("HEAD", "");
assert!(marker.exists(), "control: a trusted listing did not run uploadpack, fixture is not live");
}
}
+28 -8
Переглянути файл
@@ -638,8 +638,8 @@ mod text_encoding;
mod write;
pub use cli::{
GitCommandObserver, detected_git_executable, git_version, set_askpass_env, set_git_command_observer,
set_git_executable,
GitCommandObserver, detected_git_executable, filter_overrides_from, git_version, restrict, set_askpass_env,
set_git_command_observer, set_git_executable,
};
pub use diff::build_partial_patch;
pub use ignore::{IgnoreDestination, add_to_ignore};
@@ -674,6 +674,8 @@ pub fn clone(
let dest = dest.to_string_lossy();
// Cloning must not inherit an enclosing project's executable Git settings.
let cwd = tempfile::tempdir()?;
// No repository's config applies in that empty folder, so the user's git runs as it is.
let _scope = ExecutionScope::enter(true);
let args = clone_args(url, &dest, depth)?;
let args: Vec<&str> = args.iter().map(String::as_str).collect();
cli::run_streaming(cwd.path(), &args, cancel, on_output)
@@ -709,6 +711,8 @@ mod clone_tests {
/// Creates an empty repository in `dir` (`git init`).
pub fn init(dir: &Path) -> IdeResult<()> {
// `git init` runs nothing a repository config names, so the configured git runs as it is.
let _scope = ExecutionScope::enter(true);
cli::run(dir, &["init", "-q"], None).map(|_| ())
}
@@ -731,7 +735,8 @@ pub struct Repo {
}
impl Repo {
/// Discovers the repository containing `path`; `Ok(None)` when not under git.
/// Discovers the repository containing `path`; `Ok(None)` when not under git. The
/// repository is in Restricted Mode until [`Repo::set_execution_allowed`] says otherwise.
pub fn discover(path: &Path) -> IdeResult<Option<Self>> {
repo::discover(path)
}
@@ -986,6 +991,7 @@ impl Repo {
/// Lists one immutable revision directory. Directory names end in `/`.
pub fn revision_children(&self, revision: &str, directory: &str) -> IdeResult<Vec<String>> {
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
let repo = self.gix()?;
let id = repo::resolve(&repo, revision)?.to_string();
let directory = self.rel_path(std::path::Path::new(directory))?;
@@ -1056,7 +1062,9 @@ impl Repo {
let rel = path.map(|p| self.rel_path(p)).transpose()?;
let overrides = cli::filter_overrides(self.root(), cancel)?;
let mut args: Vec<&str> = overrides.iter().map(String::as_str).collect();
args.extend(["--literal-pathspecs", "--no-pager", "diff", "--no-ext-diff", "--no-textconv", rev, "--"]);
args.extend(["--literal-pathspecs", "--no-pager", "diff", "--no-ext-diff", "--no-textconv"]);
args.extend_from_slice(cli::submodule_guards());
args.extend([rev, "--"]);
if let Some(rel) = &rel {
args.push(rel);
}
@@ -1175,11 +1183,13 @@ impl Repo {
/// Edit Commit Message without touching the index or the working tree; returns HEAD
/// before and after.
pub fn reword(&self, commit: &str, message: &str) -> IdeResult<HeadChange> {
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
write::reword(self, commit, message).map(|(before, after)| HeadChange { before, after })
}
/// Undoes a history rewrite: HEAD goes from `after` back to `before`, local changes kept.
pub fn restore_head(&self, before: &str, after: &str) -> IdeResult<()> {
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
write::restore_head(self, before, after)
}
@@ -1301,6 +1311,7 @@ impl Repo {
/// Whether the repository has an executable hook `name` (e.g. `pre-push`), wherever
/// `core.hooksPath` puts it: the Push dialog then offers to skip it.
pub fn has_hook(&self, name: &str) -> IdeResult<bool> {
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
write::has_hook(self, name)
}
/// Pushes a local branch, or its history up to `revision`, to its upstream (Git Log
@@ -1551,11 +1562,17 @@ impl Repo {
}
}
thread_local! { static EXECUTION_ALLOWED: std::cell::Cell<bool> = const { std::cell::Cell::new(true) }; }
struct ExecutionScope(bool);
// `None`: nobody has decided yet. Git started now is restricted, and the first scope entered
// decides. The flag governs the git CLI only: gitoxide follows the repository's own flag.
thread_local! { static EXECUTION_ALLOWED: std::cell::Cell<Option<bool>> = const { std::cell::Cell::new(None) }; }
struct ExecutionScope(Option<bool>);
impl ExecutionScope {
/// The outermost scope decides; a scope inside it can only restrict.
fn enter(allowed: bool) -> Self {
Self(EXECUTION_ALLOWED.with(|current| current.replace(current.get() && allowed)))
Self(
EXECUTION_ALLOWED
.with(|current| current.replace(Some(current.get().map_or(allowed, |outer| outer && allowed)))),
)
}
}
impl Drop for ExecutionScope {
@@ -1564,9 +1581,12 @@ impl Drop for ExecutionScope {
}
}
pub(crate) fn execution_allowed() -> bool {
EXECUTION_ALLOWED.with(|current| current.get())
EXECUTION_ALLOWED.with(|current| current.get().unwrap_or(false))
}
impl Repo {
/// Whether git may run what the repository's config names (hooks, filters, the fsmonitor,
/// signature verifiers, a promisor) and use the configured git executable. A repository
/// starts without it: only a caller that knows the project is trusted grants it.
pub fn set_execution_allowed(&self, allowed: bool) {
let _scope = crate::ExecutionScope::enter(self.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
self.execution_allowed.store(allowed, std::sync::atomic::Ordering::Release);
+1 -1
Переглянути файл
@@ -71,7 +71,7 @@ pub(crate) fn discover(path: &Path) -> IdeResult<Option<Repo>> {
let pack_dir = common_dir.join("objects").join("pack");
let cache = Cache { stamp: stamp(&files), files, repo: ts, pack_stamp: pack_stamp(&pack_dir), pack_dir };
Ok(Some(Repo {
execution_allowed: std::sync::atomic::AtomicBool::new(true),
execution_allowed: std::sync::atomic::AtomicBool::new(false),
root,
git_dir,
inner: std::sync::Mutex::new(cache),
+2
Переглянути файл
@@ -200,6 +200,7 @@ pub(crate) fn unshelve(r: &Repo, shelf: &Path, id: &str, remove: bool) -> IdeRes
}
pub fn unshelve_path(r: &Repo, shelf: &Path, id: &str, remove: bool, path: Option<&str>) -> IdeResult<ApplyOutcome> {
let _scope = crate::ExecutionScope::enter(r.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
let patch = selected_patch(shelf, id, path)?;
let outcome = crate::patch::apply(r, &patch)?;
if remove && outcome.conflicts.is_empty() {
@@ -291,6 +292,7 @@ pub fn delete_path(shelf: &Path, id: &str, path: Option<&str>) -> IdeResult<()>
/// Reconstruct each side in a throwaway index; neither HEAD nor the working files are used.
pub fn diff_files(r: &Repo, shelf: &Path, id: &str) -> IdeResult<Vec<ShelfDiffFile>> {
let _scope = crate::ExecutionScope::enter(r.execution_allowed.load(std::sync::atomic::Ordering::Acquire));
let patch = patch_of(shelf, id)?;
let mut out = Vec::new();
for (section, file) in sections(&patch) {
+142 -21
Переглянути файл
@@ -5,6 +5,7 @@ use crate::{ChangeKind, FileStatus, Repo};
use gix::status::index_worktree::Item as WtItem;
use gix::status::plumbing::index_as_worktree::{Change as WtChange, EntryStatus};
use ide_core::IdeResult;
use ide_core::ignored::IgnoredNames;
use std::collections::{BTreeMap, HashSet};
#[derive(Default)]
@@ -15,10 +16,43 @@ struct Entry {
}
pub(crate) fn status(r: &Repo) -> IdeResult<Vec<FileStatus>> {
let ignored = r.ignored.read().unwrap_or_else(|p| p.into_inner()).clone();
// gitoxide stops at the first IO error (a directory it may not read), where git warns
// and lists the rest: git answers then.
let (map, conflicts) = gix_status(r, &ignored).or_else(|e| {
tracing::debug!("status via gitoxide failed, asking git: {}", e.message);
cli_status(r, &ignored)
})?;
Ok(map
.into_iter()
.filter(|(_, e)| e.staged.is_some() || e.unstaged.is_some())
// Ignored names mean "the IDE treats these as nonexistent": hide their tracked
// changes too, not just untracked hits above. Conflicts stay visible so merges
// cannot stall on a file the user cannot see.
.filter(|(rel, _)| conflicts.contains(rel) || !ignored.is_ignored_path(rel))
.map(|(rel, mut e)| {
if conflicts.contains(&rel) {
e.staged = None;
e.renamed_from = None;
e.unstaged = Some(ChangeKind::Conflicted);
}
FileStatus {
path: r.abs_path(&rel).into(),
rel_path: rel,
staged: e.staged,
unstaged: e.unstaged,
renamed_from: e.renamed_from,
}
})
.collect())
}
type Entries = (BTreeMap<String, Entry>, HashSet<String>);
fn gix_status(r: &Repo, ignored: &IgnoredNames) -> IdeResult<Entries> {
let repo = r.gix()?;
let mut map: BTreeMap<String, Entry> = BTreeMap::new();
let mut conflicts: HashSet<String> = HashSet::new();
let ignored = r.ignored.read().unwrap_or_else(|p| p.into_inner()).clone();
// Stamp before status can read the index: stamping after the load could capture
// an external `git switch`, `add` or commit's new index and allow the stale
@@ -118,28 +152,60 @@ pub(crate) fn status(r: &Repo) -> IdeResult<Vec<FileStatus>> {
}
}
Ok(map
.into_iter()
.filter(|(_, e)| e.staged.is_some() || e.unstaged.is_some())
// Ignored names mean "the IDE treats these as nonexistent": hide their tracked
// changes too, not just untracked hits above. Conflicts stay visible so merges
// cannot stall on a file the user cannot see.
.filter(|(rel, _)| conflicts.contains(rel) || !ignored.is_ignored_path(rel))
.map(|(rel, mut e)| {
if conflicts.contains(&rel) {
e.staged = None;
e.renamed_from = None;
e.unstaged = Some(ChangeKind::Conflicted);
Ok((map, conflicts))
}
/// `git status --porcelain=v2`: what git lists, unreadable directories skipped.
fn cli_status(r: &Repo, ignored: &IgnoredNames) -> IdeResult<Entries> {
let out = crate::cli::run(r.root(), &["status", "--porcelain=v2", "-z", "--untracked-files=all"], None)?;
Ok(parse_porcelain_v2(&String::from_utf8_lossy(&out.stdout), ignored))
}
fn parse_porcelain_v2(out: &str, ignored: &IgnoredNames) -> Entries {
let kind = |c: u8| match c {
b'M' | b'T' => Some(ChangeKind::Modified),
b'A' | b'C' => Some(ChangeKind::Added),
b'D' => Some(ChangeKind::Deleted),
b'R' => Some(ChangeKind::Renamed),
_ => None,
};
let mut map: BTreeMap<String, Entry> = BTreeMap::new();
let mut conflicts: HashSet<String> = HashSet::new();
let mut records = out.split('\0');
while let Some(record) = records.next() {
let (tag, rest) = record.split_once(' ').unwrap_or((record, ""));
match tag {
// `1 XY sub mH mI mW hH hI path`, `2 … Xscore path` + NUL + the original path.
"1" | "2" => {
let fields = if tag == "1" { 8 } else { 9 };
let mut parts = rest.splitn(fields, ' ');
let xy = parts.next().unwrap_or("..").as_bytes();
let Some(path) = parts.nth(fields - 2) else { continue };
let e = map.entry(path.to_owned()).or_default();
e.staged = xy.first().copied().and_then(kind);
// Intent to add shows as `.A`; worktree renames are off, as with gitoxide.
e.unstaged = xy.get(1).copied().and_then(kind).filter(|k| *k != ChangeKind::Renamed);
if tag == "2" {
let from = records.next().unwrap_or_default();
if e.staged == Some(ChangeKind::Renamed) {
e.renamed_from = Some(from.to_owned());
}
}
}
FileStatus {
path: r.abs_path(&rel).into(),
rel_path: rel,
staged: e.staged,
unstaged: e.unstaged,
renamed_from: e.renamed_from,
// `u XY sub m1 m2 m3 mW h1 h2 h3 path`
"u" => {
if let Some(path) = rest.splitn(10, ' ').nth(9) {
conflicts.insert(path.to_owned());
map.entry(path.to_owned()).or_default().unstaged = Some(ChangeKind::Conflicted);
}
}
})
.collect())
"?" if !rest.is_empty() && !ignored.is_ignored_path(rest) => {
map.entry(rest.to_owned()).or_default().unstaged = Some(ChangeKind::Untracked);
}
_ => {}
}
}
(map, conflicts)
}
/// Lets a test land an external index write in the window between the status run
@@ -227,4 +293,59 @@ mod tests {
let staged = Command::new("git").current_dir(dir).args(["diff", "--cached", "--stat"]).output().unwrap();
assert!(staged.stdout.is_empty(), "{}", String::from_utf8_lossy(&staged.stdout));
}
/// git's porcelain answer (used when gitoxide can't read the tree) matches gitoxide's.
#[test]
fn git_and_gitoxide_report_the_same_status() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
git(dir, &["init", "-q"]);
git(dir, &["config", "user.name", "Test User"]);
git(dir, &["config", "user.email", "[email protected]"]);
git(dir, &["config", "commit.gpgsign", "false"]);
for f in ["modified.txt", "staged.txt", "deleted.txt", "staged_del.txt", "both.txt", "conflict.txt"] {
std::fs::write(dir.join(f), "a\n").unwrap();
}
std::fs::write(dir.join("rename_me.txt"), "some longer content\nthat is stable\nfor rename detection\n")
.unwrap();
commit_all(dir, "base");
git(dir, &["checkout", "-q", "-b", "side"]);
std::fs::write(dir.join("conflict.txt"), "side\n").unwrap();
commit_all(dir, "side");
git(dir, &["checkout", "-q", "main"]);
std::fs::write(dir.join("conflict.txt"), "main\n").unwrap();
commit_all(dir, "main");
let merge = Command::new("git").current_dir(dir).args(["merge", "-q", "side"]).output().unwrap();
assert!(!merge.status.success());
std::fs::write(dir.join("modified.txt"), "b\n").unwrap();
std::fs::write(dir.join("staged.txt"), "b\n").unwrap();
git(dir, &["add", "staged.txt"]);
std::fs::remove_file(dir.join("deleted.txt")).unwrap();
git(dir, &["rm", "-q", "staged_del.txt"]);
std::fs::write(dir.join("both.txt"), "b\n").unwrap();
git(dir, &["add", "both.txt"]);
std::fs::write(dir.join("both.txt"), "c\n").unwrap();
git(dir, &["mv", "rename_me.txt", "renamed with space.txt"]);
std::fs::create_dir_all(dir.join("new dir")).unwrap();
std::fs::write(dir.join("new dir/untracked.txt"), "u\n").unwrap();
std::fs::write(dir.join("intent.txt"), "i\n").unwrap();
git(dir, &["add", "-N", "intent.txt"]);
let repo = Repo::discover(dir).unwrap().expect("repository found");
let ignored = IgnoredNames::default();
let show = |(map, conflicts): Entries| {
let mut c: Vec<_> = conflicts.into_iter().collect();
c.sort();
let m: Vec<_> = map
.into_iter()
.map(|(k, e)| format!("{k}: {:?} {:?} {:?}", e.staged, e.unstaged, e.renamed_from))
.collect();
(m, c)
};
let from_git = show(cli_status(&repo, &ignored).unwrap());
assert_eq!(from_git, show(gix_status(&repo, &ignored).unwrap()));
assert!(from_git.0.iter().any(|l| l.starts_with("renamed with space.txt: Some(Renamed)")), "{from_git:?}");
assert_eq!(from_git.1, ["conflict.txt"]);
}
}
+4 -1
Переглянути файл
@@ -40,8 +40,11 @@ impl Fixture {
self.git(&["add", "-A"]);
self.git(&["commit", "-q", "-m", msg]);
}
/// The repository of a trusted project.
fn repo(&self) -> Repo {
Repo::discover(&self.dir).unwrap().expect("repository found")
let repo = Repo::discover(&self.dir).unwrap().expect("repository found");
repo.set_execution_allowed(true);
repo
}
fn path(&self, rel: &str) -> PathBuf {
self.dir.join(rel)
+4 -1
Переглянути файл
@@ -103,8 +103,11 @@ impl Fixture {
fn subjects(&self) -> Vec<String> {
self.git(&["log", "--format=%s"]).lines().map(str::to_string).collect()
}
/// The repository of a trusted project.
fn repo(&self) -> Repo {
Repo::discover(&self.dir).unwrap().expect("repository found")
let repo = Repo::discover(&self.dir).unwrap().expect("repository found");
repo.set_execution_allowed(true);
repo
}
fn path(&self, rel: &str) -> PathBuf {
self.dir.join(rel)
+4 -1
Переглянути файл
@@ -58,8 +58,11 @@ impl Fixture {
self.git(&["commit", "-q", "-m", msg]);
self.git(&["rev-parse", "HEAD"])
}
/// The repository of a trusted project.
fn repo(&self) -> Repo {
Repo::discover(&self.dir).unwrap().expect("repository found")
let repo = Repo::discover(&self.dir).unwrap().expect("repository found");
repo.set_execution_allowed(true);
repo
}
fn path(&self, rel: &str) -> PathBuf {
self.dir.join(rel)
+26 -1
Переглянути файл
@@ -65,8 +65,11 @@ impl Fixture {
self.git(&["commit", "-q", "-m", msg]);
self.git(&["rev-parse", "HEAD"])
}
/// The repository of a trusted project.
fn repo(&self) -> Repo {
Repo::discover(&self.dir).unwrap().expect("repository found")
let repo = Repo::discover(&self.dir).unwrap().expect("repository found");
repo.set_execution_allowed(true);
repo
}
fn path(&self, rel: &str) -> PathBuf {
self.dir.join(rel)
@@ -724,6 +727,7 @@ fn file_at_in_a_blobless_clone() {
let url = format!("file://{}", fx.dir.display());
git_in(fx.dir.parent().unwrap(), &["clone", "-q", "--filter=blob:none", &url, "clone"]);
let repo = Repo::discover(&clone).unwrap().unwrap();
repo.set_execution_allowed(true);
// v1's blob was never fetched; reading it fetches it on demand.
assert_eq!(repo.file_at(&clone.join("f.txt"), Some(&c1), encoding_rs::UTF_8).unwrap().as_deref(), Some("v1\n"));
}
@@ -2011,3 +2015,24 @@ fn reword_refuses_commits_outside_the_current_branch() {
fx.git(&["checkout", "-q", "main"]);
assert!(fx.repo().reword(&side, "other").is_err());
}
#[cfg(unix)]
#[test]
fn status_skips_a_directory_it_cannot_read() {
// `git status` warns "could not open directory: Permission denied" and lists the rest.
use std::os::unix::fs::PermissionsExt;
let fx = Fixture::new();
fx.write("a.txt", "a\n");
fx.commit_all("init");
fx.write("a.txt", "b\n");
fx.write("new.txt", "n\n");
let locked = fx.path("locked");
std::fs::create_dir(&locked).unwrap();
std::fs::write(locked.join("inside.txt"), "x").unwrap();
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o000)).unwrap();
let st = fx.repo().status();
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o755)).unwrap();
let st = st.unwrap();
assert_eq!(status_of(&st, "a.txt").unstaged, Some(ChangeKind::Modified));
assert_eq!(status_of(&st, "new.txt").unstaged, Some(ChangeKind::Untracked));
}
+184 -4
Переглянути файл
@@ -97,8 +97,80 @@ pub fn image_mime(path: &Path) -> Option<&'static str> {
})
}
/// The whole image as a `data:` URL (allowed by the webview's CSP, unlike file URLs).
pub fn read_data_url(path: &Path) -> IdeResult<String> {
/// The server and share of a Windows network path (`\\server\share\…`, also written with `/`
/// or as `\\?\UNC\…`).
#[cfg(windows)]
fn share(path: &Path) -> Option<(&std::ffi::OsStr, &std::ffi::OsStr)> {
use std::path::{Component, Prefix};
let mut components = path.components();
let Component::Prefix(prefix) = components.next()? else { return None };
match prefix.kind() {
// `//?/UNC/server/share`, the spelling a canonical share path crosses the IPC boundary
// in (`ide_core::ipc_path`): with `/`, `std` reads the share `UNC` of a server `?`.
Prefix::UNC(device, unc) if device == "?" => {
let names = (components.next()?, components.next()?, components.next()?);
match names {
(Component::RootDir, Component::Normal(server), Component::Normal(share))
if unc.eq_ignore_ascii_case("UNC") =>
{
Some((server, share))
}
_ => None,
}
}
Prefix::UNC(server, share) | Prefix::VerbatimUNC(server, share) => Some((server, share)),
_ => None,
}
}
/// Whether `path` names a file on one of this machine's drives. On Windows that is an absolute
/// path starting at a drive letter. Every other spelling can lead to a server, and opening a
/// share signs in to its server with the user's credentials: `\\server\share`, `\\?\UNC\…`,
/// device paths, and `\??\UNC\…`, which carries no prefix `std` knows and which Windows opens
/// as written.
pub fn on_local_drive(path: &Path) -> bool {
#[cfg(windows)]
{
use std::path::{Component, Prefix};
path.is_absolute()
&& matches!(
path.components().next(),
Some(Component::Prefix(prefix)) if matches!(prefix.kind(), Prefix::Disk(_) | Prefix::VerbatimDisk(_))
)
}
#[cfg(not(windows))]
{
let _ = path;
true
}
}
/// Whether two network paths are on the same share of the same server.
pub fn same_share(a: &Path, b: &Path) -> bool {
#[cfg(windows)]
{
let same = |x: &std::ffi::OsStr, y: &std::ffi::OsStr| x.eq_ignore_ascii_case(y);
matches!((share(a), share(b)), (Some(a), Some(b)) if same(a.0, b.0) && same(a.1, b.1))
}
#[cfg(not(windows))]
{
let _ = (a, b);
false
}
}
/// Whether a path written in content (Markdown, a page, an agent's reply) may be opened: one
/// on a local drive, or on the share one of `opened` (what the user opened) is on.
pub fn content_path_allowed<P: AsRef<Path>>(path: &Path, opened: &[P]) -> bool {
on_local_drive(path) || opened.iter().any(|o| same_share(o.as_ref(), path))
}
/// The whole image as a `data:` URL (allowed by the webview's CSP, unlike file URLs). The path
/// often comes from text someone else wrote (Markdown): see [`content_path_allowed`].
pub fn read_data_url(path: &Path, opened: &[std::path::PathBuf]) -> IdeResult<String> {
if !content_path_allowed(path, opened) {
return Err(IdeError::invalid(format!("{} is not on a local drive or an opened share", path.display())));
}
let mime =
image_mime(path).ok_or_else(|| IdeError::invalid(format!("{} is not a supported image", path.display())))?;
if let Some(bytes) = archive_bytes(path, MAX_IMAGE_SIZE)? {
@@ -138,6 +210,114 @@ pub fn base64(bytes: &[u8]) -> String {
mod tests {
use super::*;
/// Security audit lead 3: an image path from agent markdown must not make the host open a
/// network share (on Windows that authenticates to the named host with the user's credentials).
#[cfg(windows)]
#[test]
fn read_data_url_refuses_network_paths() {
let dir = tempfile::tempdir().unwrap();
let local = dir.path().join("dummy.png");
std::fs::write(&local, b"\x89PNG dummy").unwrap();
// The same file reached over SMB through the loopback admin share.
let s = local.to_str().unwrap();
let back = format!(r"\\127.0.0.1\{}${}", &s[..1], &s[2..]);
// The webview hands the host the forward-slash form (`file://host/share` -> `//host/share`).
let forward = back.replace('\\', "/");
// The NT spelling: no `\\` in front, and Windows opens it all the same.
let nt = format!(r"\??\UNC{}", &back[1..]);
for unc in [forward, back, nt].map(std::path::PathBuf::from) {
assert!(
std::fs::read(&unc).is_ok(),
"control: {} is not reachable over SMB, fixture not live",
unc.display()
);
let read = read_data_url(&unc, &[]);
assert!(read.is_err(), "read_data_url opened {} over SMB: {:?}", unc.display(), read.map(|s| s.len()));
}
}
/// The exception to the above: an image on the share a project is open on, and only there.
#[cfg(windows)]
#[test]
fn read_data_url_reads_the_share_a_project_is_on() {
let dir = tempfile::tempdir().unwrap();
let local = dir.path().join("dummy.png");
std::fs::write(&local, b"\x89PNG dummy").unwrap();
let s = local.to_str().unwrap();
let on = |server: &str| std::path::PathBuf::from(format!(r"\\{server}\{}${}", &s[..1], &s[2..]));
let image = on("127.0.0.1");
let wanted = read_data_url(&local, &[]).unwrap();
// A project root as typed, and as `canonicalize` returns it.
for root in [r"\\127.0.0.1\C$\work", r"\\?\UNC\127.0.0.1\c$\work"].map(std::path::PathBuf::from) {
let opened = [std::path::PathBuf::from(r"D:\other"), root];
assert_eq!(read_data_url(&image, &opened).unwrap(), wanted);
assert_eq!(read_data_url(Path::new(&image.to_str().unwrap().replace('\\', "/")), &opened).unwrap(), wanted);
// The spelling paths of a project on a share cross the IPC boundary in.
let wire = |path: &Path| std::path::PathBuf::from(format!("//?/UNC/{}", &path.to_str().unwrap()[2..]));
assert_eq!(read_data_url(&wire(&image), &opened).unwrap(), wanted);
assert_eq!(read_data_url(&image, &[wire(Path::new(r"\\127.0.0.1\C$\work"))]).unwrap(), wanted);
// The same file through another name of this machine: a server the user did not open.
let other = on("localhost");
assert!(std::fs::read(&other).is_ok(), "control: {} is not reachable over SMB", other.display());
for other in [wire(&other), other] {
let read = read_data_url(&other, &opened);
assert!(
read.is_err(),
"read_data_url opened {} over SMB: {:?}",
other.display(),
read.map(|s| s.len())
);
}
}
}
#[cfg(windows)]
#[test]
fn only_absolute_paths_starting_at_a_drive_letter_are_local() {
for path in [r"C:\a.png", "C:/a.png", r"\\?\C:\a.png", r"z:\share\a.png"] {
assert!(on_local_drive(Path::new(path)), "{path}");
}
for path in [
r"\\server\share\a.png",
"//server/share/a.png",
r"/\server/share/a.png",
r"\\?\UNC\server\share\a.png",
"//?/UNC/server/share/a.png",
r"\\.\UNC\server\share\a.png",
r"\\?\GLOBALROOT\Device\Mup\server\share\a.png",
r"\??\UNC\server\share\a.png",
r"\??\C:\a.png",
r"\a.png",
r"C:a.png",
"a.png",
r"..\a.png",
] {
assert!(!on_local_drive(Path::new(path)), "{path}");
}
let project = Path::new(r"\\nas\work\app");
assert!(same_share(project, Path::new("//NAS/Work/app/docs/a.png")));
assert!(same_share(project, Path::new(r"\\?\UNC\nas\work\other\a.png")));
assert!(!same_share(project, Path::new(r"\\nas\private\a.png")));
assert!(!same_share(project, Path::new(r"\\evil\work\app\a.png")));
assert!(!same_share(project, Path::new(r"\??\UNC\nas\work\app\a.png")));
assert!(!same_share(Path::new(r"C:\app"), Path::new(r"C:\app\a.png")));
// Read by `std` as the share `UNC` of a server named `?`: the spelling a share's paths
// cross the IPC boundary in.
assert!(!same_share(Path::new("//?/UNC/nas/work/app"), Path::new("//?/UNC/evil/share/a.png")));
assert!(same_share(Path::new("//?/UNC/nas/work/app"), Path::new("//?/unc/NAS/Work/docs/a.png")));
assert!(same_share(project, Path::new("//?/UNC/nas/work/docs/a.png")));
assert!(same_share(Path::new(r"\\?\UNC\nas\work\app"), Path::new("//?/UNC/nas/work/docs/a.png")));
for device in ["//?/UNC/nas", "//?/UNC/../nas/work/a.png", "//?/C:/a.png", "//?/nas/work/a.png"] {
assert!(!same_share(Path::new(device), Path::new(device)), "{device}");
}
assert!(content_path_allowed(Path::new(r"C:\app\a.png"), &[project]));
assert!(content_path_allowed(Path::new(r"\\nas\work\docs\a.png"), &[Path::new(r"C:\app"), project]));
assert!(!content_path_allowed(Path::new(r"\\nas\private\a.png"), &[project]));
assert!(!content_path_allowed(Path::new(r"\\nas\work\docs\a.png"), &[Path::new(r"C:\app")]));
assert!(!content_path_allowed(Path::new(r"\\nas\work\docs\a.png"), &[] as &[&Path]));
}
#[test]
fn base64_matches_rfc4648_vectors() {
for (input, expected) in [
@@ -190,12 +370,12 @@ mod tests {
let png = dir.path().join("x.PNG");
fs::write(&png, [0x89, b'P', b'N', b'G']).unwrap();
assert_eq!(
read_data_url(&png).unwrap(),
read_data_url(&png, &[]).unwrap(),
format!("data:image/png;base64,{}", base64(&[0x89, b'P', b'N', b'G']))
);
let txt = dir.path().join("x.txt");
fs::write(&txt, "x").unwrap();
assert!(read_data_url(&txt).is_err());
assert!(read_data_url(&txt, &[]).is_err());
assert_eq!(image_mime(Path::new("a/b.svg")), Some("image/svg+xml"));
}
}
Loaded 100 of 371 files, more files were not shown because too many files have changed in this diff. Show more